July 21, 2026 · Together AI
Fine-tuning on MoE models can target expert feed-forward layers via lora_trainable_modules
Fine-tuning jobs on supported mixture-of-experts models can now target expert feed-forward layers instead of attention projections. Set lora_trainable_modules to w_up,w_gate,w_down to train a compact shared-factor adapter across experts, useful when adapting domain knowledge rather than attention patterns.