Skip to content
Back to feed

July 21, 2026 · Together AI

Fine-tuning on MoE models can target expert feed-forward layers via lora_trainable_modules

Fine-tuning jobs on supported mixture-of-experts models can now target expert feed-forward layers instead of attention projections. Set lora_trainable_modules to w_up,w_gate,w_down to train a compact shared-factor adapter across experts, useful when adapting domain knowledge rather than attention patterns.

See Target MoE expert layers.

Latest Together AI platform changes

All Together AI platform changes