Skip to content
Back to feed

September 11, 2026 · Together AI

LoRA fine-tuning no longer targets expert layers of FP4, MXFP4 and NVFP4 MoE models

LoRA fine-tuning jobs can no longer target the expert layers (w_up, w_gate, w_down) of mixture-of-experts models served in FP4, MXFP4, or NVFP4, including deepseek-ai/DeepSeek-V3.1. These models support attention-only LoRA, and creating a job that lists an expert module returns a validation error.

See Target MoE expert layers.

All Together AI platform changes