Fine-tuning context lengths increased for DeepSeek-R1-Distill, Gemma 3, Qwen and Llama models

DeepSeek models

• DeepSeek-R1-Distill-Llama-70B: SFT 8,192 → 24,576. DPO 8,192 → 8,192.
• DeepSeek-R1-Distill-Qwen-14B: SFT 8,192 → 65,536. DPO 8,192 → 12,288.
• DeepSeek-R1-Distill-Qwen-1.5B: SFT 8,192 → 131,072. DPO 8,192 → 16,384.

Google Gemma models

• gemma-3-1b-it: SFT 16,384 → 32,768. DPO 16,384 → 12,288.
• gemma-3-1b-pt: SFT 16,384 → 32,768. DPO 16,384 → 12,288.
• gemma-3-4b-it: SFT 16,384 → 131,072. DPO 16,384 → 12,288.
• gemma-3-4b-pt: SFT 16,384 → 131,072. DPO 16,384 → 12,288.
• gemma-3-12b-pt: SFT 16,384 → 65,536. DPO 16,384 → 8,192.
• gemma-3-27b-it: SFT 12,288 → 49,152. DPO 12,288 → 8,192.
• gemma-3-27b-pt: SFT 12,288 → 49,152. DPO 12,288 → 8,192.

Qwen models

• Qwen3-0.6B / Qwen3-0.6B-Base: SFT 8,192 → 32,768. DPO 8,192 → 24,576.
• Qwen3-1.7B / Qwen3-1.7B-Base: SFT 8,192 → 32,768. DPO 8,192 → 16,384.
• Qwen3-4B / Qwen3-4B-Base: SFT 8,192 → 32,768. DPO 8,192 → 16,384.
• Qwen3-8B / Qwen3-8B-Base: SFT 8,192 → 32,768. DPO 8,192 → 16,384.
• Qwen3-14B / Qwen3-14B-Base: SFT 8,192 → 32,768. DPO 8,192 → 16,384.
• Qwen3-32B: SFT 8,192 → 24,576. DPO 8,192 → 4,096.
• Qwen2.5-72B-Instruct: SFT 8,192 → 24,576. DPO 8,192 → 8,192.
• Qwen2.5-32B-Instruct: SFT 8,192 → 32,768. DPO 8,192 → 12,288.
• Qwen2.5-32B: SFT 8,192 → 49,152. DPO 8,192 → 12,288.
• Qwen2.5-14B-Instruct: SFT 8,192 → 32,768. DPO 8,192 → 16,384.
• Qwen2.5-14B: SFT 8,192 → 65,536. DPO 8,192 → 16,384.
• Qwen2.5-7B-Instruct: SFT 8,192 → 32,768. DPO 8,192 → 16,384.
• Qwen2.5-7B: SFT 8,192 → 131,072. DPO 8,192 → 16,384.
• Qwen2.5-3B-Instruct: SFT 8,192 → 32,768. DPO 8,192 → 16,384.
• Qwen2.5-3B: SFT 8,192 → 32,768. DPO 8,192 → 16,384.
• Qwen2.5-1.5B-Instruct: SFT 8,192 → 32,768. DPO 8,192 → 16,384.
• Qwen2.5-1.5B: SFT 8,192 → 32,768. DPO 8,192 → 16,384.
• Qwen2-72B-Instruct / Qwen2-72B: SFT 8,192 → 32,768. DPO 8,192 → 8,192.
• Qwen2-7B-Instruct: SFT 8,192 → 32,768. DPO 8,192 → 16,384.
• Qwen2-7B: SFT 8,192 → 131,072. DPO 8,192 → 16,384.
• Qwen2-1.5B-Instruct: SFT 8,192 → 32,768. DPO 8,192 → 16,384.
• Qwen2-1.5B: SFT 8,192 → 131,072. DPO 8,192 → 16,384.

Meta Llama models

• Llama-3.3-70B-Instruct-Reference: SFT 8,192 → 24,576. DPO 8,192 → 8,192.
• Llama-3.2-3B-Instruct: SFT 8,192 → 131,072. DPO 8,192 → 24,576.
• Llama-3.2-1B-Instruct: SFT 8,192 → 131,072. DPO 8,192 → 24,576.
• Meta-Llama-3.1-8B-Instruct-Reference: SFT 8,192 → 131,072. DPO 8,192 → 16,384.
• Meta-Llama-3.1-8B-Reference: SFT 8,192 → 131,072. DPO 8,192 → 16,384.
• Meta-Llama-3.1-70B-Instruct-Reference: SFT 8,192 → 24,576. DPO 8,192 → 8,192.
• Meta-Llama-3.1-70B-Reference: SFT 8,192 → 24,576. DPO 8,192 → 8,192.

Mistral models

• mistralai/Mistral-7B-v0.1: SFT 8,192 → 32,768. DPO 8,192 → 32,768.
• teknium/OpenHermes-2p5-Mistral-7B: SFT 8,192 → 32,768. DPO 8,192 → 32,768.