← Back to community benchmarks
Step-3.7-Flash-oQ3e
Performance
128k
tokens
492.1
PP tok/s
27.1
TG tok/s
266364
TTFT (ms)
84.1
Peak mem (GB)
Hardware
Chip
M5 Max (40c)
Memory
128 GB
GPU Cores
40
Software
oMLX
v0.5.2rc1
macOS
macOS 26.5.1
Context
131,072
Performance by Context Length
| Context | PP tok/s | TG tok/s | Peak Mem | |
|---|---|---|---|---|
| 1k | 781.2 | 54.1 | 77.8 GB | view |
| 4k | 779.9 | 53.2 | 77.9 GB | view |
| 8k | 748.4 | 51.4 | 78.1 GB | view |
| 64k | 583.1 | 34.9 | 80.8 GB | view |
| 128k | 492.1 | 27.1 | 84.1 GB | current |
Batching Results
| Batch Size | TG tok/s | Speedup |
|---|---|---|
| 1× | 54.1 | 1.00× |
| 2× | 75.2 | 1.39× |
| 4× | 98.0 | 1.81× |
| 8× | 196.2 | 3.63× |