← Back to community benchmarks
gemma-4-31b-it
Performance
16k
tokens
331.0
PP tok/s
6.5
TG tok/s
49499
TTFT (ms)
34.8
Peak mem (GB)
Hardware
Chip
M5 Pro (20c)
Memory
64 GB
GPU Cores
20
Software
oMLX
v0.3.5.dev1
macOS
macOS 26.4
Context
16,384
Performance by Context Length
| Context | PP tok/s | TG tok/s | Peak Mem | |
|---|---|---|---|---|
| 1k | 320.3 | 8.1 | 31.8 GB | view |
| 4k | 343.5 | 7.9 | 33.6 GB | view |
| 8k | 336.3 | 7.4 | 34.0 GB | view |
| 16k | 331.0 | 6.5 | 34.8 GB | current |
| 32k | 275.5 | 4.3 | 37.1 GB | view |
| 64k | 234.7 | 3.3 | 41.7 GB | view |
Batching Results
| Batch Size | TG tok/s | Speedup |
|---|---|---|
| 1× | 8.1 | 1.00× |
| 2× | 13.6 | 1.68× |
| 4× | 20.3 | 2.51× |
| 8× | 24.5 | 3.02× |