Compare Models
Select up to 4 models to compare benchmarks, pricing, and capabilities side by side.
Anthropic
Mistral AI
Shanghai AI Lab
Add Model
MMLU
Claude Haiku 3.5
85.2
Mistral Small
81.2
InternLM3 8B
72.0
HumanEval
Claude Haiku 3.5
88.1
Mistral Small
84.8
InternLM3 8B
68.0
GSM8K
Claude Haiku 3.5
91.6
Mistral Small
88.4
InternLM3 8B
78.0
GPQA
Claude Haiku 3.5
41.6
Mistral Small
37.5
InternLM3 8B
0.0
MGSM
Claude Haiku 3.5
88.5
Mistral Small
80.1
InternLM3 8B
0.0
ARC-Challenge
Claude Haiku 3.5
93.5
Mistral Small
89.5
InternLM3 8B
0.0
HellaSwag
Claude Haiku 3.5
89.5
Mistral Small
84.0
InternLM3 8B
78.0
MATH
Claude Haiku 3.5
69.2
Mistral Small
61.0
InternLM3 8B
52.0
SWE-bench
Claude Haiku 3.5
40.6
Mistral Small
18.5
InternLM3 8B
0.0
MMMLU
Claude Haiku 3.5
81.7
Mistral Small
73.2
InternLM3 8B
0.0
| Model | Input | Output | Blended* |
|---|---|---|---|
Claude Haiku 3.5 | $0.80 | $4.00 | $2.40 |
Mistral Small | $0.10 | $0.30 | $0.20 |
InternLM3 8B | $0.07 | $0.14 | $0.11 |
*Blended = average of input and output price
| Spec | Claude Haiku 3.5 | Mistral Small | InternLM3 8B |
|---|---|---|---|
| Context Window | 200K | 32K | 128K |
| Max Output | 8K | 4K | N/A |
| TTFT | 150ms | 140ms | N/A |
| Speed | 160 tok/s | 170 tok/s | N/A |
| Parameters | N/A | 24B | 8B |
| Architecture | Transformer | Transformer | Dense Transformer |
| Open Source | No | No | Yes |
| Tier | budget | budget | budget |
Quick Verdict
Best Performance
Claude Haiku 3.5
Best Value
InternLM3 8B
Fastest
Mistral Small