Model · llama · released 2024-12

What runs Llama 3.3 70B Instruct

Capacity is charged on 70.6 B total parameters. Per-token traffic is charged on 70.6 B active parameters. Every rate below is an estimate at a 1,024-token position, batch 1, single stream.

Open in the calculator → Compare the two ends of the table

Sourced architecture

Parameter accounting

Capacity is charged on total parameters. Traffic is charged on active parameters.

Fact	Value
Architecture	Dense
Total parameters	70.6 B
Active parameters per token	70.6 B
Attention	GQA
Layer groups	80 layers · 64 heads, 8 KV heads, head dim 128, full attention
Vocabulary	128,256 tokens
Tied embeddings	No
Maximum context	131,072 tokens
Native precision	bf16

Weight accounting

Size at each quantization

An exact published artifact size always beats parameters × effective bits, and the basis column says which was used. Quality is an ordinal stated in the quant table, never derived from bit width.

Quantization	Effective bits per weight	Quality	Weights (capacity)	Active weights (per-token traffic)	Size basis
BF16bf16 · quant ID bf16	16	excellentRank 100	141.2 GB	141.2 GB	Computed
FP16fp16 · quant ID fp16	16	excellentRank 100	141.2 GB	141.2 GB	Computed
Q8_0gguf_kquant · quant ID q8_0	8.5	excellentRank 96	75 GB	75 GB	Computed
Q6_Kgguf_kquant · quant ID q6_k	6.56	excellentRank 92	57.9 GB	57.9 GB	Computed
FP8fp8 · quant ID fp8	8	excellentRank 90	70.6 GB	70.6 GB	Computed
Q5_K_Mgguf_kquant · quant ID q5_k_m	5.67	goodRank 86	50 GB	50 GB	Computed
Q5_K_Sgguf_kquant · quant ID q5_k_s	5.52	goodRank 83	48.7 GB	48.7 GB	Computed
IQ4_NLgguf_iquant · quant ID iq4_nl	4.5	goodRank 76	39.7 GB	39.7 GB	Computed
Q4_K_Mgguf_kquant · quant ID q4_k_m	4.83	goodRank 74	42.6 GB	42.6 GB	Computed
IQ4_XSgguf_iquant · quant ID iq4_xs	4.25	goodRank 73	37.5 GB	37.5 GB	Computed
Q4_K_Sgguf_kquant · quant ID q4_k_s	4.58	goodRank 70	40.4 GB	40.4 GB	Computed
AWQ 4-bitawq · quant ID awq_4bit	4.25	acceptableRank 68	37.5 GB	37.5 GB	Computed
GPTQ 4-bitgptq · quant ID gptq_4bit	4.25	acceptableRank 64	37.5 GB	37.5 GB	Computed
Q3_K_Lgguf_kquant · quant ID q3_k_l	4.27	acceptableRank 58	37.7 GB	37.7 GB	Computed
IQ3_Mgguf_iquant · quant ID iq3_m	3.66	acceptableRank 56	32.3 GB	32.3 GB	Computed
Q3_K_Mgguf_kquant · quant ID q3_k_m	3.91	acceptableRank 52	34.5 GB	34.5 GB	Computed
IQ3_Sgguf_iquant · quant ID iq3_s	3.44	acceptableRank 51	30.4 GB	30.4 GB	Computed
IQ3_XXSgguf_iquant · quant ID iq3_xxs	3.06	acceptableRank 45	27 GB	27 GB	Computed
Q3_K_Sgguf_kquant · quant ID q3_k_s	3.5	degradedRank 44	30.9 GB	30.9 GB	Computed
IQ2_Mgguf_iquant · quant ID iq2_m	2.7	degradedRank 34	23.8 GB	23.8 GB	Computed
Q2_Kgguf_kquant · quant ID q2_k	3.35	degradedRank 32	29.6 GB	29.6 GB	Computed
IQ2_Sgguf_iquant · quant ID iq2_s	2.5	degradedRank 30	22.1 GB	22.1 GB	Computed
IQ2_XSgguf_iquant · quant ID iq2_xs	2.31	degradedRank 25	20.4 GB	20.4 GB	Computed
IQ2_XXSgguf_iquant · quant ID iq2_xxs	2.06	degradedRank 18	18.2 GB	18.2 GB	Computed

Estimated ranges · not measured

Configurations ranked by decode range

Every range is a sensitivity band over the selected backend target's declared bandwidth-efficiency interval, at s = 1,024, batch 1, single stream. None of it is measured. The recommended quantization is the highest-ranked one that fits at that context, with the KV cache held at fp16. The next column re-runs the same choice at this model's full declared context of 131,072 tokens, where the cache is far larger.

Rank	Configuration	Recommended quantization	Still recommended at 131,072 tokens	Memory required	Decode range (estimate)	Weights-only ceiling	Confidence	Deep link
1	NVIDIA Rubin GPU 288 GB · config ID rubin-sxm-288gb	BF16 vllm-cuda · runner-up FP16	BF16	150 GB of 288 GB usable	90.2–124.4 tok/s estimate at s = 1,024	155.8 tok/s	Tier B matched target, exposed latency uncalibrated, computed artifact size	Open
2	NVIDIA H200 SXM 141 GB · config ID h200-sxm-141gb	Q8_0 llama-cpp-cuda · runner-up Q6_K	Q8_0	79.1 GB of 141 GB usable	38.2–52.2 tok/s estimate at s = 1,024	64.0 tok/s	Tier B matched target, exposed latency uncalibrated, computed artifact size	Open
3	NVIDIA GeForce RTX 5090 32 GB · config ID geforce-rtx-5090-32gb	IQ3_XXS llama-cpp-cuda · runner-up IQ2_M	None fitsthe cache alone exhausts usable memory	28.7 GB of 32 GB usable	34.1–49.8 tok/s estimate at s = 1,024	66.4 tok/s	Tier B matched target, exposed latency uncalibrated, computed artifact size	Open
4	NVIDIA B200 SXM 180 GB · config ID b200-sxm-180gb	BF16 vllm-cuda · runner-up FP16	Q8_0 drops from BF16	150 GB of 180 GB usable	32.8–45.2 tok/s estimate at s = 1,024	56.7 tok/s	Tier B matched target, exposed latency uncalibrated, computed artifact size	Open
5	NVIDIA B300 SXM 288 GB · config ID b300-sxm-288gb	BF16 vllm-cuda · runner-up FP16	BF16	150 GB of 288 GB usable	32.8–45.2 tok/s estimate at s = 1,024	56.7 tok/s	Tier B matched target, exposed latency uncalibrated, computed artifact size	Open
6	NVIDIA H100 SXM 80 GB · config ID h100-sxm-80gb	Q8_0 llama-cpp-cuda · runner-up Q6_K	IQ3_M drops from Q8_0	79.1 GB of 80 GB usable	26.7–36.5 tok/s estimate at s = 1,024	44.7 tok/s	Tier B matched target, exposed latency uncalibrated, computed artifact size	Open
7	NVIDIA GeForce RTX 4090 24 GB · config ID geforce-rtx-4090-24gb	IQ2_S llama-cpp-cuda · runner-up IQ2_XS	None fitsthe cache alone exhausts usable memory	23.5 GB of 24 GB usable	23.4–34.2 tok/s estimate at s = 1,024	45.7 tok/s	Tier B matched target, exposed latency uncalibrated, computed artifact size	Open
8	NVIDIA GeForce RTX 3090 24 GB · config ID geforce-rtx-3090-24gb	IQ2_S llama-cpp-cuda · runner-up IQ2_XS	None fitsthe cache alone exhausts usable memory	23.5 GB of 24 GB usable	21.7–31.8 tok/s estimate at s = 1,024	42.4 tok/s	Tier B matched target, exposed latency uncalibrated, computed artifact size	Open
9	AMD Radeon RX 7900 XTX 24 GB · config ID radeon-rx-7900-xtx-24gb	IQ2_S llama-cpp-rocm · runner-up IQ2_XS	None fitsthe cache alone exhausts usable memory	23.5 GB of 24 GB usable	19.3–29.1 tok/s estimate at s = 1,024	43.5 tok/s	Tier B matched target, exposed latency uncalibrated, computed artifact size	Open
10	AMD Instinct MI300X 192 GB · config ID instinct-mi300x-192gb	BF16 vllm-rocm · runner-up FP16	Q8_0 drops from BF16	150 GB of 192 GB usable	18.7–27.3 tok/s estimate at s = 1,024	37.5 tok/s	Tier B matched target, exposed latency uncalibrated, computed artifact size	Open
11	AMD Instinct MI50 32 GB · config ID instinct-mi50-32gb	IQ3_XXS llama-cpp-rocm · runner-up IQ2_M	None fitsthe cache alone exhausts usable memory	28.7 GB of 32 GB usable	16.9–25.5 tok/s estimate at s = 1,024	37.9 tok/s	Tier B matched target, exposed latency uncalibrated, computed artifact size	Open
12	NVIDIA A100 SXM 80 GB · config ID a100-sxm-80gb	Q8_0 llama-cpp-cuda · runner-up Q6_K	IQ3_M drops from Q8_0	79.1 GB of 80 GB usable	15.9–21.8 tok/s estimate at s = 1,024	26.7 tok/s	Tier B matched target, exposed latency uncalibrated, computed artifact size	Open
13	NVIDIA RTX PRO 6000 Blackwell Workstation Edition 96 GB · config ID rtx-pro-6000-blackwell-96gb	Q8_0 llama-cpp-cuda · runner-up Q6_K	Q5_K_M drops from Q8_0	79.1 GB of 96 GB usable	14.3–19.5 tok/s estimate at s = 1,024	23.9 tok/s	Tier B matched target, exposed latency uncalibrated, computed artifact size	Open
14	NVIDIA RTX 6000 Ada Generation 48 GB · config ID rtx-6000-ada-48gb	IQ4_NL llama-cpp-cuda · runner-up Q4_K_M	None fitsthe cache alone exhausts usable memory	42 GB of 48 GB usable	12.5–18.2 tok/s estimate at s = 1,024	24.2 tok/s	Tier B matched target, exposed latency uncalibrated, computed artifact size	Open
15	Apple M5 Ultra 96 GB · config ID apple-m5-ultra-96gb	Q6_K llama-cpp-metal · runner-up Q5_K_M	IQ3_M drops from Q6_K	60.5 GB of 76.8 GB usable	12.0–16.5 tok/s estimate at s = 1,024	20.7 tok/s	Tier B matched target, exposed latency uncalibrated, computed artifact size	Open
16	NVIDIA L40S 48 GB · config ID l40s-48gb	IQ4_NL llama-cpp-cuda · runner-up Q4_K_M	None fitsthe cache alone exhausts usable memory	42 GB of 48 GB usable	11.2–16.4 tok/s estimate at s = 1,024	21.8 tok/s	Tier B matched target, exposed latency uncalibrated, computed artifact size	Open
17	Apple M5 Max 48 GB / 614 GB/s · config ID apple-m5-max-48gb-614	IQ3_M llama-cpp-metal · runner-up Q3_K_M	None fitsthe cache alone exhausts usable memory	33.9 GB of 38.4 GB usable	9.4–13.7 tok/s estimate at s = 1,024	19.0 tok/s	Tier B matched target, exposed latency uncalibrated, computed artifact size	Open
18	Apple M1 Ultra 64 GB · config ID apple-m1-ultra-64gb	Q5_K_S llama-cpp-metal · runner-up IQ4_NL	None fitsthe cache alone exhausts usable memory	51 GB of 51.2 GB usable	9.5–13.0 tok/s estimate at s = 1,024	16.4 tok/s	Tier B matched target, exposed latency uncalibrated, computed artifact size	Open
19	Apple M5 Max 36 GB / 460 GB/s · config ID apple-m5-max-36gb-460	IQ3_XXS llama-cpp-metal · runner-up IQ2_M	None fitsthe cache alone exhausts usable memory	28.4 GB of 28.8 GB usable	8.4–12.3 tok/s estimate at s = 1,024	17.0 tok/s	Tier B matched target, exposed latency uncalibrated, computed artifact size	Open
20	Apple M4 Max 48 GB / 546 GB/s · config ID apple-m4-max-48gb-546	IQ3_M llama-cpp-metal · runner-up Q3_K_M	None fitsthe cache alone exhausts usable memory	33.9 GB of 38.4 GB usable	8.4–12.2 tok/s estimate at s = 1,024	16.9 tok/s	Tier B matched target, exposed latency uncalibrated, computed artifact size	Open
21	NVIDIA Tesla P40 24 GB · config ID tesla-p40-24gb	IQ2_S llama-cpp-cuda · runner-up IQ2_XS	None fitsthe cache alone exhausts usable memory	23.5 GB of 24 GB usable	8.0–11.7 tok/s estimate at s = 1,024	15.7 tok/s	Tier B matched target, exposed latency uncalibrated, computed artifact size	Open
22	Apple M3 Ultra 96 GB · config ID apple-m3-ultra-96gb	Q6_K llama-cpp-metal · runner-up Q5_K_M	IQ3_M drops from Q6_K	60.5 GB of 76.8 GB usable	8.2–11.3 tok/s estimate at s = 1,024	14.1 tok/s	Tier B matched target, exposed latency uncalibrated, computed artifact size	Open
23	Apple M4 Max 36 GB / 410 GB/s · config ID apple-m4-max-36gb-410	IQ3_XXS llama-cpp-metal · runner-up IQ2_M	None fitsthe cache alone exhausts usable memory	28.4 GB of 28.8 GB usable	7.5–10.9 tok/s estimate at s = 1,024	15.2 tok/s	Tier B matched target, exposed latency uncalibrated, computed artifact size	Open
24	Apple M5 Max 64 GB / 614 GB/s · config ID apple-m5-max-64gb-614	Q5_K_S llama-cpp-metal · runner-up IQ4_NL	None fitsthe cache alone exhausts usable memory	51 GB of 51.2 GB usable	7.3–10.0 tok/s estimate at s = 1,024	12.6 tok/s	Tier B matched target, exposed latency uncalibrated, computed artifact size	Open
25	Apple M4 Max 64 GB / 546 GB/s · config ID apple-m4-max-64gb-546	Q5_K_S llama-cpp-metal · runner-up IQ4_NL	None fitsthe cache alone exhausts usable memory	51 GB of 51.2 GB usable	6.5–8.9 tok/s estimate at s = 1,024	11.2 tok/s	Tier B matched target, exposed latency uncalibrated, computed artifact size	Open
26	Apple M1 Ultra 128 GB · config ID apple-m1-ultra-128gb	Q8_0 llama-cpp-metal · runner-up Q6_K	Q5_K_M drops from Q8_0	78.3 GB of 102.4 GB usable	6.2–8.5 tok/s estimate at s = 1,024	10.7 tok/s	Tier B matched target, exposed latency uncalibrated, computed artifact size	Open
27	Apple M5 Ultra 256 GB · config ID apple-m5-ultra-256gb	BF16 mlx-lm-metal · runner-up FP16	BF16	147.2 GB of 204.8 GB usable	5.3–7.1 tok/s estimate at s = 1,024	8.5 tok/s	Tier B matched target, exposed latency uncalibrated, computed artifact size	Open
28	Apple M5 Ultra 512 GB · config ID apple-m5-ultra-512gb	BF16 mlx-lm-metal · runner-up FP16	BF16	147.2 GB of 409.6 GB usable	5.3–7.1 tok/s estimate at s = 1,024	8.5 tok/s	Tier B matched target, exposed latency uncalibrated, computed artifact size	Open
29	Apple M5 Pro 48 GB · config ID apple-m5-pro-48gb	IQ3_M llama-cpp-metal · runner-up Q3_K_M	None fitsthe cache alone exhausts usable memory	33.9 GB of 38.4 GB usable	4.7–6.9 tok/s estimate at s = 1,024	9.5 tok/s	Tier B matched target, exposed latency uncalibrated, computed artifact size	Open
30	Apple M5 Max 128 GB / 614 GB/s · config ID apple-m5-max-128gb-614	Q8_0 llama-cpp-metal · runner-up Q6_K	Q5_K_M drops from Q8_0	78.3 GB of 102.4 GB usable	4.7–6.5 tok/s estimate at s = 1,024	8.2 tok/s	Tier B matched target, exposed latency uncalibrated, computed artifact size	Open
31	Apple M4 Pro 48 GB · config ID apple-m4-pro-48gb	IQ3_M llama-cpp-metal · runner-up Q3_K_M	None fitsthe cache alone exhausts usable memory	33.9 GB of 38.4 GB usable	4.2–6.1 tok/s estimate at s = 1,024	8.5 tok/s	Tier B matched target, exposed latency uncalibrated, computed artifact size	Open
32	Apple M4 Max 128 GB / 546 GB/s · config ID apple-m4-max-128gb-546	Q8_0 llama-cpp-metal · runner-up Q6_K	Q5_K_M drops from Q8_0	78.3 GB of 102.4 GB usable	4.2–5.8 tok/s estimate at s = 1,024	7.3 tok/s	Tier B matched target, exposed latency uncalibrated, computed artifact size	Open
33	Apple M6 32 GB · config ID apple-m6-32gb-170	IQ2_M llama-cpp-metal · runner-up IQ2_S	None fitsthe cache alone exhausts usable memory	25.1 GB of 25.6 GB usable	3.5–5.1 tok/s estimate at s = 1,024	7.1 tok/s	Tier B matched target, exposed latency uncalibrated, computed artifact size	Open
34	Apple M5 Pro 64 GB · config ID apple-m5-pro-64gb	Q5_K_S llama-cpp-metal · runner-up IQ4_NL	None fitsthe cache alone exhausts usable memory	51 GB of 51.2 GB usable	3.6–5.0 tok/s estimate at s = 1,024	6.3 tok/s	Tier B matched target, exposed latency uncalibrated, computed artifact size	Open
35	Apple M3 Ultra 512 GB · config ID apple-m3-ultra-512gb	BF16 mlx-lm-metal · runner-up FP16	BF16	147.2 GB of 409.6 GB usable	3.6–4.9 tok/s estimate at s = 1,024	5.8 tok/s	Tier B matched target, exposed latency uncalibrated, computed artifact size	Open
36	Apple M2 Ultra 192 GB · config ID apple-m2-ultra-192gb	BF16 mlx-lm-metal · runner-up FP16	Q8_0 drops from BF16	147.2 GB of 153.6 GB usable	3.5–4.7 tok/s estimate at s = 1,024	5.7 tok/s	Tier B matched target, exposed latency uncalibrated, computed artifact size	Open
37	Apple M5 32 GB · config ID apple-m5-32gb	IQ2_M llama-cpp-metal · runner-up IQ2_S	None fitsthe cache alone exhausts usable memory	25.1 GB of 25.6 GB usable	3.2–4.6 tok/s estimate at s = 1,024	6.4 tok/s	Tier B matched target, exposed latency uncalibrated, computed artifact size	Open
38	Apple M4 32 GB · config ID apple-m4-32gb	IQ2_M llama-cpp-metal · runner-up IQ2_S	None fitsthe cache alone exhausts usable memory	25.1 GB of 25.6 GB usable	2.5–3.6 tok/s estimate at s = 1,024	5.0 tok/s	Tier B matched target, exposed latency uncalibrated, computed artifact size	Open
39	NVIDIA DGX Spark 128 GB · config ID dgx-spark-128gb	Q8_0 llama-cpp-cuda · runner-up Q6_K	Q6_K drops from Q8_0	79.1 GB of 115.2 GB usable	2.2–3.0 tok/s estimate at s = 1,024	3.6 tok/s	Tier B matched target, exposed latency uncalibrated, computed artifact size	Open
40	AMD Ryzen AI Max+ 395 Developer Platform 128 GB · config ID ryzen-ai-max-395-128gb	Q8_0 llama-cpp-rocm · runner-up Q6_K	Q5_K_M drops from Q8_0	79.1 GB of 96 GB usable	1.8–2.6 tok/s estimate at s = 1,024	3.4 tok/s	Tier B matched target, exposed latency uncalibrated, computed artifact size	Open

Why these tiers

What each confidence reason means

Confidence is resolved per operation, per configuration, per backend target.

computed artifact size - No exact published artifact size for BF16; the size is computed from parameter count and effective bits per weight (§4.3.1).
exposed latency uncalibrated - No exposed per-layer latency has been measured for this target, so the fixed overhead term contributes zero. That is a stated assumption, not a measurement (§2, §14.3).
matched target - Standard attention on a matched backend target (vllm-cuda).

Where to start

Minimum viable and cheapest qualifying

A callout appears only where a sourced price supports it.

The smallest configuration that runs Llama 3.3 70B Instruct is NVIDIA GeForce RTX 3090 - 24 GB. It has 24 GB of usable memory. The recommended quantization there is IQ2_S, whose stated quality tier is degraded.
The cheapest configuration whose whole estimated range clears 10 tokens per second is Apple M5 Ultra - 96 GB at $5,499.

Capacity verdict

Configurations this model does not fit in

This is an arithmetic result about memory, not a missing estimate. The requirement is weights plus a 1,024-token fp16 cache plus the backend target's activation reserve, at the smallest quantization any supported backend there can run.

Configuration	Usable memory	Smallest supported quantization	Memory it would still need	Shortfall
Apple M4 16 GB · config ID apple-m4-16gb	12.8 GB	IQ2_XXS degraded quality · llama-cpp-metal	19.3 GB	6.5 GB short
Apple M4 24 GB · config ID apple-m4-24gb	19.2 GB	IQ2_XXS degraded quality · llama-cpp-metal	19.3 GB	0.1 GB short
Apple M4 Pro 24 GB · config ID apple-m4-pro-24gb	19.2 GB	IQ2_XXS degraded quality · llama-cpp-metal	19.3 GB	0.1 GB short
Apple M5 16 GB · config ID apple-m5-16gb	12.8 GB	IQ2_XXS degraded quality · llama-cpp-metal	19.3 GB	6.5 GB short
Apple M5 24 GB · config ID apple-m5-24gb	19.2 GB	IQ2_XXS degraded quality · llama-cpp-metal	19.3 GB	0.1 GB short
Apple M5 Pro 24 GB · config ID apple-m5-pro-24gb	19.2 GB	IQ2_XXS degraded quality · llama-cpp-metal	19.3 GB	0.1 GB short
Apple M6 16 GB · config ID apple-m6-16gb-153	12.8 GB	IQ2_XXS degraded quality · llama-cpp-metal	19.3 GB	6.5 GB short
Apple M6 24 GB · config ID apple-m6-24gb-170	19.2 GB	IQ2_XXS degraded quality · llama-cpp-metal	19.3 GB	0.1 GB short
Intel Arc B580 12 GB · config ID arc-b580-12gb	12 GB	IQ2_XXS degraded quality · llama-cpp-vulkan	19.5 GB	7.5 GB short
NVIDIA GeForce RTX 4070 12 GB · config ID geforce-rtx-4070-12gb	12 GB	IQ2_XXS degraded quality · llama-cpp-cuda	19.5 GB	7.5 GB short
NVIDIA GeForce RTX 4070 SUPER 12 GB · config ID geforce-rtx-4070-super-12gb	12 GB	IQ2_XXS degraded quality · llama-cpp-cuda	19.5 GB	7.5 GB short
NVIDIA GeForce RTX 4070 Ti 12 GB · config ID geforce-rtx-4070-ti-12gb	12 GB	IQ2_XXS degraded quality · llama-cpp-cuda	19.5 GB	7.5 GB short
NVIDIA GeForce RTX 4070 Ti SUPER 16 GB · config ID geforce-rtx-4070-ti-super-16gb	16 GB	IQ2_XXS degraded quality · llama-cpp-cuda	19.5 GB	3.5 GB short
NVIDIA GeForce RTX 4080 16 GB · config ID geforce-rtx-4080-16gb	16 GB	IQ2_XXS degraded quality · llama-cpp-cuda	19.5 GB	3.5 GB short
NVIDIA GeForce RTX 4080 SUPER 16 GB · config ID geforce-rtx-4080-super-16gb	16 GB	IQ2_XXS degraded quality · llama-cpp-cuda	19.5 GB	3.5 GB short
NVIDIA RTX PRO 2000 Blackwell 16 GB · config ID rtx-pro-2000-blackwell-16gb	16 GB	IQ2_XXS degraded quality · llama-cpp-cuda	19.5 GB	3.5 GB short

Not estimated

Configurations with no publishable result

These rows carry a reason instead of a number. They are not a slow result. Where a reason names a quantization, that is the preferred format the resolver probed with, not the only one it tried.

No publishable result Intel Arc Pro B60 - 24 GB

Known inputs: 24 GB usable memory and 456 GB/s of sourced peak bandwidth. No backend target supports BF16 on 24 GB (Intel Arc Pro B60). This is a known-unsupported path, not a failure to estimate. Choose a quant format one of the available backends supports, or add a backend target whose applies_to covers arc-pro-b60-24gb.

No publishable result NVIDIA Jetson AGX Orin - 64 GB

Known inputs: 54.4 GB usable memory and 204.8 GB/s of sourced peak bandwidth. No backend target supports BF16 on 64 GB (NVIDIA Jetson AGX Orin). This is a known-unsupported path, not a failure to estimate. Choose a quant format one of the available backends supports, or add a backend target whose applies_to covers jetson-agx-orin-64gb.

No publishable result Raspberry Pi 5 - 8 GB

Known inputs: 6.4 GB usable memory and 17 GB/s of sourced peak bandwidth. No backend target supports BF16 on 8 GB (Raspberry Pi 5). This is a known-unsupported path, not a failure to estimate. Choose a quant format one of the available backends supports, or add a backend target whose applies_to covers raspberry-pi-5-8gb.

No publishable result AMD Ryzen Threadripper PRO 7995WX Platform - 512 GB

Known inputs: 460.8 GB usable memory and 332.8 GB/s of sourced peak bandwidth. No backend target supports BF16 on 512 GB (AMD Ryzen Threadripper PRO 7995WX Platform). This is a known-unsupported path, not a failure to estimate. Choose a quant format one of the available backends supports, or add a backend target whose applies_to covers threadripper-pro-7995wx-512gb.

Audit trail

Primary sources

Parameter counts and architecture come from the model publisher.

Model recordllama-3.3-70b

https://huggingface.co/meta-llama/Llama-3.3-70B-Instruct/raw/main/config.json

Keep exploring

Other model pages

Each linked model passed the generated-page admission gate.