Edit Models filters

Model Tree

Apps

Docker Model Runner

Unsloth Desktop

Inference Providers

OVHcloud AI Endpoints

HF Inference API

One-click Deployment

Inference Endpoints

Microsoft Foundry

Amazon SageMaker AI

Misc

text-generation-inference

Eval Results (legacy)

text-embeddings-inference

4-bit precision

8-bit precision

Mixture of Experts

Carbon Emissions

Models

15,526

Base only

Active filters: quantized

Makatia/mistral-7b-instruct-v0.2.Q8_0-Q8_0.gguf

Text Generation • 7B • Updated Mar 25 • 9

Makatia/microsoft_Phi-3-mini-4k-instruct_onnx_rpi

Text Generation • Updated Mar 25

Makatia/TinyLlama_TinyLlama-1.1B-Chat-v1.0_onnx

Text Generation • Updated Mar 25 • 2

ramblingpolymath/Qwen3-32B-W8A8

Text Generation • 33B • Updated Aug 2, 2025 • 331

steampunque/Deepseek-R1-Distill-Llama-8B-MP-GGUF

8B • Updated Feb 18 • 2

ramblingpolymath/Qwen3-14B-W8A8

Text Generation • 15B • Updated Aug 3, 2025 • 22

ramblingpolymath/Qwen3-8B-W8A8

Text Generation • 8B • Updated Aug 3, 2025 • 9

ramblingpolymath/Qwen3-4B-W8A8

Text Generation • 4B • Updated Aug 2, 2025 • 169

wavespeed/Wan2.1-T2V-14B-Diffusers-fp16-nf4

Text-to-Video • Updated 8 days ago • 18

RedHatAI/Kimi-K2-Instruct-quantized.w4a16

Text Generation • 1T • Updated Apr 28 • 125 • 12

adamrb/mpt-30b-chat-w4a16-gptq

30B • Updated Jul 15, 2025 • 10

adamrb/mpt-30b-chat-w8a8-gptq

30B • Updated Jul 15, 2025 • 8

ramblingpolymath/qwen3-30B-A3B-w8a8

Text Generation • 31B • Updated Aug 2, 2025 • 15

ramblingpolymath/Qwen3-0.6B-W8A8

Text Generation • 0.8B • Updated Aug 3, 2025 • 33

tachyphylaxis/DeepSeek-R1-0528-FP4

Text Generation • 397B • Updated Jul 18, 2025 • 7

PJEDeveloper/Mistral_Nemo_Instruct_2407-F16.gguf-Q4_K_M

12B • Updated Jul 23, 2025 • 3

sdurgi/bert_emotion_response_classifier_quantized

Text Classification • Updated Jul 19, 2025 • 4

Hariprasath28/orpheus-3b-4bit-AWQ

Text Generation • 3B • Updated Jul 19, 2025 • 374

mirekphd/whisper-large-v3-onnx-fp16

Automatic Speech Recognition • Updated Jul 19, 2025 • 6

mirekphd/whisper-large-v3-onnx-w8a16-dynamic

Automatic Speech Recognition • Updated Jul 19, 2025 • 3

mirekphd/whisper-large-v3-onnx-w4a16-dynamic

Automatic Speech Recognition • Updated Jul 19, 2025 • 4

steampunque/Qwen2.5-VL-32B-Instruct-MP-GGUF

33B • Updated Feb 18 • 55

theprint/Zeth-Gemma3-4B-GGUF

Text Generation • 5B • Updated Aug 16, 2025 • 8

ericrisco/medical-gemma-3n-4b-gguf

7B • Updated Jul 28, 2025 • 36

nvidia/DeepSeek-R1-0528-NVFP4-v2

Text Generation • 394B • Updated Sep 2, 2025 • 92.9k • 23

steampunque/GLM-Z1-9B-0414-MP-GGUF

9B • Updated Feb 18 • 55 • 2

nvidia/DeepSeek-R1-NVFP4-v2

Text Generation • 394B • Updated Jul 22, 2025 • 354 • 8

YongdongWang/llama-3.2-1b-lora-qlora-dart-llm-gguf

Text Generation • 1B • Updated Jul 16 • 21 • 3

sugiv/cardvaultplus-500m-gguf

Image-to-Text • 0.4B • Updated Jul 22, 2025 • 53 • 2

PJEDeveloper/mistralai_Mistral-7B-Instruct-v0.3-F16.gguf-Q5_K_M

7B • Updated Jul 23, 2025 • 23