Labsco
huggingface logo

huggingface-local-models

✓ Official★ 10,800

by huggingface · part of huggingface/skills

Use to select models to run locally with llama.cpp and GGUF on CPU, Mac Metal, CUDA, or ROCm. Covers finding GGUFs, quant selection, running servers, exact…

🔥🔥🔥🔥✓ VerifiedFreeQuick setup
#automation#git
🧩 One of 7 skills in the huggingface/skills package — works on its own, and pairs well with its siblings.

Use to select models to run locally with llama.cpp and GGUF on CPU, Mac Metal, CUDA, or ROCm. Covers finding GGUFs, quant selection, running servers, exact…

Inspect the full instructions your agent will receiveExpand

This is the exact playbook injected into your agent when the skill activates — shown here so you can audit it before installing. You don't need to read it to use the skill.

by huggingface

Use to select models to run locally with llama.cpp and GGUF on CPU, Mac Metal, CUDA, or ROCm. Covers finding GGUFs, quant selection, running servers, exact… npx skills add https://github.com/huggingface/skills --skill huggingface-local-models Download ZIPGitHub10.8k

Hugging Face Local Models

Search the Hugging Face Hub for llama.cpp-compatible GGUF repos, choose the right quant, and launch the model with llama-cli or llama-server.

Default Workflow

  • Search the Hub with apps=llama.cpp.

  • Open https://huggingface.co/<repo>?local-app=llama.cpp.

  • Prefer the exact HF local-app snippet and quant recommendation when it is visible.

  • Confirm exact .gguf filenames with https://huggingface.co/api/models/<repo>/tree/main?recursive=true.

  • Launch with llama-cli -hf <repo>:<QUANT> or llama-server -hf <repo>:<QUANT>.

  • Fall back to --hf-repo plus --hf-file when the repo uses custom file naming.

  • Convert from Transformers weights only if the repo does not already expose GGUF files.

Quant Choice

  • Prefer the exact quant that HF marks as compatible on the ?local-app=llama.cpp page.

  • Keep repo-native labels such as UD-Q4_K_M instead of normalizing them.

  • Default to Q4_K_M unless the repo page or hardware profile suggests otherwise.

  • Prefer Q5_K_M or Q6_K for code or technical workloads when memory allows.

  • Consider Q3_K_M, Q4_K_S, or repo-specific IQ / UD-* variants for tighter RAM or VRAM budgets.

  • Treat mmproj-*.gguf files as projector weights, not the main checkpoint.

Load References

  • Read hub-discovery.md for URL-first workflows, model search, tree API extraction, and command reconstruction.

  • Read quantization.md for format tables, model scaling, quality tradeoffs, and imatrix.

  • Read hardware.md for Metal, CUDA, ROCm, or CPU build and acceleration details.

Resources

  • llama.cpp: https://github.com/ggml-org/llama.cpp

  • Hugging Face GGUF + llama.cpp docs: https://huggingface.co/docs/hub/gguf-llamacpp

  • Hugging Face Local Apps docs: https://huggingface.co/docs/hub/main/local-apps

  • Hugging Face Local Agents docs: https://huggingface.co/docs/hub/agents-local

  • GGUF converter Space: https://huggingface.co/spaces/ggml-org/gguf-my-repo