LabscoMCP early access ↗
oussama-khMCP SERVER

LLAMA Hot Swap

Swap the llama.cpp model behind a running Claude Code session without losing the conversation.

by oussama-kh ↗Model Routing, Multi-Model Consultation & Cost ControlCommunity

LABSCO SUMMARY

Two models on one machine, without restarting the conversation.

The trick is a clean split of duties: the service manager owns the weights, the client owns the history. That is what lets you plan with a reasoning model and then implement with a coding model in the same thread. Set health_timeout above its default of 30 seconds if your models are large — a swap that times out is almost always weights still loading.

What it is

A model switcher for a local llama-server. It starts and stops model services through launchctl on macOS or systemd on Linux, so the weights behind the endpoint change while the client keeps the full message history it re-sends every turn.

What you get

  • list_models lists every configured model with its load status and the current mode
  • get_current_model returns the alias of the model loaded right now
  • swap_model unloads the current model, loads the one you name, and waits for the health check to pass
  • create_model_config writes a new launchd plist or systemd unit for a model you describe in the chat
  • Two resources, llama-swap://config and llama-swap://status, plus a swap-workflow prompt for the plan-then-implement pattern

Requirements

The PyPI package mcp-llama-swap, version 1.0.0, run as uvx mcp-llama-swap. Python 3.10+, llama-server on the machine, and launchctl or systemd. Point LLAMA_SWAP_CONFIG at an absolute path to your config.json, which maps aliases to service filenames. Claude Code has to be pointed at a LiteLLM proxy for the swap to be invisible to it.

Setup effort

One command — uvx mcp-llama-swap