Local RAG
Privacy-first local RAG server for semantic document search without external APIsRead full description
shinprLABSCO SUMMARY
Your documents stay on your hardware; only the question travels.
The four modes are really one decision — how far your data is allowed to go. Fully local keeps the embeddings, the reranker and the model on your machine; the ChatGPT and Claude modes keep the index local and let the answering model be remote.
What it is
An on-premises retrieval-augmented generation stack shipped as containers. An indexer walks a directory you nominate, embeddings go into a local Qdrant, and one of four modes decides where the answering model lives.
What you get
- Four modes: fully local with Ollama, a custom OpenAI-compatible LLM endpoint you already run, ChatGPT through a custom GPT, or Claude through MCP
- Recursive indexing of a root folder, covering
.pdf,.xls,.docx,.txt,.mdand.csv - A local chat UI on http://localhost:3000, plus an Electron desktop app
- Choice of embedding model — any Sentence Transformer — with the vector size set to match, and a reranker for the Ollama path
- A custom-LLM path that skips Ollama and the reranker entirely, so the container build does not download the reranker model and the resource footprint drops
Requirements
Docker Compose, plus Python 3.10 or newer and uv on the host for the MCP path. Copy .env.sample to .env and set at minimum LOCAL_FILES_PATH — the root folder to index — along with EMBEDDING_MODEL_ID and EMBEDDING_SIZE, which must match the model's actual embedding dimension. The Ollama path adds OLLAMA_MODEL and RERANKER_MODEL; the custom-LLM path adds LLM_BASE_URL and LLM_MODEL, with LLM_API_KEY optional. Start it with docker compose -f docker-compose-mcp.yml --env-file .env up --build, then point Claude Desktop at uv --directory <YOUR_PATH_TO_CLONED_MINIMA_PROJECT_MCP> run minima. The ChatGPT path additionally needs USER_ID and PASSWORD, used to create the account that authorizes the custom GPT.
Setup effort
One command — docker compose -f docker-compose-mcp.yml --env-file .env up --build
USE CASES
01
Asking questions about your own document folder
Point the indexer at a directory; every subfolder is indexed too, and answers come from those files rather than from the model's training data.
02
Keeping sensitive documents off the internet
The Ollama mode runs the LLM, reranker and embedding model on your own hardware, so no part of the pipeline calls out.
03
Reusing an LLM you already host
Set the base URL and model name for any OpenAI-compatible endpoint — vLLM, an Ollama server, TGI — and no Ollama container is deployed at all.
04
Querying local files from Claude Desktop
Bring the MCP compose file up and add the uv command to the Claude config; the indexer stays on your machine while Claude does the answering.
05
Reaching the same index from ChatGPT
The ChatGPT mode pairs the local indexer with a custom GPT, authorized by an emailed one-time code printed in your terminal.
BEFORE YOU INSTALL
What to know first
Requires
uv on PATH
Where it runs
Both documented clients run the server on your machine.
INSTALLATION
Connect your client
Server setup
TERMINAL
docker compose -f docker-compose-mcp.yml --env-file .env up --buildMCP Clients
Claude Desktop
CLAUDE DESKTOP · JSON
{
"mcpServers": {
"minima": {
"command": "uv",
"args": [
"--directory",
"<YOUR_PATH_TO_CLONED_MINIMA_PROJECT_MCP>",
"run",
"minima"
]
}
}
}VS Code
VS CODE · JSON
{
"servers": {
"minima": {
"type": "stdio",
"command": "<YOUR_PATH_TO_CLONED_MINIMA_PROJECT>/run_in_copilot.sh",
"args": [
"<YOUR_PATH_TO_CLONED_MINIMA_PROJECT>"
]
}
}
}
