Labsco
ViperJuice logo

Code-Index-MCP

★ 54

from ViperJuice

A local-first code indexer that enhances LLMs with deep code understanding. It integrates with AI assistants via the Model Context Protocol (MCP) and supports AI-powered semantic search.

🔥🔥🔥✓ VerifiedAccount requiredAdvanced setup
#automation#database#devops#git#security#monitoring#memory

Code-Index-MCP

Give your AI coding assistant instant, precise search across your whole codebase — so it finds the exact code it needs in milliseconds instead of burning time and tokens reading whole files.

Code-Index-MCP is a fast, local-first search index for your code. It plugs into Claude Code and other AI assistants (through the Model Context Protocol, "MCP") and lets them look up any symbol or search any text in your repository almost instantly — without your code ever leaving your machine.

New to Code-Index-MCP? Start with the Getting Started Guide.

Status: v1.2.0 — MCP tools (search_code, symbol_lookup) are the primary interface; a FastAPI admin gateway is available for diagnostics.

Stable-surface prep status: This guide targets 1.2.0 and reflects the repo-owned stable install surface prepared by GAREL. MCP STDIO remains the primary LLM surface, FastAPI remains a secondary admin surface, and final release publication plus GA release evidence remain downstream-only in GADISP.

Project Status

Version: 1.2.0 (stable surface prepared; dispatch pending) Python distribution: index-it-mcp Container image: ghcr.io/viperjuice/code-index-mcp Primary surface: MCP tools (search_code, symbol_lookup) via the STDIO runner when repository readiness is ready Secondary surface: FastAPI admin REST gateway for diagnostics and scripting — see "Admin REST Interface (secondary)" below Core features: local indexing, symbol/text search, registry-based language coverage; see docs/SUPPORT_MATRIX.md Optional features: semantic search (requires Voyage AI or a local vLLM endpoint), GitHub Artifacts index sync Performance: sub-100ms symbol lookup and sub-500ms search on indexed repos (benchmarked on this codebase; results vary by repo size and language mix) GA decision: see docs/validation/ga-final-decision.md; the current decision is ship GA, but stable release mutation and release evidence remain downstream-only in GADISP. GA readiness contract: see docs/validation/ga-readiness-checklist.md for the frozen release boundary, support-tier labels, evidence ownership, and rollback expectations that apply before dispatch. Repository model: one server can serve many unrelated repositories, with one registered worktree per git common directory. Only the tracked/default branch is indexed automatically. Indexed MCP results are authoritative only when readiness is ready; unavailable indexes return index_unavailable with safe_fallback: "native_search".

Why it exists

When an AI assistant works in a large codebase, it often reads big chunks of files just to find what it needs. That's slow, and every file it reads costs tokens (money). Code-Index-MCP builds a local index so the assistant can jump straight to the right function, class, or line — cutting token cost and making answers faster and more accurate.

Who it's for

Developers and teams using AI coding assistants on real, sizable codebases who want faster, cheaper, more accurate results — and who want their code to stay private, on their own machine.

What you get

  • ⚡ Instant lookups — sub-100ms symbol lookup, sub-500ms search on indexed repos.
  • 🔒 Local-first & private — indexing runs on your machine; your code isn't shipped to a cloud.
  • 💸 Lower token cost — the assistant searches instead of reading whole files (see the charts below).
  • 🧠 Semantic search (optional) — natural-language code search via embeddings.
  • 🌐 Many languages, many repos — one server can index multiple repositories with mixed languages.
  • 🔌 Plugin-based & extensible — add language support without touching the core.

See it in action

Benchmarks in this repository show large token/cost reductions when an assistant searches with Code-Index-MCP instead of reading files directly:

Token and cost savings — summary

Cost savings

Token reduction by language

(Charts generated from the benchmarks in reports/; numbers vary by repo size and language mix.)

🎯 Key Features

  • 🚀 Local-First Architecture: All indexing happens locally for speed and privacy
  • 📂 Local Index Storage: All indexes stored at .indexes/ (relative to MCP server)
  • 🔌 Plugin-Based Design: Easily extensible with language-specific plugins
  • 🔍 Language support: Tiered language/runtime support is documented in docs/SUPPORT_MATRIX.md
  • ⚡ Real-Time Updates: File system monitoring for instant index updates
  • 🧠 Semantic Search: AI-powered code search with Voyage AI embeddings
  • 📊 Rich Code Intelligence: Symbol resolution, type inference, dependency tracking
  • 🚀 Enhanced Performance: Sub-100ms queries with timeout protection and BM25 bypass
  • 🔄 Git Synchronization: Automatic index updates tracking repository changes
  • 📦 Portable Index Management: Zero-cost index sharing via GitHub Artifacts
  • 🔄 Automatic Index Sync: Pull indexes on clone, push on changes
  • 🎯 Smart Result Reranking: Multi-strategy reranking for improved relevance
  • 🎯 Query-Intent Routing: Symbol-pattern queries (class Foo, def bar, CamelCase) bypass BM25 and hit the symbols table directly for sub-5ms lookups
  • 🔒 Security-Aware Export: Automatic filtering of sensitive files from shared indexes
  • 🔍 Hybrid Search: BM25 + semantic search with configurable fusion
  • 🔐 Index Everything Locally: Search .env files and secrets on your machine
  • 🚫 Smart Filtering on Share: .gitignore and .mcp-index-ignore patterns applied only during export
  • 🌐 Multi-Language Indexing: Index entire repositories with mixed languages

🏗️ Architecture

The Code-Index-MCP follows a modular, plugin-based architecture designed for extensibility and performance:

System Layers

  1. 🌐 System Context (Level 1)

    • Developer interacts with Claude Code or other LLMs
    • MCP protocol provides standardized tool interface
    • Local-first processing with optional cloud features
    • Performance SLAs: <100ms symbol lookup, <500ms search

  2. 📦 Container Architecture (Level 2)

    ┌─────────────────┐     ┌──────────────┐     ┌─────────────┐
│   API Gateway   │────▶│  Dispatcher  │────▶│   Plugins   │
│   (FastAPI)     │     │              │     │ (Language)  │
└─────────────────┘     └──────────────┘     └─────────────┘
       │                        │                     │
       ▼                        ▼                     ▼
┌─────────────────┐     ┌──────────────┐     ┌─────────────┐
│  Local Index    │     │ File Watcher │     │  Embedding  │
│  (SQLite+FTS5)  │     │  (Watchdog)  │     │   Service   │
└─────────────────┘     └──────────────┘     └─────────────┘

  3. 🔧 Component Details (Level 3)

    • Gateway Controller: RESTful API endpoints
    • Dispatcher Core: Plugin routing and lifecycle
    • Plugin Base: Standard interface for all plugins
    • Language Plugins: Specialized parsers and analyzers
    • Index Manager: SQLite with FTS5 for fast searches
    • Watcher Service: Real-time file monitoring

🔐 Security

Code-Index-MCP implements defense-in-depth security hardening (Phase 15):

  • Plugin Sandboxing: Plugins execute in isolated worker processes with capability-based restrictions. See docs/security/sandbox.md.
  • Artifact Attestation: Published indexes are signed with GitHub SLSA attestations and verified at download. See docs/security/attestation.md.
  • Path Traversal Guard: Search results are validated to prevent escaping configured repository roots. See docs/security/path-guard.md.
  • Token Validation: GitHub tokens are validated for required scopes at startup (contents:read, metadata:read, actions:read, actions:write, attestations:write). See docs/security/token-scopes.md.
  • Metrics Authentication: The /metrics endpoint requires bearer token authentication.

For a comprehensive operator runbook, see docs/operations/user-action-runbook.md.

📁 Project Structure

The project follows a clean, organized structure. See docs/PROJECT_STRUCTURE.md for detailed layout.

Key directories:

  • mcp_server/ - Core MCP server implementation
  • scripts/ - Development and utility scripts
  • tests/ - Comprehensive test suite with fixtures
  • docs/ - Documentation and guides
  • architecture/ - System design and diagrams
  • docker/ - Docker configurations and compose files
  • data/ - Database files and indexes
  • logs/ - Application and test logs
  • reports/ - Generated performance reports and analysis
  • analysis_archive/ - Historical analysis and archived research

🛠️ Language Support

The current stable-surface support contract is centralized in docs/SUPPORT_MATRIX.md. It distinguishes specialized plugins, generic Tree-sitter registry coverage, default sandbox support, optional extras, semantic/rerank setup, and known alpha limitations. Do not assume every registry language has the same symbol quality or default sandbox behavior.

Using Against Many Repos

Index many unrelated repositories from a single running server instance.

Prerequisites: set MCP_ALLOWED_ROOTS to an OS-path-separator-separated list of absolute directory paths (: on Unix, ; on Windows) that the server is allowed to index before starting the server:

export MCP_ALLOWED_ROOTS=/abs/a:/abs/b

Start the server (with secrets via op run or plain python):

op run --env-file=.mcp.env -- python -m mcp_server.cli.server_commands

Register each repo — register one worktree per git common directory. The stable repo_id comes from Tier 1 git rev-parse --git-common-dir, so sibling worktrees of the same repository share identity and are not independently indexed in v3:

mcp-index repository register /abs/a
mcp-index repository register /abs/b

Scope queries per repo — pass repository=<name> (registered name or path) to search_code / symbol_lookup:

search_code(query="def parse", repository="my-repo")
symbol_lookup(symbol="Parser", repository="my-repo")

Index tracking: each repo's tracked/default branch is followed by MultiRepositoryWatcher (RefPoller every 30 s). Same-repo multiple worktrees and non-default branch queries are unsupported in v3 routing: they return index_unavailable with safe_fallback: "native_search" and readiness remediation instead of reusing another checkout's index. Check get_status or mcp-index repository list -v and trust indexed MCP results only when readiness is ready.

Path sandbox: tools search_code, symbol_lookup, summarize_sample, and reindex reject paths outside MCP_ALLOWED_ROOTS with error code path_outside_allowed_roots. Registered repo names bypass the check.

Options:

  • Set MCP_AUTO_INDEX=false in the server environment to skip background auto-indexing and call the reindex MCP tool manually (recommended for very large repos).
  • Add {"enabled": false} to .mcp-index.json in the target repo to disable indexing for that repo entirely.
  • After a full reindex or code changes, call the reindex MCP tool to rebuild the index on demand.

Semantic profiles: BM25 search requires no extra config. For semantic (vector) search, the server automatically loads code-index-mcp.profiles.yaml from its own installation directory — no need to copy it to each repo. To override with a custom profile file, set MCP_PROFILES_PATH=/abs/path/to/your-profiles.yaml in the server environment. To override individual endpoint URLs without editing the YAML, use the env vars referenced in the file (e.g. VLLM_EMBEDDING_BASE_URL, VLLM_SUMMARIZATION_BASE_URL).

⚡ Enable Semantic Search

BM25 keyword search works with zero configuration. To add vector (semantic) search, choose one path:

Option A — Voyage AI (recommended):

export VOYAGE_API_KEY=your-key   # free tier available at voyageai.com

The commercial_high profile activates automatically. Restart the MCP server — the startup log will confirm semantic search is active.

Option B — Local OSS (Qwen3-Embedding-8B via vLLM, no API key needed):

export VLLM_EMBEDDING_BASE_URL=http://localhost:8000/v1
# Start vLLM (requires ~20GB VRAM or shared CPU with --dtype float32):
docker run -p 8000:8000 vllm/vllm-openai --model Qwen/Qwen3-Embedding-8B

Both profiles and their collection names are defined in code-index-mcp.profiles.yaml and can be customized.

Costs & Optional Features

The documented container package is ghcr.io/viperjuice/code-index-mcp. BM25 code search works without provider credentials. Semantic search, reranking, artifact sync, and monitoring depend on extras, environment variables, and service configuration. See docs/SUPPORT_MATRIX.md for language/runtime support details.

🆕 Advanced Features

Search Result Reranking

Three rerankers are available, configured via the RERANKER_TYPE environment variable:

ValueRerankerNotes
flashrankFlashRankOSS, local, fast (~1–5 ms overhead)
cross-encoderCross-EncoderOSS, local, highest quality
voyageVoyage RerankerCloud API, requires VOYAGE_API_KEY
noneDisabledDefault
export RERANKER_TYPE=flashrank   # or cross-encoder, voyage, none

Reranking applies only to the semantic retrieval path. BM25/FTS results are not reranked. Implementation: mcp_server/dispatcher/reranker.py.

LLM Chunk Summarization

Semantic chunks can be augmented with LLM-generated summaries before embedding, improving retrieval of intent-based queries. Configured per-profile in code-index-mcp.profiles.yaml:

summarization:
  enabled: true
  mode: lazy           # lazy (on first query) | comprehensive (at index time)
  provider: openai_compatible
  model_name: gpt-4o-mini
  base_url: "https://api.openai.com/v1"
  api_key_env: OPENAI_API_KEY
  prompt_template: "Describe this code chunk's inputs, outputs, and purpose in 2 concise sentences."

⚠️ Security: Do not summarize untrusted code with cloud LLMs. Hidden instructions in comments can be executed by the summarizer. See the Security Notes section.

Implementation: mcp_server/indexing/summarization.py

Security-Aware Index Sharing

Prevent accidental sharing of sensitive files:

# Analyze current index for security issues
python scripts/utilities/analyze_gitignore_security.py

# Create secure index export (filters gitignored files)
python scripts/utilities/secure_index_export.py

# The secure export will:
# - Exclude all gitignored files
# - Remove sensitive patterns (*.env, *.key, etc.)
# - Create audit logs of excluded files

BM25 Hybrid Search

Combines traditional full-text search with semantic search:

# The system automatically uses hybrid search when available
# Configure weights in settings:
HYBRID_SEARCH_BM25_WEIGHT=0.3
HYBRID_SEARCH_SEMANTIC_WEIGHT=0.5
HYBRID_SEARCH_FUZZY_WEIGHT=0.2

🗂️ Index Management

Centralized Index Storage

All indexes are now stored centrally at .indexes/ (relative to the MCP project) for better organization and to prevent accidental commits:

.indexes/
├── {repo_hash}/              # Unique hash for each repository
│   ├── main_abc123.db        # Index for main branch at commit abc123
│   ├── main_abc123.metadata.json
│   └── current.db -> main_abc123.db  # Symlink to active index
├── qdrant/                   # Semantic search embeddings
│   └── main.qdrant/          # Centralized Qdrant database

Benefits:

  • Indexes never accidentally committed to git
  • Reusable across multiple clones of same repository
  • Clear separation between code and indexes
  • Automatic discovery based on git remote

Migration: For existing repositories with local indexes:

python scripts/move_indexes_to_central.py

For This Repository

This project uses GitHub Actions Artifacts for efficient index sharing, so most users start from a published index baseline instead of rebuilding locally.

# First time setup - pull latest indexes
mcp-index artifact pull --latest

# After pull, reconcile only your branch/worktree drift
mcp-index artifact sync

# Share your indexes with the team
mcp-index artifact push

# Check sync status
mcp-index artifact sync

# Optional: Install git hooks for automatic sync
mcp-index hooks install
# Now indexes upload automatically on git push
# and download automatically on git pull

For ANY Repository (MCP Index Kit)

Enable portable index management in any repository with zero GitHub compute costs:

Quick Install

npm install -g mcp-index-kit
mcp-index init

How It Works

  1. Zero-Cost Architecture:

    • All indexing happens on developer machines
    • Indexes stored as GitHub Artifacts (free for public repos)
    • Automatic download on clone, upload on push
    • No GitHub Actions compute required

  2. Portable Design:

    • Single command setup for any repository
    • Auto-detected by MCP servers and tools
    • Language/runtime behavior follows the explicit support tiers in docs/SUPPORT_MATRIX.md
    • Enable/disable per repository

  3. Usage:

    # Initialize in your repo
cd your-repo
mcp-index init

# Build index locally
mcp-index build

# Push to GitHub Artifacts
mcp-index push

# Pull latest index
mcp-index pull

# Auto sync
mcp-index sync

Configuration

Semantic Search Configuration

To enable semantic search capabilities, you need a Voyage AI API key. Get one from https://www.voyageai.com/.

Method 1: Claude Code Configuration (Recommended)

Create or edit .mcp.json in your project root:

{
  "mcpServers": {
    "code-index-mcp": {
      "command": "uvicorn",
      "args": ["mcp_server.gateway:app", "--host", "0.0.0.0", "--port", "8000"],
      "env": {
        "VOYAGE_API_KEY": "your-voyage-ai-api-key-here",
        "SEMANTIC_SEARCH_ENABLED": "true"
      }
    }
  }
}

Method 2: Claude Code CLI

claude mcp add code-index-mcp -e VOYAGE_API_KEY=your_key -e SEMANTIC_SEARCH_ENABLED=true -- uvicorn mcp_server.gateway:app

Method 3: Environment Variables

export VOYAGE_API_KEY=your_key
export SEMANTIC_SEARCH_ENABLED=true

Method 4: .env File

Create a .env file in your project root:

VOYAGE_API_KEY=your_key
SEMANTIC_SEARCH_ENABLED=true

Check Configuration

Verify your semantic search setup:

mcp-index index check-semantic
Index Configuration

Edit .mcp-index.json in your repository:

{
  "enabled": true,
  "auto_download": true,
  "artifact_retention_days": 30,
  "github_artifacts": {
    "enabled": true,
    "max_size_mb": 100
  }
}

See mcp-index-kit for full documentation

View artifact details

mcp-index artifact info 12345


#### Index Management
```bash
# Check index status
mcp-index index status

# Check compatibility
mcp-index index check-compatibility

# Rebuild indexes locally only if artifact sync cannot catch up
mcp-index index rebuild

# Create backup
mcp-index index backup my_backup

# Restore from backup
mcp-index index restore my_backup

GitHub Actions Integration

  • Pull Requests: Validates developer-provided indexes (no rebuilding)
  • Merges to Main: Promotes validated indexes to artifacts
  • Cost-Efficient: Uses free GitHub Actions Artifacts storage
  • Auto-Cleanup: Old artifacts cleaned up after 30 days

Storage & Cost

  • GitHub Actions Artifacts: FREE for public repos, included in private repo quotas
  • Retention: 7 days for PR artifacts, 30 days for main branch
  • Size Limits: 500MB per artifact (compressed)
  • Automatic Compression: ~70% size reduction with tar.gz

Developer Workflow

  1. Clone Repository

    git clone https://github.com/yourusername/Code-Index-MCP.git
cd Code-Index-MCP

  2. Get Latest Indexes

     gh auth login
 mcp-index artifact pull --latest
    • This downloads the current full GitHub artifact snapshot.
    • mcp-index artifact sync then reconciles only your local branch/worktree drift when incremental catch-up is appropriate.

  3. Make Your Changes

    • Edit code as normal
    • Indexes update automatically via file watcher

  4. Share Updates

    # Your indexes are already updated locally
 mcp-index artifact push

Embedding Model Compatibility

The system tracks embedding model versions to ensure compatibility:

  • commercial_high: voyage-code-3 — 2048 dimensions, dot product, float32
  • oss_high: Qwen/Qwen3-Embedding-8B — 4096 dimensions, dot product, l2-normalized
  • Auto-detection: System checks profile compatibility before download

Multi-profile semantic config can be provided in either:

  • SEMANTIC_PROFILES_JSON (environment variable), or
  • code-index-mcp.profiles.yaml (repository root).

Artifact Strategy

  • GitHub artifact pulls are full snapshot downloads, not partial remote patch fetches.
  • The current compressed artifact is modest enough that full downloads stay simpler than a remote delta protocol.
  • Efficiency comes from local incremental indexing after restore:
    • pull the latest full artifact
    • compare the restored artifact commit to local HEAD
    • let the watcher or local incremental reindexing reconcile added, modified, deleted, and renamed files
  • Branch-specific remote artifacts are optional. The default strategy is to use the latest main artifact as the base and reconcile branch drift locally.

Easy Semantic Setup (Docker-First)

Run onboarding with automatic local Qdrant startup:

mcp-index setup semantic

Settings precedence (highest to lowest):

  1. CLI flags (for one command run)
  2. Environment variables / .env
  3. code-index-mcp.profiles.yaml
  4. SEMANTIC_PROFILES_JSON
  5. Built-in defaults

Common controls:

# Preflight checks only
mcp-index setup semantic --dry-run

# Strict mode: fail command if semantic stack isn't ready
mcp-index setup semantic --strict

# Override local embedding endpoint
mcp-index setup semantic --openai-api-base http://127.0.0.1:8001/v1

Plugin loading is auto-optimized by default using fast repository language detection:

  • MCP_AUTO_DETECT_LANGUAGES=true
  • MCP_LANGUAGE_DETECT_MAX_FILES=5000
  • MCP_LANGUAGE_DETECT_MIN_FILES=2

For startup-sensitive environments, enable:

  • MCP_FAST_STARTUP=true (uses lazy plugin loading and skips file watcher startup)

When MCP_AUTO_DETECT_LANGUAGES=true, auto-detection takes precedence over plugins.yaml. Set MCP_AUTO_DETECT_LANGUAGES=false to force plugins.yaml language selection.

For a dual-profile setup (Voyage + local vLLM/Qwen), set:

  • VOYAGE_API_KEY
  • OPENAI_API_BASE (for example http://127.0.0.1:8000/v1)
  • OPENAI_API_KEY (placeholder accepted for local vLLM setups)

If you use a different embedding model, the system will detect incompatibility and rebuild locally with your configuration.

💻 Development

Creating a New Language Plugin

  1. Create plugin structure

    mkdir -p mcp_server/plugins/my_language_plugin
cd mcp_server/plugins/my_language_plugin
touch __init__.py plugin.py

  2. Implement the plugin interface

    from mcp_server.plugin_base import PluginBase

class MyLanguagePlugin(PluginBase):
    def __init__(self):
        self.tree_sitter_language = "my_language"
    
    def index(self, file_path: str) -> Dict:
        # Parse and index the file
        pass
    
    def getDefinition(self, symbol: str, context: Dict) -> Dict:
        # Find symbol definition
        pass
    
    def getReferences(self, symbol: str, context: Dict) -> List[Dict]:
        # Find symbol references
        pass

  3. Register the plugin

    # In dispatcher.py
from .plugins.my_language_plugin import MyLanguagePlugin

self.plugins['my_language'] = MyLanguagePlugin()

Running Tests

# Run all tests
pytest

# Run specific test
pytest test_python_plugin.py

# Run with coverage
pytest --cov=mcp_server --cov-report=html

Architecture Visualization

# View C4 architecture diagrams
docker run --rm -p 8080:8080 \
  -v "$(pwd)/architecture":/usr/local/structurizr \
  structurizr/lite

# Open http://localhost:8080 in your browser

Admin REST Interface (secondary)

The canonical surface is MCP tool calls (search_code, symbol_lookup, etc.) via the STDIO runner — see the "Quick Start" sections above. The FastAPI REST gateway documented here is a secondary admin interface for diagnostics, scripting, and clients that cannot speak MCP. Its endpoints are not the recommended path for LLM-driven workflows.

Admin REST Endpoints

GET /symbol

Get symbol definition (admin/debug surface — prefer the symbol_lookup MCP tool):

GET /symbol?symbol_name=parseFile&file_path=/path/to/file.py

Query parameters:

  • symbol_name (required): Name of the symbol to find
  • file_path (optional): Specific file to search in

GET /search

Search for code patterns (admin/debug surface — prefer the search_code MCP tool):

GET /search?query=async+def.*parse&file_extensions=.py,.js

Query parameters:

  • query (required): Search pattern (regex supported)
  • file_extensions (optional): Comma-separated list of extensions

Response Format

All API responses follow a consistent JSON structure:

Success Response:

{
  "status": "success",
  "data": { ... },
  "timestamp": "2024-01-01T00:00:00Z"
}

Error Response:

{
  "status": "error",
  "error": "Error message",
  "code": "ERROR_CODE",
  "timestamp": "2024-01-01T00:00:00Z"
}

📦 Releases & Pre-built Indexes

Using Pre-built Indexes

For quick setup, download pre-built indexes from our GitHub releases:

# List available releases
python scripts/download-release.py --list

# Download the current pre-built index artifact
python scripts/download-release.py --latest

# Download specific version
python scripts/download-release.py --tag v2024.01.15 --output ./my-index

Creating Releases

Maintainers can create new releases with pre-built indexes:

# Create a new release (as draft)
python scripts/create-release.py --version 1.2.0

# Create and publish immediately
python scripts/create-release.py --version 1.2.0 --publish

Automatic Index Synchronization

The project includes Git hooks for automatic index synchronization:

  • Pre-push: Uploads index changes to GitHub artifacts
  • Post-merge: Downloads compatible indexes after pulling

Install hooks with: mcp-index hooks install

🤝 Contributing

We welcome contributions! Please see our Contributing Guide for details.

Development Process

  1. Fork the repository
  2. Create a feature branch (git checkout -b feature/amazing-feature)
  3. Make your changes
  4. Add tests (aim for 90%+ coverage)
  5. Update documentation
  6. Submit a pull request

Code Style

  • Follow PEP 8 for Python code
  • Use type hints for all functions
  • Write descriptive docstrings
  • Keep functions small and focused

📈 Performance

Benchmarks

OperationPerformance TargetCurrent Status
Symbol Lookup<100ms (p95)✅ Achieved - All queries < 100ms
Code Search<500ms (p95)✅ Achieved - BM25 search < 50ms
File Indexing10K files/min✅ Achieved - 152K files indexed

Matrix Benchmark (2026-04-01)

MetricBM25-onlyvoyage-code-3Qwen3-Embedding-8B
Top-1 (no reranker)12/17 (70.6%)17/17 (100%)17/17 (100%)
Top-1 (flashrank)13/17 (76.5%)17/17 (100%)17/17 (100%)
Top-1 (cross-encoder)—17/17 (100%)17/17 (100%)
Top-1 (voyage-reranker)—15/17 (88.2%)—
BM25 symbol query p50~1–5 ms——
Semantic query p50 (hybrid)—~50–400 ms~50–280 ms

Full results: docs/benchmarks/matrix_benchmark.md / .json

🏗️ Architecture Overview

The system follows C4 model architecture patterns:

  • Workspace Definition: defined in architecture/workspace.dsl and validated with Structurizr CLI
  • System Context (L1): Claude Code integrates via MCP sub-agents against the STDIO primary surface
  • Container Level (L2): 8 main containers including enhanced MCP server and user documentation
  • Component Level (L3): Plugin system, memory management, and cross-repo coordination
  • Code Level (L4): 43 PlantUML diagrams documenting all system components and flows

For detailed architectural documentation, see the architecture/ directory.

🗺️ Development Roadmap

See ROADMAP.md for detailed development plans and current progress.

Current Status: 1.2.0 stable surface prepared; downstream GADISP dispatch still pending

  • ✅ Core Indexing: SQLite + FTS5 for fast local search
  • ✅ Multi-Language: Specialized and registry-backed language coverage; see docs/SUPPORT_MATRIX.md
  • ✅ MCP Protocol: Full compatibility with Claude Code and other MCP clients
  • ✅ Performance: Sub-100ms queries with BM25 optimization
  • 🔄 Index Sync: Beta support via GitHub Artifacts
  • 🔄 Semantic Search: Optional feature requiring Voyage AI API

Recent Improvements:

  • ⚡ Dispatcher Optimization: Timeout protection and BM25 bypass for reliability
  • 🔄 Hybrid Search: BM25 + semantic search with graceful degradation
  • 📊 Result Ranking: Improved relevance with score normalization
  • 🔧 CLI Tools: Full-featured mcp-index command for index management

Optimization Tips

Performance optimization features are implemented and available:

  1. Enable caching: Redis caching is implemented and configurable via environment variables
  2. Adjust batch size: Configurable via INDEXING_BATCH_SIZE environment variable
  3. Use SSD storage: Improves indexing speed significantly
  4. Limit file size: Configurable via INDEXING_MAX_FILE_SIZE environment variable
  5. Parallel processing: Multi-worker indexing configurable via INDEXING_MAX_WORKERS

🔒 Security

  • Local-first: All processing happens locally by default
  • Path validation: Prevents directory traversal attacks
  • Input sanitization: All queries are sanitized
  • Secret detection: Automatic redaction of detected secrets
  • Plugin isolation: Plugins run in restricted environments
  • ⚠️ Semantic Summary Risks: If you enable LLM-generated semantic summaries (lazy or comprehensive), be aware of prompt injection vulnerabilities. Malicious actors could place hidden instructions in code comments (e.g., in an open-source dependency) that the summarizer LLM might execute. Always review generated index metadata if summarizing untrusted code.

📄 License

This project is licensed under the MIT License - see the LICENSE file for details.

🙏 Acknowledgments

📬 Contact

<p align="center">Built with ❤️ for the developer community</p>