Labsco
MichaelYagi logo

MCP Platform

ā˜… 1

from MichaelYagi

Local MCP runtime with multi-agent orchestration, distributed tool servers, and ML-powered media recommendations.

šŸ”„šŸ”„āœ“ VerifiedAccount requiredNeeds API keys

MCP Platform

Local MCP runtime with multi-agent orchestration, distributed tool servers, ML-powered media recommendations, persistent cross-session memory, and a proactive agent scheduler.

āš ļø Experimental — intended for personal and experimental use only, not for production deployment.


Table of Contents


2. Using MCP Servers with Other Clients

Add to your MCP client config (e.g., claude_desktop_config.json):

{
    "mcpServers": {
        "code_assistant": {
            "command": "/path/to/mcp-platform/.venv/bin/python",
            "args": ["/path/to/mcp-platform/servers/code_assistant/server.py"]
        }
    }
}

Windows paths: "command": "C:\\path\\to\\mcp-platform\\.venv\\Scripts\\python.exe"

Available servers:

  • code_assistant - AI-powered code analysis, generation, and refactoring (12 tools)
  • code_review - Code review, search, and bug fixing (3 tools)
  • code_runner - Python/bash execution sandbox (4 tools)
  • discord - Discord channel notifications via webhook (2 tools) āš ļø Requires DISCORD_WEBHOOK_URL
  • github - GitHub repo clone, browse, and cleanup (4 tools) āš ļø Requires GITHUB_TOKEN for private repos
  • google - Gmail + Google Calendar (13 tools) āš ļø Requires Google setup (Apps Script or OAuth — see Google Setup)
  • image - Image search, analysis, and AI generation (6 tools) āš ļø Requires SERPER_API_KEY for search; generation is free
  • location - Weather, time, location (3 tools) — uses Open-Meteo (free, no key); falls back to OpenWeatherMap if OPENWEATHER_API_KEY is set
  • plex - Media library + ML recommendations (18 tools) āš ļø Requires PLEX_URL, PLEX_TOKEN
  • rag - Vector search and management (8 tools) āš ļø Requires Ollama + bge-large
  • system - System info and processes (3 tools)
  • text - Text processing and web search (8 tools)
  • trilium - Trilium notes integration (11 tools) āš ļø Requires TRILIUM_URL, TRILIUM_TOKEN

4. Adding Custom Tools

Step 1: Create the server file

mkdir servers/my_tool && touch servers/my_tool/server.py

Step 2: Implement with @tool_meta

from mcp.server.fastmcp import FastMCP
from tools.tool_control import check_tool_enabled
from client.tool_meta import tool_meta

mcp = FastMCP("my-tool-server")

@mcp.tool()
@check_tool_enabled(category="my_tool")
@tool_meta(
    tags=["read", "search"],
    triggers=["my keyword", "my phrase"],
    template='use my_function: arg1=""',
)
def my_function(arg1: str) -> str:
    """Short description."""
    return json.dumps({"content": f"Processed {arg1}"})

if __name__ == "__main__":
    mcp.run(transport="stdio")

Restart the client and the tool is live — routed, badged, and registered automatically.

@tool_meta field reference

FieldRequiredDescription
tagsāœ…Capability tags
triggersāœ…Natural language phrases that route to this tool
templaterecommendedPre-fill text shown in tools panel
text_fieldsif neededResponse fields containing main text
rate_limitno"100/hour", "10/day", "ollama", or None
idempotentnoTrue if side-effect free (default: True)
intent_categorynoOverride routing group name

Tag vocabulary

TagMeaning
readReads data only
writeCreates or modifies data
destructiveDeletes or irreversibly changes data
searchPrimary purpose is search
externalCalls an external API
visionProcesses image input
mediaOperates on audio/video/image
calendarInteracts with calendar data
emailInteracts with email
notesInteracts with note-taking systems
codeOperates on source code
systemInteracts with OS or hardware
ragInteracts with the RAG vector store
aiCalls an LLM or ML model

Adding External MCP Servers

Create external_servers.json in the project root:

{
    "external_servers": {
        "deepwiki": {
            "transport": "sse",
            "url": "https://mcp.deepwiki.com/mcp",
            "enabled": true
        }
    }
}

Supported transports: sse, http, stdio. Header auth env var convention:

ES_SERVERNAME_TOKEN=your_token_here

5. Distributed Mode (A2A Protocol)

python a2a_server.py    # Terminal 1 — starts on http://localhost:8010
python client.py        # Terminal 2
A2A_ENDPOINTS=http://localhost:8010,http://gpu-server:8020
A2A_EXPOSED_TOOLS=plex,location,text   # empty = expose all

All configured endpoints are discovered and registered concurrently at startup via asyncio.gather() — connection timeouts for unreachable endpoints no longer block each other.


6. Testing

Running Tests

Activate your virtualenv first, then run from the project root:

source .venv/bin/activate        # Linux/macOS
.venv\Scripts\activate           # Windows PowerShell
# WSL2: source /home/$USER/.virtualenvs/mcp_platform/bin/activate

python -m pytest                 # all tests
python -m pytest -m unit         # fast unit tests only
python -m pytest -m integration  # integration tests
python -m pytest -m e2e          # end-to-end tests
python -m pytest --no-cov        # skip coverage (faster)
python -m pytest -x              # stop on first failure
python -m pytest -k "session"    # filter by name

Test Structure

tests/
ā”œā”€ā”€ conftest.py
ā”œā”€ā”€ unit/         <- fast isolated unit tests
ā”œā”€ā”€ integration/  <- multi-component tests
└── e2e/          <- full conversation tests

tests/results/    <- generated after running tests
ā”œā”€ā”€ junit.xml
ā”œā”€ā”€ coverage.xml
ā”œā”€ā”€ test-report.html
└── coverage-report.html

CI/CD

Tests run automatically on every push and pull request via GitHub Actions (.github/workflows/ci.yml). On failure, a GitHub Issue is opened automatically with a link to the failed run.

To upload coverage to Codecov, add to the workflow:

- name: Upload coverage
  uses: codecov/codecov-action@v3
  with:
    files: results/coverage.xml

7. Architecture

Multi-Server Design

servers/
ā”œā”€ā”€ code_assistant/   12 tools  - AI-powered code analysis, generation, and refactoring
ā”œā”€ā”€ code_review/       3 tools  - Code review, search, and bug fixing
ā”œā”€ā”€ code_runner/       4 tools  - Python/bash execution sandbox
ā”œā”€ā”€ discord/           2 tools  - Discord channel notifications [requires DISCORD_WEBHOOK_URL]
ā”œā”€ā”€ github/            4 tools  - GitHub repo clone, browse, and cleanup
ā”œā”€ā”€ google/           13 tools  - Gmail + Google Calendar       [requires Google setup]
ā”œā”€ā”€ image/             6 tools  - Image search, analysis, AI generation
ā”œā”€ā”€ location/          3 tools  - Weather, time, location
ā”œā”€ā”€ plex/             18 tools  - Media + ML recommendations    [requires PLEX_URL + PLEX_TOKEN]
ā”œā”€ā”€ rag/               8 tools  - Vector search and management  [requires Ollama + bge-large]
ā”œā”€ā”€ system/            3 tools  - System info and processes
ā”œā”€ā”€ text/              8 tools  - Text processing and web search
└── trilium/          11 tools  - Trilium notes integration     [requires TRILIUM_URL + TRILIUM_TOKEN]

Total: 95 tools across 13 servers

Concurrency & Parallelism

The platform uses asyncio.gather() at several layers to run non-LLM work concurrently:

LayerWhat runs in parallel
Startup — server discoveryTCP reachability checks and OAuth probes for all external servers
Startup — A2A registrationAll A2A_ENDPOINTS are discovered and registered simultaneously
Multi-agent task executionIndependent tasks (no unmet dependencies) run as a concurrent batch each cycle
A2A subtask executionSame dependency-aware batched gather as multi-agent mode
WebSocket broadcastAll connected clients receive messages via a single gather() call

Hardware note: LLM inference serialises at the Ollama GPU layer regardless of concurrency — one inference runs at a time on a single GPU. The parallelism benefit is in I/O-bound work: HTTP tool calls, database queries, and network requests. True parallel LLM execution would require multiple GPUs or cloud-hosted sub-agents.

Directory Structure

mcp-platform/
ā”œā”€ā”€ servers/
ā”œā”€ā”€ a2a_server.py
ā”œā”€ā”€ client.py
ā”œā”€ā”€ client/
│   ā”œā”€ā”€ ui/
│   ā”œā”€ā”€ capability_registry.py  <- auto-populated from @tool_meta
│   ā”œā”€ā”€ langgraph.py
│   ā”œā”€ā”€ memory_consolidator.py  <- persistent cross-session memory
│   ā”œā”€ā”€ proactive_agent.py      <- scheduler + condition triggers
│   ā”œā”€ā”€ query_patterns.py       <- auto-populated from @tool_meta triggers
│   ā”œā”€ā”€ tool_meta.py            <- single source of truth for tool metadata
│   ā”œā”€ā”€ websocket.py
│   └── ...
ā”œā”€ā”€ data/
│   ā”œā”€ā”€ sessions.db             <- session + message history
│   ā”œā”€ā”€ memory.db               <- persistent memory (created on first run)
│   └── scheduler.db            <- scheduled jobs (created on first run)
└── tools/

8. RAG & Conversation Memory

How context works

Every LLM call receives context assembled in this exact order:

System prompt
  ā”œā”€ Persistent memory — top 5 by importance (always injected)
  ā”œā”€ Persistent memory — query-relevant (vector search, above threshold)
  └─ Original system instructions + session ID

Conversation RAG — turns that scrolled out of the window (if relevant)

Message window — last LLM_MESSAGE_WINDOW turns (always injected)

Current user message

Lookup chain — what happens on every query:

  1. Persistent memory (always) — top 5 highest-importance memories are injected unconditionally into the system prompt, regardless of query relevance. This ensures core facts (your name, family, preferences) are never lost even when no query scores well. Additional query-relevant memories are added on top via vector search.

  2. Message window (always) — the last LLM_MESSAGE_WINDOW turns of the current session are included directly as conversation history.

  3. Conversation RAG (always) — a semantic search runs against turns that have scrolled out of the window. Results above the reranker threshold are injected between the system prompt and history.

  4. LLM generates response using all of the above.

When does each layer save you:

SituationWhat helps
Asked about something from 3 messages agoMessage window
Asked about something from 20 messages ago, same sessionConversation RAG
Asked about your name in a new sessionPersistent memory (anchor)
Asked a specific fact from an old sessionPersistent memory (query-relevant)
Asked about a specific document or articleConversation RAG
Fact not yet consolidated into memoryNothing — tell the platform again

Conversation window (LLM_MESSAGE_WINDOW)

Controls how many recent turns the LLM sees directly. Set in .env:

LLM_MESSAGE_WINDOW=15   # default: 6, recommended: 15

A window of 6 is too tight for normal conversation — information shared early in a session scrolls out before you can ask about it. 15 covers a full back-and-forth without hitting token limits on qwen2.5:14b. If you share something and the LLM seems to forget it a few messages later, increase this value.

Overflow ingestion

When history exceeds the window, older turns are automatically ingested into the RAG vector database as Human + Assistant pairs. They remain searchable via semantic similarity even after scrolling out of the window.

Auto-RAG retrieval

On every message, a semantic search runs against the full RAG store using the current user message as the query. Matching chunks (from old conversation turns, ingested documents, Plex subtitles, or research) are injected into context automatically — no explicit search rag trigger needed.

What each retrieval method is good for

Query typeBest tool
"What did we discuss about X?"Auto-RAG (semantic)
"What was my first prompt?"session_history_tool (ordered DB)
"Summarise this session"session_history_tool
"What was in that article we read?"Auto-RAG
"What did I ask 10 messages ago?"session_history_tool

session_history_tool

Added to the rag server. Queries the session SQLite database directly for ordered, timestamped message history. The current session ID is always injected into the system prompt so the LLM can pass it through automatically.

use session_history_tool: session_id="<id>" [limit="20"] [order="asc"]

Triggers: first prompt, what did I ask, earlier in this session, summarise this session, session history


9. Persistent Memory & Proactive Agents

How memory and context work together

The platform has three layers of context, each serving a different purpose:

LayerWhat it isScopeSurvives session delete?
Message windowLast N turns in direct LLM contextCurrent sessionNo
Conversation RAGOlder turns ingested as vectorsPer sessionNo
Persistent memoryDistilled facts extracted by LLMAll sessionsYes — survives session deletion

What this means in practice: If you tell the platform your son's name and ask about it 3 messages later, the message window handles it. If you ask 20 messages later, RAG handles it (usually). If you start a new session tomorrow, only persistent memory has it.

The memory workflow

Step 1 — Have a conversation. Tell the platform things you want it to remember: your name, your family, your projects, your preferences. The more declarative the better ("My wife's name is Suzy" vs "what's my wife's name?").

Step 2 — Memory extracts automatically. After 15 minutes of inactivity, the InactivityWatcher fires and runs the LLM over your session transcript. It extracts facts and stores them in data/memory.db with vector embeddings.

Re-consolidation is smart: it tracks message count at last consolidation and only re-runs if new messages have been added since. Going idle overnight triggers one extraction, not dozens.

Step 3 — Memories inject on every query. On each new message, two things happen: the top 5 highest-importance memories are always injected into the system prompt unconditionally (so your name, family, and key preferences are never forgotten), then a vector search finds additional query-relevant memories and adds them on top. The combined block looks like this:


## Persistent Memory (from past sessions)

The following facts are KNOWN and TRUE. Use them to answer directly.

ā—† The user's name is Bob
ā—‹ Bob's wife is Suzy, a nurse and excellent cook
ā—‹ Bob's son Sam is 11, plays accordion, excels at hockey
...

Step 4 — Memories accumulate over time. Episodic memories accessed 3+ times are promoted to semantic (permanent) tier nightly. The platform gets more useful the longer you use it.

When memory doesn't fire automatically

The inactivity watcher fires 15 minutes after your last message. If you need memories extracted immediately:

:memory consolidate <session_id>

Use :sessions to find the session ID. The command clears the consolidation flag and re-runs extraction regardless of message count.

Memory commands

:memory                        — list all memories (sorted by relevance)
:memory semantic               — permanent memories only
:memory episodic               — session-derived memories
:memory forget <id>            — delete one memory by ID
:memory clear                  — delete all episodic memories
:memory clear session <id>     — delete memories from one session
:memory consolidate <id>       — extract memories from a session now
:memory add <fact>             — manually add a permanent memory
:memory dedup                  — remove duplicate memories

Manually added memories (:memory add) are stored as semantic tier with importance 1.0 — they always rank first in retrieval.

If the LLM forgets something mid-session

Increase LLM_MESSAGE_WINDOW in .env. The default of 6 is too tight — 15 is recommended. Information shared early in a session scrolls out of the window before you can ask about it.

Once a turn scrolls out of the window it moves into Conversation RAG — it's still there, but now retrieved by semantic similarity rather than direct context. This means the query phrasing needs to be close enough to the original content for the reranker to surface it. If the LLM still can't find something that was said earlier in the same session, try rephrasing the question to use the same keywords as the original statement.

For example: if you said "My son Sam plays accordion" and later ask "What instrument does my son play?", the semantic match is strong. But "How about Sam?" is too vague for RAG to confidently return the accordion fact — be specific.

Two memory tiers

TierHow it's createdPersists
episodicAuto-extracted from sessionsPermanent — survives session deletion
semanticPromoted from episodic (3+ accesses) or added via :memory addPermanent

Promotion threshold is configurable: MEMORY_PROMOTE_THRESHOLD=3 in .env.

Deleting a session does not delete its memories. RAG chunks are purged, but extracted facts remain. To remove memories from a specific session before deleting it:

:memory clear session <id>

Reading :memory output

PERSISTENT MEMORY
────────────────────────────────────────────────
[11] The user's name is Bob
    ā—† semantic  |  importance: 1.0  |  accessed 163x
[22] Bob's wife Suzy is a nurse and an excellent cook.
    ā—‹ episodic  |  importance: 0.9  |  accessed 89x

Each entry shows:

  • [id] — the memory ID, used with :memory forget <id> to delete it
  • Content — the extracted fact, written in plain English
  • ā—† semantic / ā—‹ episodic — tier: semantic (permanent) or episodic (session-derived)
  • importance — score assigned by the LLM at extraction time (0.0–1.0). Higher = retrieved first
  • accessed Nx — how many times this memory was returned in a retrieval search. High access count drives promotion from episodic → semantic

Memories are sorted by retrieval score (combination of vector similarity and importance), not by ID or creation date. The most relevant memories for your last query appear first.

Duplicate memories can appear when the same session is consolidated multiple times, or when two sessions contain similar facts. They don't cause errors — the LLM sees both — but they waste context space. Clean them up with:

:memory dedup        — removes exact duplicates, keeps highest importance copy
:memory forget <id>  — removes near-duplicates manually (same fact, different wording)

Persistent Memory

The platform remembers facts, preferences, and outcomes across sessions. After a session ends (or after 15 minutes of inactivity), the LLM extracts memorable information from the transcript and stores it in data/memory.db. On every new session, relevant memories are injected into the system prompt automatically — no re-explaining required.

Agentic Automation (Proactive Agent Scheduler)

The platform can act autonomously or semi-autonomously — running tools on a schedule, polling for conditions, and involving the LLM to interpret results before surfacing them to you. This is the foundation for things like daily briefings, email monitoring, and eventually fully autonomous workflows.

Just describe what you want:

show me my daily briefing every day at 6am
check for new emails every 5 minutes and summarize anything urgent
show me a random photo from my gallery every morning with a short commentary
alert me when I have unread emails
run the weather check every morning at 7am and write a friendly summary

The LLM classifies whether your message is a scheduling request, parses the intent into a job definition, and presents it for confirmation before saving anything. Ambiguous requests are clarified with a question rather than guessed.

Three trigger types:

TypeHow it worksExample
cronFires repeatedly on a fixed schedule"every day at 6am"
conditionPolls a check tool on an interval; fires only when condition is true"when I have unread emails"
onceFires once at a specific date/time, then removes itself"at 8am today", "remind me tomorrow at noon"

LLM involvement: Every job can have an llm_prompt — an instruction passed to the LLM after the tool runs. Instead of broadcasting the raw tool output, the LLM interprets it first:

  • shashin_random_tool + "write a short commentary about this photo including the location and mood"
  • gmail_get_unread + "summarize these emails and flag anything that needs a reply"
  • get_weather_tool + "write a friendly morning weather summary"

Tool pipelines: Chain multiple tools in sequence using >> in the llm_prompt. Each step's output is automatically passed to the next step — no LLM required:

use get_day_briefing >> use discord_notify
use gmail_get_unread >> use discord_notify
use calendar_get_today >> use discord_notify

When scheduling, just describe what you want:

Every day at 7am, use get_day_briefing then use discord_notify to send me the result
At 8am today, use get_day_briefing then use discord_notify to send me today's schedule

The scheduler parser will automatically produce the >> chain syntax.

Condition expressions have access to rich context from the tool's JSON output:

total_unread > 0       — fires when Gmail has unread emails
len_results > 0        — fires when a results list is non-empty
result > 10            — fires when a numeric value exceeds 10
result_len > 100       — fires when the raw output is non-trivial

Job management commands:

:jobs                    — list all scheduled jobs
:jobs pause <label>      — pause a job without deleting it
:jobs enable <label>     — resume a paused job
:jobs cancel <label>     — delete a job permanently
:jobs info <label>       — full job detail including cron and last run

Files:

  • client/memory_consolidator.py — memory extraction, storage, injection, and :memory commands
  • client/proactive_agent.py — scheduler, condition triggers, LLM-based intent parser, and :jobs commands

Required dependency:

pip install apscheduler

Data files (created automatically on first run):

data/memory.db      — persistent memory store
data/scheduler.db   — scheduled jobs store (label, tool, cron, llm_prompt, etc.)