
neuroverse
from joshua400
Multilingual intelligence + memory + safety + voice layer for autonomous AI agents
๐ What's New in v4.1
- OpenRouter Reasoning: Integrated the
stepfun/step-3.5-flash:freemodel for high-performance analytical tasks. Use the newneuroverse_reasontool for deep thinking. - Reasoning Tokens: Real-time tracking of reasoning tokens for every request.
- Voice Layer (v2.0): Built-in support for Whisper STT and Coqui TTS.
๐ What is NeuroVerse?
Every time you start a new chat with Cursor, VS Code Copilot, or any MCP-compatible AI agent, it starts from zero โ no memory, no safety, no understanding of your language. NeuroVerse is an MCP server that gives your agents:
| Feature | Description | |
|---|---|---|
| ๐ | Multilingual Intelligence | Understands mixed Indian languages โ Tamil, Hindi, Telugu, Kannada, Malayalam, Bengali + English. Code-switching? No problem. |
| ๐๏ธ | Voice Layer | STT via Whisper and TTS via Coqui. Transcribe user audio and synthesize agent responses. |
| ๐ง | Intent Extraction | LLM-first structured intent extraction with deterministic rule-based fallback. Never misses a command. |
| ๐พ | Tiered Memory | Short-term (session), Episodic (recent), Semantic (long-term facts) โ all with importance scoring. |
| ๐ก๏ธ | 3-Layer Safety (Kavach) | Keyword blocklist โ Intent risk classifier โ LLM judge. Blocks DROP DATABASE before it's too late. |
| ๐ค | Multi-Model Router (Marga) | OpenAI ยท Anthropic ยท Sarvam AI ยท Ollama ยท OpenRouter โ routes each task to the best model automatically. |
| ๐ | Agent-to-Agent (Setu) | REST+JSON agent registry with automatic fallback. Agents calling agents calling agents. |
| โก | Async Everything | FastAPI + asyncpg + httpx. Sub-millisecond safety checks. Zero blocking. |
โก NeuroVerse is a modular intelligence layer โ not a monolith. Plug in what you need. Ignore what you don't.
Table of Contents
- Quick Start
- Why NeuroVerse?
- How It Works
- Multilingual Intelligence (Vani)
- Intent Extraction (Bodhi)
- Tiered Memory (Smriti)
- Safety Layer (Kavach)
- Multi-Model Router (Marga)
- Agent-to-Agent (Setu)
- MCP Tools
- REST API
- Configuration
- Testing
- Architecture
- Roadmap
- Contributing
NeuroVerse Integration
- Use
neuroverse_processto handle any user request โ it auto-detects language, extracts intent, checks safety, and executes. - Use
neuroverse_reasonfor complex tasks requiring analytical reasoning (powered by OpenRouter). - Use
neuroverse_store/neuroverse_recallfor persistent context across sessions. - Use
neuroverse_executefor any potentially dangerous action โ it will block destructive operations.
That's it. Two commands your agent needs to know:
| Command | When | What happens |
|---|---|---|
| `neuroverse_process(text, user_id)` | Any user request | Detects language, extracts intent, safety-checks, executes |
| `neuroverse_store(user_id, intent, ...)` | End of work | Saves context for next session |
Next session, your agent picks up exactly where it left off โ like it never forgot.
### Requirements
- **npm edition:** Node.js 18+ (zero database deps โ uses JSON files)
- **Python edition:** Python 3.10+ + PostgreSQL (for persistent memory)
---
## ๐ค Why NeuroVerse?
| Without NeuroVerse | With NeuroVerse |
|---|---|
| Agent only understands English | Agent understands Tamil, Hindi, Telugu, Kannada + English code-switching |
| `"anna file ah csv convert pannu"` โ โ error | `"anna file ah csv convert pannu"` โ โ
converts file to CSV |
| Every session starts from zero | Agent remembers what it did โ across sessions, across agents |
| `DROP DATABASE` โ ๐ your data is gone | `DROP DATABASE` โ ๐ก๏ธ blocked in < 1ms, zero tokens |
| Locked to one LLM provider | Routes to the best model for each task automatically |
| Two agents = chaos | Agent A hands off to Agent B seamlessly |
### Token Efficiency
NeuroVerse's safety layer runs at **zero token cost** โ pure regex and rule matching, no LLM calls wasted:
| Safety Approach | Cost per Check | Latency |
|---|---|---|
| LLM-based safety | 500โ2,000 tokens | 1โ5 seconds |
| Embedding-based | 100โ500 tokens | 200โ500ms |
| **NeuroVerse Kavach** | **0 tokens** | **< 1ms** |
Over 100 tool calls per session, that's **50,000โ200,000 tokens saved** compared to LLM-based safety.
---
## โ๏ธ How It Works
User Input (any language) โ โโโโโโดโโโโโ โ Vani โ โ Language detection + keyword normalisation โ (เคญเคพเคทเคพ) โ Tamil/Hindi/Telugu โ normalised internal format โโโโโโฌโโโโโ โ โโโโโโดโโโโโ โ Bodhi โ โ LLM intent extraction + rule-based fallback โ (เคฌเฅเคงเคฟ) โ Returns structured JSON with confidence โโโโโโฌโโโโโ โ โโโโโโดโโโโโ โ Kavach โ โ 3-layer safety: blocklist โ risk โ LLM judge โ (เคเคตเค) โ Blocks dangerous actions at zero token cost โโโโโโฌโโโโโ โ โโโโโโดโโโโโ โ Marga โ โ Routes to best model (OpenAI/Anthropic/Sarvam/Ollama) โ (เคฎเคพเคฐเฅเค) โ Based on task type: multilingual/reasoning/local โโโโโโฌโโโโโ โ โโโโโโดโโโโโ โ Smriti โ โ Stores/recalls from tiered memory โ (เคธเฅเคฎเฅเคคเคฟ) โ Short-term + Episodic + Semantic (PostgreSQL) โโโโโโฌโโโโโ โ Tool Execution + Response
---
## ๐ Multilingual Intelligence โ Vani
**The Problem:** Every MCP server speaks only English. 70% of India code-switches daily.
"anna indha file ah csv convert pannu" โ "anna this file ah csv convert do" โ keyword normalisation (not full translation) โ Intent: convert_format { output_format: "csv" }
### Hybrid Pipeline (Rule + LLM)
Input โ Language Detect (langdetect) โ Code-Switch Split โ Keyword Normalise โ Output
**Key insight:** Don't fully translate. Only normalise domain-critical keywords. The rest stays untouched โ preserving context, tone, and nuance.
### Supported Languages
| Language | Keywords Mapped | Example |
|---|---|---|
| ๐ฎ๐ณ Tamil | `pannu` โ do, `maathru` โ change, `anuppu` โ send | `"file ah csv convert pannu"` |
| ๐ฎ๐ณ Hindi | `karo` โ do, `banao` โ create, `bhejo` โ send | `"report banao sales ka"` |
| ๐ฎ๐ณ Telugu | `cheyyi` โ do, `pampu` โ send, `chupinchu` โ show | `"data chupinchu"` |
| ๐ฎ๐ณ Kannada | Support coming in v2 | โ |
| ๐ฌ๐ง English | Pass-through | `"convert json to csv"` |
### Code-Switch Detection
```json
{
"languages": ["ta", "en"],
"confidence": 0.92,
"is_code_switched": true,
"original_text": "anna indha file ah csv convert pannu",
"normalized_text": "anna this file ah csv convert do"
}๐ง Intent Extraction โ Bodhi
LLM-first. Rule-based fallback. Never fails.
LLM succeeds (confidence โฅ 0.5)?
โโ Yes โ use LLM result
โโ No โ rule-based parser (deterministic)LLM Strategy
# Prompt to LLM:
"Extract structured intent from the following input.
Return ONLY valid JSON: {intent, parameters, confidence}"Rule-Based Fallback (7 patterns)
| Pattern | Intent | Trigger Keywords |
|---|---|---|
| Format conversion | convert_format | convert, csv, json, excel, pdf |
| Summarisation | summarize | summarise, summary, brief, tldr |
| Report generation | generate_report | report, generate report |
| Deletion | delete_data | delete, remove, drop, clean |
| Data query | query_data | query, search, find, fetch, get |
| Communication | send_message | send, share, email, notify |
| Explanation | explain | explain, describe, what is, how to |
Output
{
"intent": "convert_format",
"parameters": { "input_format": "json", "output_format": "csv" },
"confidence": 0.87,
"source": "rule"
}The key difference: the code decides โ not the LLM. If the LLM fails, hallucinates, or returns garbage, the rule engine takes over. Deterministic. Reliable.
๐พ Tiered Memory โ Smriti
The Problem: Raw logs are useless. Storing everything wastes resources. No relevance scoring.
NeuroVerse's approach: Score โ Filter โ Compress โ Store.
Three Tiers
| Tier | Storage | Lifetime | Use |
|---|---|---|---|
| Short-term | In-process dict | Current session | Active context, capped at 50 per user |
| Episodic | PostgreSQL | Recent actions | What the agent did recently |
| Semantic | PostgreSQL | Long-term facts | Persistent knowledge about users, projects, entities |
Importance Scoring
if importance_score >= 0.4:
persist_to_database() # worth remembering
else:
skip() # noiseOnly important memories survive. No bloat. No irrelevant recall.
Context Compression
โ Bad: "The user asked about sales data three times in the last hour and seemed frustrated..."
โ
Good: { "intent": "sales_query", "frequency": 3, "sentiment": "frustrated" }Structured JSON payloads, NOT raw text dumps. Compressed. Indexable. Queryable.
Memory Schema (PostgreSQL)
CREATE TABLE memory_records (
id TEXT PRIMARY KEY,
user_id TEXT NOT NULL,
tier TEXT NOT NULL, -- short_term | episodic | semantic
intent TEXT NOT NULL,
language TEXT DEFAULT 'en',
data JSONB DEFAULT '{}', -- compressed structured payload
importance REAL DEFAULT 0.5,
created_at TIMESTAMPTZ DEFAULT NOW(),
updated_at TIMESTAMPTZ DEFAULT NOW()
);
-- Indexed: user_id, intent, tier๐ก๏ธ Safety Layer โ Kavach
<p align="center"> <em>"The shield that never sleeps."</em> </p>Three Layers โ Defense in Depth
Agent calls tool โ MCP Server receives request
โ
โโโโโโโโโโโโโดโโโโโโโโโโโโ
โ Layer 1: Blocklist โ โ regex + keywords, < 0.1ms
โโโโโโโโโโโโโฌโโโโโโโโโโโโ
โ pass
โโโโโโโโโโโโโดโโโโโโโโโโโโ
โ Layer 2: Risk Score โ โ intent โ risk classification
โโโโโโโโโโโโโฌโโโโโโโโโโโโ
โ pass
โโโโโโโโโโโโโดโโโโโโโโโโโโ
โ Layer 3: LLM Judge โ โ optional model-based check
โโโโโโโโโโโโโฌโโโโโโโโโโโโ
โ pass
Execute handlerLayer 1 โ Rule-Based Blocklist (Zero Cost)
Runs inside the MCP server. Pure regex. No network. No tokens.
Blocked keywords:
delete_all_data, drop_database, drop_table, system_shutdown,
format_disk, rm -rf, truncate, shutdown, reboot, erase_all, destroyBlocked patterns (regex):
DROP (DATABASE|TABLE|SCHEMA)
DELETE FROM *
TRUNCATE TABLE
FORMAT [drive]:
rm (-rf|--force)Layer 2 โ Intent Risk Classification
| Risk Level | Intents | Action |
|---|---|---|
| ๐ข LOW | convert_format, summarize, generate_report, query_data, explain | โ Allow |
| ๐ก MEDIUM | send_message, unknown | โ ๏ธ Block if strict mode |
| ๐ด HIGH | delete_data | โ Block always |
| โ CRITICAL | drop_database, system_shutdown | โ Block always |
Layer 3 โ LLM Safety Judge (Optional)
If Layers 1โ2 pass, optionally ask an LLM: "Is this safe?"
// LLM returns:
{ "safe": false, "reason": "This action would delete all user data." }Safety Verdict
{
"allowed": false,
"risk_level": "critical",
"reason": "Blocked keyword detected: 'drop_database'",
"blocked_by": "rule"
}Token Cost: Zero
Most AI safety: Agent โ "rm -rf /" โ Safety LLM โ 2,000 tokens burned
NeuroVerse: Agent โ "rm -rf /" โ regex match โ BLOCKED (0 tokens, < 1ms)Strict Mode
# .env
SAFETY_STRICT_MODE=true # Also blocks MEDIUM risk (unknown/send)
SAFETY_STRICT_MODE=false # Only blocks HIGH and CRITICAL๐ค Multi-Model Router โ Marga
The Problem: Vendor lock-in. One model for everything. Overpaying.
NeuroVerse's approach: Route each task to the best model. Automatically.
Routing Logic
def route_task(task):
if task.type == "multilingual":
return sarvam_model # Best for Indian languages
elif task.type == "reasoning":
return claude_or_openai # Best for complex analysis
elif task.type == "local":
return ollama # Free, on-device, private
else:
return best_available # Fallback chainSupported Providers
| Provider | Default Model | Best For | Cost |
|---|---|---|---|
| ๐ฎ๐ณ Sarvam AI | sarvam-2b-v0.5 | Indian languages, multilingual | Low |
| ๐งฉ OpenRouter | stepfun/step-3.5-flash:free | High-performance reasoning | Free |
| ๐ง Anthropic | claude-sonnet-4-20250514 | Reasoning, analysis | Medium |
| ๐ค OpenAI | gpt-4o | General tasks, code | Medium |
| ๐ฆ Ollama | llama3 | Local, private, offline | Free |
Benefits
| Without Marga | With Marga | |
|---|---|---|
| Cost | Pay GPT-4 for everything | Use Ollama for simple tasks |
| Speed | Same latency for all tasks | Local models for fast tasks |
| Privacy | Everything goes to cloud | Sensitive data stays local |
| Vendor lock-in | Stuck with one provider | Switch anytime |
Fallback Chain
If your preferred provider is down or unconfigured:
OpenRouter โ Anthropic โ OpenAI โ Sarvam โ Ollama (local, always available)๐ Agent-to-Agent โ Setu
Agents calling agents calling agents.
Agent Registry
register_agent({
"agent_name": "report_agent",
"endpoint": "http://localhost:8001/generate",
"capabilities": ["generate_report", "sales_analysis"]
})Routing
{
"target_agent": "report_agent",
"task": "generate_sales_report",
"payload": { "quarter": "Q1", "year": 2026 }
}Fallback
If the target agent is unreachable:
{
"success": false,
"error": "Agent unreachable: ConnectError",
"fallback": true
}The caller can fall back to local execution. No hard failures.
๐งฉ MCP Tools
NeuroVerse exposes 6 tools via the Model Context Protocol:
| # | Tool (npm) | Tool (Python) | Description |
|---|---|---|---|
| 1 | neuroverse_process | india_mcp_process_multilingual_input | Full pipeline: detect โ normalise โ intent โ safety โ execute |
| 2 | neuroverse_store | india_mcp_store_memory | Store a memory record in the tiered system |
| 3 | neuroverse_recall | india_mcp_recall_memory | Retrieve memories by user, intent, or tier |
| 4 | neuroverse_execute | india_mcp_safe_execute | End-to-end safe execution (convenience) |
| 5 | neuroverse_route | india_mcp_route_agent | Route a task to a registered downstream agent |
| 6 | neuroverse_model | india_mcp_model_route | Query the multi-model router (optionally invoke) |
| 7 | neuroverse_transcribe | india_mcp_transcribe_audio | Transcribe audio to text via Whisper STT |
| 8 | neuroverse_synthesize | india_mcp_synthesize_speech | Synthesize speech from text via Coqui TTS |
| 9 | neuroverse_reason | N/A | High-performance reasoning via OpenRouter |
Real-World Example
โโ Session 1 (Agent Alpha, 2pm) โโโโโโโโโโโโโโโโโโโโโโโโโโโ
india_mcp_process_multilingual_input({
text: "anna indha sales data ah csv convert pannu",
user_id: "alpha",
execute: true
})
โ Language: Tamil+English (code-switched)
โ Intent: convert_format { output_format: "csv" }
โ Safety: โ
allowed (LOW risk)
โ Execution: โ
success
india_mcp_store_memory({
user_id: "alpha",
intent: "convert_format",
tier: "episodic",
data: { "file": "sales_q1.json", "output": "csv" },
importance_score: 0.8
})
โโ Session 2 (Agent Beta, next day) โโโโโโโโโโโโโโโโโโโโโโโ
india_mcp_recall_memory({
user_id: "alpha",
intent: "convert_format",
limit: 5
})
โ "Agent Alpha converted sales_q1.json to CSV yesterday"
โ Beta picks up exactly where Alpha left off๐ REST API
NeuroVerse also ships with a FastAPI REST layer โ for non-MCP clients:
python app/main.py
# โ http://localhost:8000/docs (Swagger UI)| Endpoint | Method | Description |
|---|---|---|
/health | GET | Health check |
/api/process | POST | Full multilingual pipeline |
/api/memory/store | POST | Store memory |
/api/memory/recall | POST | Recall memories |
๐งช Testing
python -m pytest tests/ -vtests/test_intent.py โ 10 passed (rule-based + async + mock LLM + fallback)
tests/test_language.py โ 10 passed (keyword normalisation + detection + code-switch)
tests/test_pipeline.py โ 8 passed (full e2e: English, Tamil, Hindi, dangerous, edges)
tests/test_safety.py โ 12 passed (blocklist, regex, risk classification, pipeline)
============================= 40 passed in 0.87s ==============================What's Tested
| Category | Tests | Coverage |
|---|---|---|
| Language Detection | 10 | Tamil, Hindi, English, empty input, code-switch flag |
| Intent Extraction | 10 | All 7 rule patterns, LLM mock, LLM failure, empty |
| Safety Engine | 12 | Keywords, regex, risk levels, full pipeline, strict mode |
| Full Pipeline | 8 | E2E English, Tamil, Hindi, dangerous commands, edge cases |
๐๏ธ Architecture
npm Edition (Node.js / TypeScript)
npm/
โโโ src/
โ โโโ core/
โ โ โโโ language.ts # Vani โ Language detection (zero deps)
โ โ โโโ intent.ts # Bodhi โ Intent extraction (LLM + fallback)
โ โ โโโ memory.ts # Smriti โ Tiered memory (JSON files)
โ โ โโโ safety.ts # Kavach โ 3-layer safety engine
โ โ โโโ router.ts # Marga โ Multi-model AI router
โ โโโ services/
โ โ โโโ executor.ts # Tool registry + retry engine
โ โ โโโ agent-router.ts # Setu โ Agent-to-Agent routing
โ โโโ types.ts # TypeScript interfaces & enums
โ โโโ constants.ts # Shared constants
โ โโโ index.ts # MCP Server โ 6 tools (McpServer + Zod)
โโโ package.json # npm publish config
โโโ tsconfig.json
โโโ LICENSE # Apache-2.0Python Edition
app/
โโโ core/
โ โโโ language.py # Vani โ Language detection (langdetect)
โ โโโ intent.py # Bodhi โ Intent extraction (LLM + fallback)
โ โโโ memory.py # Smriti โ Tiered memory (PostgreSQL)
โ โโโ safety.py # Kavach โ 3-layer safety engine
โ โโโ router.py # Marga โ Multi-model AI router
โโโ models/schemas.py # 12 Pydantic v2 models
โโโ services/
โ โโโ executor.py # Tool registry + retry engine
โ โโโ agent_router.py # Setu โ Agent-to-Agent routing
โโโ config.py # Settings from environment
โโโ main.py # FastAPI REST entry point
mcp/server.py # MCP Server (FastMCP) โ 6 tools
tests/ # 40 tests (pytest)Dependencies โ Minimal
npm (3 packages):
| Package | Purpose |
|---|---|
@modelcontextprotocol/sdk | MCP protocol |
zod | Schema validation |
axios | HTTP requests |
Python (7 packages):
| Package | Purpose |
|---|---|
mcp[cli] | Model Context Protocol SDK |
fastapi + uvicorn | REST API layer |
pydantic | Input validation (v2) |
langdetect | Statistical language identification |
asyncpg + sqlalchemy[asyncio] | PostgreSQL async driver |
httpx | Async HTTP for model APIs |
๐ Roadmap
| Phase | Status | What |
|---|---|---|
| v1.0 | โ Done | Multilingual parsing + intent extraction + 5 tools |
| v1.0 | โ Done | Tiered memory system (PostgreSQL) |
| v1.0 | โ Done | 3-layer safety engine (Kavach) |
| v1.0 | โ Done | Multi-model router (Marga) + Agent routing (Setu) |
| v2.0 | โ Done | Voice layer (Whisper/Coqui) + Extended Multilingual |
| v3.0 | โ Done | Redis caching + Embedding-based semantic retrieval |
| v4.0 | โ Done | Reinforcement learning (RLHF) + Arachne contextual indexing |
| v4.1 | โ Done | OpenRouter Reasoning Layer Integration |
| v5.0 | ๐ฎ Future | Agent marketplace & external system plugins |
๐ Security
| Measure | Implementation |
|---|---|
| API key management | Environment variables only โ never in code |
| Input sanitisation | Pydantic v2 with field constraints on all inputs |
| Rate limiting | Planned for v2.0 |
| Path traversal | N/A โ no file system access by tools |
| SQL injection | Parameterised queries via SQLAlchemy |
| Encrypted storage | Delegated to PostgreSQL TLS |
๐ค Contributing
Contributions are welcome! Here's how to get started:
- Fork the repo
- Create a feature branch (
git checkout -b feature/amazing-feature) - Commit your changes (
git commit -m 'feat: add amazing feature') - Push to the branch (
git push origin feature/amazing-feature) - Open a Pull Request
Development Setup
# npm edition
git clone https://github.com/joshua400/neuroverse.git
cd neuroverse/npm
npm install
npm run build
# Python edition
cd neuroverse
python -m pip install -e ".[dev]"
python -m pytest tests/ -v # All 40 should pass๐ License
Apache-2.0
<p align="center"> <em>"I built NeuroVerse because it broke my heart watching agents forget everything every session โ and not understand a word of Tamil."</em> </p> <p align="center"> <strong>Joshua Ragiland M</strong><br/> โ๏ธ <a href="mailto:joshuaragiland@gmail.com">joshuaragiland@gmail.com</a><br/> ๐ <a href="https://portfolio-joshua400s-projects.vercel.app/">Portfolio Website</a> </p> <p align="center"> <sub>Built with ๐ง by Joshua โ for the agents of tomorrow.</sub> </p>
npm install neuroverse๐ Quick Start
1. Install
Option A: npm (recommended) โ use anywhere
npm install neuroverseOption B: From source (Python)
git clone https://github.com/joshua400/neuroverse.git
cd neuroverse
python -m pip install -e ".[dev]"๐ก Tip: If you installed via npm, the path is node_modules/neuroverse/dist/index.js. If from source, use the absolute path to your cloned directory.
2. Add NeuroVerse to your MCP config
NeuroVerse is a standard MCP server (stdio). Add it to your host's config:
Cursor / VS Code Copilot / Claude Desktop (npm)
{
"mcpServers": {
"neuroverse": {
"command": "npx",
"args": ["neuroverse"]
}
}
}From source (Python)
{
"mcpServers": {
"neuroverse": {
"command": "python",
"args": ["mcp/server.py"],
"cwd": "/path/to/neuroverse"
}
}
}3. Tell your agent to use NeuroVerse
Add this to your agent's rules file (.md, .cursorrules, system prompt, etc.):
## โ๏ธ Configuration
All settings via environment variables (`.env`):
```bash
# Database (PostgreSQL required for persistent memory)
DATABASE_URL=postgresql+asyncpg://user:password@localhost:5432/neuroverse
# AI Model API Keys (configure the ones you have)
OPENAI_API_KEY=sk-...
ANTHROPIC_API_KEY=sk-ant-...
SARVAM_API_KEY=...
# Ollama (local, free)
OLLAMA_BASE_URL=http://localhost:11434
# Safety
SAFETY_STRICT_MODE=true # Block MEDIUM risk actions too
# MCP Transport
MCP_TRANSPORT=stdio # or streamable_http
MCP_PORT=8000No common issues documented yet. If you hit a problem, the repository's GitHub Issues page is the best place to look.