Three providers chain automatically, and the last one is hash-based — it will happily produce vectors without sentence-transformers installed, and the search results will be poor in a way that looks like a data problem rather than a setup problem. Check `get_stats` after your first ingest. Everything else about the design is sensible: local SQLite, vectors on disk, and quality filters that keep navigation chrome out of the index.
A self-hosted RAG system that pairs a local vector store with Playwright-based web scraping. Pages get scraped into a folder, processed into embeddings, and stored in SQLite with the vectors on disk as NumPy arrays. The MCP server then lets a client add, search and retrieve from that store without anything leaving the machine — unless you opt into OpenAI embeddings.
- `add_document` puts content straight into the vector store
- `search_documents` finds similar content; `get_context` returns it already formatted for a query
- `list_documents` and `get_stats` for what is in there and how it is doing
- `process_scraped_files` ingests what Playwright saved; `save_scraped_content` stashes content for later processing
- A second server in the same repository — BerryExa — adds `crawl_content` for extraction with subpage support, `extract_links` for subpage discovery, and `get_content_preview` for a quick look before committing to a full crawl
- Three embedding providers with automatic fallback: sentence-transformers locally, OpenAI if you set a key, and a hash-based last resort the README itself advises against for production
- Quality filters that drop content under 100 characters, navigation-only pages, duplicates and files over 500KB
- Chunking at 500 characters with 50 characters of overlap and sentence-boundary detection
No account required for the default path. Clone, run `npm run install-deps` and `npm run setup`, then point your client at `node` with `mcp_servers/vector_db_server.js` and the repository as the working directory. Package `berry-rag`, version 1.0.0. Set `OPENAI_API_KEY` only if you want OpenAI embeddings instead of local ones. The first run downloads the sentence-transformers model, roughly 90MB. Scraping is done by the Playwright MCP server, configured alongside this one.
One command — npx @playwright/mcp@latest