Labsco
gornskew logo

Lisply-MCP

★ 49

from gornskew

A Node.js middleware that allows AI agents to interact with Lisp-based systems using the Lisply protocol.

🔥🔥✓ VerifiedFreeAdvanced setup
#automation#payments#devops#git#maps#security#translation

Note: This branch is currently not stable for auto-starting containers. This will be corrected, but for now the more reliable way of starting a containerized Lisply-MCP environment is with the "Containerized Runnings" section of the skewed-emacs README. After the containers are started, then Claude Desktop will connect to them according to the example configurations below.

Model Context Protocol (MCP) Middleware for Lisp-based and Lisp-like Environments

Robot with Lambda machine

This project is a Model Context Protocol (MCP) middleware that enables Large Language Models (LLMs) to interact with Lisp-based development and runtime environments using a lightweight protocol called Lisply.

Who Is this Meant For?

  • AI practitioners curious about Lisp
  • Lisp practitioners curious about AI
  • Anyone interested in Neuro-Symbolic Programming
  • Mechanical/Civil Engineers and Designers interested in CAD Automation and Knowledge Based Engineering
  • Tinkerers, meddlers, and tamperers from all walks of life

What Is it Meant to Do?

The Lisply-MCP middleware connects MCP-capable AI Agent programs, or MCP Clients, such as ClaudeDesktop, to Lisp-based systems which support a REPL, or Read-Eval-Print Loop. The connection is meant to facilitate AI-assisted symbolic programming sometimes referred to as Neuro-Symbolic Programming. We have coined the term "Lisply" to refer to a lightweight protocol which most any Lisp-like system can implement to render it compatible with this Lisply-MCP middleware.

The idea is that the LLM will be able to generate and evaluate arbitrary Lisp expressions, including creating, compiling, loading, and testing entire files and projects.

System Overview

The Lisply MCP middleware is implemented as a Javascript program meant to run in Node.js, and provides a bridge between your AI Agent and any compliant Lisply backend system. This wrapper enables the AI Agent to:

  1. Evaluate Lisp code in the Lisply Backend and receive the results.
  2. Make HTTP requests to any web endpoints implemented in the backend.
  3. Access introspection and documentation lookup facilities in the LB using Lisp evaluation.
  4. Create, manipulate, compile, load, and analyze files, again using Lisp evaluation.
  5. Interact with Lisp debuggers (for locally running backends).

Lisply is a lightweight protocol that specifies a minimal yet flexible set of HTTP and standard input/output interfaces, a standard set of environment variables, Docker container image naming conventions, and several optional capabilities to facilitate AI agents controlling your running Lisp system.

Architecture

The diagram below roughly captures how the components interact:

flowchart TB
    User("User") <--> Claude("Claude Desktop")
    User <-.-> Emacs("Emacs Text Editor (Optional)")

    Claude <--> MCP("MCP Protocol")
    MCP <--> Wrapper("Node.js MCP Wrapper")

    Wrapper --> LisplyHttp("Lisply HTTP Server")
    
    subgraph Docker ["Docker Container"]
    subgraph LisplyExec["Lisply Executable"]
    LisplyHttp
    LisplySwank("Lisply SWANK Server (for Emacs connection)")
    end
    end
    
    Wrapper <-- "Manages" --> Docker

    Emacs <-.-> LisplySwank
    
    KB[("Lisply Knowledge Base")] <--> Wrapper
    
    LisplyHttp --> Endpoints("RESTful Endpoints")
    LisplyHttp --> LispEval("Lisp Evaluation")
    
    style User fill:#ff9,stroke:#333,stroke-width:2px
    style Claude fill:#f9f,stroke:#333,stroke-width:2px
    style Emacs fill:#9ff,stroke:#333,stroke-width:2px,stroke-dasharray:5
    style Wrapper fill:#bbf,stroke:#333,stroke-width:2px
    style MCP fill:#bbf,stroke:#333,stroke-width:1px
    style Docker fill:#bfb,stroke:#333,stroke-width:2px
    style LisplyExec fill:#8f8,stroke:#333,stroke-width:2px
    style LisplyHttp fill:#bfb,stroke:#333,stroke-width:1px
    style LisplySwank fill:#bfb,stroke:#333,stroke-width:1px
    style KB fill:#bfb,stroke:#333,stroke-width:1px
    style Endpoints fill:#bfb,stroke:#333,stroke-width:1px
    style LispEval fill:#bfb,stroke:#333,stroke-width:1px

The middleware handles:

  1. Starting and managing a Lisply-compliant Docker container if needed
  2. Translating Lisp evaluation requests between the MCP protocol and the backend Lisply API
  3. Error handling, Lisp debugger interaction, and logging

Security Considerations

Because Lisply-MCP allows arbitrary Lisp code to be evaluated against a running Lisp-based backend, there are certain risks in case the LLM were to go "haywire." Therefore, best practices are:

  • Allow the wrapper to connect only to a containerized version of a Lisply backend. If overriding default host/port, the wrapper will happily connect to any live Lisply-compliant http port. Avoid allowing this to happen for any http ports being served by programs running directly on your host.

  • Make sure not to mount any non-expendable directories to that container (see directory mounting configuration instructions below)

  • Consider taking steps to limit RAM and CPU usage of the container.

Code Modules/Files

  • lib/config.js: Configuration loading and environment handling
  • lib/logger.js: Logging functionality
  • lib/docker.js: Docker container management
  • lib/server.js: HTTP server and MCP wrapper implementation
  • lib/utils.js: Utility functions for response handling
  • handlers/: Tool-specific request handlers
    • initialize.js: Initialization handler
    • toolsList.js: Tools list handler
    • toolCall.js: Main tool call dispatcher
    • httpRequest.js: HTTP request handler
    • ping.js: Ping handler
    • lispEval.js: Lisp evaluation handler
  • mcp-wrapper.js: <--- Main entry point <---

Docker Integration

Lisply-MCP can interact with both local and remote Lisply backends. For the local case, the middleware can automatically run Docker commands to pull and manage the appropriate Lisply backend container.

Docker Image Selection

The middleware selects a default Docker image name based on the detected current git branch of your Lisply-MCP repository:

  1. The Lisply Docker image naming convention follows the pattern: ${DOCKER_USER}/${IMAGE_BASE}:${IMAGE_BRANCH}-${LISP_IMPL}

    • ${DOCKER_USER} Username at hub.docker.com. defaults to genworks.
    • ${IMAGE_BASE} Main name of the Lisply backend. Defaults to gendl.
    • ${IMAGE_BRANCH} defaults to the current git branch name where the wrapper script is situated, with any slashes (/) converted to double hyphens (--)
      • For example, release/1598 becomes release--1598 in the image tag
      • devo branch will use the image tag devo
      • If no git branch is detected, defaults to master.
    • ${LISP_IMPL} is the Lisp implementation in case the base Lisply backend sports multiple available Lisp flavors (e.g., ccl, sbcl are available for current public Gendl builds).

  2. The middleware will attempt to pull a newer image if one exists:

    • First tries to pull a newer image from Docker Hub.
    • If pull fails or local is up to date, uses the local one.

  3. You can override the automatic selection with:

    • The --docker-image command-line argument (overrides --image-base-name and --image-branch entirely)
    • The --image-base-name and/or --image-branch arguments
    • The LISPLY_DOCKER_IMAGE environment variable
    • The LISPLY_IMAGE_BASE and LISPLY_IMAGE_BRANCH environment variables

  4. For the Lisp implementation:

    • Specify with --lisp-impl (ccl or sbcl for current gendl builds)
    • Or use the LISPLY_LISP_IMPL environment variable
    • Defaults to ccl if not specified, sbcl is also a valid choice for the default Gendl images.

DockerHub Authentication

The wrapper will attempt to log in to DockerHub using stored credentials. However, the default container images are public and should be available anonymously without docker login.

Volume Mounting

You can mount host directories into the backend Lisply container to share files between your host system and the container (note multiple mount points can be specified):

{
  "mcpServers": {
    "lisply-gendl-4": {
      "command": "node",
      "args": [
        "/path/to/cloned/lisply-mcp/scripts/mcp-wrapper.js",
        "--mount", "/home/user/projects:/projects",
        "--mount", "/home/user/data:/data"
      ]
    }
  }
}

Or using environment variables:

LISPLY_MOUNTS=/home/user/projects:/projects,/home/user/data:/data node mcp-wrapper.js

Note the container runs with a certain UID, typically defaulting to 1000. This may cause unexpected file ownerships if the Lisply backend is writing to a mounted directory. This can be solved with docker exec by sending commands to the container to change the UID of the user running the service in the container. This behavior is expected to be automated in a future version of this project. A possible command could look like e.g.:

docker exec lisply-mcp-<hash> usermod -u 1001 lisply-user

Existing Service Detection

The wrapper will check if a Lisply service is already running on the specified http host and ports and use it if it exists, before attempting to pull and/or start a container.

Existing Services Override Local Container Settings

When an existing service is detected on the specified host and port, all Docker-related settings will be ignored:

  • --docker-image, --image-base-name, --image-branch, and --lisp-impl
  • --mount volume options
  • --start-* service flags
  • --*-port internal container port settings
  • --docker-socket path
  • --no-auto-start flag

In this case, the wrapper will log messages about which settings are being ignored.

Communication Modes

This middleware supports two primary modes of communication with configured Lisply backends: HTTP mode and stdio (Standard Input/Output) mode.

HTTP Mode

HTTP mode is the default communication method and works with both local and remote Lisply backends. This mode uses the standard HTTP endpoints that all Lisply backends are required to implement.

Characteristics:

  • Structured responses with separate result, stdout, and error fields
  • Suitable for most casual use cases
  • Response format: {Result: <result>, Stdout: <output>, Error: <any error>}

Example response in HTTP mode:

{"Result": "6", "Stdout": "This is a message to standard output"}

Stdio Mode

Stdio mode provides more of a raw REPL experience for the LLM and enables the LLM to engage in interactive debugging. This mode expects to leverage the backend's native REPL interface and any included command-driven debugger.

Characteristics:

  • Raw REPL-like output without structured formatting
  • Support for interactive debugger when errors occur
  • Only available for local containers started by this middleware
  • Ideal for development, debugging, and complex interactions
  • Captures standard output followed by return-value of evaluated expressions in same stream, so the LLM will have to distinguish these just as a human user would

Debugger Support: When an error occurs in stdio mode, the Lisp debugger can be interacted with. The wrapper detects debugger prompts and provides metadata about the debugger state to the AI Agent. This functionality relies on hardcoded prompt patterns in the wrapper code which would need to be augmented to support new Lisply backends with different REPL and debugger prompts (patches welcome).

Mode Selection:

  • Default mode is HTTP.
  • To use stdio mode for a particular tool call, ask the LLM to specify mode: "stdio" in the lisp_eval tool parameters.
  • Stdio mode can be banned for the session by configuring with the --no-use-stdio flag or LISPLY_NO_USE_STDIO=true.

If stdio mode is requested but banned or otherwise not available, the wrapper will fall back to HTTP mode. LLM callers using stdio mode need to be aware of this, because the response from the HTTP fallback comes packaged in JSON instead of in raw format.

Adding a Separate, Compatible Filesystem MCP Server

Below is a claude_desktop_config.json which sets up a filesystem mcp server as well as our lisply-mcp-1 server, with a common mount shared between the two mcp servers:

{
  "mcpServers": {
    "filesystem": {
      "command": "wsl",
      "args": [
        "docker",
        "run",
        "-i",
        "--rm",
        "-u",
        "1000:1000",
        "--mount",
        "type=bind,src=/home/user/projects,dst=/projects",
        "mcp/filesystem",
        "/projects"
      ]
    },
    "lisply-gendl": {
      "command": "wsl",
      "args": [
        "node",
        "/home/user/projects/lisply-mcp/scripts/mcp-wrapper.js",
        "--server-name", "gendl",
        "--mount", "/home/user/projects:/projects"
      ]
    }
  },
  "globalShortcut": ""
}

Tool Details for Claude

Lisp Evaluation Tool (<server>__lisp_eval)

The lisp_eval tool (prefixed with the server name, e.g., gendl__lisp_eval) allows Claude to evaluate Lisp code directly within the Lisply environment with these parameters:

  • code (required): The Lisp code to evaluate
  • package (optional): The package to use for the evaluation
  • mode (optional): The mode to use to talk to Lisply
    • http (default): Uses HTTP communication for structured responses
    • stdio: Uses standard input/output communication for a raw REPL experience

Mode Comparison:

FeatureHTTP ModeSTDIO Mode
Response FormatStructured with separate Result and Stdout fieldsRaw REPL-like output
Error HandlingTraps errors and returns them as stringsCan enter interactive debugger
Debugger SupportNo interactive debuggingSupports interactive debugger
CompatibilityWorks with local & remote serversOnly for local containers started by MCP
Use CaseClean integration, simple queriesDevelopment, debugging, complex interactions

HTTP Request Tool (<server>__http_request)

The http_request tool (prefixed with the server name, e.g., gendl__http_request) enables the AI Agent to interact with any HTTP endpoint exposed by the Lisply backend (the following example works with our default backend, which has a built-in /color-map http endpoint):

{
  "path": "/color-map",
  "method": "GET",
  "headers": {
    "Accept": "application/json"
  }
}

Ping Tool (<server>__ping_lisp)

The ping_lisp tool (prefixed with the server name, e.g., gendl__ping_lisp) lets Claude confirm that the Lisply server is running:

<server>__ping_lisp()

Real-World Examples

Example 1: Creating a Parametric Box

In this example, Claude creates a parametric box in the Lisply environment and then modifies its properties based on user requirements.

User: "Can you create a box with width 10, height 5, and depth 3? Then show me how to change the height to 8."

Claude:

I'll create a parametric box using the lisp_eval tool, then show you how to modify its height.

First, let's create the box with the specified dimensions:

(in-package :gdl-user)

;; First, create a box with the specified dimensions
(defparameter *my-box* (make-object 'box 
                         :width 10 
                         :height 5 
                         :length 3))

;; Let's verify the dimensions
(list (theo *my-box* width)
      (theo *my-box* height)
      (theo *my-box* length))

Now, let's change the height from 5 to 8:

;; Change the height using setf with accessor
(theo *my-box* (set-slot! :height 8))

;; Alternatively, we could create a new box
(defparameter *new-box* (make-object 'box 
                         :width 10 
                         :height 8  ;; Updated height
                         :length 3))

;; Verify the new dimensions of original box
(list (theo *my-box* width)
      (theo *my-box* height)
      (theo *my-box* length))

This box can be visualized by creating a web interface or using a built-in gendl visualization tool such as geysr. See Gendl Documentation for details.

Example 2: Demonstrating the Difference Between HTTP and Stdio Modes

User: "Can you show me the difference between HTTP and stdio modes by evaluating a simple expression that prints something to standard output and returns a value?"

AI Agent:

I'll demonstrate the difference between HTTP and stdio modes by evaluating a simple Lisp expression that both prints output and returns a value.

First, let's use HTTP mode (the default):

(progn 
  (format t "This is a message to standard output~%") 
  (+ 1 2 3))

Response in HTTP mode:

{"Result": "6", "Stdout": "This is a message to standard output"}

Now, let's use stdio mode for the same expression:

;; Using mode: stdio
(progn 
  (format t "This is a message to standard output~%") 
  (+ 1 2 3))

Response in stdio mode:

This is a message to standard output
6

Notice the difference: HTTP mode provides a structured response with labeled Result and Stdout sections, while stdio mode gives you the raw REPL output exactly as it appears in the Lisp REPL.

Stdio mode is particularly useful for debugging because it can show you interactive debugger prompts. For example, if we introduce an error:

;; Using mode: stdio
(progn
  (format t "About to generate an error~%")
  (/ 1 0))

In stdio mode, you might see something like:

About to generate an error
> Error: Division by zero
> While executing: /
> Type :help for debugging options

This allows the LLM to interact with the debugger directly. In HTTP mode, you would just get an error message without the interactive capabilities.

MCP Server Registries