Test Automator

from iamsnh01

An LLM-powered server for automating unit, integration, E2E, and API tests.

🔥🔥🔥✓ VerifiedAccount requiredAdvanced setup

🤖 Test Automator

A comprehensive, intelligent, and extensible testing automation platform powered by Large Language Models. Test Automator streamlines the process of generating, executing, and analyzing various types of software tests (unit, integration, end-to-end, API) for both web UI and backend logic.

✨ Features

🧠 LLM-Powered Test Generation

Intelligent Analysis: Automatically analyzes your codebase to understand structure and dependencies
Smart Test Cases: Generates comprehensive test scenarios including edge cases and error conditions
Context-Aware: Understands your code patterns and generates idiomatic tests
Multi-Language Support: Optimized for Python with extensible architecture

🔧 Comprehensive Test Types

🔬 Unit Testing

Analyzes individual functions, methods, and classes
Generates pytest test functions with proper fixtures
Includes positive, negative, and edge case scenarios
Handles both sync and async code patterns

🔗 Integration Testing

Tests interactions between different modules and services
Simulates real component interactions
Uses appropriate mocking strategies
Tests configuration and initialization flows

🌐 End-to-End (E2E) Testing

Browser automation using Playwright and browser-use
Simulates real user interactions
Tests complete user workflows
Captures screenshots and generates visual reports

🚀 API Testing

Comprehensive HTTP endpoint testing
Request/response validation
Authentication and authorization testing
Performance and timeout testing

📊 Intelligent Reporting

LLM-Enhanced Analysis: AI-powered insights from test results
Multi-Format Support: XML, JSON, and HTML report parsing
Actionable Recommendations: Specific suggestions for improvement
Risk Assessment: Identifies critical areas needing attention

🎯 Advanced Features

Large Codebase Support (100k-200k lines)

Modular Analysis: Processes code in manageable chunks
Incremental Testing: Generates tests incrementally for better performance
Parallel Execution: Supports pytest-xdist for parallel test runs
Smart Filtering: Focuses on testable units to avoid overwhelming LLM

Cross-Platform Compatibility

Windows Native: Full support with proper path handling
WSL Integration: Seamless Windows Subsystem for Linux support
Linux/macOS: Native Unix support
Event Loop Handling: Platform-specific async optimizations

Performance Optimizations

Async Operations: Non-blocking test execution
Batch Processing: Efficient handling of multiple test files
Resource Management: Proper cleanup and memory management
Timeout Handling: Configurable timeouts for different test types

📁 Project Structure

test-automator/
├── test_automator/
│   ├── __init__.py
│   ├── mcp_server.py          # Main MCP server with tools
│   ├── test_generator.py      # LLM-powered test generation
│   ├── test_runner.py         # Cross-platform test execution
│   └── report_analyzer.py     # AI-enhanced report analysis
├── pyproject.toml             # Package configuration
└── README.md                  # This file

🤝 Contributing

Fork the repository
Create a feature branch (git checkout -b feature/amazing-feature)
Commit your changes (git commit -m 'Add amazing feature')
Push to the branch (git push origin feature/amazing-feature)
Open a Pull Request

📄 License

This project is licensed under the MIT License - see the LICENSE file for details.

🙏 Acknowledgments

Gemini LLM: Google's powerful language model for intelligent test generation
Playwright: Cross-browser automation framework
pytest: Robust Python testing framework
MCP Protocol: Model Context Protocol for seamless AI integration
Claude Code: AI-powered development environment

Copy & paste — that's it

# Clone the repository
git clone https://github.com/your-repo/test-automator.git
cd test-automator

# Create virtual environment
python -m venv .venv
.venv\\Scripts\\activate # Windows
# or
source .venv/bin/activate # WSL/Linux

# Install dependencies
pip install -e .

# Install Playwright browsers
playwright install

Before it works, you'll need: GOOGLE_API_KEY

🛠️ Installation

Prerequisites

Python 3.11+
Google API Key (for Gemini LLM)
Claude Code or Cursor with MCP support
Git

Quick Install

Windows (Native or WSL)

# Clone the repository
git clone https://github.com/your-repo/test-automator.git
cd test-automator

# Create virtual environment
python -m venv .venv
.venv\\Scripts\\activate  # Windows
# or
source .venv/bin/activate  # WSL/Linux

# Install dependencies
pip install -e .

# Install Playwright browsers
playwright install

Linux/macOS

# Clone and setup
git clone https://github.com/your-repo/test-automator.git
cd test-automator

# Create virtual environment
python3.11 -m venv .venv
source .venv/bin/activate

# Install dependencies
pip install -e .

# Install Playwright browsers
playwright install

⚙️ Configuration

1. Get Google API Key

Visit Google AI Studio
Create a new API key

Set environment variable:

# Windows
set GOOGLE_API_KEY=your_api_key_here

# Linux/macOS/WSL
export GOOGLE_API_KEY=your_api_key_here

2. Configure Claude Code

Option A: CLI Configuration

claude mcp add test-automator "/path/to/test-automator/.venv/bin/test-automator" -e "GOOGLE_API_KEY=your_api_key"

Option B: Manual Configuration

Add to your Claude Code configuration (~/.claude.json):

{
  "projects": {
    "/your/project/path": {
      "mcpServers": {
        "test-automator": {
          "type": "stdio",
          "command": "/path/to/test-automator/.venv/bin/test-automator",
          "env": {
            "GOOGLE_API_KEY": "your_api_key_here"
          }
        }
      }
    }
  }
}

Windows Path Examples

Windows: C:/path/to/test-automator/.venv/Scripts/test-automator.exe
WSL: /home/username/test-automator/.venv/bin/test-automator

🚀 Usage

MCP Tools Available

`generate_tests(code_path, test_type="all")`

Generate intelligent tests for your codebase:

# Generate all test types
generate_tests("/path/to/your/code", "all")

# Generate specific test type
generate_tests("/path/to/your/api.py", "unit")
generate_tests("/path/to/your/project", "integration")
generate_tests("/path/to/your/webapp", "e2e")
generate_tests("/path/to/your/api", "api")

`run_tests(test_type="all", target_path="tests/")`

Execute generated tests:

# Run all tests
run_tests("all", "/path/to/tests")

# Run specific test type
run_tests("unit", "/path/to/tests")

`analyze_test_report(report_path)`

Get LLM-powered insights from test results:

analyze_test_report("/path/to/test_results.xml")

Example Workflow

# 1. Generate comprehensive tests
generate_tests("/home/user/my-project", "all")

# 2. Run the tests
run_tests("all", "/home/user/my-project/tests")

# 3. Analyze results
analyze_test_report("/home/user/my-project/tests/results/unit_results.xml")

🔧 Troubleshooting

Common Issues

"GOOGLE_API_KEY not found"

# Set the environment variable
export GOOGLE_API_KEY="your_api_key_here"

# Or add to shell profile
echo 'export GOOGLE_API_KEY="your_api_key"' >> ~/.bashrc

"Playwright browsers not found"

# Install browsers
playwright install

# Install system dependencies (Linux)
playwright install-deps

Windows Permission Issues

# Ensure script execution is enabled
Set-ExecutionPolicy -ExecutionPolicy RemoteSigned -Scope CurrentUser

# Check virtual environment activation
.venv\\Scripts\\activate

WSL Display Issues (for E2E tests)

# Install X11 server for Windows
# Add to ~/.bashrc:
export DISPLAY=:0.0