Labsco
SyncTek-LLC logo

GhostQA

from SyncTek-LLC

GhostQA sends AI personas through your application — they look at the screen, decide what to do, and interact like real humans. No test scripts. No selectors. You describe personas and journeys in YAML, and GhostQA handles the rest.

🔥🔥🔥🔥✓ VerifiedAccount requiredQuick setup
#email#automation#scraping#browser#git#security

SpecterQA

FTI Trust Score

AI personas walk your app so real users don't trip.

SpecterQA sends AI personas through your application — they look at the screen, decide what to do, and interact like real humans. No test scripts. No selectors. You describe personas and journeys in YAML, and SpecterQA handles the rest.

$ specterqa run -p myapp

┏━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┓
┃ SpecterQA Run                                                    ┃
┃ Product: myapp   Budget: $5.00   Viewport: 1280x720            ┃
┗━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┛

  ✓ Step 1/4: Navigate to homepage       PASS   3.2s   $0.0081
  ✓ Step 2/4: Click signup link          PASS   2.1s   $0.0043
  ✓ Step 3/4: Fill registration form     PASS   8.7s   $0.0312
  ✓ Step 4/4: Verify dashboard loads     PASS   4.5s   $0.0127

┏━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┓
┃ ALL TESTS PASSED                                                ┃
┃ Steps: 4/4   Findings: 0   Duration: 18.5s   Cost: $0.0563     ┃
┃ Run ID: GQA-RUN-20260222-143052-a1b2                            ┃
┗━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┛

What is this?

Traditional E2E tests are brittle. You write selectors, they break. You maintain scripts, they rot. SpecterQA takes a different approach: AI vision models look at your actual UI and navigate it the way a person would.

You define personas (who is using your app) and journeys (what they're trying to do). SpecterQA's engine takes a screenshot, sends it to a Claude vision model, gets back a decision ("click this button", "fill this field"), executes it via Playwright, takes another screenshot, and repeats until the goal is achieved or something goes wrong.

When something goes wrong, you get evidence: screenshots, UX observations, cost breakdowns, and findings categorized by severity.

How It Works

The core loop is simple:

screenshot --> vision model --> action decision --> execute --> repeat
  1. Screenshot -- Playwright captures the current page state as a PNG
  2. Decide -- A Claude vision model receives the screenshot + persona context + goal, returns a structured JSON action (click, fill, navigate, scroll, keyboard, wait, done, or stuck)
  3. Execute -- Playwright performs the action (click at coordinates, type text, navigate to URL, etc.)
  4. Repeat -- Loop until the goal is achieved, the agent gets stuck, or the budget runs out

The persona's profile shapes how the AI behaves. A "tech-savvy developer" explores differently than a "frustrated first-time user." Persona patience, tech comfort, and frustrations all influence the system prompt.

Model routing keeps costs down. Simple actions (click, scroll) use Haiku. Complex actions (form filling, initial assessment) use Sonnet. You can also route simple actions to a local Ollama model (llava:13b) for zero API cost on straightforward navigation.

Features

  • Persona-based testing -- Define AI users with backgrounds, goals, frustrations, and tech comfort levels. They don't just follow scripts; they react to what they see.
  • Vision-powered -- No selectors, no DOM queries. The AI interprets screenshots like a human would. Catches visual/layout issues that selector-based tests miss entirely.
  • YAML-configured -- Products, personas, and journeys are all YAML files. PMs can read them. No code to maintain.
  • Budget enforcement -- Per-run, per-day, and per-month cost caps. The engine hard-stops if you hit the limit. No surprise bills.
  • JUnit XML output -- Drop --junit-xml results.xml and plug it into any CI system.
  • Tiered model routing -- Haiku for cheap navigation, Sonnet for complex reasoning, optional local Ollama for zero-cost simple actions.
  • Multi-platform -- Web apps (via Playwright), macOS native apps (via Accessibility API + pyobjc), iOS Simulator (via simctl). Same YAML format, different runners.
  • Evidence collection -- Every run produces screenshots, a findings report, cost breakdown, and a structured JSON result. Everything is saved to an evidence directory.
  • Stuck detection -- If the AI repeats the same action or the UI stops changing, the engine escalates to a stronger model, then aborts if nothing works. No infinite loops.
  • Template variables -- Use {{persona.credentials.email}} in your journey steps. Variables resolve from persona configs at runtime.
  • Precondition checks -- Verify services are up before running tests. Fail fast with clear errors instead of wasting API calls.

YAML schema support

SpecterQA includes a JSON Schema for product YAML files at schemas/product.schema.json.

# yaml-language-server: $schema=../../schemas/product.schema.json

CI Integration

SpecterQA is built for CI. It runs headless by default and returns proper exit codes.

# Basic CI run
specterqa run -p myapp --junit-xml results.xml

# Smoke test (runs first scenario only, fast)
specterqa run -p myapp --level smoke --budget 2.00

# JSON output for programmatic consumption
specterqa run -p myapp --output json > results.json

Exit codes:

  • 0 -- all tests passed
  • 1 -- one or more tests failed
  • 2 -- configuration error
  • 3 -- infrastructure error (missing dependencies, API unreachable)

See docs/ci-integration.md for GitHub Actions, GitLab CI, and CircleCI examples.

Cost

SpecterQA uses Anthropic's Claude API. Every run costs money. Here's what to expect:

ModelRoleInput (per 1M tokens)Output (per 1M tokens)
Claude Haiku 4.5Simple navigation$0.80$4.00
Claude Sonnet 4Complex reasoning$3.00$15.00
Ollama llava:13bLocal fallbackFreeFree

Typical costs per run:

  • 3-step smoke test: ~$0.30-0.60
  • 5-step standard journey: ~$0.50-1.50
  • Complex 10-step journey with forms: ~$1.00-3.00

The default budget is $5.00 per run. The engine hard-stops if the budget is exceeded -- no silent overruns. You can set per-day and per-month caps too.

You can also set a default budget via an environment variable to avoid passing --budget every time:

export SPECTERQA_BUDGET=2.00
specterqa run -p myapp          # uses $2.00 budget
specterqa run -p myapp --budget 5.00  # uses $5.00 budget (CLI flag wins)

Model routing helps: simple clicks and scrolls use Haiku ($0.01 per action), while form fills and initial assessments use Sonnet ($0.03-0.05 per action). If you have a local Ollama instance, simple actions can route there for zero API cost.

See docs/cost-guide.md for detailed cost breakdowns and budgeting strategies.

Multi-Platform

SpecterQA isn't web-only. The same persona/journey YAML format works across platforms:

Web apps (default) -- Uses Playwright for browser automation.

macOS native apps -- Uses the macOS Accessibility API via pyobjc. The AI reads the accessibility tree and screenshots, then executes clicks and keypresses through AX actions.

product:
  name: my-mac-app
  app_type: native_macos
  app_path: /Applications/MyApp.app
  bundle_id: com.example.myapp

iOS Simulator -- Uses simctl for screenshots and touch simulation. Useful for testing iOS apps without a physical device.

product:
  name: my-ios-app
  app_type: ios_simulator
  bundle_id: com.example.myiosapp
  simulator_device: "iPhone 15 Pro"
  simulator_os: "17.2"

Native and simulator support require the native optional dependency:

pip install specterqa[native]

For AI Agents

If you're an AI agent or building agent tooling, SpecterQA provides structured interfaces for programmatic use.

CLI with JSON output

specterqa run -p myapp --output json

Returns structured JSON to stdout:

{
  "passed": true,
  "run_id": "GQA-RUN-20260222-143052-a1b2",
  "step_reports": [
    {
      "step_id": "visit_homepage",
      "passed": true,
      "duration_seconds": 12.3
    }
  ],
  "findings": [],
  "cost_usd": 0.4521
}

Python API

from specterqa.config import SpecterQAConfig
from specterqa.engine.orchestrator import SpecterQAOrchestrator

config = SpecterQAConfig()
config.project_dir = Path(".specterqa")
config.products_dir = Path(".specterqa/products")
config.personas_dir = Path(".specterqa/personas")
config.journeys_dir = Path(".specterqa/journeys")
config.evidence_dir = Path(".specterqa/evidence")
config.anthropic_api_key = "sk-ant-..."
config.budget = 5.00
config.headless = True

orchestrator = SpecterQAOrchestrator(config)
report_md, all_passed = orchestrator.run(product="myapp", level="smoke")

Federated Protocol

SpecterQA exposes a protocols.py module with Python Protocol classes (AIDecider, ActionExecutor) that let you swap in your own AI model or action backend:

from specterqa.engine.protocols import AIDecider, Decision

class MyCustomDecider:
    def decide(self, goal, screenshot_base64, **kwargs) -> Decision:
        # Your logic here
        ...

MCP Server

SpecterQA ships an MCP (Model Context Protocol) server. Any MCP-compatible agent (Claude Desktop, Cursor, Cline, custom agent tooling) can discover and invoke SpecterQA as a tool -- run tests, read results, manage configs -- without shelling out to the CLI.

Add to your MCP client config (claude_desktop_config.json or equivalent):

{
  "specterqa": {
    "command": "specterqa-mcp",
    "args": []
  }
}

Available tools:

ToolDescription
specterqa_runExecute behavioral tests against a product. Synchronous — may take 45-300s. Incurs API costs (default budget: $5.00).
specterqa_list_productsList configured products and their available journeys
specterqa_get_resultsRetrieve full structured results from a previous run by run ID
specterqa_initInitialize a new SpecterQA project directory

See docs/for-agents.md for the full programmatic API reference and MCP integration details.

API Reference

The complete API reference is available at specterqa.synctek.io/docs.

Key classes

ClassModuleDescription
SpecterQAConfigspecterqa.configRoot configuration object. Set project dirs, API key, budget, and model routing preferences.
SpecterQAOrchestratorspecterqa.engine.orchestratorMain entry point for programmatic runs. Call orchestrator.run(product, level) to execute a journey.
AIDeciderspecterqa.engine.protocolsProtocol class. Implement to swap in a custom vision model or decision backend.
ActionExecutorspecterqa.engine.protocolsProtocol class. Implement to swap in a custom action execution backend (e.g., replace Playwright).
RunReportspecterqa.modelsStructured result returned by orchestrator.run(). Contains step reports, findings, and cost breakdown.
Findingspecterqa.modelsIndividual UX issue captured during a run. Includes severity, step ID, screenshot reference, and description.

CLI reference

CommandDescription
specterqa run -p PRODUCTRun all journeys for a product
specterqa run -p PRODUCT --level smokeRun only smoke-tagged journeys
specterqa run -p PRODUCT --junit-xml results.xmlEmit JUnit XML for CI
specterqa run -p PRODUCT --output jsonEmit structured JSON to stdout
specterqa initScaffold a .specterqa/ project directory with sample configs
specterqa installDownload Playwright browser binaries
specterqa listList configured products and journeys
specterqa results RUN_IDPrint the full report for a previous run
specterqa-mcpStart the MCP server

MCP tools

ToolDescription
specterqa_runExecute behavioral tests. Parameters: product (str), level (str, optional), directory (str, optional). Returns a RunReport JSON object.
specterqa_list_productsList all products and their configured journeys. No parameters required.
specterqa_get_resultsRetrieve a previous run report by run_id.
specterqa_initInitialize a new SpecterQA project at a given directory.

For schema definitions, type stubs, and federated protocol details, see docs/for-agents.md.

Security

Directory access: When the environment variable SPECTERQA_ALLOWED_DIRS is unset, the SpecterQA MCP server permits the directory parameter of specterqa_run to point at any path on the filesystem accessible to the process. In shared or multi-user environments — or anywhere the MCP server is exposed to untrusted agents — you should set this variable to an explicit allowlist:

export SPECTERQA_ALLOWED_DIRS="/home/user/projects:/ci/workspaces"

When set, the MCP server rejects any directory value that is not under one of the listed prefixes. This mitigates the MCP directory traversal vector described in SECURITY_ADVISORY.md (GHSA-SPECTERQA-001).

Command injection fix (v0.2.1): The check_command field in product YAML service definitions has been removed. It was the source of a critical command injection vulnerability. Precondition checks are now limited to TCP connectivity and HTTP health endpoint checks, which are safe. See SECURITY_ADVISORY.md for full details.

Credential scrubbing: Run artifacts (JSON result files, log output) automatically scrub known credential patterns — API keys, tokens, passwords — from captured content before writing to disk.

Reporting vulnerabilities: Do not open public issues for security bugs. Email info@synctek.io or see SECURITY.md for the full disclosure policy.