xcomet_evaluate gives a quality score between 0 and 1, but the part you actually work from is the error spans with their minor, major and critical labels — the same score caused by one critical mistranslation and by a scattering of minor wording choices are entirely different problems. xcomet_detect_errors exists for when the score is not the question at all, and its minimum-severity filter keeps the trivia out. The batch tool is where a real evaluation run happens, returning aggregate statistics with the per-pair results, and batch_size and use_gpu are parameters rather than configuration because they belong to the run rather than the server.
A translation quality evaluation server built on the xCOMET model, with three tools: score a pair, detect its errors, and evaluate a batch.
- A score with its reasoning attached: xcomet_evaluate takes a source and a translation with an optional reference and language codes, and returns a quality score between 0 and 1, the detected error spans with minor, major or critical severity, and a readable quality summary.
- Errors on their own: xcomet_detect_errors focuses on detection and returns errors with their severity and position, filtered by a minimum severity.
- Batches with aggregates: xcomet_batch_evaluate takes an array of source and translation pairs, each with an optional reference, and returns aggregate statistics alongside the individual results, with a batch_size for fitting the run to the machine.
- Hardware and format as parameters: xcomet_evaluate, xcomet_detect_errors and xcomet_batch_evaluate each take a use_gpu flag and a response_format.
A Python environment with xCOMET available, pointed at by XCOMET_PYTHON_PATH.
One command plus a key — npm install -g xcomet-mcp-server, then supply credentials