autogpt-agents
A self-hosted platform for building AI agents as drag-and-drop workflows, or with a Python developer toolkit, that keep running continuously and respond to triggers like a schedule or an incoming webhook.
LABSCO SUMMARY
These 98 skills from Orchestra Research teach your coding assistant the tools of AI research, such as vLLM, DeepSpeed and TRL, and the work itself, from brainstorming ideas to writing the paper.
Each skill is a written guide your assistant reads when the work matches; most cover one tool, with setup steps and worked code, and many also list the usual errors with their fixes or when another tool fits better. 96 of them also come as 23 Claude Code plugins, one per research area, so you can add only the areas you work in. Orchestra's own installer adds all 98, or the areas and skills you pick, to Claude Code, Cursor, Codex and other assistants.
The skills are guides, not the tools themselves: the tools they describe install separately, under their own licences. The large-scale training skills, such as Megatron-Core or miles, assume several NVIDIA GPUs (miles expects eight H100 or H200 cards), and the skills for hosted services such as Pinecone, Modal or LangSmith need an account there. The autoresearch skill runs a whole research project without asking for approval and resumes the work every 20 minutes; it is a plugin of its own, so you can leave it out.
WHAT'S INSIDE
Showing all 98 skills
A self-hosted platform for building AI agents as drag-and-drop workflows, or with a Python developer toolkit, that keep running continuously and respond to triggers like a schedule or an incoming webhook.
Assembles a team of role-based AI agents — each with a role, goal, and backstory — that hand tasks off to each other in sequence or under a manager agent, without depending on LangChain.
The pieces an AI app usually needs — a model, your own documents, outside tools it can call, a memory of the conversation — already exist here and snap together, so a chatbot or a question-answering app gets assembled rather than written from scratch.
Reads a pile of your own documents — files, web pages, a database — into a form an AI model can search, so its answers come from what is written there instead of from whatever it half-remembers.
Runs a research project for you, from question to paper draft: it searches the literature, forms hypotheses, runs experiments, records what each one showed and writes up the findings. It works on its own without asking for approval, and its first step is a loop that resumes the work every 20 minutes, so it keeps going while you are away.
Measures how good a model is at writing code that actually runs, against the standard problem sets everyone quotes — and in 18 languages, not just Python.
Runs a language model against 60+ standardized academic benchmarks — MMLU, GSM8K, HellaSwag, TruthfulQA, and more — using the same prompts everyone else uses, so scores are directly comparable to published results.
Runs a model against 100 or more standard benchmarks — academic, coding, safety, and vision-language — from inside reproducible containers, executing locally, on a Slurm cluster, or in the cloud.
Logs the parameters, metrics, and model file from each training run, then tracks that model's versions through a shared registry as it moves from staging to production.
While a model is being trained, all you otherwise get is numbers scrolling past in a terminal; this collects them as they go and draws them as charts in a browser tab, so you can see whether the thing is actually learning.
Training a model means dozens of attempts, and by week two nobody remembers which one worked — this keeps a live record of every run, its settings and its results.
Walks you through ten ways of finding a research idea, such as hunting for trade-offs everyone accepts, revisiting methods abandoned years ago, or probing where a popular method breaks. It then narrows 10-20 raw ideas to the strongest 3-5 and turns the winner into a two-sentence pitch and a two-week pilot plan.
Eight thinking techniques from creativity research, applied to computer science and AI: combining ideas from distant fields, reframing the problem, reasoning by analogy, dropping a constraint everyone assumes, and more. It aims at new research directions rather than small extensions of existing work.
Retrains an open-source AI model on your own data from a single small config file instead of code, including the cheaper methods that only adjust part of the model and training spread across many GPUs.
Points your assistant to LLaMA-Factory's documentation for fine-tuning language models, including its no-code web screen, LoRA and 2- to 8-bit QLoRA, and support for 100+ models. The skill itself is a short index; the detail is the project's documentation copied into reference files, much of it in Chinese.
Fine-tunes a large language model by training small adapter weights instead of the full model, cutting GPU memory and storage needs to a fraction of full fine-tuning.
Points your assistant to Unsloth's documentation for faster, lower-memory fine-tuning with LoRA and QLoRA, which Unsloth puts at 2-5x faster with 50-80% less memory. The skill itself is a short index; the detail is Unsloth's own documentation, 136 pages copied into one reference file.
Rents dedicated GPU machines by the minute, from a single card up to a multi-hundred-GPU cluster, with full SSH access, a pre-installed ML software stack, and storage that survives a restart.
Runs Python functions on demand in the cloud with GPUs attached, scaling containers from zero to over a hundred instances automatically and billing only for the seconds they actually run.
Renting computing power from a cloud company normally ties your work to that one company's way of doing things; here you write the job down once and it will run on any of twenty-odd providers, picking itself back up if a cheap machine gets taken away mid-run.
Generates music and sound effects from text descriptions using Meta's AudioCraft models, with options to shape the result by melody, style reference, or stereo output.
Describes what is in a photo and answers questions about it in plain language — it joins an existing image model to an existing language model with a small bridge, so neither of them has to be retrained.
OpenAI's CLIP scores how well a picture and a piece of text go together — enough to sort photos into any categories you name, search a library by description, or flag unwanted images, with no training of your own.
Sets up and runs NVIDIA's Cosmos Policy, a model that turns camera images and a written instruction into robot movements, on two simulated robot benchmarks, LIBERO and RoboCasa. It covers a fresh GPU server without a screen, a quick one-trial check, the full benchmark runs and timing the model's responses.
Fine-tunes and evaluates OpenVLA-OFT, a version of the OpenVLA robot model that predicts smooth, continuous arm movements instead of picking from fixed steps. It follows the paper's own recipes for the LIBERO simulator and the two-armed ALOHA robot, including the settings that must match between training and use.
Fine-tunes and serves Physical Intelligence's OpenPI robot models, pi0, pi0-fast and pi0.5, which turn camera images and an instruction into robot actions. It covers setup on a fresh machine, training with JAX or PyTorch, converting checkpoints between the two, and running a policy server that your robot or simulator connects to.
An open model you can run on your own machine that looks at a picture and talks about it — you ask what is in the photo, and then keep asking follow-up questions about the same one.
Trained on a billion outlines of every kind of thing, it can trace the exact shape of an object it has never been shown before — a cell under a microscope, a roof in a satellite photo — from one mark you put on the picture.
Describe a picture in words and get it back as an image, made on your own machine rather than someone's website — and the same models will redraw a chosen patch of a photo you already have, or hold a pose or an outline while changing everything around it.
A speech-to-text model you run yourself, on your own machine, covering 99 languages and able to turn foreign-language audio straight into English.
Opens up the inside of an AI model so you can watch what each layer is doing while it works, and change a layer partway through to see how the answer shifts.
Watching one part of a model light up does not prove it is responsible for anything; this changes that part on purpose and checks whether the model's behaviour changes with it.
A single unit inside a model fires for several unrelated things at once, which is why nobody can read it directly; this pulls those tangled signals apart into a much larger set where each one stands for one recognisable idea.
The standard workbench for mapping the circuitry inside a language model: which parts feed into which, in what order, and what each of them is carrying while the model works out its answer.
Trains a smaller "student" model to imitate a larger "teacher" model's behavior, so you end up with a model that keeps most of the teacher's quality at a fraction of its size and inference cost.
A model that could only take in a short report at a time can be stretched to swallow a whole book — this is the four established ways of doing that stretching, and how much retraining each one costs.
Combines several fine-tuned models into one with mergekit, blending their strengths, such as maths, coding and chat, without further training and without a GPU. It covers the main merge methods and can choose the blend ratio without labelled test data, by picking the setting whose answers stay most consistent.
Zeroes out the least important weights in a trained model, cutting its size 40-60% with under 1% accuracy loss and no retraining required.
Instead of one big network where every part works on every word, the model is built as a panel of specialists with a router that hands each word to just a couple of them — so it can hold far more knowledge without costing much more to run.
A chatbot writes its reply one word at a time, and every single word costs a full trip through a huge model; letting a small quick model guess the next few words and having the big one merely check them gives back exactly the same reply, up to three times faster.
PyTorch's official framework for pretraining large language models from scratch, combining four kinds of parallelism — FSDP2, tensor, pipeline, and context — to scale one training run from a single node to hundreds of GPUs.
Twenty-odd well-known open AI models — Llama, Gemma, Phi, Qwen, Mistral — each written out as one readable file instead of being buried under layers of framework, so you can follow what the model actually does and change it.
A different design for the inside of an AI model: the usual kind gets slower and hungrier for memory the longer the text runs, while this one holds a steady cost per word and generates about five times faster.
A roughly 300-line, from-scratch GPT implementation that trains in minutes on a CPU for small experiments and scales up to reproduce GPT-2 on multiple GPUs, built for understanding transformers rather than production use.
A model design with two modes: it learns all at once, the fast modern way, then writes text one word at a time on a fixed amount of memory, no matter how long the piece runs.
When an AI feature gives a bad answer there is usually no way to see why it did — this records everything that happened behind each request, and lets you collect the bad cases into a test set you re-run after every change.
Everything an AI app does gets recorded step by step, on servers you run yourself rather than a vendor's — the traces, the test sets built out of them, and the scores, all in one place you own.
Makes figures for ML papers. Architecture and pipeline diagrams are drawn by Google's Gemini image model from your paper text, in one of four visual styles; charts of results are drawn with matplotlib at each conference's column width, with your method in its own colour.
Turns a research repository's code and results into a paper draft for NeurIPS, ICML, ICLR, ACL, AAAI or COLM, checking every citation against a live search instead of writing one from memory. Systems conferences such as OSDI and SOSP go to the systems-paper-writing skill in the same plugin.
Turns a finished paper into conference slides: a LaTeX Beamer PDF and an editable PowerPoint file, with speaker notes and timing for every slide. It sizes the talk to its slot, from a 3-minute poster talk to a 45-minute invited talk, and plans a demo slide for systems talks.
A section-by-section plan for a 12-page systems paper for OSDI, SOSP, ASPLOS, NSDI or EuroSys: how many pages each section gets, what each paragraph should do, and checklists drawn from guides by the conferences' own reviewers and programme chairs. It also carries LaTeX templates and each venue's page limits.
A model is only as good as the pile of text, images, video or audio it learns from — this is the cleaning step before training: duplicates thrown out, junk filtered, personal details and explicit material stripped, all of it running across graphics cards rather than ordinary processors.
The step that gets data to a model — reading it, cleaning it, resizing it, handing it over — spread across a whole cluster instead of one machine, and taken a piece at a time rather than all at once.
Teaches a model which answers people prefer — show it good examples, train a scorer on pairs of better-and-worse answers, then let the model practise against that scorer, alone or as the whole chain.
Instead of showing a model the right answers, you write rules that score its attempts — it takes several shots at every question and learns to produce more of whatever your rules rate highly.
The very largest open AI models break the ordinary reward-based training tools — the run goes unstable and the model no longer fits the hardware — and this is the industrial-grade version built for exactly that scale.
Teaching a model what people prefer takes a whole cluster running in lockstep — one part writes answers, another scores them, another retrains on the scores — and this is the machinery that coordinates it, at about twice the speed of the older option.
Teaching a model which of two answers people prefer normally means keeping a second, untouched copy of it alongside as a yardstick; this method throws that copy away and still comes out ahead of the one that keeps it.
The framework Tsinghua's team actually used to build the GLM models, released openly: it runs the stage after pre-training where a model improves by being rewarded, and leaves the logic for producing practice material up to you.
You write the rule for what counts as a good answer, and nothing else — the machines, the generating, and the keeping-everything-in-step are all handled underneath.
A training rig for teaching a language model by rewarding its good answers, built to run across a whole cluster of GPUs rather than one machine.
Turns research material, such as a PDF paper, a code repository, experiment logs or rough notes, into a research record. The record is a folder of linked files listing the claims, the experiments behind them, the exact results, the code settings and the dead ends. It copies numbers exactly, marks what the material does not say, and checks the folder's structure before reporting back.
Keeps a research log for you: at the end of each coding or research session it reads back through the conversation and records the decisions, experiments, dead ends, claims and changes of direction in an ara folder. Each entry is marked by who it came from: you, an AI suggestion, or an action the AI took.
Reviews a research record the way a careful referee would: does the evidence really support each claim, could the claims be proven wrong, are failures recorded honestly, are the experiments sound. It scores six areas from 1 to 5, ranks problems by severity and ends with a recommendation from Strong Accept to Reject.
Anthropic's method for making a model safer without people having to label harmful content: the model judges its own answers against a short written set of principles and rewrites them, then is trained on its own judgements.
A second model from Meta that sits between a chatbot and its users, reading each message and each reply and marking anything that falls into one of six risk areas — violence and hate, sexual content, weapons, drugs, self-harm, or crime.
Every message going in and every reply coming out passes a set of checks first — for attempts to talk the bot out of its rules, for made-up facts, for personal details, and for abusive content.
A small, fast checker for one specific danger: text that is trying to hijack a model's instructions, whether a user typed it in or it was hidden inside a web page or document the model is about to read.
The store an AI app keeps its documents in so it can pull back the ones closest in meaning to a question — a free, open-source vector database that runs in a notebook or on a server.
Facebook's library for finding, among millions or billions of items, the handful most like the one you are holding — with knobs to trade accuracy for speed or memory, and an option to run on the graphics card.
A hosted database that finds things by meaning rather than by matching words: each document goes in as a numeric fingerprint, and a query comes back with the closest ones — scaling to billions of them without you running a single server.
A search engine that matches on meaning rather than on the words themselves, and that you run on your own machines — kept on your premises, under your control, instead of handed to a hosted service.
Text and images get turned into lists of numbers that land close together when the meaning is close — which is the trick behind "find me something like this" — and all of it runs on your own machine.
The program that actually runs an AI chat model on hardware people already own, with no Nvidia graphics card required and no stack of other software to install underneath it.
Answering one request at a time leaves most of an expensive graphics card sitting idle; this keeps many conversations in flight at once and gets roughly twenty-four times the work out of the same hardware.
Nearly every request to a chatbot or an agent opens with the same block of text, and most servers redo that work every single time — this one remembers it, so only the genuinely new part of a message costs anything.
Rather than running a model as it comes, this compiles it ahead of time into a build tuned for the exact NVIDIA card it will sit on — the fastest option available, at the price of that build step and of being tied to one make of hardware.
Shrinks a large AI model to about a quarter of the memory so it fits on a smaller graphics card and runs roughly three times faster, protecting the few numbers inside that matter most so quality barely drops.
Turns a downloaded AI model into a single self-contained file, shrunk by as much or as little as you choose, that ordinary desktop apps can open and run on your own machine.
A very large AI model normally wants more graphics-card memory than anyone has spare — this cuts what it needs to roughly a quarter and makes it answer three to four times faster, with the answers themselves barely changed.
Other ways of shrinking an AI model want a pile of sample text first and hours of processing; this one needs neither — point it at the model and a much smaller version comes back in minutes.
A field guide to training neural networks in PyTorch: which model to start from for your kind and amount of data, optimizer and learning-rate settings, and step-by-step fixes when loss explodes, training stalls or the GPU runs out of memory. It covers language, vision and diffusion models, and biomedical work such as proteins and medical images.
Inside every modern AI model, one step compares every word against every other word, and that step is what devours the memory; this swaps it for a version that needs ten to twenty times less and returns exactly the same answers.
Loads a large language model in 8-bit or 4-bit precision instead of full precision, cutting GPU memory use by 50 to 75 percent with little accuracy loss.
Instead of hand-writing prompts, you declare what goes in and what should come out — then a search over your own examples and your own scoring finds the wording that actually performs best.
You normally ask a model for output in a particular format and hope it obeys; this locks the format in while the text is being written, so a reply that does not fit simply cannot come out.
A layer that sits over an AI model and turns its answer into a filled-in form rather than a paragraph — and when the answer doesn't fit the form, it sends it back to the model to try again.
A model running on your own hardware can be pinned to a shape you define in advance, so the text it produces always fits — nothing left to parse afterwards, and no retry loop for when it does not.
Text has to be chopped into small pieces and turned into numbers before a model can read it; this does the chopping at about a gigabyte every twenty seconds, and can learn a set of pieces tailored to your own kind of writing.
Chopping text into pieces normally starts by splitting on spaces, which falls apart for Chinese, Japanese and Korean, where there are none; this works straight off the raw characters instead, so one method covers every language.
A model too big for one graphics card, trained anyway — Microsoft's DeepSpeed splits it and everything training needs across many GPUs, and spills to ordinary memory or disk when even that is not enough.
You write the training script once and choose afterwards where it runs — one graphics card, eight of them, several machines, or a TPU — instead of rewriting it for each.
When a model is too big to fit on one graphics card, its parts have to be cut up and spread across several — this is how to make that split correctly in PyTorch, including the order of steps it has to be done in.
Training code written by hand is mostly the same plumbing every time; put yours into this standard shape and the plumbing comes written for you, whether the run ends up on a laptop or on a supercomputer.
Training across a room full of separate machines is more an organising problem than a coding one — who runs what, who reports to whom, what happens when one of them dies — and this is the part that handles all of that.
Building a model from scratch at the largest sizes means splitting the work across thousands of graphics cards in several directions at once — this is the machinery that does the splitting, tuned so as little of that hardware as possible sits idle.
Teaches your assistant A-Evolve, a framework that improves an existing AI agent automatically. The agent works through a batch of benchmark tasks, a language model rewrites its prompts, skills and memory from what failed, and every change is kept in git so it can be undone. It is not part of any of the collection's plugins, so it is installed on its own.
Teaches your assistant SwanLab, an open-source tracker for training runs: it logs metrics, settings, images and audio, and compares runs side by side. Runs can go to SwanLab's cloud, to a server you host yourself, or stay on your own machine. It is not part of any of the collection's plugins, so it is installed on its own.
LLM architectures and implementations including LitGPT, Mamba, NanoGPT, RWKV, and TorchTitan. Use when implementing, training, or understanding transformer and alternative architectures.
Text tokenization for LLMs including HuggingFace Tokenizers and SentencePiece. Use when training custom tokenizers or handling multilingual text.
LLM fine-tuning frameworks including Axolotl, LLaMA-Factory, PEFT, and Unsloth. Use when fine-tuning models with LoRA, QLoRA, or full fine-tuning.
Neural network interpretability tools including TransformerLens, SAELens, NNSight, and pyvene. Use when analyzing model internals, finding circuits, or understanding how models compute.
Data curation and processing at scale including NeMo Curator and Ray Data. Use when preparing training datasets or processing large-scale data.
RLHF and preference alignment including TRL, GRPO, OpenRLHF, SimPO, verl, slime, miles, and torchforge. Use when aligning models with human preferences, training reward models, or large-scale RL training.
AI safety and content moderation including Constitutional AI, LlamaGuard, NeMo Guardrails, and Prompt Guard. Use when implementing safety filters, content moderation, or prompt injection detection.
Multi-GPU and multi-node training including DeepSpeed, PyTorch FSDP, Accelerate, Megatron-Core, PyTorch Lightning, and Ray Train. Use when training large models across GPUs.
GPU cloud and compute orchestration including Modal, Lambda Labs, and SkyPilot. Use when deploying training jobs or managing GPU resources.
Model optimization and quantization including Flash Attention, bitsandbytes, GPTQ, AWQ, GGUF, and HQQ. Use when reducing memory, accelerating inference, or quantizing models.
LLM benchmarking and evaluation including lm-evaluation-harness, BigCode Evaluation Harness, and NeMo Evaluator. Use when benchmarking models or measuring performance.
Production LLM inference including vLLM, TensorRT-LLM, llama.cpp, and SGLang. Use when deploying models for production inference.
ML experiment tracking and lifecycle including Weights & Biases, MLflow, and TensorBoard. Use when tracking experiments or managing models.
LLM agent frameworks including LangChain, LlamaIndex, CrewAI, and AutoGPT. Use when building chatbots, autonomous agents, or tool-using systems.
Retrieval-Augmented Generation including Chroma, FAISS, Pinecone, Qdrant, and Sentence Transformers. Use when building semantic search or document retrieval systems.
Structured LLM outputs including DSPy, Instructor, Guidance, and Outlines. Use when extracting structured data or constraining LLM outputs.
LLM application monitoring including LangSmith and Phoenix. Use when debugging LLM apps or monitoring production systems.
Vision, audio, and multimodal models including CLIP, Whisper, LLaVA, BLIP-2, Segment Anything, Stable Diffusion, AudioCraft, Cosmos Policy, OpenPI, and OpenVLA-OFT. Use when working with images, audio, multimodal tasks, or vision-language-action robot policies.
Advanced ML techniques including MoE Training, Model Merging, Long Context, Speculative Decoding, Knowledge Distillation, and Model Pruning. Use when implementing cutting-edge optimization or architecture techniques.
Autonomous research orchestration using a two-loop architecture. Manages the full research lifecycle from literature survey to paper writing, routing to domain-specific skills for execution. Use when starting a research project, running autonomous experiments, or managing multi-hypothesis research.
Write publication-ready ML/AI/Systems papers for NeurIPS, ICML, ICLR, ACL, AAAI, COLM, OSDI, NSDI, ASPLOS, SOSP. Includes LaTeX templates, citation verification, reviewer guidelines, publication-quality figure generation, systems paper structural blueprints, and conference presentation slides.
Research ideation frameworks including structured brainstorming and creative thinking. Use when exploring new research directions, generating novel ideas, or seeking fresh angles on existing work.
Agent-Native Research Artifact (ARA) tooling: compile any research input (paper, repo, notes) into a structured artifact, record session provenance as a post-task epilogue, and run Seal Level 2 epistemic review. Use when ingesting research into a falsifiable, agent-traversable artifact, capturing how a research project actually evolved, or auditing an ARA for evidence-claim alignment.
INSTALL
SINGLE SKILL
Run this outside your AI tool, using Node.js 22.20.0 or newer. Skills CLI 1.7.2 skips skill selection; prompts cancel installation without an interactive terminal. If choosing Project when asked about installation scope, open your project's folder before running this.
Install checklist
Additional agents. ↑/↓ moves, Space toggles selections, Enter confirms.Installing to: for your intended tool. Stop if it names a different tool.Symlink (Recommended) for shared files or Copy to all agents for separate copies; review the summary before confirming installation.Install the find-skills skill? defaults to Yes, adding another skill for all projects. Choose No if you don't want that extra installation.Find the skill name in the Skills tab. Open that skill's panel before copying a command: some skills need files or other skills from this package, and the panel shows the install route to use.
npx skills add orchestra-research/ai-research-skills --skill <YOUR_SKILL_NAME> --full-depthClaude Code
/plugin marketplace add orchestra-research/AI-research-SKILLs/plugin install distributed-training@ai-research-skillsType both lines in a Claude Code session, not at your shell prompt. They add Orchestra Research's marketplace, which is this repository, and open distributed-training, the research-area plugin for multi-GPU training with tools such as DeepSpeed and PyTorch FSDP.
Orchestra installer
npx @orchestra-research/ai-research-skillsRun it in a terminal. It opens a menu where you choose everything, a quick-start set, whole research areas or single skills, then which of the coding agents it found on your computer get them.