← Back to feed

Microsoft has released Agent Lightning v1.0, a framework for improving AI agents using other AI agents. The package installs as a skill into Claude Code, Codex, or GitHub Copilot and operates by taking two inputs: an editable agent definition and a benchmark. From there, it runs a structured loop that proposes and evaluates changes to that agent’s prompts, tools, workflows, model selection, and reasoning settings.

The underlying technique is what Microsoft calls harnessed agentic reinforcement learning. Unlike conventional RL fine-tuning, which requires a separate training environment, Agent Lightning’s approach lets the production harness manage the environment interaction loop directly. The agent being optimised runs inside its own operational context; improvements are measured against a real benchmark rather than a synthetic reward function. The result is that you can iterate on an agent without setting up a training cluster or risking changes to a live system.

What the Optimization Loop Does

Given a benchmark and an agent, Agent Lightning evaluates current performance, proposes a change, tests it, and measures the delta across four dimensions: accuracy, cost, latency, and reliability. Changes that improve the target metric without degrading others are accepted. Changes that degrade non-target metrics beyond a configured threshold are rejected.

The optimization surface includes:

• System prompt edits — rewriting instructions or adding context
• Tool configuration — modifying which tools are available or how they are called
• Workflow changes — restructuring multi-step sequences
• Model selection — switching between models if the agent is model-agnostic
• Reasoning settings — adjusting thinking depth or chain-of-thought configuration

Install via:

gh skill install microsoft/agent-lightning agent-lightning --agent <agent>

Why This Matters

The standard way to improve a production agent is to eyeball its failures, write a fix, and deploy. That process is manual, slow, and depends on human intuition about what prompts or tools are causing the problem. Agent Lightning automates the hypothesis-test loop, making it possible to run dozens of improvement iterations systematically.

The RL framing is deliberate. Rather than treating agent improvement as a prompt engineering exercise, the framework treats it as a policy search problem: find the agent configuration that maximises performance on a given benchmark. That shifts the bottleneck from human judgment to benchmark quality — a problem that is easier to distribute and parallelize.

The practical constraint is that Agent Lightning is only as good as the benchmark you give it. A poorly constructed benchmark will produce an optimised agent that performs well on the benchmark and poorly in production. The framework provides the optimization loop; defining what to optimise for remains a human responsibility.

Agent Lightning v1.0.1 is available on GitHub under microsoft/agent-lightning.