← Back to feed

Every major AI benchmark resets. Atari resets. MuJoCo resets. SWE-bench gives the model a repository, a task, and a clean slate. Real enterprise environments do not reset — they evolve, accumulate constraints, and shift the rules on running systems. Morpheus, a new evaluation platform from Skyfall AI, is built around that gap.

The result: frontier models that post strong scores on conventional benchmarks fail to adapt in dynamic enterprise simulations. The paper calls this a fundamental failure of continual learning.

What Morpheus Measures

Morpheus is described as the first reinforcement learning environment built on real-world enterprise workflow dynamics rather than game-like abstractions. The platform creates persistent simulations where resource allocation and scheduling constraints change over time — mimicking how a business actually operates. No resets. No clean-slate evaluations.

The benchmark breaks into two tasks:

Task 1 (Resource Allocation): Conditions are relatively stable. Models learn what resources to allocate under a given constraint set and maintain that behavior across simulation time.

Task 2 (Scheduling Under Drift): The environment shifts. Scheduling constraints, resource costs, and reward signals all change as the simulation progresses. Models must detect that something has changed and revise their behavior accordingly.

What Happened

Task 1 results across frontier models are stable and strong — comparable to what static benchmarks would predict. Task 2 tells a different story. Performance starts near Task 1 levels, becomes volatile as drift accumulates, and in multiple runs collapses near the end of the simulation.

The core finding: frontier models are not updating when rules change. They are pattern-matching against training distributions rather than learning from in-context feedback about what has shifted. Strong static benchmark averages conceal this brittleness because static benchmarks never present the model with a moving target.

Skyfall AI frames the distinction precisely: Morpheus separates continual learning from broad pretraining coverage. A model can have excellent coverage of enterprise scheduling problems from pretraining without being able to adapt when the specific constraints in front of it drift away from that coverage.

Why It Matters for Deployed Agents

The implication for production AI systems is direct. An agent deployed into a long-running enterprise workflow — supply chain scheduling, resource planning, capacity management — will encounter drift. Business conditions change. Pricing changes. Demand changes. Vendor relationships change. If the model cannot revise its behavior as those conditions shift, it will degrade.

This is a different failure mode from the one documented in AgingBench, which tracks degradation from stale model weights. Morpheus finds degradation even with current weights, caused by the model’s inability to update its behavior within a session as environmental rules change.

No specific model-by-model scores have been published from the Morpheus runs. The Skyfall AI team has released the benchmark environment and the two-task graph showing aggregate performance trends. Full leaderboard results are expected to follow.

The Broader Picture

Morpheus joins a growing set of evaluations that stress-test AI systems under conditions that move beyond single-turn capability measurement. ByteDance’s EdgeBench found agent performance follows a scaling law over 12-hour runs. The AutoLab benchmark found that persistence, not raw capability, predicts research agent performance across 36 tasks. Morpheus adds a third dimension: whether models can detect and respond to rule changes in real time.

The question for enterprise AI teams is whether strong SWE-bench and agent benchmark scores are sufficient signal for production deployment in dynamic environments. Morpheus suggests they are not.