A small eval beats a big rewrite: improving skills in development

Morgane MossML Engineering — Nous Research

Nous Research has one of the best agent harnesses for performing deep research and other common business tasks. To do this effectively, they rely on bundling the best skills with their model and harness. Researchers have recently demonstrated that applying an evolutionary algorithm to skills can take small, cheap and lightweight models and have them outperform the most expensive tools to date.

“Jetty helped me run hundreds of long agent trajectories in parallel — cleanly and organized for analysis.”

01 · Situation

For Morgane at Nous Research, using Jetty to optimize the skill was the shortest path to making significant improvements by leveraging the stored runs and runtime configuration, paired with the evaluation framework that ships with Jetty AI Workloads.

As the project scope increased, Morgane used Jetty to shift the job from her laptop to managed agents running parallel, with shared memory and failure modes that could be read and turned into evaluations for future runs.

The key unlock was a research loop that kept every config and every run in context and ready for skill self-improvement.

02 · Solution

Morgane was able to run and test her skill with different versions of the Hermes Agent. Jetty’s ability to support a range of models and allow her to bring her own MCP tools. That’s the same bring-the-job, we-handle-the-infra model Jetty runs on generally: pick your agent and model pair, and Jetty tracks the configs, outputs, and logs underneath it.

Every run’s configuration was stored automatically. She launched runs of 30 to 90 minutes, many at a time, and left them. Set the AI workload, close the laptop, walk the dog and pick it back up when it’s done.

Onboarding was quick: she ran against her own agent and model.
Parallel runs and saved runtime configuration: every result reproducible and traceable.
Answered questions along the way: find failure modes and pick the best changes to the eval suite.

03 · Outcome

Each run’s failure modes told Morgane exactly what to adjust for the next iteration. She was able to quickly prove out different algorithms and ultimately settle on a variant of the GEPA optimization mechanism.

The best part was the skill was ready the moment the model landed: no agent-sitting. Using Jetty meant that runs finished on their own, freeing Morgane to work on the next iteration instead of watching a terminal.