Microsoft Research has introduced EvoLib, a test-time learning framework that enables API-hosted large language models to improve from their own experience without ever touching the model’s parameters. Described in a new paper and accompanying blog post, the system builds a constantly evolving knowledge library from an agent’s successes and failures, allowing it to get smarter at tasks like code generation and mathematical reasoning over time—all while the underlying model remains frozen.
How EvoLib turns experiences into reusable knowledge
Most AI agents today have memory that’s little more than a running transcript. They can recall past interactions, but they don’t truly learn from them. EvoLib changes that by distilling raw reasoning traces into two distinct types of knowledge: modular skills and reflective insights. Skills capture reusable procedures from successful attempts—like a debugging strategy or a coding pattern. Insights, on the other hand, extract lessons from mistakes, such as common pitfalls or error-correction techniques.
Once captured, these knowledge units aren’t left to gather dust. EvoLib continuously refines them. When a new skill or insight arrives, the system checks the library for similar existing entries and, if possible, merges them into a more generalizable form. The result isn’t a bigger pile of memories; it’s a smarter, more compact library. Microsoft calls this process “consolidation,” and it’s paired with a dynamic weighting mechanism that evaluates each knowledge unit not just by its immediate usefulness but by its potential to strengthen future learning. Over time, the most impactful strategies rise to the top, while one-off workarounds or outdated advice naturally fade.
Crucially, EvoLib operates entirely at inference time. There’s no fine-tuning, no gradient updates, no need for ground-truth labels or external reward signals. It works with any model accessible via API—think Azure OpenAI Service, third-party LLMs, or any black-box model you can prompt. The model itself never changes; only the context provided alongside each new task evolves. For organizations that rely on cloud-hosted models and can’t modify their internal weights, this is a big deal.
Microsoft tested EvoLib on three challenging domains: mathematical reasoning, code generation under efficiency constraints, and multi-turn agent tasks that require exploration and interaction. According to the research paper, EvoLib outperformed retrieval-based memory systems and other test-time learning baselines across all three. In static experiments, it delivered an 11% to 20% improvement over basic repeated sampling, and a 5% to 10% edge over the best existing test-time learning method. Moreover, EvoLib used fewer tokens in some code benchmarks, meaning it achieved better results with less computation. The team also found that performance was robust to random task orderings—an important sign that the system can handle the messy, interleaved requests typical of real-world use.
Who stands to gain from self-improving AI agents
The promise of EvoLib extends differently to various groups, but the common thread is letting AI agents learn from experience without requiring expensive retraining.
For Windows developers building AI assistants: If you’re integrating an API-bound LLM into a long-running desktop application, EvoLib offers a pattern for making that assistant improve over time. Imagine a code-review bot that remembers which refactoring steps worked last time, or a PowerShell helper that learns the most efficient command sequences for your specific environment. Instead of starting from scratch with every query, the agent draws on a growing library of proven techniques. The GitHub repository accompanying the research provides code you can experiment with today, though adapting it to a production tool would require careful engineering.
For IT administrators and power users: Many organizations are deploying AI agents to triage support tickets, generate reports, or monitor systems. Right now, most of these agents are memoryless or rely on simple retrieval that can’t distinguish a good solution from a bad one. EvoLib’s approach suggests a future where an agent that fumbles a ticket might actually learn from that fumble—and share that lesson across similar tickets handled by other agents. The catch is governance. An evolving skill library that persists over months could accidentally embed stale information, security-sensitive text, or faulty logic. Admins would need tools to audit, roll back, or isolate individual knowledge entries—something the current research prototype doesn’t address fully.
For enterprise decision-makers: EvoLib aligns with a broader trend: getting more value from inference compute. Instead of spending extra tokens solely on refining a single answer (through chain-of-thought or multiple samples), EvoLib lets some of that compute generate reusable knowledge that amortizes across future tasks. For organizations paying per-token API costs, this could mean lower overall bills as agents become more efficient. However, any production use would need thorough testing; the research results were achieved on controlled benchmarks, not messy enterprise environments with shifting tools, permissions, and data.
From memory to learning: how EvoLib got here
Agent memory isn’t a new idea. Systems like retrieval-augmented generation (RAG) have long allowed LLMs to pull in relevant documents or past conversations. But as Microsoft Research argues, “memory alone is not learning.” Storing every past interaction doesn’t guarantee better outcomes—it might just create noise. Earlier attempts at abstract memory, such as storing summaries or key-value pairs, helped but still couldn’t match human-like skill extraction.
What sets EvoLib apart is its focus on knowledge evolution. Rather than freezing knowledge after creation, it continually re-evaluates and reshapes it. The approach was inspired by cognitive science: humans don’t remember every detail; we remember what matters—strategies that work, mistakes to avoid, and skills that transfer. EvoLib tries to give AI a similar capacity.
The timing is critical. As foundation models have become more capable, the bottleneck has shifted from acquiring a model to customizing it for specific workflows. Fine-tuning is powerful but often impractical for organizations using cloud APIs, where model weights are inaccessible. Test-time methods that adapt behaviour without altering the model have gained traction, but many still treat each task in isolation. EvoLib’s contribution is a persistent library that evolves across sessions, making it a step toward truly continuous learning.
Putting EvoLib to work: what you can try today
EvoLib is a research project, not a managed service. But that doesn’t mean you can’t explore its potential:
- Read the paper and explore the code. The research team has published their work on arxiv and released code on GitHub. The repository includes implementations for the benchmark tasks, which can serve as a starting point for understanding the framework.
- Experiment in a sandbox. If you have access to an LLM API (like Azure OpenAI), you can replicate the basic idea: capture your agent’s outputs, distill them into reusable templates, and insert those into future prompts. A simple proof-of-concept might use a vector database to store and retrieve distilled skills.
- Audit your current agent memory. If you’re already using retrieval-augmented memory, ask: how do you decide what to store? Is it just raw transcripts, or have you abstracted lessons? Do you prune or update old entries? EvoLib provides a mental model for upgrading that memory to something more dynamic.
- Prepare for governance early. Even if EvoLib isn’t ready for production, its concepts highlight upcoming challenges. Start thinking about how you’d manage an evolving knowledge base: versioning, access control, auditing, and correction mechanisms will be essential before any self-improving agent touches real user data or critical systems.
What’s next for EvoLib and lifelong agent learning
Microsoft hasn’t announced any product integrations for EvoLib, but the research aligns with its broader push into autonomous agents and Copilot technologies. It’s easy to imagine a future where Copilot in Windows or Azure AI Foundry agents begin to learn from interactions across a fleet of users, carefully anonymized and governed. The research also opens questions for the wider industry: How do you prevent an agent from learning bad habits? Can you share skill libraries between organizations without leaking proprietary methods? And how do you balance a library that’s compact enough to include in a prompt without exceeding token limits?
The EvoLib paper suggests that test-time compute can be an investment, not just an expense. By turning today’s reasoning effort into tomorrow’s reusable skill, AI agents might finally break out of the “stateless employee” model and become assistants that genuinely improve with age. The code is available now, and while the production road is long, the foundational idea—letting black-box models learn from their own experience—is one every developer and IT leader should watch closely.