CopilotKit Releases CLiF: A Continual Learning Platform That Turns Agent Traces and User Actions into Skills, Evals, and Fine-Tuning Data

Introduction Teams shipping AI agents at scale keep running into the same pattern. Even agents built on top-tier models like Fable and Astra struggle to carry out complex, end-to-end tasks inside a specific product consistently enough to deploy. So teams grind out evals for the workflows that matter most, often hundreds or thousands of hand-written cases, and then improve the agent bit by bit through prompt edits, harness fixes, and special cases until it passes enough of them to go live. Meanwhile, the product’s own users carry out those same tasks every day, and know which actions to take and in what order. CopilotKit, the company behind the open-source CopilotKit framework and the AG-UI protocol, released CLiF , a platform for continual learning from full-stack trajectories, in early access . CLiF securely captures what the agent does and what users do in the product, and turns that record into skills, production-based evals, and fine-tuning datasets, all exportable along with the raw trajectories. CopilotKit is open source , with more than 38,000 stars on GitHub. TL;DR Agent tracing shows where an agent went wrong, while the correct path lives in what users do in the product, which most stacks never capture. CLiF, now in early access, learns from full-stack trajectories: the agent trace and the user’s actions in the product, securely captured as one timeline over AG-UI. A four-stage loop, built on CopilotKit’s Automatic Learning, turns those full-stack trajectories into skills, production-based evals, fine-tuning datasets, and raw exports, with a human reviewing every skill. A worked example shows how agents learn the way team leads actually close out a sprint. CLiF targets the pains slowing agent teams, including evals, context bloat, cost, fragmented stacks, and shadow AI usage, and runs fully self-hosted. What Agent Tracing Misses Most agent observability today is centered on the agent. Tracing tools capture prompts, tool calls, and model outputs, which shows where the agent went wrong while leaving the correct path out of view. Product analytics covers the other side with clicks, funnels, and session replays, though that data is built for a person reading a dashboard rather than for teaching an agent a task or testing it. In most production agent stacks, the two never end up on the same timeline. That gap matters because the agent learns a product from prompts, skills, tool descriptions, and exposed APIs, and has to work out on its own how those pieces fit together for a real task. The people using the product have already solved that, often hundreds of times. CopilotKit’s view, laid out in its explainer on agent learning , is that how users actually get work done is the most valuable dataset any agent team has, and that it is rarely captured in a form an agent can learn from. Full-Stack Trajectories: The Training Signal CLiF Is Built On CLiF is part of CopilotKit Intelligence and is built around a single primitive, the full-stack trajectory. It is captured through AG-UI (Agent-User Interaction Protocol) , the open protocol CopilotKit created to connect agents to user-facing applications. Because AG-UI sits between the agent and the user on any surface, it can see both sides of a task: Agent trajectory (agent trace): every run, tool call, and state change the agent made, and what each one returned. Product trajectory (user actions): what was on the user’s screen, the buttons they clicked, and the server requests they triggered. If the user talks to the agent along the way, that conversation lands on the same timeline, in order. Together, that is a full-stack trajectory. AG-UI already streams the agent’s events to the frontend, so securely capturing user actions over the same protocol puts both halves in one stream from the start, with no separate analytics pipeline to stitch them together later. Each user session becomes a single trajectory that a team can open in CopilotKit Intelligence and replay step by step, with clicks, server requests, and agent messages in the order they happened. Inside CLiF: From Trajectories to Skills, Evals, and Training Data CopilotKit describes the learning loop in four stages. 1. Capture. Trajectories stream into CopilotKit Intelligence and are assigned to learning containers, which scope learning to one kind of work, such as billing support or customer onboarding. The runtime picks the container for each session through the getTrajectoryLearningContainerIds callback, and sessions can be moved in or out of a container later in the Intelligence UI. Each container learns only from its own sessions. 2. Find patterns. Analysis runs on a schedule once a container passes a set threshold of new sessions, and teams can also trigger it manually. Each run works from a frozen snapshot of the container’s sessions and surfaces Insights: recurring requests, repeated corrections, and the friction users quietly work around. CopilotKit sees that friction as the strongest signal, since it tends to repeat across thousands of sessions and shows exactly what the agent should learn next. Every Insight cites the specific events behind it and links straight to the source trajectory. 3. Review. When the evidence supports a reusable lesson, CLiF proposes it as a skill, written as a plain SKILL.md file. The analysis service cannot publish anything on its own: a person approves or rejects each proposal, either in the Intelligence UI or through the CopilotKit CLI. Published skills are versioned and can be retired at any time. 4. Deliver and export. Published skills reach the agent without a redeploy, and teams that need stability can pin the agent to a fixed skill revision. For any other agent or MCP server, skills can be downloaded as plain files with copilotkit skills download –output . The trajectories themselves can be exported as raw event data from the Intelligence dashboard. For early access teams, the same pipeline also produces eval candidates and fine-tuning datasets from these trajectories. Agent Trajectories: Automatic Learning in CopilotKit Intelligence On the agent side, CLiF builds on Automatic Learning , the capability in CopilotKit Intelligence that CopilotKit also positions as the foundation for self-improving agents . Automatic Learning learns from the conversations users have with the agent. CLiF extends the same loop to product trajectories, so it also learns from the work users do directly in the product, outside the chat. Every agent conversation is stored in full through AG-UI Streams : messages, tool calls, generative UI, state, approvals, and files. Developers choose which threads count as evidence by returning a container ID from the getLearningContainerId callback on the CopilotKit runtime, which is the agent-side counterpart to the trajectory callback. Teams that already have an agent in production can import existing history from a LangGraph Server or LangSmith Deployment instead of starting from zero. Skill delivery works through adapters for LangGraph (Python and TypeScript), Mastra, Google ADK, Microsoft Agent Framework, CopilotKit’s built-in agent, and more. The adapter adds a catalog and two tools, copilotkit_load_skill and copilotkit_read_skill_file, and the model loads a skill when the task calls for it. Developer instructions keep precedence, so skills cannot override the agent’s role, safety rules, or tool restrictions. Beyond Skills: Evals, Fine-Tuning Data, and Raw Trajectories Skills are only the first output. The same trajectories feed four outputs in total, all exportable into a team’s existing stack: Skills for agents or MCP servers, built from patterns that recur across many sessions. Production-based evals , built from tasks users actually performed, including the inputs and edge cases nobody thought to write by hand. These can be exported to the eval platform a team already uses. Fine-tuning datasets drawn from real task trajectories, intended for training smaller, faster op