LLM drift monitoring is the practice of watching AI agents in production for slow changes in behaviour, quality and cost, so that a decline is caught and corrected before it reaches customers. It is one of the core controls in Nestack Agent Care, the managed AI agent operations service from Nestack Technologies.

What drift looks like in an AI agent

An AI agent depends on many moving parts: the goal it is given, the data it retrieves, the workflow and tasks it follows, the tools it calls, the language model underneath, and the guardrails and human reviewers around it. A change in any of these layers can shift results. Answers may become less accurate, an agent may start using a tool differently, or token usage and cost may creep upward. Nestack Agent Care operates production agents across all of these layers so that performance drift and policy risks are detected as conditions evolve.

How Nestack Agent Care monitors drift

Cost and drift controls run alongside end-to-end tracing, evaluation runs, verification loops, monitored tool actions, verified guardrails, staffed human review and ready incident response. Monitoring is continuous, around the clock. Each of the 23 industry playbooks lists the failure modes to watch and the evaluation sets to run, drawn from a catalogue of 5,814 failure modes, so drift is measured against the specific risks of that sector.

From an alert to a verified fix

When drift triggers an alert or a failed evaluation, the case is acknowledged according to its response target. The team correlates the goal, retrieval, plan, workflow, task, tool, LLM, evaluation and guardrail records to identify the affected slice and the earliest decision that went wrong. The relevant runbook is applied to correct the prompt, workflow, tool configuration, data source or guardrail, and the affected slice is checked with targeted output, step or trajectory evaluations before autonomy is restored.

A post-incident review follows within five business days, and the incident closes only after the evaluation case, guardrail or runbook has been updated. Each drift event therefore strengthens future monitoring.

Reporting

Clients receive a monthly audit-ready scorecard. Pricing is per agent, per month, for agents actively running in production; see Agent Care pricing or read more at nestackagents.com.