What is AgentSim — and when to use it
Define AgentSim in plain terms, the problem it solves, and the 3 signs you need it.
What do people ask about AgentSim?
What does AgentSim do?
It runs your agent definition against a batch of simulated scenarios and reports the success rate and failure modes — a pre-release safety gate that catches problems before users do.
What kind of scenarios does it generate?
Task-style scenarios drawn from your agent description: edge cases, ambiguous inputs, and multi-step goals where an agent is most likely to fail.
Is AgentSim a replacement for production testing?
No. It is a pre-release gate. Use it to catch failure modes early; real traffic still needs monitoring in production.
How is success rate measured?
Across the scenario batch: share of runs that reached the goal. Failure modes are grouped so you can see which step breaks most often.
Does it need my own model key?
Free and demo run without live calls; Pro uses the platform key. Enterprise can bring its own key (BYOK).
Which countries and regions can I use AgentSim in?
As a cloud web app it is reachable from any country with internet access; there is no region lock by default.
Is my agent definition private?
Your definition is used only to generate and run scenarios and is not sold. Details are on our Privacy page.
What languages does it support?
Interface and reports are in English; scenario input in other languages is supported where the underlying model understands them.