How can I track my custom agent's performance, usage stats, and user adoption?

Q&A
,
1

Hi team! :waving_hand:

I’ve created a custom chat agent (MAP Process Expert) and I’d like to understand how it’s performing. Specifically, I’m looking for answers to the following:

  1. Is there a way to see who is using my agent and how frequently?

  2. Can I get a breakdown of usage percentage or adoption rate among users who have access?

  3. Are there any built-in analytics or dashboards that show conversation volume, response quality, or user satisfaction for custom agents?

  4. If not natively available, is there a workaround or admin report that can provide these insights?

I want to use this data to improve the agent’s knowledge base and measure its impact on the team. Any guidance on where to find these metrics would be really appreciated!

I am a regular user not an admin

Thanks! :folded_hands:

2

Hi @searnab and welcome to the community!

As to my knowledge, there’s no built-in analytics dashboard or usage metrics panel for custom chat agents today. You can’t natively see who’s using your agent, how frequently, conversation volume, or satisfaction scores from within Quick Sight as a regular user.

As a non-admin workaround, you could add a simple feedback mechanism within your agent’s instructions, for example, ask users to rate responses or include a short “was this helpful?” prompt at the end of interactions. This gives you qualitative data you can review manually.

If you can get your admin involved, I believe they can check CloudWatch metrics and CloudTrail logs for API-level activity related to your agent (Monitoring data in Amazon Quick Sight using CloudWatch). This won’t give you a polished dashboard out of the box, but it can show interaction counts and which users are triggering the agent.

Hope this helps!