Nebuly allows you to automatically understand why some of your users are not satisfied with the answers they are getting from your LLMs. This can help you in the following use cases:

  • Explore why some LLM answers are insufficient according to your users
  • Pinpoint user actions and questions that lead to dissatisfaction
  • Determine topics where LLMs’ answers are inadequate for your users
  • Understand the time your users need to get the answers they were looking for
  • Evaluate the efficacy of RAG data sources for your users
  • Analyze user behavior after (or before) receiving an unsatisfactory answer

Understanding frustration reasons

The best way to easily understand why your users are dissatisfied with your LLM answers is using the Analytics chart filtering users or interactions by frustration cause. Alt Text

Examples of frustration reasons are:

Frustration reasonDescription
VerbosityUsers are frustrated by overly lengthy responses
Limited knowledgeUsers are unsatisfied when the LLM doesn’t have the information they seek
Lack of understandingUsers feel misunderstood when the LLM misinterprets their query
Lack of clarityUsers are confused by unclear or ambiguous answers from the LLM
Lack of depthUsers find the LLM’s responses superficial and not insightful

Pinpointing user actions and questions that lead to dissatisfaction