User properties

Nebuly’s SDK automatically detects the behaviours of your LLM users. In particular, the concepts we use to build your users data model are as follows:

Interactions: An interaction is a data point that comprehends the user input prompt and the answer of the LLM.
Actions: An action is the task (or question) the user performs during an interaction. Examples of actions are “ask to summarize”, “ask to change tone”, “ask to write again” or “copy and paste” the answer of the LLM, etc
Users: Users are the ones performing the actions.
Properties: Properties are attributes that define the specifics of an interaction, an action or a user. Examples of properties are: topic of an interaction (legal, finance, marketing, etc), style, tone, language, satisfaction, frustration, etc

List of actions

Here is the list of user actions that Nebuly automatically detects:

Action	Description
Information Retrieval	Users pose direct questions, with the model sourcing answers from its memory or external RAG sources
Content Creation	Users prompt the model to generate fictional text, unrelated to information retrieval
Summary Request	Users seek concise summaries of extensive articles or documents
Clarification Request	Users ask for further explanations, elaborations, or details on a prior query
Alternative Answers	Users request the model to provide varied responses to a previous query for selection
Translation Request	Users request language translations
Improvement Request	Users instruct the model to enhance the quality of specific text or make designated modifications (e.g., word replacements, character renaming)
List Creation	Users direct the model to present text in a bullet-point format
Styling Request	Users specify a desired style for the model to format the target text
Coding Assistance	Users seek guidance on coding or debugging issues
Problem Solving	Users ask for solutions to mathematical equations or general problems
Prompt Elaboration	Users provide supplementary information to clarify a previous query
Answer Acceptance	Users convey satisfaction with a prior response
Error Reporting	Users report mistakes to the LLM
Question Reiteration	Users repeat a previously posed question
Drop Off	Users don’t pose a follow-up to the LLM within a set time frame (default is 30 minutes)

List of properties

Here is the list of properties of each user interaction that Nebuly automatically detects:

Property	Description
Topic	The main theme the user is talking about during the interaction
Text Format	The structural layout of the model’s response, such as paragraph, bullet points, article, email, tweet, blogpost, notes, etc
Text Style	The tone of the response, indicating if it’s formal, casual, humorous, etc.
Verbosity	The length of the response, indicating whether it’s concise or detailed
Text Language	The specific language (like English or Spanish) in which the model generates its response
User Satisfaction	A measure indicating whether the user found the LLM’s response satisfactory or not
Frustration Causes	Specific reasons a user might be frustrated, such as the response being too wordy (verbosity), the model having insufficient knowledge, the model not understanding the query, the answer lacking clarity, or the answer not delving deep enough
Latency	The time taken between the user’s query and the LLM’s response
Trace Cost	The computational cost associated with generating a response
RAG Sources	The external reference or knowledge sources the LLM might use to fetch or validate information

Getting started

Instrument (Core)

Instrument (Optional)

Eligible Users

Insights API

Reports API

Utilities

List of actions

List of properties

​List of actions

​List of properties

List of actions

List of properties