Welcome
Tracking
- Getting started
- API Reference
- Examples
Embedded Pages
AB Testing
Platform users
- Getting started
- API Reference
SDK
- Getting started
- Python
- Node.js
- Ruby
- Javascript
Get Interaction Aggregates
curl --request POST \
--url https://backend.nebuly.com/api/external/get-interaction-aggregates \
--header 'Authorization: Bearer <token>' \
--header 'Content-Type: application/json' \
--data '{
"time_range": {
"start": "<string>",
"end": "<string>"
},
"filters": [
{
"kind": "<string>",
"tag": "<string>",
"values": [
"<string>"
],
"is_exclude": true
}
],
"group_by": {
"kind": "<string>",
"tag": "<string>",
"field": "<string>"
},
"variables": [
"<string>"
],
"additional_group_bys": [
{
"kind": "<string>"
}
],
"limit": 123,
"offset": 123
}'
{
"total": 123,
"offset": 123,
"limit": 123,
"data": [
{
"group_name": "<string>",
"n_interactions": 123,
"n_users": 123,
"n_feedback": 123,
"n_positive_implicit_user_feedback": 123,
"n_negative_implicit_user_feedback": 123,
"n_positive_explicit_user_feedback": 123,
"n_negative_explicit_user_feedback": 123,
"n_users_negative": 123,
"n_users_negative_implicit": 123,
"n_users_negative_explicit": 123,
"explicit_feedback_ratio": 123,
"n_paste_output": 123,
"n_interactions_negative": 123,
"n_interactions_negative_implicit": 123,
"n_interactions_negative_explicit": 123,
"n_conversations_positive_explicit": 123,
"n_conversations_negative_explicit": 123,
"n_intents": 123,
"cost": 123,
"latency_seconds": 123,
"retention": 123,
"topics": [
"<string>"
],
"session_time": 123,
"session_length": 123,
"start_session": "<string>",
"end_session": "<string>",
"percentage_negative_feedback": 123,
"percentage_negative_feedback_explicit": 123,
"conversation_percentage_negative_feedback": 123,
"conversation_percentage_negative_feedback_explicit": 123,
"sentiment_score": 123,
"conversation_sentiment_score": 123,
"n_positive_messages": 123,
"n_negative_messages": 123,
"n_topics": 123,
"n_topics_conversations": 123,
"n_user_actions": 123,
"n_user_actions_conversations": 123,
"n_negative_intent": 123,
"n_keywords": 123,
"n_data_sources": 123,
"n_emotions": 123,
"tags": [
{
"tag_name": "<string>",
"n_tag_values": 123
}
],
"bounce_rate": 123,
"penetration": 123,
"user_rating": 123
}
]
}
This endpoint returns aggregated values for interactions. The specific aggregates to compute are defined by the variables
field in the request body.
You can optionally use the group_by
field to aggregate results by various dimensions—for example, by intent
or positive_implicit_feedback
.
If no group_by
is provided, the response returns overall totals across all interactions.
You need an API key to authenticate. See here for more information about the API keys.
Filters you can use to get data based only on the interactions you are interested in.
Show properties
Show properties
The filter kind. It can be one of the following values:
- user_intent
- negative_user_intent_v2
- keyword
- user_feedback_v2
- user_rating
- type_of_problem_v2
- type_of_problem_mid
- positive_comment
- user
- cohort
- data_source
- pii
- language
- emotions
- type_of_risk
- tag
- topic
- user
- user_query
- conversation_id
Optional field to be used only when the kind is tag. It should specify the tag name, which is the same as the key you used in the tags dictionary during the interaction submission.
The values of the selected filter you want to filter on. All values in the list are combined with an “OR” condition.
For example, if kind
is "topic"
and the values are "value A"
and "value B"
,
the condition is interpreted as: topic="value A"
OR topic="value B"
.
Optional field to be used to exclude the interactions that match the given filter. If not specified, the default value is false, meaning that the interactions matching the filter will be included. If set to true, the interactions matching the filter will be excluded.
Groups you can use to aggregate data based on the interactions you’re interested in.
If no value is provided, total aggregated values for the selected interactions will be returned.
When a group_by is provided, the selected group will appear as group_name
in the response.
If not, the response will contain a single row with the overall aggregated values.
Show properties
Show properties
The group kind. It can be one of the following values:
- user_intent
- negative_user_intent_v2
- keyword
- user_feedback_v2
- user_rating
- type_of_problem_v2
- type_of_problem_mid
- positive_comment
- user
- cohort
- data_source
- pii
- language
- emotions
- type_of_risk
- tag
- topic
- user
- user_query
- conversation_id
Optional field to be used only when the kind is tag. It should specify the tag name, which is the same as the key you used in the tags dictionary during the interaction submission.
Optional field used only when kind is conversation_id. It’s used to pick a field of each conversation as group_name. Available values:
- conversation_id (default)
- user_query
- negative_intent
The variables to compute aggregated values for. If not specified, all available variables will be computed.
Show child attributes
Show child attributes
The supported variables are
- n_interactions: number of interactions
- n_users: number of distinct users
- user_feedback_v2: details about the user feedback, including:
- n_feedback: number of total feedback
- n_positive_implicit_user_feedback: number of positive implicit feedback
- n_negative_implicit_user_feedback: number of negative implicit feedback
- n_positive_explicit_user_feedback: number of positive explicit feedback
- n_negative_explicit_user_feedback: number of negative explicit feedback
- n_users_negative: number of users that provided negative feedback
- n_users_negative_implicit: number of users that provided negative implicit feedback
- n_users_negative_explicit: number of users that provided negative explicit feedback
- explicit_feedback_ratio: ratio of explicit feedback over total feedback
- n_paste_output: number of messages where user pasted the output of a previous interaction
- interaction_negative_feedback_v2: details about the negative feedback on interactions, including:
- n_interactions_negative: number of interactions with negative feedback
- n_interactions_negative_implicit: number of interactions with negative implicit feedback
- n_interactions_negative_explicit: number of interactions with negative explicit feedback
- conversation_user_feedback: details about the user feedback at the conversation level, including:
- n_conversations_positive_explicit: number of conversations with positive explicit feedback
- n_conversations_negative_explicit: number of conversations with negative explicit feedback
- n_intents: number of distinct intents
- cost: total cost of the aggregated interactions
- latency_seconds: average latency (time-to-response) of the aggregated interactions, in seconds
- retention: percentage of users who return to engage with your LLMs at least once within the defined time period
- topics: list of distinct topics for the selected aggregation
- session_time: average session time in seconds, calculated as the difference between the start of the first interaction and the end of the last interaction in a conversation
- session_length: average length of a session, defined as the number of interactions in a conversation
- start_session: timestamp of the start of the first interaction in the requested aggregation.
- end_session: timestamp of the end of the last interaction in the requested aggregation
- percentage_negative_feedback: percentage of interactions where users express dissatisfaction with the LLM’s responses, encompassing both explicit feedback (e.g., thumbs down) and implicit signals.
- percentage_negative_feedback_explicit: percentage of interactions with explicit negative feedback, such as thumbs down or other forms of direct user dissatisfaction.
- conversation_percentage_negative_feedback: same definition as
percentage_negative_feedback
, but calculated at the conversation level, meaning it considers the entire conversation rather than individual interactions. - conversation_percentage_negative_feedback_explicit: same definition as
percentage_negative_feedback_explicit
, but calculated at the conversation level, meaning it considers the entire conversation rather than individual interactions. - sentiment_score: average sentiment score of the interactions, where a higher score indicates a more positive sentiment.
- conversation_sentiment_score: average sentiment score of the conversations, where a higher score indicates a more positive sentiment.
- n_positive_messages: number of messages in the interactions that are classified with a positive sentiment
- n_negative_messages: number of messages in the interactions that are classified with a negative sentiment
- n_topics: number of distinct topics in the interactions
- n_topics_conversations: number of distinct conversation topics. A conversation topic is the first topic corresponding to the first interaction in a conversation.
- n_user_actions: number of distinct user actions in the interactions
- n_user_actions_conversations: number of distinct user actions in the conversations. A user action is the first action corresponding to the first interaction in a conversation.
- n_negative_intent_v2: number of distinct negative intents in the interactions
- n_keywords: number of distinct keywords in the interactions
- n_data_sources: number of distinct data sources in the interactions
- n_emotions: number of distinct emotions in the interactions
- tags: returns the variable n_tag_values which returns the distinct tags with their relative occurrences
Note: if the
conversation_id
filter is applied, n_tag_values will return the distinct tags with their distinct values rather than count - bounce_rate: percentage of users having just a one-single message conversation in the given time-range
- penetration (activation rate): percentage of users who have used the platform
- user_rating: average user rating left by the user
Optional additional group_by
filters to further refine the aggregations.
These filters allow you to split the selected variables in the main group_by
into more detailed subcategories.
Show properties
Show properties
See group by for available values
Parameter needed for pagination. It indicates the number of rows per page.
Parameter needed for pagination. It indicates the offset to be applied to the first rows returned.
Response
Needed for pagination. The total number of rows available.
Needed for pagination. It indicates the offset to be applied to the first rows returned.
Needed for pagination. It indicates the number of rows per page.
The data aggregated using the given group_by field.
Show properties
Show properties
The value of the selected group_by
filter.
The total number of interactions for the selected group.
The total number of distinct users for the selected group.
The total number of feedback entries for the selected group.
The total number of positive implicit feedback entries for the selected group.
The total number of negative implicit feedback entries for the selected group.
The total number of positive explicit feedback entries for the selected group.
The total number of negative explicit feedback entries for the selected group.
The total number of users who provided negative feedback for the selected group.
The total number of users who provided negative implicit feedback for the selected group.
The total number of users who provided negative explicit feedback for the selected group.
The ratio of explicit feedback to total feedback for the selected group.
The total number of times the output was pasted by users for the selected group.
The total number of interactions with negative feedback for the selected group.
The total number of interactions with negative implicit feedback for the selected group.
The total number of interactions with negative explicit feedback for the selected group.
The total number of conversations with positive explicit feedback for the selected group.
The total number of conversations with negative explicit feedback for the selected group.
The total number of distinct intents for the selected group.
The total cost for the selected group.
The average latency (time-to-response) in seconds for the selected group.
The percentage of users who returned to engage with your LLMs at least once within the defined time period for the selected group.
The list of distinct topics for the selected group.
The average session time in seconds for the selected group, calculated as the difference between the start of the first interaction and the end of the last interaction in a conversation.
The average length of a session for the selected group, defined as the number of interactions in a conversation.
The timestamp of the start of the first interaction in the requested aggregation for the selected group.
The timestamp of the end of the last interaction in the requested aggregation for the selected group.
The percentage of interactions where users expressed dissatisfaction with the LLM’s responses for the selected group, encompassing both explicit feedback (e.g., thumbs down) and implicit signals.
The percentage of interactions with explicit negative feedback, such as thumbs down or other forms of direct user dissatisfaction, for the selected group.
The same definition as percentage_negative_feedback
, but calculated at the conversation level, meaning it considers the entire conversation rather than individual interactions for the selected group.
The same definition as percentage_negative_feedback_explicit
, but calculated at the conversation level, meaning it considers the entire conversation rather than individual interactions for the selected group.
The average sentiment score of the interactions for the selected group, where a higher score indicates a more positive sentiment.
The average sentiment score of the conversations for the selected group, where a higher score indicates a more positive sentiment.
The total number of messages in the interactions that are classified with a positive sentiment for the selected group.
The total number of messages in the interactions that are classified with a negative sentiment for the selected group.
The total number of distinct topics in the interactions for the selected group.
The total number of distinct conversation topics for the selected group. A conversation topic is the first topic corresponding to the first interaction in a conversation.
The total number of distinct user actions in the interactions for the selected group.
The total number of distinct user actions in the conversations for the selected group. A user action is the first action corresponding to the first interaction in a conversation.
The total number of distinct negative intents in the interactions for the selected group.
The total number of distinct keywords in the interactions for the selected group.
The total number of distinct data sources in the interactions for the selected group.
The total number of distinct emotions in the interactions for the selected group.
The distinct tags with their relative occurrences for the selected group.
The percentage of users who had only a single-message conversation in the given time range for the selected group.
The percentage of users who have used the platform for the selected group.
The average user rating left by users for the selected group.
curl --request POST \
--url https://backend.nebuly.com/api/external/get-interaction-aggregates \
--header 'Authorization: Bearer <token>' \
--header 'Content-Type: application/json' \
--data '{
"time_range": {
"start": "<string>",
"end": "<string>"
},
"filters": [
{
"kind": "<string>",
"tag": "<string>",
"values": [
"<string>"
],
"is_exclude": true
}
],
"group_by": {
"kind": "<string>",
"tag": "<string>",
"field": "<string>"
},
"variables": [
"<string>"
],
"additional_group_bys": [
{
"kind": "<string>"
}
],
"limit": 123,
"offset": 123
}'
{
"total": 123,
"offset": 123,
"limit": 123,
"data": [
{
"group_name": "<string>",
"n_interactions": 123,
"n_users": 123,
"n_feedback": 123,
"n_positive_implicit_user_feedback": 123,
"n_negative_implicit_user_feedback": 123,
"n_positive_explicit_user_feedback": 123,
"n_negative_explicit_user_feedback": 123,
"n_users_negative": 123,
"n_users_negative_implicit": 123,
"n_users_negative_explicit": 123,
"explicit_feedback_ratio": 123,
"n_paste_output": 123,
"n_interactions_negative": 123,
"n_interactions_negative_implicit": 123,
"n_interactions_negative_explicit": 123,
"n_conversations_positive_explicit": 123,
"n_conversations_negative_explicit": 123,
"n_intents": 123,
"cost": 123,
"latency_seconds": 123,
"retention": 123,
"topics": [
"<string>"
],
"session_time": 123,
"session_length": 123,
"start_session": "<string>",
"end_session": "<string>",
"percentage_negative_feedback": 123,
"percentage_negative_feedback_explicit": 123,
"conversation_percentage_negative_feedback": 123,
"conversation_percentage_negative_feedback_explicit": 123,
"sentiment_score": 123,
"conversation_sentiment_score": 123,
"n_positive_messages": 123,
"n_negative_messages": 123,
"n_topics": 123,
"n_topics_conversations": 123,
"n_user_actions": 123,
"n_user_actions_conversations": 123,
"n_negative_intent": 123,
"n_keywords": 123,
"n_data_sources": 123,
"n_emotions": 123,
"tags": [
{
"tag_name": "<string>",
"n_tag_values": 123
}
],
"bounce_rate": 123,
"penetration": 123,
"user_rating": 123
}
]
}