ROI settings

ROI settings determine how Nebuly calculates the economic impact of your AI applications. You can configure three aspects of the calculation:

• Hourly cost – How much an hour of employee time is worth.
• Time per action – How long each task would take without AI.
• Success criteria – Which conversations count as successful and contribute to ROI.

Changes are applied immediately and automatically recompute ROI across the project.

​

Hourly cost

The Hourly cost represents the average hourly cost of the employees performing the work and is used to estimate how much the organization/employee would have spent completing the task manually, per hour. Nebuly then subtracts the AI execution cost (based on the tokens consumed to perform that task) to calculate the net value generated. In other words:

Value Saved = Hourly Cost − AI Execution Cost

By default, Nebuly applies a single hourly cost across the entire project. The default value is $75/hour, but you can adjust it to reflect your organization’s actual labor costs.

​

Time per action

The Time per action defines how long a skilled employee would take to complete a task without using AI. This value is the foundation of the Time Saved calculation. Nebuly automatically classifies every interaction into your taxonomy (read more here), grouping it into a Topic and a more granular class (sub-topic). This allows ROI to be calculated at a much finer level than using a single average time across all tasks. For each action, Nebuly uses proprietary AI models to estimate the average time required for a person to complete that task manually. These estimates are provided out of the box, allowing you to start measuring ROI immediately. You remain in full control of these values. Time estimates can be overridden at any level of the taxonomy, from an entire Topic down to an individual sub-topic, allowing you to tailor the calculations to your organization’s processes and expertise. Changes are applied immediately and automatically update all ROI calculations.

​

Success criteria

Success criteria determine which conversations contribute to ROI. Nebuly evaluates every conversation against a set of predefined failure conditions, such as:

• Empty response – The AI failed to provide a meaningful answer.
• Language problems – The response is in the wrong language or contains significant language errors.
• Unhandled requests – The AI could not complete or answer the user’s request.
• User frustration – The conversation indicates the user was dissatisfied or repeatedly attempted to correct the AI.
• Off-topic responses – The AI’s response is unrelated or not relevant to the user’s request.
• In-scope clarification – The AI asks follow-up questions to gather missing information before completing a valid request.
• Unclear intent – The user’s request is too vague or ambiguous for the AI to determine what action is expected.
• Knowledge base – The AI cannot answer because the required information is missing or unavailable in the connected knowledge base.
• Intent misunderstanding – The AI misinterprets the user’s intent and responds to a different request than the one originally asked.

When a failure condition is enabled, conversations matching that condition are excluded from ROI calculations. Disable a condition if you want those conversations to be treated as successful and included in the results.

ROI is only as accurate as its configuration. Periodically reviewing your hourly costs, task durations, and success criteria ensures your ROI calculations continue to reflect the real business value generated by your AI applications.