The Nebuly SDK is a Python-based library, meticulously crafted to integrate
seamlessly into your application. This document offers a comprehensive overview of the SDK’s architecture
and its functionality within your application.
The Observer is a component that monitors the calls to various LLM providers.
It gathers the input and output of these calls, collectively referred to as an
“interaction”. An interaction may encompass multiple calls to the same or
different LLM providers, termed as “spans”.
The Interaction Queue serves as a temporary storage for interactions. It
retains the interactions until they are ready for processing by the Interaction
Parser. This mechanism decouples the Primary Thread from the Nebuly Thread,
enabling concurrent operation.
The Interaction Parser is tasked with preparing the interactions for
transmission to the Nebuly Server. It operates on a separate Nebuly Thread,
ensuring that the primary application thread remains unblocked. The Parser is
equipped to capture input and output based on the specific LLM providers.
The Nebuly Server is the destination where the interactions are received and
processed. It calculates various metrics such as user satisfaction, cost,
topic, frustration level, and latency.
The Nebuly SDK is engineered to have a minimal impact on your application’s
performance. It operates on a separate thread, ensuring that your primary
application thread remains unblocked. In the event of any issues, the SDK is
capable of restarting its thread, ensuring uninterrupted operation.