> ## Documentation Index
> Fetch the complete documentation index at: https://docs.nebuly.com/llms.txt
> Use this file to discover all available pages before exploring further.

# Gemini Enterprise

> Sync Gemini Enterprise assistant interactions into Nebuly from Google Cloud Logging and Cloud Trace

The Gemini Enterprise integration is a self-hosted utility script that reads end-user activity via **Cloud Logging**, enriches token counts from **Cloud Trace**, and ingests each prompt/response interaction into Nebuly.

The data flow is:

Cloud Logging → (Optional BigQuery layer) → Cloud Trace enrichment → Nebuly ingestion

The script is incremental and resumable, so you can re-run it safely at any time without creating duplicates.

The source repository is available [here](https://github.com/nebuly-ai/nebuly-integration-examples/tree/main/gemini_enterprise).

## Prerequisites

* A Google Cloud project with a **Gemini Enterprise** app configured.
* The `{PROJECT_ID}` of the Google Cloud project.
* The [Google Cloud CLI](https://cloud.google.com/sdk/docs/install) (`gcloud`) installed and authenticated.
* **Python ≥ 3.12** and [Poetry](https://python-poetry.org/docs/#installing-with-the-official-installer) installed on the machine that runs the script.
* A Google Cloud admin who can grant IAM roles and enable audit logging on the Gemini Enterprise app.

## Install & run

Clone the integration examples repository, install dependencies, configure your `.env` file, and run the sync:

```bash theme={null}
git clone https://github.com/nebuly-ai/nebuly-integration-examples.git
cd nebuly-integration-examples/gemini_enterprise
poetry install
cp .env.example .env
# Fill in NEBULY_API_KEY, GCP_PROJECT_ID, GCP_LOCATION, GCP_ENGINE_ID
poetry run python -m gemini_enterprise_sync --from-date 2026-01-01
```

Follow the guide below to retrieve the necessary credentials and permissions.

The first run **requires** `--from-date`. Set it to the earliest date you want to import.

For the complete environment variable reference, available CLI flags, payload customization and resumable-coverage details, check the project README.

## Retrieve Gemini Enterprise Application information

<Steps>
  <Step title="Finding your GCP_PROJECT_ID">
    Open the [Gemini Enterprise console](https://console.cloud.google.com/gemini-enterprise/).

    Click on the Project Dropdown Menu at the top-left of the page (next to the Google Cloud logo) and select the project hosting your gemini enterprise application.

    Copy the value listed under the ID column (e.g., gemini-enterprise-17828149).
  </Step>

  <Step title="Finding your GCP_ENGINE_ID and GCP_LOCATION">
    Open the [Gemini Enterprise console](https://console.cloud.google.com/gemini-enterprise/).

    Under **Apps**, copy the value listed under the ID column (e.g., gemini-enterprise-17828149\_1782814981868) for your **GCP\_ENGINE\_ID**.
    Copy the value listed under the Location column (e.g., eu) for your **GCP\_LOCATION**.
  </Step>

  <Step title="Setting your GCP Collection (GCP_COLLECTION)">
    For 99% of Google Cloud setups, this is hardcoded to default\_collection.
    Unless your company has a highly customized multi-tenant setup managed via GCP APIs, you can safely keep this set as:

    ```bash theme={null}
    GCP_COLLECTION=default_collection
    ```
  </Step>
</Steps>

## Enable audit logs and traces

Before the sync can read interactions, Gemini Enterprise must emit usage/audit logs and agent traces.

<Steps>
  <Step title="Enable required APIs">
    Enable the APIs the sync depends on:

    ```bash theme={null}
    gcloud services enable \
      logging.googleapis.com \
      cloudtrace.googleapis.com \
      discoveryengine.googleapis.com \
      telemetry.googleapis.com
    ```
  </Step>

  <Step title="Enable usage and audit logging">
    In the [Gemini Enterprise console](https://console.cloud.google.com/gemini-enterprise/),
    select your app, click on the **Configurations** button (gear icon).

    Head to **Observability** and toggle on:

    * **Enable logging of prompt inputs and response outputs**, to ingest the prompt and response into Nebuly. **This is mandatory for the sync to work.**
    * **Enable instrumentation of OpenTelemetry traces and logs**, to retrieve input/output token counts from Cloud Trace.

    This requires the **Gemini Enterprise Admin** role (`roles/discoveryengine.agentspaceAdmin`). When enabled, Google Cloud writes the `discoveryengine.googleapis.com/gemini_enterprise_user_activity` log stream that the sync reads.

    See [Set up usage and audit logs](https://docs.cloud.google.com/gemini/enterprise/docs/set-up-usage-audit-logs) for the full console walkthrough.
  </Step>

  <Step title="Verify logs are flowing">
    Open [Logs Explorer](https://console.cloud.google.com/logs/query) and run:

    ```
    logName="projects/{PROJECT_ID}/logs/discoveryengine.googleapis.com%2Fgemini_enterprise_user_activity"
    ```

    Send a test prompt through your Gemini Enterprise app, then confirm new log entries appear within a few minutes.
  </Step>
</Steps>

## Set up authentication and permissions

<Steps>
  <Step title="Nebuly secret key → NEBULY_API_KEY">
    In Nebuly, go to **Settings → Projects → View Nebuly keys → Secret keys** and copy the secret key for the project you want to sync.
  </Step>

  <Step title="Create a service account and grant read roles">
    Create a dedicated service account for the sync machine and grant it read-only access to the data sources it needs:

    * **`roles/logging.viewer`** (or **`roles/logging.privateLogViewer`** if audit data is private) — `logging.logEntries.list`
    * **`roles/cloudtrace.user`** — `cloudtrace.traces.get`
    * **BigQuery source only:** **`roles/bigquery.jobUser`** + **`roles/bigquery.dataViewer`** — `bigquery.jobs.create` + `bigquery.tables.getData`

    Example (adjust the service account name as needed):

    ```bash theme={null}
    PROJECT_ID="{PROJECT_ID}"

    gcloud iam service-accounts create nebuly-gemini-sync \
      --display-name="Nebuly Gemini Enterprise sync"

    SA="nebuly-gemini-sync@${PROJECT_ID}.iam.gserviceaccount.com"

    gcloud projects add-iam-policy-binding ${PROJECT_ID} \
      --member="serviceAccount:${SA}" --role="roles/logging.viewer"

    gcloud projects add-iam-policy-binding ${PROJECT_ID} \
      --member="serviceAccount:${SA}" --role="roles/cloudtrace.user"
    ```

    If you use BigQuery as the log source, also grant `roles/bigquery.jobUser` and `roles/bigquery.dataViewer` on the dataset or project.

    ```bash theme={null}
    gcloud projects add-iam-policy-binding ${PROJECT_ID} \
      --member="serviceAccount:${SA}" --role="roles/bigquery.jobUser"

    gcloud projects add-iam-policy-binding ${PROJECT_ID} \
      --member="serviceAccount:${SA}" --role="roles/bigquery.dataViewer"
    ```

    See [Export logs to BigQuery](#export-logs-to-bigquery-recommended) for more details.
  </Step>

  <Step title="Authenticate the machine">
    **Production (recommended):** download a JSON key for the service account and set in your `.env` file:

    ```bash theme={null}
    GOOGLE_APPLICATION_CREDENTIALS=/path/to/service-account-key.json
    ```

    To get the service account key, go to the [Service Accounts](https://console.cloud.google.com/iam-admin/serviceaccounts) page,
    select your project and the newly created service account, and click on the **Keys** tab.
    Create a new JSON key and download it.

    **Local / dev:** use Application Default Credentials:

    ```bash theme={null}
    gcloud auth application-default login
    ```

    <Warning>
      Store service account keys in a secrets manager. On GKE or Cloud Run, prefer [Workload Identity](https://cloud.google.com/kubernetes-engine/docs/how-to/workload-identity) over long-lived JSON keys.
    </Warning>
  </Step>
</Steps>

## Export logs to BigQuery (recommended)

By default, the sync uses Cloud Logging as the log source. While this is a good option for development and testing, BigQuery is the recommended ongoing log source for production syncs.
BigQuery is a more reliable and scalable log storage solution, and it's much faster to retrieve data from it compared to Cloud Logging.

It's possible to export logs to BigQuery as they arrive using a Cloud Logging sink.

<Warning>
  A **new sink holds no historical data** — it only exports logs written **after** the sink is created.

  Cloud Logging retains entries for a limited window (default **30 days**); older data cannot be recovered.

  Plan your backfill before relying on BigQuery alone (see [Backfill historical data](#backfill-historical-data) below).
</Warning>

<Steps>
  <Step title="Create a BigQuery dataset">
    Create a dataset in the region where you want to store the logs (for example, `EU` for Europe).

    ```bash theme={null}
    bq mk --dataset --location=EU ${PROJECT_ID}:gemini_enterprise_logs_nebuly
    ```

    Choose a location that matches your compliance requirements.
  </Step>

  <Step title="Create the Cloud Logging sink">
    Create a sink with the filter the sync expects:

    ```bash theme={null}
    gcloud logging sinks create gemini-enterprise-logs-nebuly \
      bigquery.googleapis.com/projects/${PROJECT_ID}/datasets/gemini_enterprise_logs_nebuly \
      --log-filter="logName=\"projects/${PROJECT_ID}/logs/discoveryengine.googleapis.com%2Fgemini_enterprise_user_activity\" AND (jsonPayload.serviceTextReply:* OR protoPayload.response.reply:*)"
    ```

    You can use the `--use-partitioned-tables` flag to create partitioned tables instead of date-sharded tables.
    For more details, see [Partitioned tables](https://docs.cloud.google.com/bigquery/docs/partitioned-tables).

    The filter matches user-activity log entries that contain an assistant reply, whether in `jsonPayload` or `protoPayload` form.

    By default the sink should create a table named
    `discoveryengine_googleapis_com_gemini_enterprise_user_activity_<date>`
    for each day. Check the BigQuery console to verify.
  </Step>

  <Step title="Grant the sink writer identity access">
    Describe the sink to get its **writer identity**, then grant it **`roles/bigquery.dataEditor`** on the dataset:

    ```bash theme={null}
    SINK_WRITER_IDENTITY=$(gcloud logging sinks describe gemini-enterprise-logs-nebuly --format='value(writerIdentity)')

    bq query --use_legacy_sql=false "
    GRANT \`roles/bigquery.dataEditor\`
    ON SCHEMA \`${PROJECT_ID}\`.gemini_enterprise_logs_nebuly
    TO \"${SINK_WRITER_IDENTITY}\";
    "
    ```

    The console often grants this automatically when you are a project Owner; verify it if exports do not appear in BigQuery.

    <Warning>
      It might take up to 10 minutes for the sink to be created and the data to appear in BigQuery.
    </Warning>
  </Step>

  <Step title="Configure the sync for BigQuery">
    In your `.env` file, set:

    ```bash theme={null}
    GCP_LOG_SOURCE=bigquery
    GCP_BIGQUERY_TABLE=gemini_enterprise_logs_nebuly.discoveryengine_googleapis_com_gemini_enterprise_user_activity_*
    ```

    * Use a `_*` suffix when the sink creates **date-sharded** tables (the default for log sinks).
    * Use a plain table name (no `_*`) for **partitioned** tables.
    * Optionally set `GCP_BIGQUERY_LOCATION` if your dataset is not in the default region.

    ### Backfill historical data

    Because the sink only captures new logs, it's required to backfill historical data using the logging source before switching to BigQuery.

    1. Leave `GCP_LOG_SOURCE=logging` (the default) and run a one-time backfill:

       ```bash theme={null}
       poetry run python -m gemini_enterprise_sync --from-date 2026-01-01
       ```

    2. Switch to `GCP_LOG_SOURCE=bigquery` for ongoing runs.

    Nebuly deduplication and the script's resumable coverage mean there is no gap and no duplicates when the sources hand off.

    <Warning>
      The backfill process can take a while, depending on the amount of data to process.
    </Warning>
  </Step>
</Steps>

## FAQ

**What are the Google Cloud costs associated with this integration?**

The sync uses Cloud Logging, Cloud Trace and optionally BigQuery.\
Costs are based on the amount of data ingested, but should be negligible for most use cases.

As of July 2026:

* Cloud Logging: Provides 50 GiB of free ingestion per month. Then it's \$0.50 per GiB.
* Cloud Trace: Provides 2.5 M spans for free per month. Then it's \$0.20 per 1M spans.
* BigQuery: Provides 1 TiB of free query usage per month. Then it's \$6.25 per TiB.

For more details, see the [Google Cloud pricing page](https://cloud.google.com/pricing).

**Where does the synced data appear?**\
Interactions imported by the script show up on the **Interactions** page in your Nebuly project.

**When will my data appear?**\
Cloud Logging and trace data can take a few minutes to settle after a user interaction. Once ingested, there can be a delay of up to **30 minutes** before data becomes visible in Nebuly. Subsequent updates follow the same polling interval.

**Is it safe to re-run the script?**\
Yes. The script maintains resumable coverage in `.cache/coverage.json`. Re-running it picks up only new data, and Nebuly deduplicates interactions so duplicates are not created.

**How can I see what would be sent without triggering ingestion?**\
Pass the `--dry-run` flag. The script fetches and converts data but skips the POST to Nebuly's ingestion endpoint, printing a summary instead.

**Why do some interactions show 0 tokens?**\
Token counts come from Cloud Trace lookups. If a trace is missing, not yet available, or the lookup fails, the script reports **0 tokens** for that interaction.
For example, image generation traces do not contain token counts.

**Why do some interactions show a low output token count?**\
Google-specific tools, like `Google Deep Research`, do not expose their token consumption details.
The script will report a low output token (just the amount of tokens required to call these tools) count for these interactions.

## Resources

<CardGroup cols={2}>
  <Card title="Gemini Enterprise Nebuly integration" icon="github" href="https://github.com/nebuly-ai/nebuly-integration-examples/tree/main/gemini_enterprise">
    Full install guide, env-var table, CLI flags, and customization docs
  </Card>

  <Card title="Set up usage and audit logs" icon="google" href="https://docs.cloud.google.com/gemini/enterprise/docs/set-up-usage-audit-logs">
    Enable Gemini Enterprise usage and audit logging in Google Cloud
  </Card>

  <Card title="Gemini Enterprise traces and observability" icon="google" href="https://docs.cloud.google.com/gemini-enterprise-agent-platform/optimize/observability/traces">
    Enable agent tracing and view traces in Cloud Trace
  </Card>

  <Card title="Export logs to BigQuery" icon="google" href="https://docs.cloud.google.com/logging/docs/export/configure_export_v2">
    Configure Cloud Logging sinks to route entries to BigQuery
  </Card>

  <Card title="Nebuly Interaction API" icon="book" href="/tracking/api-reference/events/post-events-interaction-with-trace-v2">
    API reference for the Interaction ingestion endpoint
  </Card>
</CardGroup>