Self-hosting Nebuly on AWS

Overview

Nebuly on AWS is deployed on an Elastic Kubernetes Service (EKS) cluster provisioned via a Terraform module, with all platform components managed through an official Helm chart. The setup involves two layers:

Infrastructure — Terraform provisions EKS, RDS (PostgreSQL), S3, and supporting IAM/networking resources.
Platform — the nebuly-platform Helm chart deploys all Nebuly services onto the provisioned cluster, configured using the values produced by Terraform.

A bootstrap Helm chart (bootstrap-aws) handles EKS-specific dependencies (ingress, secret management, storage classes) and must be installed before the platform chart.

Terraform module

Terraform module for provisioning Nebuly Platform resources on AWS. Available on Terraform Registry.

Helm chart

The Nebuly Platform Helm chart is the official way to deploy and manage the Nebuly Platform on Kubernetes. It packages all platform components and is configured using the values produced by the Nebuly Terraform module. Available on GHCR.

Prerequisites

Nebuly Credentials

Before using this Terraform module, ensure that you have your Nebuly credentials ready. These credentials are necessary to activate your installation and should be provided as input via the nebuly_credentials input.

Quickstart

To get started with Nebuly installation on AWS, you can follow the steps below. These instructions will guide you through the installation using Nebuly’s default standard configuration with the Nebuly Helm Chart. For specific configurations or assistance, reach out to the Nebuly Slack channel or email support@nebuly.ai.

1. Terraform setup

Import Nebuly into your Terraform root module, provide the necessary variables, and apply the changes. For configuration examples, you can refer to the Examples. Once the Terraform changes are applied, proceed with the next steps to deploy Nebuly on the provisioned Elastic Kubernetes Service (EKS) cluster.

2. Connect to the EKS cluster

Prerequisites: install the AWS CLI.

Fetch the command for retrieving the credentials from the module outputs:

terraform output eks_cluster_get_credentials

Run the command you got from the previous step

3. Create image pull secret

The auto-generated Helm values use the name defined in the k8s_image_pull_secret_name input variable for the Image Pull Secret. If you prefer a custom name, update either the Terraform variable or your Helm values accordingly. Create a Kubernetes Image Pull Secret for authenticating with your Docker registry and pulling the Nebuly Docker images. Example:

kubectl create secret generic nebuly-docker-pull \
  -n nebuly \    
  --from-file=.dockerconfigjson=dockerconfig.json \
  --type=kubernetes.io/dockerconfigjson

4. Bootstrap EKS cluster

Retrieve the auto-generated values from the Terraform outputs and save them to a file named values-bootstrap.yaml:

terraform output helm_values_bootstrap

Install the bootstrap Helm chart to set up all the dependencies required for installing the Nebuly Platform Helm chart on EKS. Refer to the chart documentation for all the configuration details.

helm install oci://ghcr.io/nebuly-ai/helm-charts/bootstrap-aws \
  --namespace nebuly-bootstrap \
  --generate-name \
  --create-namespace \
  -f values-bootstrap.yaml

5. Create Secret Provider Class

Create a Secret Provider Class to allow EKS to fetch credentials from the provisioned Key Vault.

Get the Secret Provider Class YAML definition from the Terraform module outputs:
terraform output secret_provider_class
Copy the output of the command into a file named secret-provider-class.yaml.

Run the following commands to install Nebuly in the Kubernetes namespace nebuly:

kubectl create ns nebuly
kubectl apply --server-side -f secret-provider-class.yaml

6. Install nebuly-platform chart

Retrieve the auto-generated values from the Terraform outputs and save them to a file named values.yaml:

terraform output helm_values

Install the Nebuly Platform Helm chart. Refer to the chart documentation for detailed configuration options.

helm install <your-release-name> oci://ghcr.io/nebuly-ai/helm-charts/nebuly-platform \
  --namespace nebuly \
  -f values.yaml \
  --timeout 30m 

ℹ️ During the initial installation of the chart, all required Nebuly LLMs are uploaded to your model registry. This process can take approximately 5 minutes. If the helm install command appears to be stuck, don’t worry: it’s simply waiting for the upload to finish.

7. Access Nebuly

Retrieve the external Load Balancer DNS name to access the Nebuly Platform:

kubectl get svc -n nebuly-bootstrap -o jsonpath='{range .items[?(@.status.loadBalancer.ingress)]}{.status.loadBalancer.ingress[0].ip}{"\n"}{end}'

You can then register a DNS CNAME record pointing to the Load Balancer DNS name to access Nebuly via the custom domain you provided in the input variable platform_domain.

Upgrade to a new version

You can upgrade the Nebuly Platform Helm chart by running the following command:

helm upgrade <your-release-name> oci://ghcr.io/nebuly-ai/helm-charts/nebuly-platform \
  --namespace nebuly \
  -f values.yaml \ 
  -- version=<version> \ 
  --timeout 30m

You can find the list of available versions and their changelogs in the releases page.

Examples

You can find examples of code that uses this Terraform module in the examples directory.

​Overview

​Terraform module

​Helm chart

​Prerequisites

​Nebuly Credentials

​Quickstart

​1. Terraform setup

​2. Connect to the EKS cluster

​3. Create image pull secret

​4. Bootstrap EKS cluster

​5. Create Secret Provider Class

​6. Install nebuly-platform chart

​7. Access Nebuly

​Upgrade to a new version

​Examples