Upgrade self-hosted Neptune to `2.5`#

Neptune 2.5 introduces support for the following:

Azure Blob Storage for object storage (to store data logged with Neptune)
Elasticsearch ≥8.5.1 (<9.x)
ArgoCD
MariaDB

For a summary of configuration and installation, see the Upgrade procedure section.

Neptune 2.5 also supports a simplified CLI procedure for adding users to your workspace.

What's new in Neptune 2.5

To learn more about introduced changes, see also Neptune 2.5 user guide .

Before you start#

Neptune 2.5 introduces no significant changes to requirements. To upgrade, you don't necessarily need to change anything in your configuration.

If you're using the default installation and have no need for customizations, you can just perform the two upgrade steps.
If you wish to make use of the new supported features, edit the Neptune Helm chart accordingly before upgrading.

Pre-upgrade checklist

Upgrading to version 2.5 is supported only from the previous version (2.4.x).

If you have an older version installed, contact us in advance at support@neptune.ai.
Prepare a back-up of your Neptune deployment before upgrading. This means creating a snapshot of any persistent volume or database that you provide for Neptune.
Scheduling the upgrade during a window when no experiments are running. If this is not possible, at least let Neptune end users know when the upgrade is planned to be performed.

A typical upgrade takes from a few minutes to an hour, depending on your internet connection speed. Upgrades are usually imperceptible to the end users (the data scientists running model-training experiments). However:

The Neptune app may be unavailable for some time. This risk is mitigated on the Neptune client side: If the Neptune server is unavailable, started runs switch to offline mode and the logged metadata is stored on the local disk. The data can be later manually uploaded using the neptune sync CLI command.
We recommend that Neptune users upgrade the Neptune Python client (neptune ) on the systems where the metadata is logged. To learn more, see Compatibility with Neptune client packages.

Upgrade procedure#

This section shows how to:

Optionally configure your deployment for the new supported components.
Perform the upgrade.

For complete installation instructions, see Default installation or Custom installation.

If you're upgrading from 2.3, contact support at support@neptune.ai.

Configuring components#

Azure Blob Storage#

In addition to S3 and PVC, Neptune 2.5 supports Azure Blob Storage (ABS) as a storage backend.

To enable ABS, use the following configuration:

neptune.yaml

leaderboard:
  storage:
    azureBlob:
      connectionString: DefaultEndpointsProtocol=https;AccountName=exampleneptune;AccountKey=REDACTED;EndpointSuffix=core.windows.net
      container: neptune-example-data
      clientThreadPoolSize: 400

For details, see Chart reference: Storage configuration.

Elasticsearch#

Neptune 2.5 supports Elasticsearch 7.17 and 8.5.1+ (not 9.x).

In Elasticsearch 8, HTTPS is enabled by default, so additional configuration might be needed:

neptune.yaml

leaderboard:
  elasticsearch:
    address: "https://elasticsearch-neptune:9200"
    username: "elastic"
    password: "elastic-password"
    clusterName: "elasticsearch"
    insecureSSL: false  # set to 'true' to accept any SSL certificate from ES
    shards: 5
    replicas: 0
    attributes:
      shards: 5
      replicas: 0

For details, see Chart reference: Leaderboard.

ArgoCD#

You can use https://helm.neptune.ai as the Helm repository URL in ArgoCD and the neptune Helm chart.

Due to how ArgoCD handles random secret generation, set the following values for your deployment:

keycloak:
  users:
    instanceAdmin:
      username: admin
      password: admin
    serviceAccount:
      username: neptune
      password: neptune
  clients:
    management:
      id: "neptune-management"
      secret: "4b84b477-2e60-474e-89e2-e7fc3f5145f8" # random UUID

For added security, you can provide your own secrets and reference them in the values.

For details, see Chart reference: Keycloak.

Example of full minimal values for a self-hosted Neptune instance

Note: Requires external MySQL, Elasticsearch and Ingress Controller.

imagePullPolicy: IfNotPresent
imagePullSecrets:
  - regcred
database:
  host: mysql
  port: 3306
  username: neptune_user
  password: neptune_password
ingress:
  host: neptune.example.org
  annotations:
    nginx.ingress.kubernetes.io/proxy-body-size: 10G
    nginx.ingress.kubernetes.io/proxy-read-timeout: '86400'
    kubernetes.io/ingress.class: neptune
keycloak:
  users:
    instanceAdmin:
      username: admin
      password: admin
    serviceAccount:
      username: neptune
      password: neptune
  clients:
    management:
      id: "neptune-management"
      secret: "4b84b477-2e60-474e-89e2-e7fc3f5145f8" # random UUID

init:
  workspaceName: neptune-self-hosted
  administrator:
    username: administrator
    password: change_me
leaderboard:
  elasticsearch:
    address: http://elasticsearch-neptune:9200
storage:
  pvc:
    size: 100Gi

MariaDB#

To enable MariaDB, set the following values for your Helm deployment:

database:
  type: mariadb
  host: your.mariadb.address

Upgrading Neptune#

The following procedure upgrades your Neptune installation.

If you use custom Helm chart values, ensure that the values are configured for your specific deployment needs. For instructions, see the Neptune Helm chart reference.

Perform a repository update:

helm repo update

Expected output

Hang tight while we grab the latest from your chart repositories...
...Successfully got an update from the "neptune" chart repository
Update Complete. ⎈Happy Helming!⎈

Upgrade the Helm chart with the new Neptune version:

helm upgrade -i \
--create-namespace -n neptune neptune-onprem neptune/neptune-onprem \
--version 2.5.0 \
--set dockerRegistryCredentials="$DOCKER_CREDENTIALS"