Upgrade self-hosted Neptune to 2.5
#
Neptune 2.5
introduces support for the following:
- Azure Blob Storage for object storage (to store data logged with Neptune)
- Elasticsearch
≥8.5.1
(<9.x
) - ArgoCD
- MariaDB
For a summary of configuration and installation, see the Upgrade procedure section.
Neptune 2.5
also supports a simplified CLI procedure for adding users to your workspace.
Before you start#
Neptune 2.5
introduces no significant changes to requirements. To upgrade, you don't necessarily need to change anything in your configuration.
- If you're using the default installation and have no need for customizations, you can just perform the two upgrade steps.
- If you wish to make use of the new supported features, edit the Neptune Helm chart accordingly before upgrading.
Pre-upgrade checklist
-
Upgrading to version
2.5
is supported only from the previous version (2.4.x
).If you have an older version installed, contact us in advance at support@neptune.ai.
-
Prepare a back-up of your Neptune deployment before upgrading. This means creating a snapshot of any persistent volume or database that you provide for Neptune.
- Scheduling the upgrade during a window when no experiments are running. If this is not possible, at least let Neptune end users know when the upgrade is planned to be performed.
A typical upgrade takes from a few minutes to an hour, depending on your internet connection speed. Upgrades are usually imperceptible to the end users (the data scientists running model-training experiments). However:
- The Neptune app may be unavailable for some time. This risk is mitigated on the Neptune client side: If the Neptune server is unavailable, started runs switch to offline mode and the logged metadata is stored on the local disk. The data can be later manually uploaded using the
neptune sync
CLI command. - We recommend that Neptune users upgrade the Neptune Python client (
neptune
) on the systems where the metadata is logged. To learn more, see Compatibility with Neptune client packages.
Upgrade procedure#
This section shows how to:
- Optionally configure your deployment for the new supported components.
- Perform the upgrade.
For complete installation instructions, see Default installation or Custom installation.
If you're upgrading from
2.3
, contact support at support@neptune.ai.
Configuring components#
Azure Blob Storage#
In addition to S3 and PVC, Neptune 2.5
supports Azure Blob Storage (ABS) as a storage backend.
To enable ABS, use the following configuration:
leaderboard:
storage:
azureBlob:
connectionString: DefaultEndpointsProtocol=https;AccountName=exampleneptune;AccountKey=REDACTED;EndpointSuffix=core.windows.net
container: neptune-example-data
clientThreadPoolSize: 400
For details, see Chart reference: Storage configuration.
Elasticsearch#
Neptune 2.5
supports Elasticsearch 7.17
and 8.5.1+
(not 9.x
).
In Elasticsearch 8, HTTPS is enabled by default, so additional configuration might be needed:
leaderboard:
elasticsearch:
address: "https://elasticsearch-neptune:9200"
username: "elastic"
password: "elastic-password"
clusterName: "elasticsearch"
insecureSSL: false # set to 'true' to accept any SSL certificate from ES
shards: 5
replicas: 0
attributes:
shards: 5
replicas: 0
For details, see Chart reference: Leaderboard.
ArgoCD#
You can use https://helm.neptune.ai
as the Helm repository URL in ArgoCD and the neptune
Helm chart.
Due to how ArgoCD handles random secret generation, set the following values for your deployment:
keycloak:
users:
instanceAdmin:
username: admin
password: admin
serviceAccount:
username: neptune
password: neptune
clients:
management:
id: "neptune-management"
secret: "4b84b477-2e60-474e-89e2-e7fc3f5145f8" # random UUID
For added security, you can provide your own secrets and reference them in the values.
For details, see Chart reference: Keycloak.
Example of full minimal values for a self-hosted Neptune instance
Note: Requires external MySQL, Elasticsearch and Ingress Controller.
imagePullPolicy: IfNotPresent
imagePullSecrets:
- regcred
database:
host: mysql
port: 3306
username: neptune_user
password: neptune_password
ingress:
host: neptune.example.org
annotations:
nginx.ingress.kubernetes.io/proxy-body-size: 10G
nginx.ingress.kubernetes.io/proxy-read-timeout: '86400'
kubernetes.io/ingress.class: neptune
keycloak:
users:
instanceAdmin:
username: admin
password: admin
serviceAccount:
username: neptune
password: neptune
clients:
management:
id: "neptune-management"
secret: "4b84b477-2e60-474e-89e2-e7fc3f5145f8" # random UUID
init:
workspaceName: neptune-self-hosted
administrator:
username: administrator
password: change_me
leaderboard:
elasticsearch:
address: http://elasticsearch-neptune:9200
storage:
pvc:
size: 100Gi
MariaDB#
To enable MariaDB, set the following values for your Helm deployment:
Upgrading Neptune#
The following procedure upgrades your Neptune installation.
If you use custom Helm chart values, ensure that the values are configured for your specific deployment needs. For instructions, see the Neptune Helm chart reference.
-
Perform a repository update:
-
Upgrade the Helm chart with the new Neptune version:
For complete installation instructions, see Default installation or Custom installation.
For older upgrade guides, see: