Set up Neptune credentials in AWS Secrets#

This guide shows how to store your Neptune credentials in AWS Secrets. This lets you smoothly use the Neptune client library with Amazon SageMaker.

About AWS secrets

AWS Secrets Manager is a secure way to share sensitive information across AWS services. The secrets are encrypted and access to them is controlled by access policies. It's like using a password manager that stores your password in an encrypted form and pastes it to website forms when needed.

Before you start#

Set up your Neptune project, where the runs and metadata will go: Create a Neptune project
(optional) Instead of using a Neptune user account, you can create a service account for use in automated pipelines.

Creating the AWS secret#

To store your Neptune information in an AWS secret:

In AWS Secrets Manager, navigate to Secrets.
Click Store new secret.
Under Secret type, select Other type of secret.
Under Key/value pairs, add the following entries:

Key Value

api_token The Neptune API token of your account

project The name of your Neptune project

How do I find my API token?

In the bottom-left corner of the Neptune app, open the user menu and select Get your API token.

You can copy your token from the dialog that opens. It's very long – make sure to copy and paste it in full!
How do I find my project name?

Your full project name has the form workspace-name/project-name.

For example, if your workspace name (shown in the top-left corner) is "ml-team" and your project is named "classification", your project string is: "ml-team/classification".
- To copy the name to your clipboard, navigate to the project details ( → Details & privacy).
- You can also find a pre-filled project string in Experiments → Create a new run.
Click Next to continue.
In the Configure secret step, under Secret name and description, enter a secret name starting with AmazonSageMaker-.
Optionally enter more information about the secret, then click Next and leave the default settings.

Accessing the AWS secret#

The SageMaker resource that needs access to the secret must have the following permissions:

secretsmanager:ListSecrets
secretsmanager:GetSecretValue

The default AmazonSageMakerFullAccess policy used in SageMaker services does have these permissions. The relevant part of the policy looks like this:

{
    "Effect": "Allow",
    "Action": [
        ...
        "secretsmanager:ListSecrets",
        ...
    ],
    "Resource": "*"
},
{
    "Effect": "Allow",
    "Action": [
        "secretsmanager:DescribeSecret",
        "secretsmanager:GetSecretValue",
        "secretsmanager:CreateSecret"
    ],
    "Resource": [
        "arn:aws:secretsmanager:*:*:secret:AmazonSageMaker-*"
    ]
},

With the Neptune-AWS integration#

To access the secrets from your code (such as a notebook or training script):

Install the neptune-aws integration package.

Initialize Neptune with the following:

from neptune.integrations.aws import init_run 

run = init_run(
    secret="neptune-secret",  # Use your secret name here
    region="us-west-1",  # Use the appropriate region here
)

Without the integration#

To access the secrets from your code without the Neptune-AWS integration, you can also use the following code:

import boto3
from botocore.exceptions import ClientError
import json

secret_name = "AmazonSageMaker-name-of-your-secret"
region_name = "eu-west-1"

# Create a Secrets Manager client
session = boto3.session.Session()
client = session.client(
    service_name="secretsmanager",
    region_name=region_name,
)

get_secret_value_response = client.get_secret_value(SecretId=secret_name)

json.loads(get_secret_value_response["SecretString"])  #(1)!

Dictionary containing the saved secrets.

Key	Value
`api_token`	The Neptune API token of your account
`project`	The name of your Neptune project