Core concepts
Core concepts in Neptune
Neptune Client Library (API): The suite of libraries to help you log, query, and download model-building metadata. You can use Neptune client for Python, R, and one of many integrations with ML tools.
Neptune Web Interface (UI): The visual interface where you can display, filter, compare, and organize ML runs and model building metadata.
Workspace: The space inside Neptune where you can manage Projects, people, and your subscription. When you create an account you are assigned to your individual workspace. You can belong to many team workspaces at the same time.
Project: The space inside a Workspace where you put all the ML runs and metadata generated while working on a single ML model. You can assign different people from your Workspace to different Projects.
Run: A namespace inside a Project where you log model building metadata. Typically, you create a Run every time you execute a script that does model training, re-training, or inference.
Field: A place inside the Run namespace where you log various types of model building metadata.
Metadata: Metrics and losses, hyperparameters, figures, image predictions, hardware consumption logs, code snapshots, and any other information you need to feel in control of your model building.

Client library (Neptune API)

Neptune client is an open-source Python library that helps you do two things:
It is designed to be:
    lightweight: easy connect to your workflow and integrate with other tools in your MLOps tech stack,
    flexible: capable of logging any kind of ML metadata in any structure you need,
    straightforward: you define what you want to log for each ML run.
Neptune API in 30 seconds

Step 1: Install the neptune-client library

Depending on your operating system open a terminal or CMD and run this command:
pip
conda
1
pip install neptune-client
Copied!
1
conda install -c conda-forge neptune-client
Copied!
For more help see installing neptune-client.

Step 2: Add logging into your model training script

train.py
1
import neptune.new as neptune
2
3
run = neptune.init(project='#', # your project
4
api_token='#', # your api token
5
)
6
7
# Track metadata and hyperparameters of your run
8
run["JIRA"] = "NPT-952"
9
10
params = {
11
"batch_size": 64,
12
"dropout": 0.2,
13
"optimizer":{ "learning_rate": 0.001,
14
"optimizer": "Adam"},
15
}
16
run["parameters"] = params
17
18
# Track the training process by logging your training metrics
19
for epoch in range(100):
20
run["train/accuracy"].log(epoch * 0.6)
21
run["train/loss"].log(epoch * 0.4)
22
23
# Log the final results
24
run["f1_score"] = 0.66
Copied!

Step 3: Run model training

1
python train.py
Copied!

Step 4: Query whatever model building metadata you need

1
import neptune.new as neptune
2
3
run = neptune.init(project='#', # your project
4
api_token='#', # your api token
5
run='#', # id of the run you want to access
6
)
7
8
params = run["parameters"].fetch()
Copied!

Web Interface (Neptune UI)

Neptune comes with a Web Interface designed for working with ML model building metadata.
It makes it easy to:
There are a few essential components of the Neptune UI:
    Projects: where you can create and manage all your projects,
    Runs Table: where you can compare, filter, and organize your runs,
    Single run: where you can explore and display rich metadata,
    People: where you can deal with user access control (ACL),
    Subscription: where you can manage your plan and see current usage.
Neptune UI Tour
Examples of Neptune UI in action
Description
Example in Neptune
Runs table
Comparison of runs
Logged learning curves
Logged hardware consumption
Logged interactive charts
Logged images
Custom Dashboard
Projects

Workspace

Workspace is a space inside Neptune where you can manage projects, people, and your subscription.
There are two types of workspaces:
    Individual
      When you create an account you are assigned an individual workspace called your username,
      You can create an unlimited number of public and private projects,
      You cannot invite other people to this workspace.
    Team
      Comes in handy when you want to collaborate on projects with other people,
      You can invite as many people as you want to your team workspace,
      You can belong to many team workspaces,
      Team workspace is paid but you can try it for free, for details check our pricing page,
      For academic purposes or Kaggle, you can use the team workspace for free. Create a team workspace and claim non-profit /educational type of organization in the setup process. We suggest you use your edu/non-profit email for this type of account.
Note: You can be a member of multiple workspaces. For example, you can have:
    your individual workspace,
    a team workspace at work,
    a team workspace at the university.

Project

A Project is a collection of Runs created by a user (or users) assigned to the Project. Typically you should create a Project per machine learning task to make it easy to compare runs that are connected to building a single ML model.
There are two types of projects:
    Private projects: Only people assigned to the project can see it,
    Public projects: freely available to view by everyone who has access to the Internet.
You can assign different people from your Workspace to different Projects.
To log to a Project via Neptune API you need a full project name of the format workspace/project:
1
run = neptune.init(project='my_workspace/my_project')
Copied!

Run

A Run is a namespace inside a Project where you log model building metadata. Typically, you create a run every time you execute a script that does model training, re-training, or inference.
It has a dictionary-like structure with Fields to which any type of ML metadata can be logged.
You can create a nested structure in your Runs to organize your parameters, metrics, or other metadata.
You start a Run with neptune.init():
1
import neptune.new as neptune
2
3
run = neptune.init(project='#', api_token='#') # your credentials
Copied!
Log model building metadata to various fields of the run:
1
run["JIRA"] = "NPT-952"
2
3
run["parameters"] = {
4
"batch_size": 64,
5
"dropout": 0.2,
6
"optimizer":{ "learning_rate": 0.001,
7
"optimizer": "Adam"},
8
}
9
10
for epoch in range(100):
11
run["train/accuracy"].log(epoch * 0.6)
12
run["train/loss"].log(epoch * 0.4)
13
14
# Log the final results
15
run["f1_score"] = 0.66
Copied!
Explore your Run in the UI:
Run in the Neptune UI
Access Run programmatically (after it has finished) :
1
run = neptune.init(project="#", api_token="#", # pass credentials
2
run="SUN-123") # pass a run ID
3
4
# update with new data
5
run["test_accuracy"] = 0.42
6
7
# fetch metadata
8
run["parameters/batch_size"].fetch()
Copied!
The run stops when the script finishes or when you explicitly call. Explicitly stopping the run is mandatory for notebooks and interactive environments.
1
run.stop()
Copied!

Field

A Field of a Run is a namespace to which various types of model building metadata can be logged.
Run field can be flat or nested:
1
# flat
2
run["f1_score"] = 0.66
3
run["model"].upload("model.pkl")
4
5
# nested
6
run["parameters/batch_size"] = 64
7
run["parameters/optimizer/learning_rate"] = 0.001
8
run["parameters/optimizer/optimizer"] = "Adam"
Copied!
You can also assign values to a Field of a Run with a dictionary:
1
run["parameters"] = {
2
"batch_size": 64,
3
"optimizer":{ "learning_rate": 0.001,
4
"optimizer": "Adam"},
5
}
Copied!
Neptune Field types and logging methods
Model building metadata are displayed in the Neptune UI based on the type you choose to assign to your Field.
Depending on what metadata you want to log and how you wish to display it you should use different Neptune field types and methods:
Metadata
Field Type
Logging Method
Example
Single metric
Float
=, assign()
run["f1_score"] = 0.66
Series of loss values
.log()
for epoch in range(100):
run["loss"].log(loss_value)
Hyperparameters
Float, String
=, assign()
run["learning_rate"] = 0.001
run["optimizer"] = "Adam"
Image predictions
.log()
for epoch in range(100):
run["preds"].log(image)
Charts
File
run["results"].upload(File.as_image(fig))
Model weights
File
run["model"].upload("model.pkl")
Query and download metadata from the Field
You can access information from the Field of a Run programmatically with .fetch() and .download() methods.
Field type
Method
What it does
Example
.fetch()
get value from the field
run["train/epoch"].fetch()
download file from the field
run["predictions/img2"].download()
get all values from a series field
run["train/loss"].fetch_values()
download the last file from a series field
run["predictions"].download_last()

Metadata

When we talk about metadata in the context of Neptune we usually mean model building metadata.
There are three main types of ML metadata that you can log and display in Neptune.
Experiment and model training metadata
    Metrics
    Hyperparameters
    Learning curves
    Training code and configuration files
    Predictions (images, tables, etc)
    Diagnostic charts (Confusion matrix, ROC curve, etc)
    Console logs
    Hardware logs
    And more
Artifact (datasets, predictions, and models) metadata
    Paths to the dataset or model (s3 bucket, filesystem)
    Dataset hash
    Dataset/prediction preview (head of the table, snapshot of the image folder)
    Description
    Feature column names (for tabular data)
    Who created/modified
    When last modified
    Size of the dataset
    And more
(Trained) Model metadata
    Model binary or location to your model asset
    Dataset versions
    Links to recorded model training runs and experiments
    Who trained the model
    Model descriptions and notes
    Links to observability dashboards (Grafana)
    And more
Most model building metadata types can be easily logged and nicely displayed in the Neptune UI:
Have a look at the complete list of metadata types you can log.

What's next?

Last modified 1mo ago