You should know
How-to guides
Integrations and Supported Tools
API reference
Core concepts
Core concepts in Neptune
​Neptune Client Library (API): The suite of libraries to help you log, query, and download model-building metadata. You can use Neptune client for Python, R, and one of many integrations with ML tools.
​Neptune Web Interface (UI): The visual interface where you can display, filter, compare, and organize ML runs and model building metadata.
​Workspace: The space inside Neptune where you can manage Projects, people, and your subscription. When you create an account you are assigned to your individual workspace. You can belong to many team workspaces at the same time.
​Project: The space inside a Workspace where you put all the ML runs and metadata generated while working on a single ML model. You can assign different people from your Workspace to different Projects.
​Run: A namespace inside a Project where you log model building metadata. Typically, you create a Run every time you execute a script that does model training, re-training, or inference.
​Metadata: Metrics and losses, hyperparameters, figures, image predictions, hardware consumption logs, code snapshots, and any other information you need to feel in control of your model building.
Client library (Neptune API)
It is designed to be:
- lightweight: easy connect to your workflow and integrate with other tools in your MLOps tech stack,
- flexible: capable of logging any kind of ML metadata in any structure you need,
- straightforward: you define what you want to log for each ML run.
Neptune API in 30 seconds
Step 1: Install the neptune-client library
Depending on your operating system open a terminal or CMD and run this command:
pip
conda
1
pip install neptune-client
Copied!
1
conda install -c conda-forge neptune-client
Copied!
Step 2: Add logging into your model training script
train.py
1
import neptune.new as neptune
2
​
3
run = neptune.init(project='#', # your project
4
api_token='#', # your api token
5
)
6
​
7
# Track metadata and hyperparameters of your run
8
run["JIRA"] = "NPT-952"
9
​
10
params = {
11
"batch_size": 64,
12
"dropout": 0.2,
13
"optimizer":{ "learning_rate": 0.001,
14
"optimizer": "Adam"},
15
}
16
run["parameters"] = params
17
​
18
# Track the training process by logging your training metrics
19
for epoch in range(100):
20
run["train/accuracy"].log(epoch * 0.6)
21
run["train/loss"].log(epoch * 0.4)
22
​
23
# Log the final results
24
run["f1_score"] = 0.66
Copied!
Step 3: Run model training
1
python train.py
Copied!
Step 4: Query whatever model building metadata you need
1
import neptune.new as neptune
2
​
3
run = neptune.init(project='#', # your project
4
api_token='#', # your api token
5
run='#', # id of the run you want to access
6
)
7
​
8
params = run["parameters"].fetch()
Copied!
Web Interface (Neptune UI)
Neptune comes with a Web Interface designed for working with ML model building metadata.
It makes it easy to:
There are a few essential components of the Neptune UI:
- Projects: where you can create and manage all your projects,
- Runs Table: where you can compare, filter, and organize your runs,
- Single run: where you can explore and display rich metadata,
- People: where you can deal with user access control (ACL),
- Subscription: where you can manage your plan and see current usage.
Neptune UI Tour
Examples of Neptune UI in action
Workspace
Workspace is a space inside Neptune where you can manage projects, people, and your subscription.
There are two types of workspaces:
- Individual
- When you create an account you are assigned an individual workspace called your
username, - You can create an unlimited number of public and private projects,
- You cannot invite other people to this workspace.
- Team
- Comes in handy when you want to collaborate on projects with other people,
- You can invite as many people as you want to your team workspace,
- You can belong to many team workspaces,
- For academic purposes or Kaggle, you can use the team workspace for free. Create a team workspace and claim non-profit /educational type of organization in the setup process. We suggest you use your edu/non-profit email for this type of account.
Note:
You can be a member of multiple workspaces. For example, you can have:
- your individual workspace,
- a team workspace at work,
- a team workspace at the university.
Project
A Project is a collection of Runs created by a user (or users) assigned to the Project. Typically you should create a Project per machine learning task to make it easy to compare runs that are connected to building a single ML model.
There are two types of projects:
- Private projects: Only people assigned to the project can see it,
- Public projects: freely available to view by everyone who has access to the Internet.
You can assign different people from your Workspace to different Projects.
To log to a
Project via Neptune API you need a full project name of the format workspace/project: 1
run = neptune.init(project='my_workspace/my_project')
Copied!
Run
A Run is a namespace inside a Project where you log model building metadata. Typically, you create a run every time you execute a script that does model training, re-training, or inference.
You can create a nested structure in your Runs to organize your parameters, metrics, or other metadata.
1
import neptune.new as neptune
2
​
3
run = neptune.init(project='#', api_token='#') # your credentials
Copied!
Log model building metadata to various fields of the run:
1
run["JIRA"] = "NPT-952"
2
​
3
run["parameters"] = {
4
"batch_size": 64,
5
"dropout": 0.2,
6
"optimizer":{ "learning_rate": 0.001,
7
"optimizer": "Adam"},
8
}
9
​
10
for epoch in range(100):
11
run["train/accuracy"].log(epoch * 0.6)
12
run["train/loss"].log(epoch * 0.4)
13
​
14
# Log the final results
15
run["f1_score"] = 0.66
Copied!
Explore your Run in the UI:
Run in the Neptune UI
Access Run programmatically (after it has finished) :
1
run = neptune.init(project="#", api_token="#", # pass credentials
2
run="SUN-123") # pass a run ID
3
​
4
# update with new data
5
run["test_accuracy"] = 0.42
6
​
7
# fetch metadata
8
run["parameters/batch_size"].fetch()
Copied!
The run stops when the script finishes or when you explicitly call. Explicitly stopping the run is mandatory for notebooks and interactive environments.
1
run.stop()
Copied!
Field
A Field of a Run is a namespace to which various types of model building metadata can be logged.
Run field can be flat or nested:
1
# flat
2
run["f1_score"] = 0.66
3
run["model"].upload("model.pkl")
4
​
5
# nested
6
run["parameters/batch_size"] = 64
7
run["parameters/optimizer/learning_rate"] = 0.001
8
run["parameters/optimizer/optimizer"] = "Adam"
Copied!
You can also assign values to a Field of a Run with a dictionary:
1
run["parameters"] = {
2
"batch_size": 64,
3
"optimizer":{ "learning_rate": 0.001,
4
"optimizer": "Adam"},
5
}
Copied!
Neptune Field types and logging methods
Model building metadata are displayed in the Neptune UI based on the type you choose to assign to your Field.
Depending on what metadata you want to log and how you wish to display it you should use different Neptune field types and methods:
Metadata
Field Type
Logging Method
Example
Series of loss values
for epoch in range(100): run["loss"].log(loss_value)Hyperparameters
run["learning_rate"] = 0.001run["optimizer"] = "Adam"Query and download metadata from the Field
You can access information from the Field of a Run programmatically with .fetch() and .download() methods.
Field type
Method
What it does
Example
download file from the field
run["predictions/img2"].download()get all values from a series field
run["train/loss"].fetch_values()download the last file from a series field
run["predictions"].download_last()Metadata
When we talk about metadata in the context of Neptune we usually mean model building metadata.
There are three main types of ML metadata that you can log and display in Neptune.
Experiment and model training metadata
- Metrics
- Hyperparameters
- Learning curves
- Training code and configuration files
- Predictions (images, tables, etc)
- Diagnostic charts (Confusion matrix, ROC curve, etc)
- Console logs
- Hardware logs
- And more
Artifact (datasets, predictions, and models) metadata
- Paths to the dataset or model (s3 bucket, filesystem)
- Dataset hash
- Dataset/prediction preview (head of the table, snapshot of the image folder)
- Description
- Feature column names (for tabular data)
- Who created/modified
- When last modified
- Size of the dataset
- And more
(Trained) Model metadata
- Model binary or location to your model asset
- Dataset versions
- Links to recorded model training runs and experiments
- Who trained the model
- Model descriptions and notes
- Links to observability dashboards (Grafana)
- And more
Most model building metadata types can be easily logged and nicely displayed in the Neptune UI:
What's next?
​
Last modified 7d ago
Export as PDF
Copy link