Neptune objects overview#

Neptune objects refer to entities that can contain metadata: runs, models, model versions, and projects.

The type of Neptune object you initialize in your code determines where the data goes and what you can do with it in the app.

Object type	Represents	Description	Section in app
`Run`	A tracked experiment	A namespace inside a project where you log ML metadata. Typically, you create a run every time you execute a script that does model training, retraining, or inference.	Runs
`Model`	An ML model	Collection of metadata common to all versions of a model you're working on.	Models
`ModelVersion`	A version of an ML model	Collection of metadata specific to a version of a model.	Models
`Project`	A Neptune project (an ML task you're solving)	An object that can be used to log and query project-level metadata.	Project metadata

How it works in the code#

RunsModelsProject metadata

The below code creates a run in your Neptune project, as specified by the NEPTUNE_PROJECT environment variable. Run objects can contain any metadata you want to track, plus some automatically logged system metrics.

Sample code

run = neptune.init_run()
run["params"] = {"lr": 0.01, "batch_size": 64, "activation": "ReLU"}
run["data/sample"].upload("datasets/sample.csv")
...
run.stop()

Once the run is stopped, you get a link to the run in the Neptune app.

Create a run
API reference ≫ Run

Register a model once, then create as many versions as you need.

Just like with runs, you can create new model objects or resume existing ones, and store metadata in a structure of your choosing.

Registering a model

model = neptune.init_model(key="FOREST")
model["signature"].upload("model_signature.json")

Creating a new model version

model_version = neptune.init_model_version(model="CLS-FOREST")
model_version["model/binary"].upload("model.pt")
model_version.change_stage("production")

Model registry overview
API reference ≫ Model object – general model metadata
API reference ≫ ModelVersion object – version-specific model metadata

A Neptune project can itself be treated as a Neptune object, which lets you track and query metadata that applies on project level.

You can use the same logging methods as for runs and model objects.

Tracking dataset version for the whole project

project = neptune.init_project(project="ml-team/classification")
project["dataset/v0.1"].track_files("s3://datasets/images")
project["dataset/sample"].upload("s3://datasets/sample.csv")

Project metadata
API reference ≫ Project

Anatomy of a Neptune object#

Namespace: A "folder" inside a Neptune object.
Field: Location inside a namespace where a piece of metadata is stored.

What you logWhat you seeHow to query

run["params/activation"] = "ReLU"

Result

Namespace: params
Field: activation
Field value: "ReLU"

When viewing the run in the app:

Text field in run metadata view

To fetch the value of the field through the Neptune API:

>>> run = neptune.init_run(with_id="HELLO-23", mode="read-only")  # (1)!
>>> activation = run["params/activation"].fetch()
>>> print(activation)
'ReLU'

The Neptune ID of the run. It's stored in the sys/id field. You can also use a custom run ID.

In your code, you work with the objects in much the same way:

You initialize the object with its own init() function, either to create a new object or log to an existing one.
- You can resume a connection to an object any number of times.
It's up to you to define what types of metadata to log and in what structure.
When you're done logging, you should stop the connection to the object.

The main differences are:

The available methods and properties. For example, managing the stage is only available for model versions.
Model and project objects do not monitor things in the background, like runs do.
Run and model version objects are designed for multiple trials and comparison, so their Neptune IDs include a counter.

Neptune objects overview#

How it works in the code#

Anatomy of a Neptune object#

Similarities and differences between Neptune objects#