Skip to content

Neptune objects overview#

Neptune objects refer to entities that can contain metadata: runs, models, model versions, and projects.

The type of Neptune object you initialize in your code determines where the data goes and what you can do with it in the app.

Object type    Represents Description Section in app
Run A tracked experiment A namespace inside a project where you log ML metadata. Typically, you create a run every time you execute a script that does model training, re-training, or inference. Runs
Model An ML model Collection of metadata common to all versions of a model you're working on. Models
ModelVersion A version of an ML model Collection of metadata specific to a version of a model. Models
Project A Neptune project (an ML task you're solving) An object that can be used to log and query project-level metadata. Project metadata

How it works in the code#

The below code creates a run in your Neptune project, as specified by the NEPTUNE_PROJECT environment variable. Run objects can contain any metadata you want to track, plus some automatically logged system metrics.

Sample code
run = neptune.init_run()
run["params"] = {"lr": 0.01, "batch_size": 64, "activation": "ReLU"}
run["data/sample"].upload("datasets/sample.csv")
...
run.stop()

Once the run is stopped, you get a link to the run in the Neptune app.

Related

Register a model once, then create as many versions as you need.

Just like with runs, you can create new model objects or resume existing ones, and store metadata in a structure of your choosing.

Registering a model
model = neptune.init_model(key="FOREST")
model["signature"].upload("model_signature.json")
Creating a new model version
model_version = neptune.init_model_version(model="CLS-FOREST")
model_version["model/binary"].upload("model.pt")
model_version.change_stage("production")

Related

A Neptune project can itself be treated as a Neptune object, which lets you track and query metadata that applies on project level.

You can use the same logging methods as for runs and model objects.

Tracking dataset version for the whole project
project = neptune.init_project(name="ml-team/classification")
project["dataset/v0.1"].track_files("s3://datasets/images")
project["dataset/sample"].upload("s3://datasets/sample.csv")

Related

Anatomy of a Neptune object#

  • Namespace: A "folder" inside a Neptune object.
  • Field: Location inside a namespace where a piece of metadata is stored.
run["params/activation"] = "ReLU"

Result

  • Namespace: params
  • Field: activation
  • Field value: "ReLU"

When viewing the run in the app:

Text field in run metadata view

To fetch the value of the field through the Neptune API:

>>> run = neptune.init_run(with_id="CLS-23")  # (1)
>>> activation = run["params/activation"].fetch()
>>> print(activation)
'ReLU'
  1. The Neptune ID of the run. It's stored in the sys/id field. You can also use a custom run ID.

Similarities and differences between Neptune objects#

In your code, you work with the objects in much the same way:

  1. You initialize the object with its own init() function, either to create a new object or log to an existing one.
    • You can resume a connection to an object any number of times.
  2. It's up to you to define what types of metadata to log and in what structure.
  3. When you're done logging, you should stop the connection to the object.

The main differences are:

  • The available methods and properties. For example, managing the stage is only available for model versions.
  • Model and project objects do not monitor things in the background, like runs do.
  • Run and model version objects are designed for multiple trials and comparison, so their Neptune IDs include a counter.