Logging and managing runs results

Log and manage data in Neptune via neptune-client (Python and R clients are available).

What is neptune-client?

Neptune client is an open source Python library that serves three purposes:

  1. update existing run with new data and visualizations,

  2. download runs data from Neptune to local machine.

It is designed to be:

  • lightweight: low setup effort,

  • generic: capable of logging any kind of machine learning work,

  • straightforward: user defines what to keep track of during the run.

You can log runs from anywhere (local machine, cluster, cloud, Colab, etc.) and they will be tracked in the same, standardized way. You will be able to quickly compare runs made by you on your workstation with runs executed on AWS by your team-mate.

What you need to add to your code to start logging

Bare minimum is one statement:

import neptune.new as neptune
run = neptune.init(project='common/quickstarts', api_token='ANONYMOUS')

These are usually just copy&paste into your existing project code.

Remember to create project and setup API token before you create a run using snippet above.

Now, that the run is created you can start logging metrics, losses, images, model weights or whatever metadata you feel relevant to keep track of in your run.

Runs can be viewed as dictionary-like structures - namespaces - that you can define in your code. You can apply hierarchical structure to your metadata that will be reflected in the UI as well. Thanks to this you can easily organize your metadata in a way you feel is most convenient.

Overall idea for logging is to use statements similar to this:

# logging score (single value)
run['score'] = 0.97
run['test/acc'] = 0.97
# logging metric (series of values)
for epoch in range(100):
acc = ...
loss = ...
metric = ...
run["train/accuracy"].log(acc)
run["train/loss"].log(loss)
run['metric'].log(metric)

Generic recipe being:

# single value
run['some/structure'] = val
# for series of values
run['some/structure'].log(val)

Essential Neptune client concepts

Project and Run are two important entities in Neptune.

Basic snippet below, sets project and creates new run in that project.

import neptune.new as neptune
# Create run in project
run = neptune.init(project='workspace/project',
api_token='ANONYMOUS')

Project

It is a collection of Runs, created by user (or users) assigned to the project.

You can log runs to the project or fetch all runs that satisfy some criteria.

import neptune.new as neptune
# Create run in project
run = neptune.init(project='my_workspace/my_project', api_token='ANONYMOUS')
# Use project to get runs data from the project
my_project = neptune.get_project('my_workspace/my_project')
runs_df = my_project.fetch_runs_table(tag=['maskRCNN']).to_pandas()

Learn more about downloading data from Neptune.

Run

Run is everything that you log to Neptune, by using run = neptune.init().

Creating run is easy:

import neptune.new as neptune
# Create run in project
run = neptune.init(project='my_workspace/my_project', api_token='ANONYMOUS')

You can now log various metadata to the run including metrics, losses, model weights, images, predictions and much more. Have a look at the complete list of what you can log to the run

Besides logging data, you can also download run data to you local machine or update an existing run even when it’s offline.