Log model building metadata

Before you start

Make sure you meet the following prerequisites before starting:

Step 1: Connect Neptune to your script

At the top of your script add
import neptune.new as neptune
run = neptune.init_run(
This creates a new run in Neptune to which you can log metadata.
You need to tell Neptune who you are and where you want to log things. To do that you specify:
  • project=my_workspace/my_project: your workspace name and project name,
  • api_token=YOUR_API_TOKEN : your Neptune API token.
If you configured your Neptune API token correctly, as described here, you can skip the api_tokenargument.
Runs can be viewed as dictionary-like structures - namespaces - that you can define in your code. You can apply hierarchical structure to your metadata that will be reflected in the UI as well. Thanks to this you can easily organize your metadata in a way you feel is most convenient.

Step 2: Log parameters

PARAMS = {"lr": 0.1, "epoch_nr": 10, "batch_size": 32}
run["parameters"] = PARAMS
It logs your PARAMS dictionary with all the parameters that you want to keep track of.

See also:

How to Track Hyperparameters of Machine Learning Models? - neptune.ai

Step 3: Log dataset version

Log dataset as a versioned artifact with the .track_files() method:

Step 4: Add logging for metrics and losses

To log a metric or loss during training you should use:
loss = ...
Few explanations here:
  • "train/loss" is a name of the log with a hierarchical structure.
  • "train/loss" is a series of values - you can log multiple values to this log.
  • You can have one or multiple log names like "train/acc", "val/f1_score", "train/log-loss", "test/acc").
  • The argument of the log() method is the actual value you want to log.
Typically during training, there will be some sort of a loop where those losses are logged. You can simply call run["train/loss"].log(loss) multiple times at each step.
for i in range(epochs):

Step 5: Add logging of test score

run["test/acc"] = 0.76
In this way, you will log a single value, not a series of values. Again, you can make use of the hierarchical structure of the run to organize scores - "test/acc".
You can also update runs after the script is done running. Read about updating existing runs.

Step 6: Add logging of performance charts

# confusion_matrix is matplotlib figure object
confusion_matrix = ...
# log image predictions
from neptune.new.types import File
img_pred = File("path/to/image")

Step 7: Add logging of model files

Log your model as a versioned artifact with .track_files() method. Just pass the path to the file you want to log to Neptune.
There are many other objects that you can log to Neptune:

Step 7: Stop logging

Once you are done logging, you should stop tracking the run using the stop() method. This is needed only while logging from a notebook environment. While logging through a script, Neptune automatically stops tracking once the script has completed execution.