Skip to content

Essential logging methods#

Depending on your type of metadata, there are a few different logging methods to choose from.

The method determines what type of field the metadata is stored in and how you can interact with it.

Metadata type Example Logging method    Sample code
Single value Parameters, final scores, text = run["f1_score"] = 0.86
Single file Image, plot file, data sample upload() model_version["binaries"].upload("model.pkl")
Series of values Metrics, series of text entries append()* (inside for loop) run["train/loss"].append(loss)
Series of files Image series append()* (inside for loop) run["misclassified_images"].append(File("misclassified_image.png"))
Set of files Large number of files upload_files() model["preprocessing_scripts"].upload_files("./preprocessing/*.py")
Tags Text tags to annotate objects add() run.add(["RNN", "finetune"])
Artifact, externally stored file Dataset, model file track_files() model["data/val"].track_files("validation.csv")

* Previously log()

Related

For the full list of field types and their methods, see Field types reference.

Simple value assignment#

Single value#

To log single-valued metadata, like a hyperparameter or evaluation metric, assign the value with an equals sign (=):

run["max_epochs"] = 5
run["optimizer"] = "Adam"

You query a single value from a Neptune object with the fetch() method:

>>> print(run["max_epochs"].fetch())
5

Dictionary of values#

To log metadata from a Python dictionary, like a training configuration, assign the value with an equals sign (=). Your Python dictionary will be parsed into a Neptune namespace automatically.

run["parameters"] = {
    "batch_size": 64,
    "dropout": 0.2,
    "optim": {"learning_rate": 0.001, "optimizer": "Adam"},
}

When fetching, the namespace structure would look like this:

>>> print(run["parameters/optim/optimizer"].fetch())
'Adam'

Creating a series: append()#

append() replaces log()

As of neptune-client 0.16.14, append() and extend() are the preferred methods for logging series of values.

You can upgrade your installation with pip install -U neptune-client or continue using log().

Use the append() method to create a series of metrics or other values, like loss during training or text logs after every iteration. Each append() call adds a new value to the series.

for iteration in range(100):
    run["train/loss"].append(loss)
    run["logs"].append(iteration_config)

You query entries from a series of values with the fetch_last() method:

>>> final_loss = run["train/loss"].fetch_last()
>>> print(final_loss)
0.13

Series of images or figures: append()#

You can also use the append() method to log a series of figures, like image predictions after every epoch.

You can pass PIL, Matplotlib, or Seaborn figure objects as the argument:

for iteration in range(100):
    pil_image = ...
    run["train/distribution"].append(pil_image)

Passing a file path

If you supply a file path, use the File constructor:

from neptune.types import File

for i in range(100):
    run["image_preds"].append(File(f"image_pred_{i}.png"))

Related

Tracking artifact metadata: track_files()#

To track and version a dataset, model, or any other artifact stored in a file, folder, or S3-compatible storage, use the track_files() method:

run["dataset/train"].track_files("./datasets/train/images")

Related

Tracking artifacts

Uploading files or objects: upload()#

To log a single file or object, like a sample of data or confusion matrix figure, use the upload() method:

run["data_sample"].upload("sample_data.csv")

For how to upload files from binary or text content, see also from_content() and from_stream().

Displaying arrays or tensors: upload() + as_image()#

Display your arrays or tensors as a series of images with the as_image() method:

from neptune.types import File

run["prediction_example"].upload(File.as_image(numpy_array))

Related

Logging arrays and tensors

Interactive HTML: as_html()#

You can convert, for example, dataframe objects to interactive HTML with the as_html() method:

from neptune.types import File

run["results"].upload(File.as_html(df_predictions))

By default, Neptune logs Matplotlib figures as static images. You can use as_html() to log the figure as an interactive visualization instead:

from neptune.types import File

fig = ...
run["visuals/matplotlib-fig"].upload(File.as_html(fig))

Related

Python object: as_pickle()#

Log Python objects as pickles with the as_pickle() method:

from neptune.types import File

run["pickled_model"].upload(File.as_pickle(trained_model))