Essential logging methods#
Depending on your type of metadata, there are a few different logging methods to choose from.
The method determines what type of field the metadata is stored in and how you can interact with it.
Metadata type | Example | Logging method | Sample code |
---|---|---|---|
Single value | Parameters, final scores, text | = |
run["f1_score"] = 0.86 |
Single file | Image, plot file, data sample | upload() |
model_version["binaries"].upload("model.pkl") |
Series of values | Metrics, series of text entries | append() * |
(inside for loop) run["train/loss"].append(loss) |
Series of files | Image series | append() * |
(inside for loop) run["misclassified_images"].append(File("misclassified_image.png")) |
Set of files | Large number of files | upload_files() |
model["preprocessing_scripts"].upload_files("./preprocessing/*.py") |
Tags | Text tags to annotate objects | add() |
run.add(["RNN", "finetune"]) |
Artifact, externally stored file | Dataset, model file | track_files() |
model["data/val"].track_files("validation.csv") |
* Previously log()
Related
For the full list of field types and their methods, see Field types reference.
Simple value assignment#
Single value#
To log single-valued metadata, like a hyperparameter or evaluation metric, assign the value with an equals sign (=
):
You query a single value from a Neptune object with the fetch()
method:
Dictionary of values#
To log metadata from a Python dictionary, like a training configuration, assign the value with an equals sign (=
). Your Python dictionary will be parsed into a Neptune namespace automatically.
run["parameters"] = {
"batch_size": 64,
"dropout": 0.2,
"optim": {"learning_rate": 0.001, "optimizer": "Adam"},
}
When fetching, the namespace structure would look like this:
Creating a series: append()
#
Use the append()
method to create a series of metrics or other values, like loss during training or text logs after every iteration. Each append()
call adds a new value to the series.
You query entries from a series of values with the fetch_last()
method:
Series of images or figures: append()
#
You can also use the append()
method to log a series of figures, like image predictions after every epoch.
You can pass PIL, Matplotlib, or Seaborn figure objects as the argument:
Passing a file path
If you supply a file path, use the File
constructor:
Related
Tracking artifact metadata: track_files()
#
To track and version a dataset, model, or any other artifact stored in a file, folder, or S3-compatible storage, use the track_files()
method:
Related
Uploading files or objects: upload()
#
To log a single file or object, like a sample of data or confusion matrix figure, use the upload()
method:
For how to upload files from binary or text content, see also from_content()
and from_stream()
.
Displaying arrays or tensors: upload()
+ as_image()
#
Display your arrays or tensors as a series of images with the as_image()
method:
Related
Interactive HTML: as_html()
#
You can convert, for example, dataframe objects to interactive HTML with the as_html()
method:
By default, Neptune logs Matplotlib figures as static images. You can use as_html()
to log the figure as an interactive visualization instead:
Python object: as_pickle()
#
Log Python objects as pickles with the as_pickle()
method: