What can you log and display?
Neptune supports the logging and display for many different types of model building metadata.
On this page, you can see how each ML metadata type can be logged via Neptune API and how it will be displayed in the Neptune UI.
Supported ML metadata types include:
Remember to create a Run before you start logging:
1
import neptune.new as neptune
2
3
run = neptune.init(api_token="YOUR_API_TOKEN", project="YOUR_WORKSPACE/YOUR_PROJECT")
Copied!

Metrics and losses

Log metrics and losses to Neptune using =and .log() methods:
1
# logging score (single value)
2
run["score"] = 0.97
3
run["test/acc"] = 0.97
4
5
# logging metric (series of values)
6
for epoch in range(100):
7
acc = ...
8
loss = ...
9
metric = ...
10
11
run["train/accuracy"].log(acc)
12
run["train/loss"].log(loss)
13
run["metric"].log(metric)
Copied!
Metrics and losses in Neptune
Metric can be accuracy, MSE, or any numerical value. All metrics are visualized as Charts in the Run UI. You can also check grouped values in the All Metadata section of the Run UI and add metrics as columns to the Runs Table.

Parameters and Model Configuration

Log parameters one-by-one to Fields of a Run.
1
run["parameters/epoch_nr"] = 5
2
run["parameters/batch_size"] = 32
3
run["parameters/dense"] = 512
4
run["parameters/optimizer"] = "sgd"
5
run["parameters/metrics"] = ["accuracy", "mae"]
6
run["parameters/activation"] = "relu"
Copied!
Define parameters as a Python dictionary and log them all at once to a Namespace of a Run.
1
# Define parameters
2
PARAMS = {
3
"epoch_nr": 5,
4
"batch_size": 32,
5
"dense": 512,
6
"optimizer": "sgd",
7
"metrics": ["accuracy", "binary_accuracy"],
8
"activation": "relu",
9
}
10
11
# Pass parameters
12
run["parameters"] = PARAMS
Copied!
Logged parameters appear in the All metadata section of a single Run UI:
Parameters in the single Run UI
You can also add parameters as columns to the Runs Table:
Parameters in the Runs Table

Model checkpoints

You can log model checkpoints to a Run.
1
# Log PyTorch model weights
2
my_model = ...
3
torch.save(my_model, "my_model.pt")
4
run["model_checkpoints/my_model"].upload("model_checkpoints/my_model.pt")
Copied!
You can save model weights from any deep learning framework. Model checkpoints will appear in the All metadata section of the Run UI, for example in the ‘model_checkpoints’ Namespace.

Code

Neptune supports code versioning in three different ways.

Git information

If you start a Run from a directory that is a part of the git repo, Neptune will automatically find the .git directory and log information from it.
It creates a summary in the source_code/git section of the Run UI with the following Fields:
  • Dirty: whether the commit was dirty or not
  • Commit message: what was the commit message
  • Commit ID: Git SHA signature
  • Commit author: the author of the last commit
  • Commit date: the data of the last commit
  • Current branch: what is the current git branch
  • Remotes: list of git remotes your local .git is connected to

Code Snapshot

Neptune automatically snapshots code when you create a Run with neptune.init().
By default, it will only save the entry point file (e.g.main.py if you run python main.py), but you can pass a list of files or wildcard patterns ("glob") like: **/*.py to specify more files:
1
# Snapshot model.py and prep_data.py
2
run = neptune.init(..., source_files=["model.py", "prep_data.py"])
3
4
# Snapshot all python files and 'config.yaml' file
5
run = neptune.init(..., source_files=["**/*.py", "config.yaml"])
Copied!
You can turn off logging of source code by passing an empty list to the source_files argument of neptune.init():
1
run = neptune.init(..., source_files=[])
Copied!
Tip: You can always log file contents manually, for example, if you need a customized structure or you're working with non-run objects:
1
model_version["source_files"].upload_files("**/*.py")
Copied!
When using pattern expansion, such as "**/*.py", make sure that your expression does not log too many files or non-source code files.
For example, using "*" as a pattern will upload all files and directories from the current working directory (cwd). It may result in logging files that you did not want to upload and cluttering your project.
You will have all the files in the Source code section of the Run UI.
Neptune also logs the entry point filename so that you know which script was actually executed.

Notebook Code Snapshot

Neptune can automatically snapshot your Jupyter notebook checkpoint every time you create a Run in that notebook.
You can also log a Jupyter notebook checkpoint without creating a Neptune Run to log your Exploratory Data Analysis (EDA) or other work.

Data Versions

You can log and display metadata about datasets, models, and other artifacts in Neptune.
The hash depends on the file contents and metadata like path, size, and last modification time. Any change to any of these will result in a different hash, even if the file contents are exactly the same. Read more about this here.

Local filesystem

You can version artifacts stored on your local filesystem with Neptune. For each artifact you version you will see:
  • URL: a path of an artifact on your local filesystem
  • Hash: md5 signature of your artifact
To use artifacts you need neptune-client>=0.10.10
1
pip install neptune-client>=0.10.10
Copied!
To do that, use .track_files() method passing a path to file or folder:
1
# single file
2
run["train_dataset"].track_files("./datasets/train.csv")
3
4
# Folder
5
run["train/images"].track_files("./datasets/images")
Copied!
Go to the All metadata section of the Run UI and find your data versions and other dataset metadata.
Viewing versioned local files as Artifacts in the Neptune UI

S3 compatible storage

You can version datasets or models stored on S3 compatible storage ("s3://..." ) including AWS S3, GCP, MinIO, and others in Neptune. For each artifact you version you will see:
  • URL: a path of an artifact on your local filesystem
  • Hash: md5 signature of your artifact
To use artifacts you need neptune-client>=0.10.10
1
pip install neptune-client>=0.10.10
Copied!
To do that, use .track_files() method passing a path to file or folder:
1
# single file
2
run["train_dataset"].track_files("s3://datasets/train.csv")
3
4
# Folder
5
run["train/images"].track_files("s3://datasets/images")
Copied!
Remember to pass credentials as environment variables.
For example, on AWS S3 you should configure an IAM group policy with S3ReadAccessOnly permissions. Then export the user access keys:
1
export AWS_SECRET_ACCESS_KEY='YOUR_AWS_KEY'
2
export AWS_ACCESS_KEY_ID='YOUR_AWS_ID'
Copied!
Viewing versioned S3 files as Artifacts in the Neptune UI

Data Version Control (DVC)

Neptune displays DVC files in the UI, you just need to specify which DVC files you would like to log.
For example, to log all the DVC files from your working directory and all subdirectories pass "**/*.dvc" to the source_files argument of neptune.init():
1
# Snapshot all .dvc files from any directory
2
run = neptune.init(..., source_files=["**/*.dvc"])
Copied!

Hardware consumption

Automatically monitor hardware utilization for your Runs:
  • cpu: total CPU consumption of your machine
  • memory: total memory consumption of your machine
  • gpu: GPU consumption of your machine
  • gpu_memory: GPU memory consumption of your machine
  • stderr: standard error logs from your console
  • stdout: standard output logs from your console
All that information is visualized in the Monitoring section of the Run UI.
To turn off the logging of hardware consumption use capture_hardware_ metrics argument of neptune.init():
1
# Turn off hardware monitoring
2
run = neptune.init(..., capture_hardware_metrics=False)
Copied!
To log hardware consumption to a different Namespace use monitoring_namespace argument argument of neptune.init():
1
# Assign custom name to the monitoring namespace
2
run = neptune.init(..., monitoring_namespace="worker-7-monitoring")
Copied!
If you see "Info (NVML): NVML Error: NVML Shared Library Not Found - GPU usage metrics may not be reported." your GPU consumption is not being logged.
It means that either:
  • there are no GPUs on your machine
  • your NVIDIA NVML library is not installed or configured properly. See how to install and configure NVML.
Logging GPU on Windows
On Windows, Neptune searches for the nvml.dll file in the standard locations:
  • C:\Program Files\NVIDIA Corporation\NVSMI\nvml.dll
  • C:\Windows\System32\nvml.dll
If you are having trouble logging GPU metrics on Windows, please check that your NVML installation is correct and that you have the nvml.dll file in either of those locations.
Alternatively, you can set a custom location of nvml.dll on Windows by setting the NVML_DLL_PATH environment variable.

Files

You can log any file or directory you want to Neptune by using .upload()and .upload_files()methods:
1
# Log file
2
run["aux/data"].upload("auxiliary-data.zip")
3
4
# Save multiple files under a path
5
run["config_files"].upload_files([path_to_config_1, path_to_config_2])
6
7
# You can also use wildcard patterns ("glob") with .upload_files()
8
run["preprocessing_scripts"].upload_files("./preprocessing/*.py")
Copied!
You can browse and download files in the All metadata section of the Run UI.
Files are displayed in the UI based on the file format:
  • PNG, JPG, JPEG, GIF, BMP, and WebP image formats are displayed as Images
  • HTML files are displayed as Interactive visualizations
  • MP4, OGG, OGV, MOV, M4V, MKV, WEBM are displayed as players for Video
  • MP3, M4A, OGA, WAVE are displayed as players for Audio
  • CSV files are displayed as Tables
  • IPYNB files are displayed as fully rendered Notebooks
  • Text files and source code files are displayed as text with appropriate syntax coloring if applicable
Make sure that you define the correct path to files that you want to upload when using wildcards like '*.png' or '**/*.yaml' as it may result in the unintended logging of a large amount of data and cluttering your storage.
You can delete files from a Run by using .pop()method:
1
run.pop("aux/data")
Copied!

Console logs

stdout and stderr

Automatically save console logs for your Runs :
  • stderr: standard error logs from your console
  • stdout: standard output logs from your console
All that information is visualized in the Monitoring section of the Run UI.
To turn off the saving of console logs use capture_stdout capture_stderarguments of neptune.init():
1
# Turn off saving console logs
2
run = neptune.init(
3
...,
4
capture_stdout=False,
5
capture_stderr=False,
6
)
Copied!

Python's Logger

You can also capture Python's Logger logs. Simply create and add Neptune handler to your Logger object:
1
import logging
2
from neptune.new.integrations.python_logger import NeptuneHandler
3
4
logger = logging.getLogger("my_python_logger")
5
6
logger.addHandler(NeptuneHandler(run=run))
7
8
logger.debug("Starting training")
Copied!
When initializing the NeptuneHandler you can customize what level of logs you want to capture or the name of the field where the logs will be stored.

Images

You can log either a single image or a series of images to Neptune.
Several data formats are available:
In all cases, you will have images in the All metadata section of the Run UI (under Namespace defined by you), where you can browse and download them.
To log images use .upload() or .log(File) methods:
1
from neptune.new.types import File
2
3
# single image
4
run["train/bounding_boxes"].upload("bboxes.png")
5
6
# for series of images
7
for i in range(valid_nr):
8
ith_path = ...
9
run["valid/misclassified"].log(File(ith_path))
Copied!
You can log an unlimited number of images either in the single image log or as a series of image logs.

Image file

Neptune currently supports PNG, JPEG, JPG, GIF, BMP, and WebP formats for image files.
You can log image format files directly from the disk.
Log single image from disk with the .upload() method:
1
run["train/bounding_boxes"].upload("bboxes.png")
Copied!
Log a series of images in a for loop with the .log(File) method:
1
from neptune.new.types import File
2
3
for name in misclassified_images_names:
4
y_pred = ...
5
y_true = ...
6
run["misclassified_images"].log(File("misclassified_image.png"))
Copied!
Notice the difference between .upload() for single images and .log() for series of images.

Matplotlib figure

Log a Matplotlib figure (matplotlib.figure.Figure) as an image:
1
# Import matplotlib
2
import matplotlib.pyplot as plt
3
4
# Generate figure
5
fig = plt.figure(figsize=(7, 9))
6
...
7
8
# Log figure to run
9
run["matplotlib-fig"].upload(fig)
Copied!
You can also log a series of Matplotlib figures:
1
for epoch in range(params["iterations"]):
2
plt_fig = get_histogram()
3
run["train/distribution"].log(plt_fig)
Copied!
You will have the Matplotlib figures displayed in the All metadata section of the Run UI, where you can browse and download them.
For more information, see How to use Neptune and Matplotlib.

Seaborn figure

Log a Seaborn figure as an image:
1
# Import Seaborn
2
import seaborn as sns
3
4
# Generate chart
5
seaborn_fig = ...
6
7
# Convert Seaborn object to Matplotlib format (matplotlib.figure.Figure)
8
figure = seaborn_fig.fig
9
10
# Log figure to run
11
run["seaborn-img"].upload(neptune.types.File.as_image(figure))
Copied!
You will have the Seaborn figure displayed in the All metadata section of the Run UI, where you can browse and download it.

PIL Image

You can log a PIL image directly from the memory:
1
image1 = Image.open("Representation-learning.jpg")
2
image2 = Image.open("RL-agents-1.jpg")
3
4
# Log image to run
5
run["representation_learning"].upload(image1)
6
run["reinforcement_learning"].upload(image2)
Copied!
You can also log a series of PIL images:
1
from neptune.new.types import File
2
3
for epoch in range(images_nr):
4
pil_image = Image.open("path_to_image")
5
run["train/distribution"].log(File(pil_image))
Copied!
You will have the images in the All metadata section of the Run UI, where you can browse and download them.

NumPy array

You can log NumPy arrays (2d or 3d) directly from the memory, and have them visualized as an image:
1
import numpy as np
2
from neptune.new.types import File
3
4
im_array = np.random.rand(100, 100, 3) * 255
5
run["np_image"].upload(File.as_image(im_array))
Copied!
You can also log a series of NumPy arrays:
1
from neptune.new.types import File
2
3
for epoch in range(data):
4
im_array = ...
5
run["train/distribution"].log(File.as_image(im_array))
Copied!
You will have the NumPy images in the All metadata section of the Run UI, where you can browse and download them.

TensorFlow tensor

You can log TensorFlow Tensors (2d or 3d) directly from the memory, and have them visualized as an image:
1
import tensorflow as tf
2
from neptune.new.types import File
3
4
tf_tensor = tensorflow.random.uniform(shape=[10, 20, 3])
5
run["tf_image"].upload(File.as_image(tf_tensorrray))
Copied!
You can also log a series of TensorFlow Tensors:
1
from neptune.new.types import File
2
3
for epoch in range(data):
4
tf_tensor = ...
5
run["train/distribution"].log(File.as_image(tf_tensor))
Copied!
You will have the TensorFlow Tensor images in the All metadata section of the Run UI, where you can browse and download them.

PyTorch tensor

You can log PyTorch Tensors (2d or 3d) directly from the memory, and have them visualized as an image:
1
import torch
2
from neptune.new.types import File
3
4
torch_tensor = torch.rand(30, 30)
5
run["torch_tensor"].upload(File.as_image(torch_tensor))
Copied!
You can also a log series of PyTorch Tensors:
1
from neptune.new.types import File
2
3
for epoch in range(data):
4
torch_tensor = ...
5
run["train/distribution"].log(File.as_image(torch_tensor))
Copied!
You will have the PyTorch Tensor images in the All metadata section of the Run UI, where you can browse and download them.

Interactive visualizations

You can log interactive charts and they will be rendered interactively in the Neptune UI. You will find it in the All metadata section of the Run UI.
Common visualization libraries are supported:
For a full-screen view, you can open visualization in the new browser tab, by clicking on the “arrow-pointing-top-right” icon, located right above your visualization.

HTML file

You can log any HTML file, and it will be rendered in the UI.
1
run["html"] = neptune.types.File("button_example.html")
Copied!
The HTML will appear in the All metadata section of the Run UI.

Matplotlib figure

This option is tested with matplotlib==3.2.0 and plotly==4.12.0. Make sure that you have the correct versions installed. See the plotly installation guide.
You can log Matplotlib figure (matplotlib.figure.Figure) as an interactive visualization, by using neptune.types.File.as_html():
1
from neptune.new.types import File
2
3
fig = ...
4
run["visuals/matplotlib-fig"] = File.as_html(fig)
Copied!
The interactive chart will appear in the All metadata section of the Run UI, where you can explore, open in full-screen and download it.
For more information, see How to use Neptune and Matplotlib.

Altair chart

You can log Altair charts as interactive visualizations, by using neptune.types.File.as_html():
1
from neptune.new.types import File
2
3
fig = ...
4
run["visuals/altair-fig"] = File.as_html(fig)
Copied!
The interactive Altair chart will appear in the All metadata section of the Run UI, where you can explore, open it in full screen, and download it.

Bokeh plot

You can log Bokeh plots as interactive visualizations, by using neptune.types.File.as_html():
1
from neptune.new.types import File
2
3
fig = ...
4
run["visuals/bokeh-fig"] = File.as_html(fig)
Copied!
The interactive Bokeh plot will appear in the All metadata section of the Run UI, where you can explore, open in full-screen and download it.

Plotly figure

You can log Plotly figures (2D or 3D) as interactive visualizations, by using neptune.types.File.as_html():
1
from neptune.new.types import File
2
3
fig = ...
4
run["visuals/plotly-fig"] = File.as_html(fig)
Copied!
Below is an example of how to log a 3D point cloud created using Plotly to Neptune:
The interactive Plotly figure will appear in the All metadata section of the Run UI, where you can explore, open in full-screen and download it.
For more information, see How to use Neptune and Plotly.

Notes and comments

You can add notes and comments to your Runs by using the description argument of neptune.init():
1
run = neptune.init(
2
description="neural net trained on Fashion-MNIST with high LR and low dropout"
3
)
Copied!
Run description appears in the All metadata > sys section of the Run UI and it can also be added as a column to the Runs Table.
You can also edit the description directly in the Neptune UI.

Tags

You can add tags to your Runs to organize them in a meaningful way:
  • Specify tags at Run initialization
1
run = neptune.init(tags=["maskRCNN", "finetune", "prod_v2.0.1"])
Copied!
  • Add tags to a Run
1
run["sys/tags"].add("maskRCNN")
2
run["sys/tags"].add(["finetune", "prod_v2.0.1"])
Copied!
Tags are a convenient way to organize or group runs. They appear in the All metadata > sys section of the Run UI and can be also be added as columns to the Runs Table.
You can also edit tags directly in the Neptune UI.

Text

You can log textual information to the Run by using:
1
# Log single String
2
run["aux/text"] = "text I keep track of, like query or tokenized word"
3
4
# Log series of String to one log
5
for epoch in range(epochs_nr):
6
token = str(...)
7
run["train/tokens"].log(token)
Copied!
You will have it in the All metadata section of the Run UI, where you can browse and download it.
A single line of text log is limited to 1000 characters but the number of lines is not limited. To log longer texts split your text into lines of under 1000 characters.

Video

Neptune currently supports MP4, OGG, OGV, MOV, M4V, MKV, and WEBM formats for video.
You can log video files and watch them in the Neptune UI.
1
# Log video file from disk
2
run["video"] = neptune.types.File("/path/to/video-file.mp4")
Copied!
A video player will be rendered in the All metadata section of the Run UI, where you can watch, open in full screen and download it.

Audio

Neptune currently supports MP3, M4A, OGA, and WAVE formats for audio.
Log audio files and listen to them right in the UI.
1
# Log audio file from disk
2
run["audio"] = neptune.types.File("/path/to/audio-file.mp3")
Copied!
An audio player will be rendered in the All metadata section of the Run UI, where you can watch, open it in full screen, and download it.

Tables

When you log tabular data, such as CSV files or Pandas DataFrames, Neptune will display it as a table.

CSV files

In the Neptune app, CSV files are displayed as an interactive table.
To record a CSV file, use the upload() method:
1
run["test/preds"].upload("path/to/test_preds.csv")
Copied!
The CSV file is displayed in the All metadata section of the single run view.
Table view of a CSV file
To adjust the number of displayed rows, use the drop-down menu in the bottom-right corner.
By switching to the Raw data tab, you can view and copy the raw contents of the file.

Pandas DataFrame

You can log Pandas DataFrames and have them displayed as a table in the Neptune UI.
1
from neptune.new.types import File
2
3
# Pandas DataFrame
4
run["data/sample"].upload(File.as_html(sample_df))
Copied!
The Pandas DataFrame is displayed in the All metadata section of the Run UI.

What's next?