documentation
ExamplesAPI referenceIntegrations
Search…
Home
Changelog
Getting Started
Installation and setup
Quickstart
Examples
How to add Neptune to your code
Getting help
You should know
Core concepts
Logging metadata
Displaying metadata
What can you log and display?
Best practices
Comparing runs
Organizing and filtering Runs
Querying and downloading metadata
Connection modes
Collaboration in Neptune
How-to guides
Experiment tracking
Model registry
Model monitoring
CI/CD
Data versioning
Neptune API
Neptune UI
Integrations and Supported Tools
Introduction
Languages
Model training
Hyperparameter Optimization
Model visualization and debugging
Automation pipelines
Experiment Tracking
IDEs and notebooks
Model monitoring
API reference
API reference guide
Neptune
Project
Run
System ("sys") namespace
Model
Model version
Field types
Management
Integrations
Environment variables
Command Line Interface
Administration
Workspaces
Projects
User management
Service accounts
Logging in to Neptune with a third-party account
Deploying Neptune on your server (on-premises)
Troubleshooting
Waiting for remaining operations
Max string attribute length exceeded
Float attribute value malformed
Links
Neptune Website
Neptune App
Legacy Neptune Documentation
Migrating to neptune.new
Powered By GitBook
What can you log and display?
Neptune supports the logging and display for many different types of model building metadata.
On this page, you can see how each ML metadata type can be logged via Neptune API and how it will be displayed in the Neptune UI.
Supported ML metadata types include:
Remember to create a Run before you start logging:
import neptune.new as neptune
​
run = neptune.init(api_token="YOUR_API_TOKEN", project="YOUR_WORKSPACE/YOUR_PROJECT")

Metrics and losses

Log metrics and losses to Neptune using =and .log() methods:
# logging score (single value)
run["score"] = 0.97
run["test/acc"] = 0.97
​
# logging metric (series of values)
for epoch in range(100):
acc = ...
loss = ...
metric = ...
​
run["train/accuracy"].log(acc)
run["train/loss"].log(loss)
run["metric"].log(metric)
Metrics and losses in Neptune
Metric can be accuracy, MSE, or any numerical value. All metrics are visualized as Charts in the Run UI. You can also check grouped values in the All Metadata section of the Run UI and add metrics as columns to the Runs Table.

Parameters and Model Configuration

Log parameters one-by-one to Fields of a Run.
run["parameters/epoch_nr"] = 5
run["parameters/batch_size"] = 32
run["parameters/dense"] = 512
run["parameters/optimizer"] = "sgd"
run["parameters/metrics"] = ["accuracy", "mae"]
run["parameters/activation"] = "relu"
Define parameters as a Python dictionary and log them all at once to a Namespace of a Run.
# Define parameters
PARAMS = {
"epoch_nr": 5,
"batch_size": 32,
"dense": 512,
"optimizer": "sgd",
"metrics": ["accuracy", "binary_accuracy"],
"activation": "relu",
}
​
# Pass parameters
run["parameters"] = PARAMS
Logged parameters appear in the All metadata section of a single Run UI:
Parameters in the single Run UI
You can also add parameters as columns to the Runs Table:
Parameters in the Runs Table

Model checkpoints

You can log model checkpoints to a Run.
# Log PyTorch model weights
my_model = ...
torch.save(my_model, "my_model.pt")
run["model_checkpoints/my_model"].upload("model_checkpoints/my_model.pt")
You can save model weights from any deep learning framework. Model checkpoints will appear in the All metadata section of the Run UI, for example in the ‘model_checkpoints’ Namespace.

Code

Neptune supports code versioning in three different ways.

Git information

If you start a Run from a directory that is a part of the git repo, Neptune will automatically find the .git directory and log information from it.
It creates a summary in the source_code/git section of the Run UI with the following Fields:
  • Dirty: whether the commit was dirty or not
  • Commit message: what was the commit message
  • Commit ID: Git SHA signature
  • Commit author: the author of the last commit
  • Commit date: the data of the last commit
  • Current branch: what is the current git branch
  • Remotes: list of git remotes your local .git is connected to

Code snapshot

Neptune automatically snapshots code when you create a run and execute the script.
By default, Neptune saves the name and contents of the entry-point file (e.g.main.py if that's the script you execute).
You can snapshot code from Jupyter notebooks with the help of an extension. For instructions, see JupyterLab and Jupyter Notebook. Neptune can't snapshot code from other interactive sessions, such as Google Colab.
The snapshotted files are listed in the Source code section of the run view.

Customizing which files to snapshot

You can pass a list of files or wildcard patterns to specify exactly which files to snapshot:
Example
# Snapshot model.py and prep_data.py
run = neptune.init(..., source_files=["model.py", "prep_data.py"])
​
# Snapshot all python files and 'config.yaml' file
run = neptune.init(..., source_files=["**/*.py", "config.yaml"])
When using pattern expansion, such as "**/*.py", make sure that your expression does not log too many files or non-source code files.
For example, using "*" as a pattern will upload all files and directories from the current working directory (cwd). It may result in logging files that you did not want to upload and cluttering your project.
You can also use the upload_files() method to log file contents manually, for example, if you need a customized structure or you're working with non-run objects:
Example
model_version["source_files"].upload_files("**/*.py")

Turning off source code logging

You can turn off logging of source code by passing an empty list to the source_files argument of neptune.init():
run = neptune.init(..., source_files=[])

Notebook code snapshot

Neptune can automatically snapshot your Jupyter notebook checkpoint every time you create a Run in that notebook.
You can also log a Jupyter notebook checkpoint without creating a Neptune Run to log your Exploratory Data Analysis (EDA) or other work.
See Integration with Jupyter Notebook and Jupyter Lab.

Data Versions

You can log and display metadata about datasets, models, and other artifacts in Neptune.
The hash depends on the file contents and metadata like path, size, and last modification time. Any change to any of these will result in a different hash, even if the file contents are exactly the same. Read more about this here.

Local filesystem

You can version artifacts stored on your local filesystem with Neptune. For each artifact you version you will see:
  • URL: a path of an artifact on your local filesystem
  • Hash: md5 signature of your artifact
To use artifacts you need neptune-client>=0.10.10
pip install neptune-client>=0.10.10
To do that, use .track_files() method passing a path to file or folder:
# single file
run["train_dataset"].track_files("./datasets/train.csv")
​
# Folder
run["train/images"].track_files("./datasets/images")
Go to the All metadata section of the Run UI and find your data versions and other dataset metadata.
Viewing versioned local files as Artifacts in the Neptune UI

S3 compatible storage

You can version datasets or models stored on S3 compatible storage ("s3://..." ) including AWS S3, GCP, MinIO, and others in Neptune. For each artifact you version you will see:
  • URL: a path of an artifact on your local filesystem
  • Hash: md5 signature of your artifact
To use artifacts you need neptune-client>=0.10.10
pip install neptune-client>=0.10.10
To do that, use .track_files() method passing a path to file or folder:
# single file
run["train_dataset"].track_files("s3://datasets/train.csv")
​
# Folder
run["train/images"].track_files("s3://datasets/images")
Remember to pass credentials as environment variables.
For example, on AWS S3 you should configure an IAM group policy with S3ReadAccessOnly permissions. Then export the user access keys:
export AWS_SECRET_ACCESS_KEY='YOUR_AWS_KEY'
export AWS_ACCESS_KEY_ID='YOUR_AWS_ID'
Viewing versioned S3 files as Artifacts in the Neptune UI

Data Version Control (DVC)

Neptune displays DVC files in the UI, you just need to specify which DVC files you would like to log.
For example, to log all the DVC files from your working directory and all subdirectories pass "**/*.dvc" to the source_files argument of neptune.init():
# Snapshot all .dvc files from any directory
run = neptune.init(..., source_files=["**/*.dvc"])

Hardware consumption

Automatically monitor hardware utilization for your Runs:
  • cpu: total CPU consumption of your machine
  • memory: total memory consumption of your machine
  • gpu: GPU consumption of your machine
  • gpu_memory: GPU memory consumption of your machine
  • stderr: standard error logs from your console
  • stdout: standard output logs from your console
All that information is visualized in the Monitoring section of the Run UI.
To turn off the logging of hardware consumption use capture_hardware_ metrics argument of neptune.init():
# Turn off hardware monitoring
run = neptune.init(..., capture_hardware_metrics=False)
To log hardware consumption to a different Namespace use monitoring_namespace argument argument of neptune.init():
# Assign custom name to the monitoring namespace
run = neptune.init(..., monitoring_namespace="worker-7-monitoring")
If you see "Info (NVML): NVML Error: NVML Shared Library Not Found - GPU usage metrics may not be reported." your GPU consumption is not being logged.
It means that either:
  • there are no GPUs on your machine
  • your NVIDIA NVML library is not installed or configured properly. See how to install and configure NVML.
Logging GPU on Windows
On Windows, Neptune searches for the nvml.dll file in the standard locations:
  • C:\Program Files\NVIDIA Corporation\NVSMI\nvml.dll
  • C:\Windows\System32\nvml.dll
If you are having trouble logging GPU metrics on Windows, please check that your NVML installation is correct and that you have the nvml.dll file in either of those locations.
Alternatively, you can set a custom location of nvml.dll on Windows by setting the NVML_DLL_PATH environment variable.

Files

You can log any file or directory you want to Neptune by using .upload()and .upload_files()methods:
# Log file
run["aux/data"].upload("auxiliary-data.zip")
​
# Save multiple files under a path
run["config_files"].upload_files([path_to_config_1, path_to_config_2])
​
# You can also use wildcard patterns ("glob") with .upload_files()
run["preprocessing_scripts"].upload_files("./preprocessing/*.py")
You can browse and download files in the All metadata section of the Run UI.
Files are displayed in the UI based on the file format:
  • PNG, JPG, JPEG, GIF, BMP, and WebP image formats are displayed as Images​
  • HTML files are displayed as Interactive visualizations​
  • MP4, OGG, OGV, MOV, M4V, MKV, WEBM are displayed as players for Video​
  • MP3, M4A, OGA, WAVE are displayed as players for Audio​
  • CSV files are displayed as Tables​
  • IPYNB files are displayed as fully rendered Notebooks
  • Text files and source code files are displayed as text with appropriate syntax coloring if applicable
Make sure that you define the correct path to files that you want to upload when using wildcards like '*.png' or '**/*.yaml' as it may result in the unintended logging of a large amount of data and cluttering your storage.
You can delete files from a Run by using .pop()method:
run.pop("aux/data")

Console logs

stdout and stderr

Automatically save console logs for your Runs :
  • stderr: standard error logs from your console
  • stdout: standard output logs from your console
All that information is visualized in the Monitoring section of the Run UI.
To turn off the saving of console logs use capture_stdout capture_stderarguments of neptune.init():
# Turn off saving console logs
run = neptune.init(
...,
capture_stdout=False,
capture_stderr=False,
)

Python's Logger

You can also capture Python's Logger logs. Simply create and add Neptune handler to your Logger object:
import logging
from neptune.new.integrations.python_logger import NeptuneHandler
​
logger = logging.getLogger("my_python_logger")
​
logger.addHandler(NeptuneHandler(run=run))
​
logger.debug("Starting training")
When initializing the NeptuneHandler you can customize what level of logs you want to capture or the name of the field where the logs will be stored.

Images

You can log either a single image or a series of images to Neptune.
Several data formats are available:
In all cases, you will have images in the All metadata section of the Run UI (under Namespace defined by you), where you can browse and download them.
To log images use .upload() or .log(File) methods:
from neptune.new.types import File
​
# single image
run["train/bounding_boxes"].upload("bboxes.png")
​
# for series of images
for i in range(valid_nr):
ith_path = ...
run["valid/misclassified"].log(File(ith_path))
You can log an unlimited number of images either in the single image log or as a series of image logs.

Image file

Neptune currently supports PNG, JPEG, JPG, GIF, BMP, and WebP formats for image files.
You can log image format files directly from the disk.
Log single image from disk with the .upload() method:
run["train/bounding_boxes"].upload("bboxes.png")
Log a series of images in a for loop with the .log(File) method:
from neptune.new.types import File
​
for name in misclassified_images_names:
y_pred = ...
y_true = ...
run["misclassified_imgs"].log(File("misclassified_image.png"))
Notice the difference between .upload() for single images and .log() for series of images.

Matplotlib figure

Log a Matplotlib figure (matplotlib.figure.Figure) as an image:
# Import matplotlib
import matplotlib.pyplot as plt
​
# Generate figure
fig = plt.figure(figsize=(7, 9))
...
​
# Log figure to run
run["matplotlib-fig"].upload(fig)
You can also log a series of Matplotlib figures:
for epoch in range(params["iterations"]):
plt_fig = get_histogram()
run["train/distribution"].log(plt_fig)
You will have the Matplotlib figures displayed in the All metadata section of the Run UI, where you can browse and download them.
For more information, see How to use Neptune and Matplotlib.

Seaborn figure

Log a Seaborn figure as an image:
# Import Seaborn
import seaborn as sns
​
# Generate chart
seaborn_fig = ...
​
# Convert Seaborn object to Matplotlib format (matplotlib.figure.Figure)
figure = seaborn_fig.fig
​
# Log figure to run
run["seaborn-img"].upload(figure)
You will have the Seaborn figure displayed in the All metadata section of the Run UI, where you can browse and download it.

PIL Image

You can log a PIL image directly from the memory:
image1 = Image.open("Representation-learning.jpg")
image2 = Image.open("RL-agents-1.jpg")
​
# Log image to run
run["representation_learning"].upload(image1)
run["reinforcement_learning"].upload(image2)
You can also log a series of PIL images:
for epoch in range(images_nr):
pil_image = Image.open("path_to_image")
run["train/distribution"].log(pil_image)
You will have the images in the All metadata section of the Run UI, where you can browse and download them.

NumPy array

You can log NumPy arrays (2d or 3d) directly from the memory, and have them visualized as an image:
import numpy as np
from neptune.new.types import File
​
im_array = np.random.rand(100, 100, 3) * 255
run["np_image"].upload(File.as_image(im_array))
You can also log a series of NumPy arrays:
from neptune.new.types import File
​
for epoch in range(data):
im_array = ...
run["train/distribution"].log(File.as_image(im_array))
You will have the NumPy images in the All metadata section of the Run UI, where you can browse and download them.

TensorFlow tensor

You can log TensorFlow Tensors (2d or 3d) directly from the memory, and have them visualized as an image:
import tensorflow as tf
from neptune.new.types import File
​
tf_tensor = tensorflow.random.uniform(shape=[10, 20, 3])
run["tf_image"].upload(File.as_image(tf_tensorrray))
You can also log a series of TensorFlow Tensors:
from neptune.new.types import File
​
for epoch in range(data):
tf_tensor = ...
run["train/distribution"].log(File.as_image(tf_tensor))
You will have the TensorFlow Tensor images in the All metadata section of the Run UI, where you can browse and download them.

PyTorch tensor

You can log PyTorch Tensors (2d or 3d) directly from the memory, and have them visualized as an image:
import torch
from neptune.new.types import File
​
torch_tensor = torch.rand(30, 30)
run["torch_tensor"].upload(File.as_image(torch_tensor))
You can also a log series of PyTorch Tensors:
from neptune.new.types import File
​
for epoch in range(data):
torch_tensor = ...
run["train/distribution"].log(File.as_image(torch_tensor))
You will have the PyTorch Tensor images in the All metadata section of the Run UI, where you can browse and download them.

Interactive visualizations

You can log interactive charts and they will be rendered interactively in the Neptune UI. You will find it in the All metadata section of the Run UI.
Common visualization libraries are supported:
For a full-screen view, you can open visualization in the new browser tab, by clicking on the “arrow-pointing-top-right” icon, located right above your visualization.

HTML file

You can log any HTML file, and it will be rendered in the UI.
run["html"].upload("button_example.html")
The HTML will appear in the All metadata section of the Run UI.

Matplotlib figure

This option is tested with matplotlib==3.2.0 and plotly==4.12.0. Make sure that you have the correct versions installed. See the plotly installation guide.
You can log Matplotlib figure (matplotlib.figure.Figure) as an interactive visualization, by using neptune.types.File.as_html():
from neptune.new.types import File
​
fig = ...
run["visuals/matplotlib-fig"].upload(File.as_html(fig))
The interactive chart will appear in the All metadata section of the Run UI, where you can explore, open in full-screen and download it.
For more information, see How to use Neptune and Matplotlib.

Altair chart

You can log Altair charts as interactive visualizations with the upload() method:
fig = ... # Altair chart
run["visuals/altair-fig"].upload(fig)
The interactive Altair chart will appear in the All metadata section of the Run UI, where you can explore, open it in full screen, and download it.

Bokeh plot

You can log Bokeh plots as interactive visualizations with the upload() method:
fig = ... # Bokeh chart
run["visuals/bokeh-fig"].upload(fig)
The interactive Bokeh plot will appear in the All metadata section of the Run UI, where you can explore, open in full-screen and download it.

Plotly figure

You can log Plotly figures (2D or 3D) as interactive visualizations with the upload() method:
fig = ... # Plotly chart
run["visuals/plotly-fig"].upload(fig)
Below is an example of how to log a 3D point cloud created using Plotly to Neptune:
The interactive Plotly figure will appear in the All metadata section of the Run UI, where you can explore, open in full-screen and download it.
For more information, see How to use Neptune and Plotly.

Notes and comments

You can add notes and comments to your Runs by using the description argument of neptune.init():
run = neptune.init(
description="neural net trained on Fashion-MNIST with high LR and low dropout"
)
Run description appears in the All metadata > sys section of the Run UI and it can also be added as a column to the Runs Table.
You can also edit the description directly in the Neptune UI.

Tags

You can add tags to your Runs to organize them in a meaningful way:
  • Specify tags at Run initialization
run = neptune.init(tags=["maskRCNN", "finetune", "prod_v2.0.1"])
  • Add tags to a Run
run["sys/tags"].add("maskRCNN")
run["sys/tags"].add(["finetune", "prod_v2.0.1"])
Tags are a convenient way to organize or group runs. They appear in the All metadata > sys section of the Run UI and can be also be added as columns to the Runs Table.
You can also edit tags directly in the Neptune UI.

Text

You can log textual information to the Run by using:
# Log single String
run["aux/text"] = "text I keep track of, like query or tokenized word"
​
# Log series of String to one log
for epoch in range(epochs_nr):
token = str(...)
run["train/tokens"].log(token)
You will have it in the All metadata section of the Run UI, where you can browse and download it.
A single line of text log is limited to 1000 characters but the number of lines is not limited. To log longer texts split your text into lines of under 1000 characters.

Video

Neptune currently supports MP4, OGG, OGV, MOV, M4V, MKV, and WEBM formats for video.
You can log video files and watch them in the Neptune UI.
# Log video file from disk
run["video"].upload("/path/to/video-file.mp4")
A video player will be rendered in the All metadata section of the Run UI, where you can watch, open in full screen and download it.

Audio

Neptune currently supports MP3, M4A, OGA, and WAVE formats for audio.
Log audio files and listen to them right in the UI.
# Log audio file from disk
run["audio"].upload("/path/to/audio-file.mp3")
An audio player will be rendered in the All metadata section of the Run UI, where you can watch, open it in full screen, and download it.

Tables

When you log tabular data, such as CSV files or Pandas DataFrames, Neptune will display it as a table.

CSV files

In the Neptune app, CSV files are displayed as an interactive table.
To record a CSV file, use the upload() method:
run["test/preds"].upload("path/to/test_preds.csv")
The CSV file is displayed in the All metadata section of the single run view.
Table view of a CSV file
To adjust the number of displayed rows, use the drop-down menu in the bottom-right corner.
By switching to the Raw data tab, you can view and copy the raw contents of the file.

Pandas DataFrame

You can log Pandas DataFrames and have them displayed as a table in the Neptune UI.
from neptune.new.types import File
​
# Pandas DataFrame
run["data/sample"].upload(File.as_html(sample_df))
The Pandas DataFrame is displayed in the All metadata section of the Run UI.

What's next?

Previous
Smoothing in Neptune charts
Next - You should know
Best practices
Last modified 11d ago
Export as PDF
Copy link
Outline
Metrics and losses
Parameters and Model Configuration
Model checkpoints
Code
Git information
Code snapshot
Notebook code snapshot
Data Versions
Local filesystem
S3 compatible storage
Data Version Control (DVC)
Hardware consumption
Files
Console logs
stdout and stderr
Python's Logger
Images
Image file
Matplotlib figure
Seaborn figure
PIL Image
NumPy array
TensorFlow tensor
PyTorch tensor
Interactive visualizations
HTML file
Matplotlib figure
Altair chart
Bokeh plot
Plotly figure
Notes and comments
Tags
Text
Video
Audio
Tables
CSV files
Pandas DataFrame
What's next?