Skip to content

Log system metrics#

The following is logged automatically by default:

  • Hardware consumption: CPU, GPU (only NVIDIA), and memory.
  • Console logs: stdout and stderr.

You'll find these in the monitoring namespace of each run.

The metrics are organized into subfolders per process – monitoring/<hash>/ – where the hash is calculated based on environment information, to ensure that it's unique for each process.

How to disable

To turn off logging of system metrics, use one or more of the following parameters with init_run():

run = neptune.init_run(
    capture_hardware_metrics=False,
    capture_stdout=False,
    capture_stderr=False,
)

See in Neptune 

Setting a custom monitoring namespace name#

You can provide your custom monitoring namespace name by passing it to the monitoring_namespace argument at initialization:

Example
import neptune

run = neptune.init_run(monitoring_namespace="my_namespace_name")

This custom name will replace the monitoring/<hash>/ pattern. If you're logging metrics from multiple processes, ensure that your custom name is unique for each process.

Logging Python Logger output#

You can also capture logs from a Python Logger .

Create and add a Neptune handler to your Logger:

import logging
from neptune.integrations.python_logger import NeptuneHandler

logger = logging.getLogger("my_python_logger")

logger.addHandler(NeptuneHandler(run=run))  # a previously initialized run

logger.debug("Starting training")

Tip

You can customize what level of logs you want to capture or the name of the field where the logs will be stored.

For details, see API reference → Python Logger.

Enabling monitoring in interactive sessions#

By default, monitoring of system metrics is turned off for interactive Python kernels, such as Jupyter notebooks. This includes logging of hardware consumption and standard streams (stdout and stderr).

To turn it on, you need to explicitly set the related initialization parameters to a value of True:

>>> import neptune
>>> run = neptune.init_run(
...     capture_hardware_metrics=True,
...     capture_stderr=True,
...     capture_stdout=True,
... )
Relation to pricing

We changed this with neptune 1.0.0 to avoid unintentionally using up logging hours in the background during interactive sessions.

This is no longer relevant for workspaces that follow new pricing model, which is based on the number of active projects.