Log metadata with Neptune

To log metadata, use:

• log_configs() for single values.
• log_metrics() for a series of numerical values.

Pass the metadata as a dictionary {key: value} with

• key: path to where the metadata should be stored in the run.
• value: a numerical, string, datetime, or Boolean value. If logging metrics, a float or int value to append to the series.

In the attribute path, each forward slash (/) nests the attribute under a namespace. Use namespaces to structure the metadata into meaningful categories.

Example

from neptune_scale import Run


if __name__ == "__main__":
    run = Run(experiment_name=...)

    run.log_configs(
        {
            "parameters/use_preprocessing": True,
            "parameters/learning_rate": 0.001,
            "parameters/batch_size": 64,
            "parameters/optimizer": "Adam",
        }
    )

    for step in epoch:
        # your training loop
        run.log_metrics(
            data={
                "train/accuracy": 0.87,
                "train/loss": 0.14,
            }
            step=step,
        )

In the above example, the attributes are organized into two namespaces:

• parameters, containing all the config attributes
• train, containing all the metric attributes

A sensible structure makes it more convenient to work with the metadata later.

Related

• Logging from different processes
• Log configs – save single values and display them in the runs table, custom dashboards, or reports.
• Log metrics – log metrics and losses as series of values and visualize them as charts.
• Histograms – log histograms as series with steps.
• Attribute types – the type determines how you can display and interact with the attribute in Neptune.
• API cheat sheet

Date and time

You can track dates and times by assigning a Python datetime object to an attribute of your choice.

Logging custom training end time

from datetime import datetime

run = Run(...)
run.log_configs(
    {"train/end": datetime.now()}
)

Logging other time-related metadata

run.log_configs(
    {"dataset/creation_date": datetime.fromisoformat("1998-11-01")}
)

note

Any datetime values that don't have the tzinfo attribute set are assumed to be in the local timezone.