Skip to content

Namespaces and fields#

In a Neptune object, such as a run, fields and namespaces provide a folder-like structure for the metadata you log.

  • A field represents a location for storing a piece of metadata.
  • Fields can be organized under namespaces.

A field can be flat (stored at the root of the object) or nested (stored under one or more namespaces).

object
|-- namespace
    |-- field
        |-- Float
    |-- field
        |-- FileSeries
|-- namespace
    |-- namespace
        |-- field
            |-- String
        |-- field
            |-- StringSeries

Example:

Python code
# Flat
run["f1_score"] = 0.66
run["model"].upload("model.pkl")

# Nested
run["params/batch_size"] = 64

# Nested twice
run[
    "params/optimization/learning_rate"
] = 0.001
run[
    "params/optimization/algorithm"
] = "Adam"
Resulting metadata structure
run
|-- "f1_score"
    |-- 0.66
|-- "model"
    |-- model.pkl
|-- "params"
    |-- "batch_size"
        |-- 64
    |-- "optimization"
        |-- "learning_rate"
            |-- 0.001
        |-- "algorithm"
            |-- "Adam"

Field types and limitations#

Fields can contain values of various types: strings, dictionaries, floats, files, or artifacts. A field can also contain a series of values.

Namespaces (folders) can contain multiple fields of various types, but a field is always fixed to a type.

Good to know

When you assign some metadata to a new path in the Neptune object, that creates a field of a certain type.

If you reassign a value of the same type, it'll override the previous value stored in the field. Attempting to assign data of a different type to an existing field will result in an error.

Example
# Create new Float field
run["parameters/momentum"] = 0.9

# Update the value of the field
run["parameters/momentum"] = 0.8

# It's no longer possible to store a File under a Float field
run["parameters/momentum"].upload("sample.csv")  # Error

In the above example, in order to upload a file, you'd need to change the "parameters/momentum" path to something else. You could keep the "parameters" namespace, but at minimum, "momentum" should be changed to something different.

There are several options for storing metadata, such as simple assignment (=) for single values, upload() for uploading files, and track_files() for tracking file metadata, to name a few. Each yields a different field type.

The most suitable logging method depends on:

  • The data type of metadata you're logging.
  • The options you want for visualizing, storing, and interacting with the data in Neptune.

    Example: For large files, you may want to simply track their metadata rather than upload them in full.

Learn more: