Setting a base namespace#
You can organize your metadata into folder-like structures with the help of namespaces and handler objects.
If you assign a run namespace path to a variable, you get a namespace handler object. The namespace handler exposes similar methods as other Neptune objects – however, all field paths are relative to the namespace.
Start by creating a
Handler object and turn it into a namespace by organizing metadata inside it:
In the pipeline logging example below, we show how to use a handler to organize metadata from a particular step of a run into a given namespace.
Pipeline logging example#
Let's say we have a data preprocessing step in our pipeline, and we would like to ensure that all of the metadata generated during this part is organized under a folder named "preprocessing".
We would start a Neptune run early in the script:
To collect the metadata from this step into one place, we set up a
preprocessing namespace inside the run. This will be the base namespace where all the preprocessing metadata is logged.
We define the path
run["preprocessing"] and assign it to a handler object, which we'll then use to log metadata as if it were a run.
This way, whenever we log something to
preprocessing_handler, it'll end up under the path
Setting base namespace in integrations#
Most integrations automatically organize the metadata logged by the integration framework into a dedicated namespace.
You can usually customize the name of this namespace with the help of a parameter.
Even if there is no automatic namespace implementation, whenever you can pass a Neptune
Run object to a callback, you can instead pass a namespace handler.
For the exact implementations, check the specific Integrations and their API references.