How to query metadata via the Neptune API#
You can also download metadata through the Neptune web app.
With the Neptune API, you can query and fetch your logged metadata to the local machine.
You can download metadata logged in your project in bulk, or extract data from individual objects.
This page outlines some common ways to access typical metadata. For the full list of ways to interact with metadata fields, see Field types reference.
Downloading runs table as DataFrame#
The full project name. For example,
To copy it, navigate to the project settings in the top-right () and select Edit project details.
The same applies to models stored in the model registry. Fetch them with
runs_table_df is a pandas DataFrame, where
- each row represents a run.
- each column represents a metadata field, such as metrics, text logs, and parameters.
To include only certain fields as columns in the table, you can specify the names of namespaces or fields with the
# Fetch table of runs, including only the "f1_score" and "sys/running_time" # fields as columns >>> filtered_runs_table = project.fetch_runs_table( ... columns=["f1_score", "sys/running_time"] ... ) >>> filtered_runs_df = filtered_runs_table.to_pandas() >>> print(filtered_runs_df) sys/id sys/running_time f1_score 0 CLS-8 5.436 0.95 1 CLS-7 12.342 0.92 2 CLS-6 318.538 0.80 3 CLS-5 9.560 0.80 ...
- API reference ≫
Querying metadata from object#
To query metadata from a given run or other object, you fetch or download the metadata in a similar fashion as you logged it.
The methods correspond in the following way:
|Logging method||Querying method||Field type to use on|
||Any single value field except (
For the full list, see API reference ≫ Field types.
Initializing existing object#
Start by initializing Neptune with the ID of the object:
How do I find the ID?
The Neptune ID is a unique identifier for the object. In the table view, it's displayed in the leftmost column.
The ID is stored in the system namespace (
sys/id). If the object is active, you can obtain its ID with
neptune_object["sys/id"].fetch(). For example:
Setting the mode to "read only" ensures that no data is added or changed.
Querying single value#
fetch() method to query any single-valued metadata of the object:
The same method works for fields in the system namespace:
Fetching artifact metadata#
You can access the tags of an object by using
fetch() on the
Querying values from series field#
For value series logged with the
log()) method, you can query either the last value or the full list of values.
Retrieve all the values and their indexes as a pandas DataFrame with
Similarly, from FileSeries fields, you can download the last file to the disk with
download_last() or download all files with
download() to query uploaded files:
You can also download files from artifact fields:
For details, see Download artifact metadata.
Query nested namespaces or fields#
To query namespaces or fields that are nested under another namespace, you can access the object structure with
To get the monitoring sub-namespaces, which are generated automatically by Neptune for each process: