Skip to main content
App version: 3.20250811

Query metadata

To query the logged metadata programmatically, use the Neptune Query API. You can:

  • Fetch experiment metadata as a data frame.
  • Define filters to query experiments, runs, and attributes that meet certain criteria.
  • List experiments, runs, and attributes of a project.

The extended regular expression syntax is supported for matching names and other string values.

Setup​

To install the Neptune Query API client:

pip install "neptune-query>=1.0.0,<2.0.0"

Set your Neptune API token and project name as environment variables:

In your .profile or other shell initialization file, use the export commands:

export NEPTUNE_API_TOKEN="h0dHBzOi8aHR0cHM.4kl0jvYh3Kb8...ifQ=="
export NEPTUNE_PROJECT="workspace-name/project-name"

For help, see Get started.

You can also pass the project path to the project argument of any querying function.

Usage​

import neptune_query as nq

Available functions:

note

You can also target individual runs by ID instead of experiment runs by name. To use the corresponding methods for runs, import the runs module:

import neptune_query.runs as nq_runs


nq_runs.fetch_metrics(...)

Example 1: Fetch metric values per step​

To fetch numerical values at each step, use fetch_metrics():

  • To specify or filter experiments to return, use the experiments parameter.
  • To specify attributes to include as columns, use the attributes parameter.
  • To limit the returned values, use the available parameters.
Fetch last 5 values from specific experiment's matching attributes
nq.fetch_metrics(
    experiments=["exp_dczjz"],
    attributes=r"metrics/val_.+_estimated$",
    tail_limit=5,
)
Sample output
                    metrics/val_accuracy_estimated  metrics/val_loss_estimated
experiment    step
exp_dczjz   1000.0                         0.72187                     0.14989
            1001.0                         0.72683                     0.14781
            1002.0                         0.76981                     0.14233
            1003.0                         0.76995                     0.13552
            1004.0                         0.78161                     0.12020

Example 2: Fetch metadata as one run per row​

To fetch experiment metadata from your project, use fetch_experiments_table(). The output mimics the runs table of the web app.

  • To specify or filter experiments to return, use the experiments parameter.
  • To specify attributes to include as columns, use the attributes parameter.
Fetch specific attribute values from matching experiments
nq.fetch_experiments_table(
    experiments=r"^exp_",
    attributes=["metrics/train_accuracy", "metrics/train_loss", "learning_rate"],
)
Sample output
            metrics/train_accuracy   metrics/train_loss   learning_rate
experiment
exp_ergwq                 0.278149             0.336344            0.01
exp_qgguv                 0.160260             0.790268            0.02
exp_hstrj                 0.365521             0.459901            0.01

For series attributes, the value of the last logged step is returned.

Example 3: Define filters​

List my experiments that have a "dataset_version" attribute and "val/loss" less than 0.1:

from neptune_query.filters import Filter


owned_by_me = Filter.eq("sys/owner", "sigurd")
dataset_check = Filter.exists("dataset_version")
loss_filter = Filter.lt("val/loss", 0.1)

interesting = owned_by_me & dataset_check & loss_filter
nq.list_experiments(experiments=interesting)
Sample output
['exp_ergwq', 'exp_qgguv', 'exp_hstrj']

Then fetch configs from the experiments, including also the interesting metric:

nq.fetch_experiments_table(
    experiments=interesting,
    attributes=r"config/ | val/loss",
)
Sample output
            config/optimizer  config/batch_size  config/learning_rate   val/loss
experiment
exp_ergwq               Adam                 32                 0.001     0.0901
exp_qgguv           Adadelta                 32                 0.002     0.0876
exp_hstrj           Adadelta                 64                 0.001     0.0891

For details, see Filter.