Migrate from Old Fetcher to Query API
To use the new Query API instead of the old Fetcher API, you need to update your scripts.
The following sections show the code differences side by side.
This guide is about the first experimental neptune-fetcher
API for Neptune 3.x
.
To migrate from the newer neptune_fetcher.alpha
to neptune_query
, see Migrate from Fetcher Alpha to Query API.
Installation
pip install neptune-fetcher
pip install "neptune-query>=1.2.0,<2.0.0"
Imports
from neptune_fetcher import (
ReadOnlyProject,
ReadOnlyRun,
)
import neptune_query as nq
For more advanced filtering than name search, import filters from the filters
module:
from neptune_query.filters import (
Attribute,
AttributeFilter,
Filter,
)
To work with runs instead of experiments, import the runs
module:
import neptune_query.runs as nq_runs
Setting the project and API token
project = ReadOnlyProject(
project="team-alpha/project-x",
api_token="h0dHBzOi8aHR0cHM6...Y2MifQ==",
)
any_fetching_method(
project="team-alpha/project-x",
...
)
In the Query API, the context is set automatically from environment variables. You can use the above method to override the default project.
Listing experiments or runs of the project
project = ReadOnlyProject()
for exp in project.list_experiments():
print(exp)
for run in project.list_runs():
print(run)
nq.list_experiments()
nq_runs.list_runs()
In the Query API, you can also use
list_attributes()
for experiments or runs.
Fetching metadata as table
You can use a filter object to set criteria for the experiments to search, or for the attributes to include as columns in the returned table.
project.fetch_experiments_df()
nq.fetch_experiments_table()
Match experiment names:
project.fetch_experiments_df(
names_regex=r"exp.*"
)
nq.fetch_experiments_table(r"exp.*")
Match run IDs:
project.fetch_experiments_df(
custom_id_regex=r"run.*"
)
nq_runs.fetch_runs_table(r"run.*")
Filter by metadata value:
project.fetch_experiments_df(
query="(last(`acc`:floatSeries) > 0.88)"
)
nq.fetch_experiments_table(
experiments=Filter.gt("acc", 0.88)
)
Specify columns to include:
project.fetch_experiments_df(
columns_regex=r"acc"
)
nq.fetch_experiments_table(
attributes=r"acc"
)
Fetching metric values
With alpha
, you can fetch metric values from multiple experiments or runs at once.
# For each experiment or run
run = ReadOnlyRun(
ReadOnlyProject(...),
experiment_name="exp123",
)
run.prefetch(["loss"])
run["loss"].fetch_values()
nq.fetch_metrics(
experiments=r"exp.*",
attributes=r"loss",
)
Fetching a config or single value
To fetch the f1
attribute value from a particular experiment:
run = ReadOnlyRun(
ReadOnlyProject(...),
experiment_name="exp123",
)
f1_value = run["f1"].fetch()
f1_value = nq.fetch_experiments_table(
experiments=["exp-123"],
attributes=["f1"],
).iat[0, 0]
Using the old API
For how to use the old version of the Neptune Fetcher API, see: