API reference: Optuna integration#
The Neptune-Optuna integration provides a NeptuneCallback
class and two utility functions.
NeptuneCallback
#
A callback that logs metadata from an Optuna study to Neptune.
With this callback, you can log and display:
- Values and parameters for each trial.
- Current best values and parameters for the study.
- Visualizations from the
optuna.visualizations
module. - Parameter distributions for each trial.
- The study object itself, to load it later.
Parameters
Name | Type | Default | Description |
---|---|---|---|
run |
Run or Handler |
- | (required) An existing run reference, as returned by neptune.init_run() , or a namespace handler. |
base_namespace |
str , optional |
"" |
Namespace under which all metadata logged by the Neptune callback will be stored. If omitted, defaults to an empty string, in which case the metadata is logged directly under the run, without a common "root" namespace. |
plots_update_freq |
int or str , optional |
1 |
Frequency at which plots are logged and updated in Neptune.
|
study_update_freq |
int or str , optional |
1 |
Frequency at which a study object is logged and updated in Neptune.
|
visualization_backend |
str , optional |
"plotly" |
Which visualization backend is used for optuna.visualizations plots. Can be set to either "matplotlib" or "plotly" . |
log_plot_contour |
bool , optional |
True |
Whether to log the optuna.visualizations.plot_contour visualization to Neptune. |
log_plot_edf |
bool , optional |
True |
Whether to log the optuna.visualizations.plot_edf visualization to Neptune. |
log_plot_parallel_coordinate |
bool , optional |
True |
Whether to log the optuna.visualizations.plot_parallel_coordinate visualization to Neptune. |
log_plot_param_importances |
bool , optional |
True |
Whether to log the optuna.visualizations.plot_param_importances visualization to Neptune. |
log_plot_pareto_front |
bool , optional |
True |
Whether to log the optuna.visualizations.plot_pareto_front visualization to Neptune.If your study is not multi-objective, this plot is not logged. |
log_plot_slice |
bool , optional |
True |
Whether to log the optuna.visualizations.plot_slice visualization to Neptune. |
log_plot_intermediate_values |
bool , optional |
True |
Whether to log the optuna.visualizations.plot_intermediate_values visualization to Neptune.If your study is not using pruners, this plot is not logged. |
log_plot_optimization_history |
bool , optional |
True |
Whether to log the optuna.visualizations.plot_optimization_history visualization to Neptune. |
target_names |
list of str |
None |
List of one or more study objective names to log (see example). |
Examples
Create a Neptune run and initialize a Neptune callback:
import neptune
import neptune.integrations.optuna as npt_utils
run = neptune.init_run()
neptune_callback = npt_utils.NeptuneCallback(run)
If Neptune can't find your project name or API token
As a best practice, you should save your Neptune API token and project name as environment variables:
export NEPTUNE_API_TOKEN="h0dHBzOi8aHR0cHM6Lkc78ghs74kl0jvYh3Kb8"
export NEPTUNE_PROJECT="ml-team/classification"
You can, however, also pass them as arguments when initializing Neptune:
run = neptune.init_run( # (1)!
api_token="h0dHBzOi8aHR0cHM6Lkc78ghs74kl0jvYh3Kb8", # your token here
project="ml-team/classification", # your full project name here
)
-
Also works for
init_model()
,init_model_version()
, andinit_project()
. -
API token: In the bottom-left corner, expand the user menu and select Get my API token.
- Project name: in the top-right menu: → Edit project details.
If you haven't registered, you can also log anonymously to a public project (make sure not to publish sensitive data through your code!):
Or optionally pass a list of objective names:
neptune_callback = npt_utils.NeptuneCallback(
run,
target_names=["FLOPS", "accuracy"],
)
Log single and multi objective study metadata to Neptune by passing the callback to the Optuna study:
study = optuna.create_study(direction="maximize")
study.optimize(objective, n_trials=5, callbacks=[neptune_callback])
log_study_metadata()
#
Logs the metadata from an Optuna study to Neptune.
With this function, you can log and display:
- Values and parameters for each trial.
- Current best values and parameters for the study.
- Visualizations from the
optuna.visualizations
module. - Parameter distributions for each trial.
- The study object itself, to load it later.
Parameters
Name | Type | Default | Description |
---|---|---|---|
study |
optuna.Study |
- | (required) An Optuna study object. |
run |
Run |
- | (required) An existing run reference, as returned by neptune.init_run() . |
base_namespace |
str , optional |
"" |
Namespace under which all metadata logged by the Neptune callback will be stored. If omitted, defaults to an empty string, in which case the metadata is logged directly under the run, without a common "root" namespace. |
log_plots |
bool , optional |
True |
Whether the visualizations from optuna.visualizations will be logged to Neptune. |
log_study |
bool , optional |
True |
Whether the study will be logged to Neptune. The objects that are logged depend on the study storage type used: If InMemoryStorage is used, the pickled study object will be logged to Neptune. Otherwise the database URL will be logged. |
log_all_trials |
bool , optional |
True |
Whether to log all trials. |
log_distributions |
bool , optional |
True |
Whether to log the distributions for all trials. |
visualization_backend |
str , optional |
"plotly" |
Which visualization backend is used for optuna.visualizations plots. Can be set to either "matplotlib" or "plotly" . |
log_plot_contour |
bool , optional |
True |
Whether to log the optuna.visualizations.plot_contour visualization to Neptune. |
log_plot_edf |
bool , optional |
True |
Whether to log the optuna.visualizations.plot_edf visualization to Neptune. |
log_plot_parallel_coordinate |
bool , optional |
True |
Whether to log the optuna.visualizations.plot_parallel_coordinate visualization to Neptune. |
log_plot_param_importances |
bool , optional |
True |
Whether to log the optuna.visualizations.plot_param_importances visualization to Neptune. |
log_plot_pareto_front |
bool , optional |
True |
Whether to log the optuna.visualizations.plot_pareto_front visualization to Neptune.If your study is not multi-objective, this plot is not logged. |
log_plot_slice |
bool , optional |
True |
Whether to log the optuna.visualizations.plot_slice visualization to Neptune. |
log_plot_intermediate_values |
bool , optional |
True |
Whether to log the optuna.visualizations.plot_intermediate_values visualization to Neptune.If your study is not using pruners, this plot is not logged. |
log_plot_optimization_history |
bool , optional |
True |
Whether to log the optuna.visualizations.plot_optimization_history visualization to Neptune. |
target_names |
list of str |
None |
List of one or more study objective names to log (see example). |
Examples
Create a Neptune run:
import neptune
import neptune.integrations.optuna as npt_utils
run = neptune.init_run(project="workspace-name/project-name") # (1)!
-
The full project name. For example,
"ml-team/classification"
.To copy it, navigate to the project settings in the top-right () and select Edit project details.
Create and run the study:
Log single and multi objective study metadata to Neptune:
Or optionally pass a list of objective names:
load_study_from_run()
#
Loads Optuna study from an existing Neptune run.
Loading mechanics depend on the study storage type used during the run:
- If the study used InMemoryStorage, it will be loaded from the logged pickled study object.
- If the study used database storage, it will be loaded from the logged database URL.
Parameters
Name | Type | Default | Description |
---|---|---|---|
run |
Run |
- | (required) An existing run reference, as returned by neptune.init_run() . |
Returns
An Optuna study object.
Examples
Initialize an existing run by passing the run ID:
How do I find the ID?
The Neptune ID is a unique identifier for the object. In the table view, it's displayed in the leftmost column.
The ID is stored in the system namespace (sys/id
). If the object is active, you can obtain its ID with neptune_object["sys/id"].fetch()
. For example:
Load the study from the run and continue optimization:
import neptune.integrations.optuna as npt_utils
study = npt_utils.load_study_from_run(run)
study.optimize(objective, n_trials=20)
See also
neptune-optuna repo on GitHub