API reference: Optuna integration#

The Neptune-Optuna integration provides a NeptuneCallback class and two utility functions.

`NeptuneCallback`#

A callback that logs metadata from an Optuna study to Neptune.

With this callback, you can log and display:

Values and parameters for each trial.
Current best values and parameters for the study.
Visualizations from the optuna.visualizations module.
Parameter distributions for each trial.
The study object itself, to load it later.

Parameters

Name	Type	Default	Description
`run`	`Run` or `Handler`	-	(required) An existing run reference, as returned by `neptune.init_run()`, or a namespace handler.
`base_namespace`	`str`, optional	`""`	Namespace under which all metadata logged by the Neptune callback will be stored. If omitted, defaults to an empty string, in which case the metadata is logged directly under the run, without a common "root" namespace.
`plots_update_freq`	`int` or `str`, optional	`1`	Frequency at which plots are logged and updated in Neptune. If you pass an integer `k`, plots will be updated every k iterations. If you pass the string `"never"`, plots will not be logged.
`study_update_freq`	`int` or `str`, optional	`1`	Frequency at which a study object is logged and updated in Neptune. If you pass an integer `k`, the study will be updated every k iterations. If you pass the string `"never"`, the study will not be logged.
`visualization_backend`	`str`, optional	`"plotly"`	Which visualization backend is used for optuna.visualizations plots. Can be set to either `"matplotlib"` or `"plotly"`.
`log_plot_contour`	`bool`, optional	`True`	Whether to log the `optuna.visualizations.plot_contour` visualization to Neptune.
`log_plot_edf`	`bool`, optional	`True`	Whether to log the `optuna.visualizations.plot_edf` visualization to Neptune.
`log_plot_parallel_coordinate`	`bool`, optional	`True`	Whether to log the `optuna.visualizations.plot_parallel_coordinate` visualization to Neptune.
`log_plot_param_importances`	`bool`, optional	`True`	Whether to log the `optuna.visualizations.plot_param_importances` visualization to Neptune.
`log_plot_pareto_front`	`bool`, optional	`True`	Whether to log the `optuna.visualizations.plot_pareto_front` visualization to Neptune. If your study is not multi-objective, this plot is not logged.
`log_plot_slice`	`bool`, optional	`True`	Whether to log the `optuna.visualizations.plot_slice` visualization to Neptune.
`log_plot_intermediate_values`	`bool`, optional	`True`	Whether to log the `optuna.visualizations.plot_intermediate_values` visualization to Neptune. If your study is not using pruners, this plot is not logged.
`log_plot_optimization_history`	`bool`, optional	`True`	Whether to log the `optuna.visualizations.plot_optimization_history` visualization to Neptune.
`target_names`	list of `str`	`None`	List of one or more study objective names to log (see example).
`log_all_trials`	`bool`, optional	`True`	Whether to log all trials.

Examples

Create a Neptune run and initialize a Neptune callback:

import neptune
import neptune.integrations.optuna as npt_utils

run = neptune.init_run()
neptune_callback = npt_utils.NeptuneCallback(run)

If Neptune can't find your project name or API token

As a best practice, you should save your Neptune API token and project name as environment variables:

export NEPTUNE_API_TOKEN="h0dHBzOi8aHR0cHM6Lkc78ghs74kl0jv...Yh3Kb8"

export NEPTUNE_PROJECT="ml-team/classification"

Alternatively, you can pass the information when using a function that takes api_token and project as arguments:

run = neptune.init_run( # (1)!
    api_token="h0dHBzOi8aHR0cHM6Lkc78ghs74kl0jv...Yh3Kb8",  # your token here
    project="ml-team/classification",  # your full project name here
)

Also works for init_model(), init_model_version(), init_project(), and integrations that create Neptune runs underneath the hood, such as NeptuneLogger or NeptuneCallback.
API token: In the bottom-left corner, expand the user menu and select Get my API token.
Project name: You can copy the path from the project details ( → Edit project details).

If you haven't registered, you can log anonymously to a public project:

api_token=neptune.ANONYMOUS_API_TOKEN
project="common/quickstarts"

Make sure not to publish sensitive data through your code!

Or optionally pass a list of objective names:

Single objective

neptune_callback = npt_utils.NeptuneCallback(
    run,
    target_names=["accuracy"],
)

Multi-objective

neptune_callback = npt_utils.NeptuneCallback(
    run,
    target_names=["FLOPS", "accuracy"],
)

Log single and multi objective study metadata to Neptune by passing the callback to the Optuna study:

study = optuna.create_study(direction="maximize")
study.optimize(objective, n_trials=5, callbacks=[neptune_callback])

`log_study_metadata()`#

Logs the metadata from an Optuna study to Neptune.

With this function, you can log and display:

Values and parameters for each trial.
Current best values and parameters for the study.
Visualizations from the optuna.visualizations module.
Parameter distributions for each trial.
The study object itself, to load it later.

Parameters

Name	Type	Default	Description
`study`	`optuna.Study`	-	(required) An Optuna study object.
`run`	`Run`	-	(required) An existing run reference, as returned by `neptune.init_run()`.
`base_namespace`	`str`, optional	`""`	Namespace under which all metadata logged by the Neptune callback will be stored. If omitted, defaults to an empty string, in which case the metadata is logged directly under the run, without a common "root" namespace.
`log_plots`	`bool`, optional	`True`	Whether the visualizations from `optuna.visualizations` will be logged to Neptune.
`log_study`	`bool`, optional	`True`	Whether the study will be logged to Neptune. The objects that are logged depend on the study storage type used: If InMemoryStorage is used, the pickled study object will be logged to Neptune. Otherwise the database URL will be logged.
`log_all_trials`	`bool`, optional	`True`	Whether to log all trials.
`log_distributions`	`bool`, optional	`True`	Whether to log the distributions for all trials.
`visualization_backend`	`str`, optional	`"plotly"`	Which visualization backend is used for optuna.visualizations plots. Can be set to either `"matplotlib"` or `"plotly"`.
`log_plot_contour`	`bool`, optional	`True`	Whether to log the `optuna.visualizations.plot_contour` visualization to Neptune.
`log_plot_edf`	`bool`, optional	`True`	Whether to log the `optuna.visualizations.plot_edf` visualization to Neptune.
`log_plot_parallel_coordinate`	`bool`, optional	`True`	Whether to log the `optuna.visualizations.plot_parallel_coordinate` visualization to Neptune.
`log_plot_param_importances`	`bool`, optional	`True`	Whether to log the `optuna.visualizations.plot_param_importances` visualization to Neptune.
`log_plot_pareto_front`	`bool`, optional	`True`	Whether to log the `optuna.visualizations.plot_pareto_front` visualization to Neptune. If your study is not multi-objective, this plot is not logged.
`log_plot_slice`	`bool`, optional	`True`	Whether to log the `optuna.visualizations.plot_slice` visualization to Neptune.
`log_plot_intermediate_values`	`bool`, optional	`True`	Whether to log the `optuna.visualizations.plot_intermediate_values` visualization to Neptune. If your study is not using pruners, this plot is not logged.
`log_plot_optimization_history`	`bool`, optional	`True`	Whether to log the `optuna.visualizations.plot_optimization_history` visualization to Neptune.
`target_names`	list of `str`	`None`	List of one or more study objective names to log (see example).

Examples

Create a Neptune run:

import neptune
import neptune.integrations.optuna as npt_utils

run = neptune.init_run(project="workspace-name/project-name") # (1)!

The full project name. For example, "ml-team/classification".

To copy it to your clipboard, navigate to the project settings in the top-right () and select Edit project details.

Create and run the study:

study = optuna.create_study(direction="maximize")
study.optimize(objective, n_trials=5)

Log single and multi objective study metadata to Neptune:

import neptune.integrations.optuna as npt_utils

npt_utils.log_study_metadata(study, run)

Or optionally pass a list of objective names:

Single objective

npt_utils.log_study_metadata(
    study,
    run,
    target_names=["accuracy"],
)

Multi-objective

npt_utils.log_study_metadata(
    study,
    run,
    target_names=["FLOPS", "accuracy"],
)

`load_study_from_run()`#

Loads Optuna study from an existing Neptune run.

Loading mechanics depend on the study storage type used during the run:

If the study used InMemoryStorage, it will be loaded from the logged pickled study object.
If the study used database storage, it will be loaded from the logged database URL.

Parameters

Name	Type	Default	Description
`run`	`Run`	-	(required) An existing run reference, as returned by `neptune.init_run()`.

Returns

An Optuna study object.

Examples

Initialize an existing run by passing the run ID:

import neptune

run = neptune.init_run(run="PRO-123")

How do I find the ID?

The Neptune ID is a unique identifier for the run. In the table view, it's displayed in the leftmost column.

The ID is stored in the system namespace (sys/id).

If the run is active, you can obtain its ID with run["sys/id"].fetch(). For example:

>>> run = neptune.init_run(project="ml-team/classification")
>>> run["sys/id"].fetch()
'CLS-26'

Load the study from the run and continue optimization:

import neptune.integrations.optuna as npt_utils

study = npt_utils.load_study_from_run(run)
study.optimize(objective, n_trials=20)

API reference: Optuna integration#

NeptuneCallback#

log_study_metadata()#

load_study_from_run()#

`NeptuneCallback`#

`log_study_metadata()`#

`load_study_from_run()`#