Skip to content

API reference: Optuna integration#

The Neptune-Optuna integration provides a NeptuneCallback class and two utility functions.


NeptuneCallback#

A callback that logs metadata from an Optuna study to Neptune.

With this callback, you can log and display:

  • Values and parameters for each trial.
  • Current best values and parameters for the study.
  • Visualizations from the optuna.visualizations module.
  • Parameter distributions for each trial.
  • The study object itself, to load it later.

Parameters

Name         Type Default Description
run Run or Handler - (required) An existing run reference, as returned by neptune.init_run(), or a namespace handler.
base_namespace str, optional "" Namespace under which all metadata logged by the Neptune callback will be stored. If omitted, defaults to an empty string, in which case the metadata is logged directly under the run, without a common "root" namespace.
plots_update_freq int or str, optional 1 Frequency at which plots are logged and updated in Neptune.
  • If you pass an integer k, plots will be updated every k iterations.
  • If you pass the string "never", plots will not be logged.
study_update_freq int or str, optional 1 Frequency at which a study object is logged and updated in Neptune.
  • If you pass an integer k, the study will be updated every k iterations.
  • If you pass the string "never", the study will not be logged.
visualization_backend str, optional "plotly" Which visualization backend is used for optuna.visualizations plots. Can be set to either "matplotlib" or "plotly".
log_plot_contour bool, optional True Whether to log the optuna.visualizations.plot_contour visualization to Neptune.
log_plot_edf bool, optional True Whether to log the optuna.visualizations.plot_edf visualization to Neptune.
log_plot_parallel_coordinate bool, optional True Whether to log the optuna.visualizations.plot_parallel_coordinate visualization to Neptune.
log_plot_param_importances bool, optional True Whether to log the optuna.visualizations.plot_param_importances visualization to Neptune.
log_plot_pareto_front bool, optional True Whether to log the optuna.visualizations.plot_pareto_front visualization to Neptune.

If your study is not multi-objective, this plot is not logged.

log_plot_slice bool, optional True Whether to log the optuna.visualizations.plot_slice visualization to Neptune.
log_plot_intermediate_values bool, optional True Whether to log the optuna.visualizations.plot_intermediate_values visualization to Neptune.

If your study is not using pruners, this plot is not logged.

log_plot_optimization_history bool, optional True Whether to log the optuna.visualizations.plot_optimization_history visualization to Neptune.
target_names list of str None List of one or more study objective names to log (see example).
log_all_trials bool, optional True Whether to log all trials.

Examples

Create a Neptune run and initialize a Neptune callback:

import neptune
import neptune.integrations.optuna as npt_utils

run = neptune.init_run()
neptune_callback = npt_utils.NeptuneCallback(run)
If Neptune can't find your project name or API token

As a best practice, you should save your Neptune API token and project name as environment variables:

export NEPTUNE_API_TOKEN="h0dHBzOi8aHR0cHM6Lkc78ghs74kl0jv...Yh3Kb8"
export NEPTUNE_PROJECT="ml-team/classification"

Alternatively, you can pass the information when using a function that takes api_token and project as arguments:

run = neptune.init_run( # (1)!
    api_token="h0dHBzOi8aHR0cHM6Lkc78ghs74kl0jv...Yh3Kb8",  # your token here
    project="ml-team/classification",  # your full project name here
)
  1. Also works for init_model(), init_model_version(), init_project(), and integrations that create Neptune runs underneath the hood, such as NeptuneLogger or NeptuneCallback.

  2. API token: In the bottom-left corner, expand the user menu and select Get my API token.

  3. Project name: You can copy the path from the project details ( Edit project details).

If you haven't registered, you can log anonymously to a public project:

api_token=neptune.ANONYMOUS_API_TOKEN
project="common/quickstarts"

Make sure not to publish sensitive data through your code!

Or optionally pass a list of objective names:

Single objective
neptune_callback = npt_utils.NeptuneCallback(
    run,
    target_names=["accuracy"],
)
Multi-objective
neptune_callback = npt_utils.NeptuneCallback(
    run,
    target_names=["FLOPS", "accuracy"],
)

Log single and multi objective study metadata to Neptune by passing the callback to the Optuna study:

study = optuna.create_study(direction="maximize")
study.optimize(objective, n_trials=5, callbacks=[neptune_callback])

log_study_metadata()#

Logs the metadata from an Optuna study to Neptune.

With this function, you can log and display:

  • Values and parameters for each trial.
  • Current best values and parameters for the study.
  • Visualizations from the optuna.visualizations module.
  • Parameter distributions for each trial.
  • The study object itself, to load it later.

Parameters

Name         Type Default Description
study optuna.Study - (required) An Optuna study object.
run Run - (required) An existing run reference, as returned by neptune.init_run().
base_namespace str, optional "" Namespace under which all metadata logged by the Neptune callback will be stored. If omitted, defaults to an empty string, in which case the metadata is logged directly under the run, without a common "root" namespace.
log_plots bool, optional True Whether the visualizations from optuna.visualizations will be logged to Neptune.
log_study bool, optional True Whether the study will be logged to Neptune.

The objects that are logged depend on the study storage type used: If InMemoryStorage is used, the pickled study object will be logged to Neptune. Otherwise the database URL will be logged.

log_all_trials bool, optional True Whether to log all trials.
log_distributions bool, optional True Whether to log the distributions for all trials.
visualization_backend str, optional "plotly" Which visualization backend is used for optuna.visualizations plots. Can be set to either "matplotlib" or "plotly".
log_plot_contour bool, optional True Whether to log the optuna.visualizations.plot_contour visualization to Neptune.
log_plot_edf bool, optional True Whether to log the optuna.visualizations.plot_edf visualization to Neptune.
log_plot_parallel_coordinate bool, optional True Whether to log the optuna.visualizations.plot_parallel_coordinate visualization to Neptune.
log_plot_param_importances bool, optional True Whether to log the optuna.visualizations.plot_param_importances visualization to Neptune.
log_plot_pareto_front bool, optional True Whether to log the optuna.visualizations.plot_pareto_front visualization to Neptune.

If your study is not multi-objective, this plot is not logged.

log_plot_slice bool, optional True Whether to log the optuna.visualizations.plot_slice visualization to Neptune.
log_plot_intermediate_values bool, optional True Whether to log the optuna.visualizations.plot_intermediate_values visualization to Neptune.

If your study is not using pruners, this plot is not logged.

log_plot_optimization_history bool, optional True Whether to log the optuna.visualizations.plot_optimization_history visualization to Neptune.
target_names list of str None List of one or more study objective names to log (see example).

Examples

Create a Neptune run:

import neptune
import neptune.integrations.optuna as npt_utils

run = neptune.init_run(project="workspace-name/project-name") # (1)!
  1. The full project name. For example, "ml-team/classification".

    To find the required string in the Neptune app, click How to create a new run. You can copy the project argument from the modal that opens.

Create and run the study:

study = optuna.create_study(direction="maximize")
study.optimize(objective, n_trials=5)

Log single and multi objective study metadata to Neptune:

import neptune.integrations.optuna as npt_utils

npt_utils.log_study_metadata(study, run)

Or optionally pass a list of objective names:

Single objective
npt_utils.log_study_metadata(
    study,
    run,
    target_names=["accuracy"],
)
Multi-objective
npt_utils.log_study_metadata(
    study,
    run,
    target_names=["FLOPS", "accuracy"],
)

load_study_from_run()#

Loads Optuna study from an existing Neptune run.

Loading mechanics depend on the study storage type used during the run:

  • If the study used InMemoryStorage, it will be loaded from the logged pickled study object.
  • If the study used database storage, it will be loaded from the logged database URL.

Parameters

Name Type Default Description
run Run - (required) An existing run reference, as returned by neptune.init_run().

Returns

An Optuna study object.

Examples

Initialize an existing run by passing the run ID:

import neptune

run = neptune.init_run(run="PRO-123")
How do I find the ID?

The Neptune ID is a unique identifier for the run. In the table view, it's displayed in the leftmost column.

The ID is stored in the system namespace (sys/id).

If the run is active, you can obtain its ID with run["sys/id"].fetch(). For example:

>>> run = neptune.init_run(project="ml-team/classification")
>>> run["sys/id"].fetch()
'CLS-26'

Load the study from the run and continue optimization:

import neptune.integrations.optuna as npt_utils

study = npt_utils.load_study_from_run(run)
study.optimize(objective, n_trials=20)

See also

neptune-optuna repo on GitHub