neptunecontrib.monitoring.optuna

Module Contents

Classes

NeptuneCallback(experiment=None, log_study=False, log_charts=False, log_optimization_history=False, log_contour=False, log_parallel_coordinate=False, log_slice=False, params=None)

Logs hyperparameter optimization process to Neptune.

Functions

log_study_info(study, experiment=None, log_study=True, log_charts=True, log_optimization_history=False, log_contour=False, log_parallel_coordinate=False, log_slice=False, params=None)

Logs runs results and parameters to neptune.

class neptunecontrib.monitoring.optuna.NeptuneCallback(experiment=None, log_study=False, log_charts=False, log_optimization_history=False, log_contour=False, log_parallel_coordinate=False, log_slice=False, params=None)[source]

Logs hyperparameter optimization process to Neptune.

For each iteration it logs run metric and run parameters as well as the best score to date.

Parameters
  • experiment (neptune.experiments.Experiment) – Neptune experiment. Default is None.

  • log_study ('bool') – Whether optuna study object should be pickled and logged. By default it is not.

  • log_charts ('bool') – Deprecated argument. Whether all optuna visualizations plots should be logged. By default they are not. To log all the charts set log_charts=True. If you want to log a particular chart change the argument for that chart explicitly. For example log_charts=False and log_slice=True will log only the slice plot to Neptune.

  • log_optimization_history ('bool') – Whether optuna optimization history chart should be logged. By default it is not.

  • log_contour ('bool') – Whether optuna contour plot should be logged. By default it is not.

  • log_parallel_coordinate ('bool') – Whether optuna parallel coordinate plot should be logged. By default it is not.

  • log_slice ('bool') – Whether optuna slice chart should be logged. By default it is not.

  • params (list) – List of parameters to be visualized. Default is all parameters.

Examples

Initialize neptune_monitor:

import neptune
import neptunecontrib.monitoring.optuna as opt_utils

neptune.init(api_token='ANONYMOUS',
             project_qualified_name='shared/showroom')
neptune.create_experiment(name='optuna sweep')

neptune_callback = opt_utils.NeptuneCallback()

Run Optuna training passing neptune_callback as callback:

...
study = optuna.create_study(direction='maximize')
study.optimize(objective, n_trials=100, callbacks=[neptune_callback])

You can explore an example experiment in Neptune: https://ui.neptune.ai/o/shared/org/showroom/e/SHOW-1016/artifacts

You can also log optuna visualization charts and study object after every iteration:

neptune_callback = opt_utils.NeptuneCallback(log_charts=True, log_study=True)
__call__(self, study, trial)[source]
neptunecontrib.monitoring.optuna.log_study_info(study, experiment=None, log_study=True, log_charts=True, log_optimization_history=False, log_contour=False, log_parallel_coordinate=False, log_slice=False, params=None)[source]

Logs runs results and parameters to neptune.

Logs all hyperparameter optimization results to Neptune. Those include best score (‘best_score’ metric), best parameters (‘best_parameters’ property), the study object itself as artifact, and interactive optuna charts (‘contour’, ‘parallel_coordinate’, ‘slice’, ‘optimization_history’) as artifacts in ‘charts’ sub folder.

Parameters
  • study ('optuna.study.Study') – Optuna study object after training is completed.

  • experiment (neptune.experiments.Experiment) – Neptune experiment. Default is None.

  • log_study ('bool') – Whether optuna study object should be logged as pickle. Default is True.

  • log_charts ('bool') – Deprecated argument. Whether all optuna visualizations charts should be logged. By default all charts are sent. To not log any charts set log_charts=False. If you want to log a particular chart change the argument for that chart explicitly. For example log_charts=False and log_slice=True will log only the slice plot to Neptune.

  • log_optimization_history ('bool') – Whether optuna optimization history chart should be logged. Default is True.

  • log_contour ('bool') – Whether optuna contour plot should be logged. Default is True.

  • log_parallel_coordinate ('bool') – Whether optuna parallel coordinate plot should be logged. Default is True.

  • log_slice ('bool') – Whether optuna slice chart should be logged. Default is True.

  • params (list) – List of parameters to be visualized. Default is all parameters.

Examples

Initialize neptune_monitor:

import neptune
import neptunecontrib.monitoring.optuna as opt_utils

neptune.init(project_qualified_name='USER_NAME/PROJECT_NAME')
neptune.create_experiment(name='optuna sweep')

neptune_callback = opt_utils.NeptuneCallback()

Run Optuna training passing monitor as callback:

...
study = optuna.create_study(direction='maximize')
study.optimize(objective, n_trials=100, callbacks=[neptune_callback])
opt_utils.log_study_info(study)

You can explore an example experiment in Neptune: https://ui.neptune.ai/o/shared/org/showroom/e/SHOW-1016/artifacts