Neptune-Scikit-Optimize Integration

This integation lets you monitor Scikit-Optimize (skopt) hyperparameter optimization in Neptune.

Scikit Optimize Neptune integration


To use Neptune + Scikit-Optimize integration you need to have installed is neptune-client and neptune-contrib.

pip install neptune-client neptune-contrib['monitoring']

Initialize Neptune and create an experiment

import neptune

neptune.create_experiment(name='skopt sweep')

Create NeptuneCallback

Pass the experiment object as the first argument.

import neptunecontrib.monitoring.skopt as skopt_utils

neptune_callback = skopt_utils.NeptuneCallback()

Pass neptune_callback to skopt.forest_minimize or others

This causes the metrics, parameters and results pickle logged after every iteration. Everything can be inspected live.

results = skopt.forest_minimize(objective, space, callback=[neptune_callback],
                                base_estimator='ET', n_calls=100, n_random_starts=10)

Log all results

You can log additional information from skopt results after the sweep has completed. By running:


You log the following things to Neptune:

  • Best score

  • Best parameters

  • Figures from plots module: plot_evaluations, plot_convergence, plot_objective, and plot_regret

  • Pickled results object


Monitor your Scikit-Optimize training in Neptune

Now you can watch your Scikit-Optimize hyperparameter optimization in Neptune!

Check out this example experiment.

Scikit-Optimize monitoring in Neptune

Full script

import lightgbm as lgb
import skopt
from sklearn.datasets import load_breast_cancer
from sklearn.metrics import roc_auc_score
from sklearn.model_selection import train_test_split

import neptune
import neptunecontrib.monitoring.skopt as skopt_utils


neptune_callback = skopt_utils.NeptuneCallback()

space = [, 0.5, name='learning_rate', prior='log-uniform'),, 30, name='max_depth'),, 100, name='num_leaves'),, 1000, name='min_data_in_leaf'),, 1.0, name='feature_fraction', prior='uniform'),, 1.0, name='subsample', prior='uniform'),

def objective(**params):
    data, target = load_breast_cancer(return_X_y=True)
    train_x, test_x, train_y, test_y = train_test_split(data, target, test_size=0.25)
    dtrain = lgb.Dataset(train_x, label=train_y)

    param = {
        'objective': 'binary',
        'metric': 'binary_logloss',

    gbm = lgb.train(param, dtrain)
    preds = gbm.predict(test_x)
    accuracy = roc_auc_score(test_y, preds)
    return -1.0 * accuracy

results = skopt.forest_minimize(objective, space, n_calls=100, n_random_starts=10,