neptunecontrib.monitoring.sklearn

Module Contents

Functions

log_regressor_summary(regressor, X_train, X_test, y_train, y_test, model_name=None, nrows=1000, experiment=None)

Log sklearn regressor summary.

log_classifier_summary(classifier, X_train, X_test, y_train, y_test, model_name=None, nrows=1000, experiment=None)

Log sklearn classifier summary.

log_estimator_params(estimator, experiment=None)

Log estimator parameters.

log_pickled_model(estimator, model_name=None, experiment=None)

Log pickled estimator.

log_test_predictions(estimator, X_test, y_test, y_pred=None, nrows=1000, experiment=None)

Log test predictions.

log_test_preds_proba(classifier, X_test, y_pred_proba=None, nrows=1000, experiment=None)

Log test predictions probabilities.

log_scores(estimator, X, y, y_pred=None, name=None, experiment=None)

Log estimator scores on X.

log_learning_curve_chart(regressor, X_train, y_train, experiment=None)

Log learning curve chart.

log_feature_importance_chart(regressor, X_train, y_train, experiment=None)

Log feature importance chart.

log_residuals_chart(regressor, X_train, X_test, y_train, y_test, experiment=None)

Log residuals chart.

log_prediction_error_chart(regressor, X_train, X_test, y_train, y_test, experiment=None)

Log prediction error chart.

log_cooks_distance_chart(regressor, X_train, y_train, experiment=None)

Log feature importance chart.

log_classification_report_chart(classifier, X_train, X_test, y_train, y_test, experiment=None)

Log classification report chart.

log_confusion_matrix_chart(classifier, X_train, X_test, y_train, y_test, experiment=None)

Log confusion matrix.

log_roc_auc_chart(classifier, X_train, X_test, y_train, y_test, experiment=None)

Log ROC-AUC chart.

log_precision_recall_chart(classifier, X_test, y_test, y_pred_proba=None, experiment=None)

Log precision recall chart.

log_class_prediction_error_chart(classifier, X_train, X_test, y_train, y_test, experiment=None)

Log class prediction error chart.

log_kmeans_clustering_summary(model, X, nrows=1000, experiment=None, **kwargs)

Log sklearn kmeans summary.

log_cluster_labels(model, X, nrows=1000, experiment=None, **kwargs)

Log index of the cluster label each sample belongs to.

log_kelbow_chart(model, X, experiment=None, **kwargs)

Log K-elbow chart for KMeans clusterer.

log_silhouette_chart(model, X, experiment=None, **kwargs)

Log Silhouette Coefficients charts for KMeans clusterer.

_validate_experiment(experiment)

neptunecontrib.monitoring.sklearn.log_regressor_summary(regressor, X_train, X_test, y_train, y_test, model_name=None, nrows=1000, experiment=None)

Log sklearn regressor summary.

This method automatically logs all regressor parameters, pickled estimator (model), test predictions as table, model performance visualizations and test metrics.

Regressor should be fitted before calling this function.

Make sure you created an experiment by using neptune.create_experiment() before you use this method.

Tip

Check Neptune documentation for the full example.

Parameters
  • regressor (regressor) –

    Fitted sklearn regressor object

  • X_train (ndarray) –

    Training data matrix

  • X_test (ndarray) –

    Testing data matrix

  • y_train (ndarray) –

    The regression target for training

  • y_test (ndarray) –

    The regression target for testing

  • model_name (str, optional, default is None) –

    If logging picked model, define a name of the file to be logged to model/<model_name>
    If None - model/estimator.skl is used.

  • nrows (int, optional, default is 1000) –

    Log first nrows rows of test predictions.

  • experiment (neptune.experiments.Experiment, optional, default is None) –

    Neptune Experiment object to control to which experiment you log the data.
    If None, log to currently active, and most recent experiment.

Returns

None

Examples

Log random forest regressor summary

rfr = RandomForestRegressor()
rfr.fit(X_train, y_train)

neptune.init('my_workspace/my_project')
neptune.create_experiment()

log_regressor_summary(rfr, X_train, X_test, y_train, y_test)
neptunecontrib.monitoring.sklearn.log_classifier_summary(classifier, X_train, X_test, y_train, y_test, model_name=None, nrows=1000, experiment=None)

Log sklearn classifier summary.

This method automatically logs all classifier parameters, pickled estimator (model), test predictions, predictions probabilities as table, model performance visualizations and test metrics.

Classifier should be fitted before calling this function.

Make sure you created an experiment by using neptune.create_experiment() before you use this method.

Tip

Check Neptune documentation for the full example.

Parameters
  • classifier (classifier) –

    Fitted sklearn classifier object

  • X_train (ndarray) –

    Training data matrix

  • X_test (ndarray) –

    Testing data matrix

  • y_train (ndarray) –

    The classification target for training

  • y_test (ndarray) –

    The classification target for testing

  • model_name (str, optional, default is None) –

    If logging picked model, define a name of the file to be logged to model/<model_name>
    If None - estimator.skl is used.

  • nrows (int, optional, default is 1000) –

    Log first nrows rows of test predictions and predictions probabilities.

  • experiment (neptune.experiments.Experiment, optional, default is None) –

    Neptune Experiment object to control to which experiment you log the data.
    If None, log to currently active, and most recent experiment.

Returns

None

Examples

Log random forest classifier summary

rfc = RandomForestClassifier()
rfc.fit(X_train, y_train)

neptune.init('my_workspace/my_project')
neptune.create_experiment()

log_classifier_summary(rfc, X_train, X_test, y_train, y_test)
neptunecontrib.monitoring.sklearn.log_estimator_params(estimator, experiment=None)

Log estimator parameters.

Log all estimator parameters as experiment properties.

Make sure you created an experiment by using neptune.create_experiment() before you use this method.

Tip

Check Neptune documentation for the full example.

Parameters
  • estimator (estimator) –

    Scikit-learn estimator from which to log parameters.

  • experiment (neptune.experiments.Experiment, optional, default is None) –

    Neptune Experiment object to control to which experiment you log the data.
    If None, log to currently active, and most recent experiment.

Returns

None

Examples

rfr = RandomForestRegressor()

neptune.init('my_workspace/my_project')
neptune.create_experiment()

log_estimator_params(rfr)
neptunecontrib.monitoring.sklearn.log_pickled_model(estimator, model_name=None, experiment=None)

Log pickled estimator.

Log estimator as pickled file to Neptune artifacts.

Estimator should be fitted before calling this function.

Path to file in the Neptune artifacts is model/<model_name>.

Make sure you created an experiment by using neptune.create_experiment() before you use this method.

Tip

Check Neptune documentation for the full example.

Parameters
  • estimator (estimator) –

    Scikit-learn estimator to log.

  • model_name (str, optional, default is None) –

    Name of the file.
    If None - estimator.skl is used.

  • experiment (neptune.experiments.Experiment, optional, default is None) –

    Neptune Experiment object to control to which experiment you log the data.
    If None, log to currently active, and most recent experiment.

Returns

None

Examples

rfr = RandomForestRegressor()
rfr.fit(X_train, y_train)

neptune.init('my_workspace/my_project')
neptune.create_experiment()

log_pickled_model(rfr, 'my_model')
neptunecontrib.monitoring.sklearn.log_test_predictions(estimator, X_test, y_test, y_pred=None, nrows=1000, experiment=None)

Log test predictions.

Calculate and log test predictions and have them as csv file in the Neptune artifacts.

If you pass y_pred, then predictions are logged without computing from X_test data.

Estimator should be fitted before calling this function.

Path to predictions in the Neptune artifacts is ‘csv/test_predictions.csv’.

Make sure you created an experiment by using neptune.create_experiment() before you use this method.

Tip

Check Neptune documentation for the full example.

Parameters
  • estimator (estimator) –

    Scikit-learn estimator to compute predictions.

  • X_test (ndarray) –

    Testing data matrix.

  • y_test (ndarray) –

    Target for testing.

  • y_pred (ndarray, optional, default is None) –

    Estimator predictions on test data.

  • nrows (int, optional, default is 1000) –

    Number of rows to log.

  • experiment (neptune.experiments.Experiment, optional, default is None) –

    Neptune Experiment object to control to which experiment you log the data.
    If None, log to currently active, and most recent experiment.

Returns

None

Examples

rfr = RandomForestRegressor()
rfr.fit(X_train, y_train)

neptune.init('my_workspace/my_project')
neptune.create_experiment()

log_test_predictions(rfr, X_test, y_test)
neptunecontrib.monitoring.sklearn.log_test_preds_proba(classifier, X_test, y_pred_proba=None, nrows=1000, experiment=None)

Log test predictions probabilities.

Calculate and log test predictions probabilities and have them as csv file in the Neptune artifacts.

If you pass y_pred_proba, then predictions probabilities are logged without computing from X_test data.

Estimator should be fitted before calling this function.

Path to predictions probabilities in the Neptune artifacts is ‘csv/test_preds_proba.csv’.

Make sure you created an experiment by using neptune.create_experiment() before you use this method.

Tip

Check Neptune documentation for the full example.

Parameters
  • classifier (classifier) –

    Scikit-learn classifier to compute predictions probabilities.

  • X_test (ndarray) –

    Testing data matrix.

  • y_pred_proba (ndarray, optional, default is None) –

    Classifier predictions probabilities on test data.

  • nrows (int, optional, default is 1000) –

    Number of rows to log.

  • experiment (neptune.experiments.Experiment, optional, default is None) –

    Neptune Experiment object to control to which experiment you log the data.
    If None, log to currently active, and most recent experiment.

Returns

None

Examples

rfc = RandomForestClassifier()
rfc.fit(X_train, y_train)

neptune.init('my_workspace/my_project')
neptune.create_experiment()

log_test_preds_proba(rfc, X_test, y_test)
neptunecontrib.monitoring.sklearn.log_scores(estimator, X, y, y_pred=None, name=None, experiment=None)

Log estimator scores on X.

Calculate and log scores on data and have them as metrics in Neptune. If you pass y_pred, then predictions are not computed from X data.

Estimator should be fitted before calling this function.

Make sure you created an experiment by using neptune.create_experiment() before you use this method.

Regressor

For regressors that outputs single value, following scores are logged:

  • explained variance

  • max error

  • mean absolute error

  • r2

For multi-output regressor:

  • r2

Classifier

For classifier, following scores are logged:

  • precision

  • recall

  • f beta score

  • support

Tip

Check Neptune documentation for the full example.

Parameters
  • estimator (estimator) –

    Scikit-learn estimator to compute scores.

  • X (ndarray) –

    Data matrix.

  • y (ndarray) –

    Target for testing.

  • y_pred (ndarray, optional, default is None) –

    Estimator predictions on data.

  • name (str, optional, default is None) –

    Use ‘train’, ‘valid’, ‘test’ to better define on what data scores are logged.

  • experiment (neptune.experiments.Experiment, optional, default is None) –

    Neptune Experiment object to control to which experiment you log the data.
    If None, log to currently active, and most recent experiment.

Returns

None

Examples

rfc = RandomForestClassifier()
rfc.fit(X_train, y_train)

neptune.init('my_workspace/my_project')
neptune.create_experiment()

log_scores(rfc, X, y, name='test', experiment=exp)
neptunecontrib.monitoring.sklearn.log_learning_curve_chart(regressor, X_train, y_train, experiment=None)

Log learning curve chart.

Make sure you created an experiment by using neptune.create_experiment() before you use this method.

Tip

Check Neptune documentation for the full example.

Parameters
  • regressor (regressor) –

    Fitted sklearn regressor object

  • X_train (ndarray) –

    Training data matrix

  • y_train (ndarray) –

    The regression target for training

  • experiment (neptune.experiments.Experiment, optional, default is None) –

    Neptune Experiment object to control to which experiment you log the data.
    If None, log to currently active, and most recent experiment.

Returns

None

Examples

rfr = RandomForestRegressor()
rfr.fit(X_train, y_train)

neptune.init('my_workspace/my_project')
neptune.create_experiment()

log_learning_curve_chart(rfr, X_train, y_train)
neptunecontrib.monitoring.sklearn.log_feature_importance_chart(regressor, X_train, y_train, experiment=None)

Log feature importance chart.

Make sure you created an experiment by using neptune.create_experiment() before you use this method.

Tip

Check Neptune documentation for the full example.

Parameters
  • regressor (regressor) –

    Fitted sklearn regressor object

  • X_train (ndarray) –

    Training data matrix

  • y_train (ndarray) –

    The regression target for training

  • experiment (neptune.experiments.Experiment, optional, default is None) –

    Neptune Experiment object to control to which experiment you log the data.
    If None, log to currently active, and most recent experiment.

Returns

None

Examples

rfr = RandomForestRegressor()
rfr.fit(X_train, y_train)

neptune.init('my_workspace/my_project')
neptune.create_experiment()

log_feature_importance_chart(rfr, X_train, y_train)
neptunecontrib.monitoring.sklearn.log_residuals_chart(regressor, X_train, X_test, y_train, y_test, experiment=None)

Log residuals chart.

Make sure you created an experiment by using neptune.create_experiment() before you use this method.

Tip

Check Neptune documentation for the full example.

Parameters
  • regressor (regressor) –

    Fitted sklearn regressor object

  • X_train (ndarray) –

    Training data matrix

  • X_test (ndarray) –

    Testing data matrix

  • y_train (ndarray) –

    The regression target for training

  • y_test (ndarray) –

    The regression target for testing

  • experiment (neptune.experiments.Experiment, optional, default is None) –

    Neptune Experiment object to control to which experiment you log the data.
    If None, log to currently active, and most recent experiment.

Returns

None

Examples

rfr = RandomForestRegressor()
rfr.fit(X_train, y_train)

neptune.init('my_workspace/my_project')
exp = neptune.create_experiment()

log_residuals_chart(rfr, X_train, X_test, y_train, y_test, experiment=exp)
neptunecontrib.monitoring.sklearn.log_prediction_error_chart(regressor, X_train, X_test, y_train, y_test, experiment=None)

Log prediction error chart.

Make sure you created an experiment by using neptune.create_experiment() before you use this method.

Tip

Check Neptune documentation for the full example.

Parameters
  • regressor (regressor) –

    Fitted sklearn regressor object

  • X_train (ndarray) –

    Training data matrix

  • X_test (ndarray) –

    Testing data matrix

  • y_train (ndarray) –

    The regression target for training

  • y_test (ndarray) –

    The regression target for testing

  • experiment (neptune.experiments.Experiment, optional, default is None) –

    Neptune Experiment object to control to which experiment you log the data.
    If None, log to currently active, and most recent experiment.

Returns

None

Examples

rfr = RandomForestRegressor()
rfr.fit(X_train, y_train)

neptune.init('my_workspace/my_project')
neptune.create_experiment()

log_prediction_error_chart(rfr, X_train, X_test, y_train, y_test)
neptunecontrib.monitoring.sklearn.log_cooks_distance_chart(regressor, X_train, y_train, experiment=None)

Log feature importance chart.

Make sure you created an experiment by using neptune.create_experiment() before you use this method.

Tip

Check Neptune documentation for the full example.

Parameters
  • regressor (regressor) –

    Fitted sklearn regressor object

  • X_train (ndarray) –

    Training data matrix

  • y_train (ndarray) –

    The regression target for training

  • experiment (neptune.experiments.Experiment, optional, default is None) –

    Neptune Experiment object to control to which experiment you log the data.
    If None, log to currently active, and most recent experiment.

Returns

None

Examples

rfr = RandomForestRegressor()
rfr.fit(X_train, y_train)

neptune.init('my_workspace/my_project')
neptune.create_experiment()

log_cooks_distance_chart(rfr, X_train, y_train)
neptunecontrib.monitoring.sklearn.log_classification_report_chart(classifier, X_train, X_test, y_train, y_test, experiment=None)

Log classification report chart.

Make sure you created an experiment by using neptune.create_experiment() before you use this method.

Tip

Check Neptune documentation for the full example.

Parameters
  • classifier (classifier) –

    Fitted sklearn classifier object

  • X_train (ndarray) –

    Training data matrix

  • X_test (ndarray) –

    Testing data matrix

  • y_train (ndarray) –

    The classification target for training

  • y_test (ndarray) –

    The classification target for testing

  • experiment (neptune.experiments.Experiment, optional, default is None) –

    Neptune Experiment object to control to which experiment you log the data.
    If None, log to currently active, and most recent experiment.

Returns

None

Examples

rfc = RandomForestClassifier()
rfc.fit(X_train, y_train)

neptune.init('my_workspace/my_project')
exp = neptune.create_experiment()

log_classification_report_chart(rfc, X_train, X_test, y_train, y_test, experiment=exp)
neptunecontrib.monitoring.sklearn.log_confusion_matrix_chart(classifier, X_train, X_test, y_train, y_test, experiment=None)

Log confusion matrix.

Make sure you created an experiment by using neptune.create_experiment() before you use this method.

Tip

Check Neptune documentation for the full example.

Parameters
  • classifier (classifier) –

    Fitted sklearn classifier object

  • X_train (ndarray) –

    Training data matrix

  • X_test (ndarray) –

    Testing data matrix

  • y_train (ndarray) –

    The classification target for training

  • y_test (ndarray) –

    The classification target for testing

  • experiment (neptune.experiments.Experiment, optional, default is None) –

    Neptune Experiment object to control to which experiment you log the data.
    If None, log to currently active, and most recent experiment.

Returns

None

Examples

rfc = RandomForestClassifier()
rfc.fit(X_train, y_train)

neptune.init('my_workspace/my_project')
neptune.create_experiment()

log_confusion_matrix_chart(rfc, X_train, X_test, y_train, y_test)
neptunecontrib.monitoring.sklearn.log_roc_auc_chart(classifier, X_train, X_test, y_train, y_test, experiment=None)

Log ROC-AUC chart.

Make sure you created an experiment by using neptune.create_experiment() before you use this method.

Tip

Check Neptune documentation for the full example.

Parameters
  • classifier (classifier) –

    Fitted sklearn classifier object

  • X_train (ndarray) –

    Training data matrix

  • X_test (ndarray) –

    Testing data matrix

  • y_train (ndarray) –

    The classification target for training

  • y_test (ndarray) –

    The classification target for testing

  • experiment (neptune.experiments.Experiment, optional, default is None) –

    Neptune Experiment object to control to which experiment you log the data.
    If None, log to currently active, and most recent experiment.

Returns

None

Examples

rfc = RandomForestClassifier()
rfc.fit(X_train, y_train)

neptune.init('my_workspace/my_project')
exp = neptune.create_experiment()

log_roc_auc_chart(rfc, X_train, X_test, y_train, y_test, experiment=exp)
neptunecontrib.monitoring.sklearn.log_precision_recall_chart(classifier, X_test, y_test, y_pred_proba=None, experiment=None)

Log precision recall chart.

Make sure you created an experiment by using neptune.create_experiment() before you use this method.

Tip

Check Neptune documentation for the full example.

Parameters
  • classifier (classifier) –

    Fitted sklearn classifier object

  • X_test (ndarray) –

    Testing data matrix

  • y_test (ndarray) –

    The classification target for testing

  • y_pred_proba (ndarray, optional, default is None) –

    Classifier predictions probabilities on test data.

  • experiment (neptune.experiments.Experiment, optional, default is None) –

    Neptune Experiment object to control to which experiment you log the data.
    If None, log to currently active, and most recent experiment.

Returns

None

Examples

rfc = RandomForestClassifier()
rfc.fit(X_train, y_train)

neptune.init('my_workspace/my_project')
neptune.create_experiment()

log_precision_recall_chart(rfc, X_test, y_test)
neptunecontrib.monitoring.sklearn.log_class_prediction_error_chart(classifier, X_train, X_test, y_train, y_test, experiment=None)

Log class prediction error chart.

Make sure you created an experiment by using neptune.create_experiment() before you use this method.

Tip

Check Neptune documentation for the full example.

Parameters
  • classifier (classifier) –

    Fitted sklearn classifier object

  • X_train (ndarray) –

    Training data matrix

  • X_test (ndarray) –

    Testing data matrix

  • y_train (ndarray) –

    The classification target for training

  • y_test (ndarray) –

    The classification target for testing

  • experiment (neptune.experiments.Experiment, optional, default is None) –

    Neptune Experiment object to control to which experiment you log the data.
    If None, log to currently active, and most recent experiment.

Returns

None

Examples

rfc = RandomForestClassifier()
rfc.fit(X_train, y_train)

neptune.init('my_workspace/my_project')
exp = neptune.create_experiment()

log_class_prediction_error_chart(rfc, X_train, X_test, y_train, y_test, experiment=exp)
neptunecontrib.monitoring.sklearn.log_kmeans_clustering_summary(model, X, nrows=1000, experiment=None, **kwargs)

Log sklearn kmeans summary.

This method fit KMeans model to data and logs cluster labels, all kmeans parameters and clustering visualizations: KMeans elbow chart and silhouette coefficients chart.

Make sure you created an experiment by using neptune.create_experiment() before you use this method.

Tip

Check Neptune documentation for the full example.

Parameters
  • model (KMeans) –

    KMeans object.

  • X (ndarray) –

    Training instances to cluster.

  • nrows (int, optional, default is 1000) –

    Number of rows to log in the cluster labels

  • experiment (neptune.experiments.Experiment, optional, default is None) –

    Neptune Experiment object to control to which experiment you log the data.
    If None, log to currently active, and most recent experiment.

  • kwargs – KMeans parameters.

Returns

None

Examples

km = KMeans(n_init=11, max_iter=270)
X, y = make_blobs(n_samples=579, n_features=17, centers=7, random_state=28743)

neptune.init('my_workspace/my_project')
neptune.create_experiment()

log_kmeans_clustering_summary(km, X=X)
neptunecontrib.monitoring.sklearn.log_cluster_labels(model, X, nrows=1000, experiment=None, **kwargs)

Log index of the cluster label each sample belongs to.

Make sure you created an experiment by using neptune.create_experiment() before you use this method.

Tip

Check Neptune documentation for the full example.

Parameters
  • model (KMeans) –

    KMeans object.

  • X (ndarray) –

    Training instances to cluster.

  • nrows (int, optional, default is 1000) –

    Number of rows to log.

  • experiment (neptune.experiments.Experiment, optional, default is None) –

    Neptune Experiment object to control to which experiment you log the data.
    If None, log to currently active, and most recent experiment.

  • kwargs – KMeans parameters.

Returns

None

Examples

km = KMeans(n_init=11, max_iter=270)
X, y = make_blobs(n_samples=579, n_features=17, centers=7, random_state=28743)

neptune.init('my_workspace/my_project')
neptune.create_experiment()

log_cluster_labels(km, X=X)
neptunecontrib.monitoring.sklearn.log_kelbow_chart(model, X, experiment=None, **kwargs)

Log K-elbow chart for KMeans clusterer.

Make sure you created an experiment by using neptune.create_experiment() before you use this method.

Tip

Check Neptune documentation for the full example.

Parameters
  • model (KMeans) –

    KMeans object.

  • X (ndarray) –

    Training instances to cluster.

  • experiment (neptune.experiments.Experiment, optional, default is None) –

    Neptune Experiment object to control to which experiment you log the data.
    If None, log to currently active, and most recent experiment.

  • kwargs – KMeans parameters.

Returns

None

Examples

km = KMeans(n_init=11, max_iter=270)
X, y = make_blobs(n_samples=579, n_features=17, centers=7, random_state=28743)

neptune.init('my_workspace/my_project')
neptune.create_experiment()

log_kelbow_chart(km, X=X)
neptunecontrib.monitoring.sklearn.log_silhouette_chart(model, X, experiment=None, **kwargs)

Log Silhouette Coefficients charts for KMeans clusterer.

Charts are computed for j = 2, 3, …, n_clusters.

Make sure you created an experiment by using neptune.create_experiment() before you use this method.

Tip

Check Neptune documentation for the full example.

Parameters
  • model (KMeans) –

    KMeans object.

  • X (ndarray) –

    Training instances to cluster.

  • experiment (neptune.experiments.Experiment, optional, default is None) –

    Neptune Experiment object to control to which experiment you log the data.
    If None, log to currently active, and most recent experiment.

  • kwargs – KMeans parameters.

Returns

None

Examples

km = KMeans(n_init=11, max_iter=270)
X, y = make_blobs(n_samples=579, n_features=17, centers=7, random_state=28743)

neptune.init('my_workspace/my_project')
neptune.create_experiment()

log_silhouette_chart(km, X=X, n_clusters=12)
neptunecontrib.monitoring.sklearn._validate_experiment(experiment)