Dalex Integration

This integration enables you to log DALEX global and local explanations as well as the dalex.expainer object to Neptune.

DALEX Neptune integration

Requirements

To use Neptune + DALEX integration you need to have neptune-client and neptune-contrib installed.

pip install neptune-client neptune-contrib

You also need to have DALEX python client installed.

pip install dalex

Note

dalex client is available for python versions 3.6 and later.

Initialize Neptune and create an experiment

import neptune

neptune.init(api_token='ANONYMOUS',
             project_qualified_name='shared/dalex-integration')

neptune.create_experiment(name='logging explanations')

Train model and create explainer

...
clf.fit(X, y)
expl = dx.Explainer(clf, X, y, label="Titanic MLP Pipeline")

Log global explanations

Explainer object contains all the information to create global explanation plots so you can simply log that to Neptune and charts will be created automatically.

If you are dealing with categorical variables you need to pass the column names explicitly.

The following charts can be logged to Neptune:

  • variable importance

  • partial dependence if numerical_features are specified

  • accumulated dependence if categorical_features are specified

import neptunecontrib.api import log_global_explanations

log_global_explanations(expl, categorical_features=["gender", "class"], numerical_features=["age", "fare"])

Log local explanations

To log local explanations you need to pass the observation and the explainer object.

The following charts can be logged to Neptune:

  • break down

  • break down with interactions

  • shap

  • ceteris paribus

  • ceteris paribus for categorical variables

import neptunecontrib.api import log_local_explanations

new_observation = pd.DataFrame({'gender': ['male'],
                                'age': [25],
                                'class': ['1st'],
                                'embarked': ['Southampton'],
                                'fare': [72],
                                'sibsp': [0],
                                'parch': 0},
                                index=['John'])

log_local_explanations(expl, new_observation)

Log explainer object

You can also log the explainer object in case you may need it later.

import neptunecontrib.api import log_explainer

log_explainer('explainer.pkl', expl)

See explanations in Neptune

Now you can go to the experiment Artifacts section and see your interactive DALEX explanations.

Check out this example experiment.

DALEX explanation charts in Neptune

Full script

import dalex as dx
import neptune
import pandas as pd
from sklearn.compose import ColumnTransformer
from sklearn.impute import SimpleImputer
from sklearn.neural_network import MLPClassifier
from sklearn.pipeline import Pipeline
from sklearn.preprocessing import StandardScaler, OneHotEncoder

from neptunecontrib.api import log_explainer, log_local_explanations, log_global_explanations

project = neptune.init(project_qualified_name='shared/dalex-integration',
                       api_token='ANONYMOUS')

data = dx.datasets.load_titanic()

X = data.drop(columns='survived')
y = data.survived

numerical_transformer = Pipeline(
    steps=[
        ('imputer', SimpleImputer(strategy='median')),
        ('scaler', StandardScaler())
    ]
)

categorical_transformer = Pipeline(
    steps=[
        ('imputer', SimpleImputer(strategy='constant', fill_value='missing')),
        ('onehot', OneHotEncoder(handle_unknown='ignore'))
    ]
)

preprocessor = ColumnTransformer(
    transformers=[
        ('num', numerical_transformer, ['age', 'fare', 'sibsp', 'parch']),
        ('cat', categorical_transformer, ['gender', 'class', 'embarked'])
    ]
)

classifier = MLPClassifier(hidden_layer_sizes=(100, 200, 20), max_iter=400, random_state=0)

clf = Pipeline(steps=[('preprocessor', preprocessor),
                      ('classifier', classifier)])

neptune.create_experiment('explanations')

clf.fit(X, y)

new_observation = pd.DataFrame({'gender': ['male'],
                                'age': [25],
                                'class': ['1st'],
                                'embarked': ['Southampton'],
                                'fare': [72],
                                'sibsp': [0],
                                'parch': 0},
                               index=['John'])

expl = dx.Explainer(clf, X, y, label="Titanic MLP Pipeline")

log_local_explanations(expl, new_observation)
log_global_explanations(expl, categorical_features=["gender", "class"], numerical_features=["age", "fare"])
log_explainer('explainer.pkl', expl)