Keras integration guide#

Custom dashboard displaying metadata logged with Keras

Keras is an API built on top of TensorFlow. The Neptune-Keras integration logs the following metadata automatically:

Model summary
Model diagram
Parameters of the optimizer used for training the model
Parameters passed to model.fit during the training
Model accuracy and loss at the end of each batch and epoch
Learning rate at every epoch
Hardware consumption and console output during training
Training code and git information

You can also use set the log_model_diagram option to True to save the model visualization produced by the Keras functions model_to_dot() and plot_model().

See example in Neptune Code examples

Before you start#

Sign up at neptune.ai/register.
Create a project for storing your metadata.
Have Keras installed.

Installing the integration#

Install integration onlyInstall Neptune + integration

To use your preinstalled version of Neptune together with the integration:

pip

pip install -U neptune-tensorflow-keras

conda

conda install -c conda-forge neptune-tensorflow-keras

To install both Neptune and the integration:

pip

pip install -U "neptune[tensorflow-keras]"

conda

conda install -c conda-forge neptune neptune-tensorflow-keras

Passing your Neptune credentials

Once you've registered and created a project, set your Neptune API token and full project name to the NEPTUNE_API_TOKEN and NEPTUNE_PROJECT environment variables, respectively.

export NEPTUNE_API_TOKEN="h0dHBzOi8aHR0cHM.4kl0jvYh3Kb8...6Lc"

To find your API token: In the bottom-left corner of the Neptune app, expand the user menu and select Get my API token.

export NEPTUNE_PROJECT="ml-team/classification"

Your full project name has the form workspace-name/project-name. You can copy it from the project settings: Click the menu in the top-right → Edit project details.

On Windows, navigate to Settings → Edit the system environment variables, or enter the following in Command Prompt: setx SOME_NEPTUNE_VARIABLE 'some-value'

While it's not recommended especially for the API token, you can also pass your credentials in the code when initializing Neptune.

run = neptune.init_run(
    project="ml-team/classification",  # your full project name here
    api_token="h0dHBzOi8aHR0cHM6Lkc78ghs74kl0jvYh...3Kb8",  # your API token here
)

For more help, see Set Neptune credentials.

If you'd rather follow the guide without any setup, you can run the example in Colab .

Keras logging example#

This example shows how to use NeptuneCallback to log metadata as you train your model with Keras.

Create a run:

import neptune

run = neptune.init_run() # (1)!

If you haven't set up your credentials, you can log anonymously:

neptune.init_run(
    api_token=neptune.ANONYMOUS_API_TOKEN,
    project="common/tf-keras-integration",
)

Initialize the Neptune callback and pass it to model.fit():
```
from neptune.integrations.tensorflow_keras import NeptuneCallback

neptune_callback = NeptuneCallback(run=run) # (1)!

model.fit(
    ...,
    callbacks=[neptune_callback],
)
```
1. You can customize the base namespace (folder) of your logged metadata here, by supplying the base_namespace argument and setting it to a name of your choice. The default is "training".
Run your script as you normally would.

To open the run, click the Neptune link that appears in the console output.

Example link: https://app.neptune.ai/o/common/org/tf-keras-integration/e/TFK-18
Monitor your Keras training in Neptune.

In the Run details view, select Charts to watch the training metrics live.

Tip

You can monitor the hardware consumption in the Monitoring section.
To stop the connection to Neptune and sync all data, call the stop() method:
```
run.stop()
```

More options#

Enabling logging on batch#

You can set the callback to log metrics for each batch, in addition to each epoch, by tweaking the callback:

neptune_callback = NeptuneCallback(run=run, log_on_batch=True)

Saving the model visualization#

You can save the model visualization diagram by setting the log_model_diagram argument to True.

Note

This option requires pydot to be installed.

neptune_callback = NeptuneCallback(run=run, log_model_diagram=True)

Disable logging of model summary#

The model summary is logged by default. To disable it, set the log_model_summary argument to False.

neptune_callback = NeptuneCallback(run=run, log_model_summary=False)

Logging model weights#

You can log model weights to Neptune both during and after training.

To have all the metadata in a single place, you can log model metadata to the same run you created earlier.
To manage your model metadata separately, you can use the Neptune model registry.

Logging to the runUsing the model registry

import glob

model.save("my_model.keras")

run["saved_model"].upload("my_model.keras")

Initialize a ModelVersion object.

You first need to create a Model object that functions as an umbrella for all the versions. You can create and manage each model version separately.

neptune_model = neptune.init_model(key="PRETRAINED")

Create a specific version of that model by passing the model ID:

model_id = neptune_model["sys/id"].fetch()
model_version = neptune.init_model_version(model=model_id)

Log metadata to the model version, just like you would for runs:

import glob

model.save("my_model.keras")
model_version["saved_model"].upload("my_model.keras")

Result

The model metadata will now be displayed in the Models section of the project.

For more, see the Model registry overview.

Uploading checkpoints#

You can set up the ModelCheckpoint to save a new checkpoint every epoch:

checkpoint_cbk = ModelCheckpoint(
    "checkpoints/ep{epoch:02d}-acc{accuracy:.3f}.keras",
    save_best_only=False,
    save_weights_only=False,
    save_freq="epoch",
) 

...
neptune_cbk = NeptuneCallback(...)
model.fit(...)

Then upload the checkpoint files after training:

run["checkpoints"].upload_files("checkpoints")

Logging test sample images#

You can log sample images to Neptune with the append() method, which will create a series of images.

from neptune.types import File

for image in x_test[:100]:
    run["test/sample_images"].append(File.as_image(image))

See in Neptune

Manually logging metadata#

If you have other types of metadata that are not covered in this guide, you can still log them using the Neptune client library.

When you initialize the run, you get a run object, to which you can assign different types of metadata in a structure of your own choosing.

import neptune
from neptune.integrations.tensorflow_keras import NeptuneCallback

# Create a new Neptune run
run = neptune.init_run()

# Log metrics using NeptuneCallback
neptune_callback = NeptuneCallback(run=run)
model.fit(x_train, y_train, batch_size=32, callbacks=[neptune_callback])

# Use the same run to upload files
run["test/preds"].upload("path/to/test_preds.csv")

# Use the same run track and version artifacts
run["train/images"].track_files("./datasets/images")

# Use the same run to log numbers or text
run["tokenizer"] = "regexp_tokenize"