TensorFlow / Keras

Neptune integrates with both TensorFlow / Keras directly and via TensorBoard. If you are already using TensorBoard you may want to see the TensorBoard integration page.

What will you get with this integration?

TensorFlow is an open-source deep learning framework commonly used for building neural network models. Keras is an official higher-level API on top of TensorFlow. Neptune helps with keeping track of model training metadata.

With Neptune + TensorFlow / Keras integration you can:

  • log hyperparameters for every run,

  • see learning curves for losses and metrics during training,

  • see hardware consumption and stdout/stderr output during training,

  • log TensorFlow tensors as images to see model predictions live,

  • log training code and git commit information,

  • log model weights.

You can log many other run metadata like interactive charts, video, audio, and more. See the full list.

Where to start?

To get started with this integration, follow the quickstart below. You can also skip the basics and take a look at how to log model weights and prediction images in the more options section.

If you want to try things out and focus only on the code you can:

Quickstart

This quickstart will show you how to:

  • Install the necessary Neptune packages

  • Connect Neptune to your TensorFlow / Keras model training code and create the first run

  • Log metrics, training scripts, and .git info to Neptune

  • Explore learning curves in the Neptune UI

This quickstart shows only Neptune-specific code snippets. You can see the full code example in Colab/Github or Neptune. Click on one of the links above.

Before you start

Make sure that:

Install neptune-client, tensorflow, and neptune-tensorflow-keras

Depending on your operating system open a terminal or CMD and run this command. All required libraries are available via pip and conda:

pip
conda
pip
pip install tensorflow neptune-client neptune-tensorflow-keras
conda
conda install -c conda-forge neptune-client tensorflow neptune-tensorflow-keras

For more help see installing neptune-client.

This integration is tested with neptune-client==0.9.4, tensorflow==2.4.1, and neptune-tensorflow-keras==0.9.1.

You also need minimal familiarity with TensorFlow/Keras. Have a look at this TensorFlow tutorial to get started.

Step 1: Initialize Neptune and create a new run

Add to your script (at the top):

import neptune.new as neptune
run = neptune.init(project='common/tf-keras-integration',
api_token='ANONYMOUS')

This opens a new “run” in Neptune to which you can log various objects.

You need to tell Neptune who you are and where you want to log things. To do that you specify:

  • project=my_workspace/my_project: your workspace name and project name,

  • api_token=YOUR_API_TOKEN : your Neptune API token.

If you configured your Neptune API token correctly, as described in this docs page, you can skip the api_token argument.

Step 2: Add NeptuneCallback to model.fit()

Import and instantiate NeptuneCallback and pass it to the callbacks argument of model.fit().

from neptune.new.integrations.tensorflow_keras import NeptuneCallback
neptune_cbk = NeptuneCallback(run=run, base_namespace='metrics')
model.fit(x_train, y_train,
epochs=5,
batch_size=64,
callbacks=[neptune_cbk])

Step 3: Run your training script

Run your script as you normally would:

python train.py

Step 4: Stop logging

Once you are done logging, you should stop tracking the run using the stop() method. This is needed only while logging from a notebook environment. While logging through a script, Neptune automatically stops tracking once the script has completed execution.

run.stop()

Step 5: Monitor your TensorFlow / Keras training in Neptune

Now you can switch to the Neptune tab which you had opened previously to watch the training live!

More Options

Log hardware consumption and stderr/stdout

Neptune can automatically log your CPU and GPU consumption during training as well as stderr and stdout from your console. To do that you just need to install psutil.

pip
conda
pip
pip install psutil
conda
conda install psutil

Log hyperparameters

If you want to log hyperparameters of your model training run you just need to pass them to the base_namespace of your choice.

PARAMS = {'lr':0.005,
'momentum':0.9,
'epochs':10,
'batch_size':32}
# log hyper-parameters
run['hyper-parameters'] = PARAMS

Log model weights

You can log model weights to Neptune both during and after training. To do that just use the upload()method on the saved model file.

import glob
model.save('my_model')
run_2['my_model/saved_model'].upload('my_model/saved_model.pb')
for name in glob.glob('my_model/variables/*'):
run_2[name].upload(name)

Log test sample images

You can log sample images to Neptune, by using the .log() method, that will create series of images.

for image in x_test[:100]:
run_2['test/sample_images'].log(neptune.types.File.as_image(image))

You can log many other run metadata like interactive charts, video, audio, and more. See the full list.

Remember that you can try it out with zero setup:

How to ask for help?

Please visit the Getting help page. Everything regarding support is there.

Other pages you may like

You may also find the following pages useful: