What will you get with this integration?

​TensorFlow is an open-source deep learning framework commonly used for building neural network models. Keras is an official higher-level API on top of TensorFlow. Neptune helps with keeping track of model training metadata.

With Neptune + TensorFlow / Keras integration you can:

• log hyperparameters for every run,

• see learning curves for losses and metrics during training,

• see hardware consumption and stdout/stderr output during training,

• log TensorFlow tensors as images to see model predictions live,

• log training code and git commit information,

• log model weights.

Where to start?

To get started with this integration, follow the quickstart below. You can also skip the basics and take a look at how to log model weights and prediction images in the more options section.

If you want to try things out and focus only on the code you can either:

Quickstart

This quickstart will show you how to:

• Install the necessary Neptune packages

• Connect Neptune to your TensorFlow / Keras model training code and create the first run

• Log metrics, training scripts and .git info to Neptune

• Explore learning curves in the Neptune UI

Before you start

You have Python 3.x and following libraries installed:

• neptune-client. See neptune-client installation guide.

• neptune-tensorflow-keras.

  • pip install neptune-tensorflow-keras

• tensorflow. See TensorFlow installation.

pip install tensorflow==2.4.1 neptune-client==0.9.4  neptune-tensorflow-keras==0.9.1

You also need minimal familiarity with TensorFlow / Keras. Have a look at this TensorFlow tutorial to get started.

Step 1: Initialize Neptune and create new run

Add to your script (at the top):

import neptune.new as neptune
​
run = neptune.init(project='common/tf-keras-integration',
                   api_token='ANONYMOUS')

This opens a new “run” in Neptune to which you can log various objects.

You need to tell Neptune who you are and where you want to log things. To do that you specify:

• project=my_workspace/my_project: your workspace name and project name,

• api_token=YOUR_API_TOKEN : your Neptune API token.

Step 2: Add NeptuneCallback to model.fit()

Import and instantiate NeptuneCallback and pass it to the callbacks argument of model.fit().

from neptune.new.integrations.tensorflow_keras import NeptuneCallback
​
neptune_cbk = NeptuneCallback(run=run, base_namespace='metrics')
​
model.fit(x_train, y_train,
          epochs=5,
          batch_size=64,
          callbacks=[neptune_cbk])

Step 3: Run your training script

Run your script as you normally would:

python train.py

Step 4: Monitor your TensorFlow / Keras training in Neptune

Now you can switch to the Neptune tab which you had opened previously to watch the training live!

More Options

Log hardware consumption and stderr/stdout

Neptune can automatically log your CPU and GPU consumption during training as well as stderr and stdout from your console. To do that you just need to install psutil.

pip install psutil

Log hyperparameters

If you want to log hyperparameters of your model training run you just need to pass them to the base_namespace of your choice.

PARAMS = {'lr':0.005, 
          'momentum':0.9, 
          'epochs':10,
          'batch_size':32}
​
# log hyper-parameters
run['hyper-parameters'] = PARAMS

Log model weights

You can log model weights to Neptune both during and after training. To do that just use a upload()method on the saved model file.

import glob
​
model.save('my_model')
​
run_2['my_model/saved_model'].upload('my_model/saved_model.pb')
​
for name in glob.glob('my_model/variables/*'):
    run_2[name].upload(name)

Log test sample images

You can log sample images to Neptune, by using the .log() method, that will create series of images.

for image in x_test[:100]:
    run_2['test/sample_images'].log(neptune.types.File.as_image(image))

Remember that you can try it out with zero setup:

How to ask for help?

Please visit the Getting help page. Everything regarding support is there.

Other pages you may like

You may also find the following pages useful:

• ​TensorBoard integration​

• ​Full list of objects you can log and display in Neptune​

• ​Optuna integration​

• ​Logging Plotly/Bokeh/Altair/Matplotlib charts to Neptune​