TensorFlow / Keras

Neptune integrates with both TensorFlow / Keras directly and via TensorBoard. If you are already using TensorBoard you may want to see the TensorBoard integration page.
​TensorFlow is an open-source deep learning framework commonly used for building neural network models. Keras is an official higher-level API on top of TensorFlow. Neptune helps with keeping track of model training metadata.
What will you get with this integration?
With Neptune + TensorFlow / Keras integration you can:
log hyperparameters for every run,
see learning curves for losses and metrics during training,
see hardware consumption and stdout/stderr output during training,
log TensorFlow tensors as images to see model predictions live,
log training code and git commit information,
log model weights.
You can log many other run metadata like interactive charts, video, audio, and more.
See the full list.​
Where to start?
To get started with this integration, follow the quickstart below. You can also skip the basics and take a look at how to log model weights and prediction images in the more options section.
If you want to try things out and focus only on the code you can:
​​
​​
​​
​​
​​
​​
​See code examples on GitHub​
​See results in Neptune​
​Run example in Google Colab​
Quickstart
This quickstart will show you how to:
Install the necessary Neptune packages
Connect Neptune to your TensorFlow / Keras model training code and create the first run
Log metrics, training scripts, and .git info to Neptune
Explore learning curves in the Neptune UI
This quickstart shows only Neptune-specific code snippets. You can see the full code example in Colab/Github or Neptune. Click on one of the links above.
Before you start
Make sure that:
You have Python 3.7+ in your system,
You are already registered user, so that you can log metadata to your private projects.
You have your NEPTUNE_API_TOKEN set to the environment variable: check this docs page.
Install neptune-client, tensorflow, and neptune-tensorflow-keras
Depending on your operating system open a terminal or CMD and run this command. All required libraries are available via pip and conda:
pip
conda
1
pip install tensorflow neptune-client neptune-tensorflow-keras
Copied!
1
conda install -c conda-forge neptune-client tensorflow neptune-tensorflow-keras
Copied!
For more help see installing neptune-client.
This integration is tested with neptune-client==0.9.4, tensorflow==2.4.1, and neptune-tensorflow-keras==0.9.1.
You also need minimal familiarity with TensorFlow/Keras. Have a look at this TensorFlow tutorial to get started.
Step 1: Initialize Neptune and create a new run
Add to your script (at the top):
1
import neptune.new as neptune
2
​
3
run = neptune.init(project='common/tf-keras-integration',
4
                   api_token='ANONYMOUS')
Copied!
This opens a new “run” in Neptune to which you can log various objects.
You need to tell Neptune who you are and where you want to log things. To do that you specify:
project=my_workspace/my_project: your workspace name and project name,
api_token=YOUR_API_TOKEN : your Neptune API token.
If you configured your Neptune API token correctly, as described in this docs page, you can skip the api_token argument.
Step 2: Add NeptuneCallback to model.fit()
Import and instantiate NeptuneCallback and pass it to the callbacks argument of model.fit().
1
from neptune.new.integrations.tensorflow_keras import NeptuneCallback
2
​
3
neptune_cbk = NeptuneCallback(run=run, base_namespace='metrics')
4
​
5
model.fit(x_train, y_train,
6
          epochs=5,
7
          batch_size=64,
8
          callbacks=[neptune_cbk])
Copied!
Step 3: Run your training script
Run your script as you normally would:
1
python train.py
Copied!
Step 4: Stop logging
Once you are done logging, you should stop tracking the run using the stop() method. This is needed only while logging from a notebook environment. While logging through a script, Neptune automatically stops tracking once the script has completed execution.
1
run.stop()
Copied!
Step 5: Monitor your TensorFlow / Keras training in Neptune
Now you can switch to the Neptune tab which you had opened previously to watch the training live!
More Options
Log hardware consumption and stderr/stdout
Neptune can automatically log your CPU and GPU consumption during training as well as stderr and stdout from your console.
Log hyperparameters
If you want to log hyperparameters of your model training run you just need to pass them to the base_namespace of your choice.
1
PARAMS = {'lr':0.005, 
2
          'momentum':0.9, 
3
          'epochs':10,
4
          'batch_size':32}
5
​
6
# log hyper-parameters
7
run['hyper-parameters'] = PARAMS
Copied!
Log model weights
You can log model weights to Neptune both during and after training. To do that just use the upload()method on the saved model file.
1
import glob
2
​
3
model.save('my_model')
4
​
5
run_2['my_model/saved_model'].upload('my_model/saved_model.pb')
6
​
7
for name in glob.glob('my_model/variables/*'):
8
    run_2[name].upload(name)
Copied!
Log test sample images
You can log sample images to Neptune, by using the .log() method, that will create series of images.
1
for image in x_test[:100]:
2
    run_2['test/sample_images'].log(neptune.types.File.as_image(image))
Copied!
You can log many other run metadata like interactive charts, video, audio, and more.
See the full list.​
Remember that you can try it out with zero setup:
​​
​​
​​
​​
​​
​​
​See code examples on GitHub​
​See results in Neptune​
​Run example in Google Colab​
How to ask for help?
Please visit the Getting help page. Everything regarding support is there.
Other pages you may like
You may also find the following pages useful:
​TensorBoard integration​
​Full list of objects you can log and display in Neptune​
​Optuna integration​
​Logging Plotly/Bokeh/Altair/Matplotlib charts to Neptune​