Skip to content

Neptune legacy API#

The legacy client is deprecated

With the upcoming version 1.0.0 release of neptune-client, we're ending active maintenance and support for the Neptune legacy API.

We strongly encourage you to migrate to the new client at your earliest convenience.

Legacy docs#

Neptune legacy client documentation

Migrating to neptune.new#

This guide walks you through migrating your legacy code to the new Python API.

Initialization#

Legacy
import neptune

neptune.init(project_qualified_name="my-workspace/my-project")

neptune.create_experiment(tags=["resnet"])
New
import neptune.new as neptune

run = neptune.init_run(project="my-workspace/my-project", tags=["resnet"])

Note

The names of the environment variables have not changed.

Instead of specifying the project name or API token in the code, you can provide them by setting the NEPTUNE_PROJECT and NEPTUNE_API_TOKEN environment variables.

Hierarchical structure#

Runs can be viewed as nested dictionary-like structures that you can define in your code. This lets you organize your metadata in a way that is most convenient for you.

The hierarchical structure that you apply to your metadata is reflected when you view the metadata in the app.

The structure of a run consists of fields that are organized into namespaces.

The path of a field is a combination of the namespaces and its name: if you store a value (such as a float, 0.8) in a field named momentum under a namespace params, the full path of the field is params/momentum.

You can organize any type of metadata this way – such as images, parameters, metrics, scores, model checkpoints, and CSV files.

Let's look at the following code:

import neptune.new as neptune

run = neptune.init_run(project="my_workspace/my_project")

run["about/JIRA"] = "NPT-952"

run["parameters/batch_size"] = 5
run["parameters/algorithm"] = "ConvNet"

for epoch in range(100):
    acc_value = ...
    loss_value = ...
    run["train/accuracy"].log(acc_value)
    run["train/loss"].log(loss_value)

exp["trained_model"].upload("model.pt")

The resulting structure of the run will be:

'about':
    'JIRA': String
'parameters': 
    'batch_size': Float
    'algorithm': String
'train':
    'accuracy': FloatSeries
    'loss': FloatSeries
'trained_model': File

Batch assign#

You can assign values to multiple fields in batch by using a dictionary.

With the legacy API, you had to pass parameters when creating an experiment and it was not possible to change them afterward. In addition, nested dictionaries were not fully supported.

Legacy
import neptune

PARAMS = {"epoch_nr": 100, "lr": 0.005, "use_nesterov": True}

neptune.init()
neptune.create_experiment(params=PARAMS)

With the neptune.new API, it's up to you when and where you want to specify parameters. You can also update them later:

New
import neptune.new as neptune

PARAMS = {"epoch_nr": 100, "lr": 0.005, "use_nesterov": True}

run = neptune.init_run()

run["my_params"] = PARAMS

# You can also specify parameters one by one
run["my_params/batch_size"] = 64

# Update lr value
run["my_params/lr"] = 0.007

The artificial distinction between parameters and properties is also gone. You can log and access them in a single, unified way.

Interacting with files (artifacts)#

You are no longer bound to store files only in the artifacts folder.

Whether it's a model checkpoint, custom interactive visualization, or audio file, you can track it in the same hierarchical structure with the rest of the metadata:

Legacy New
neptune.log_artifact('model_viz.png') run['model/viz'].upload('model_viz.png')
neptune.log_artifact('model.pt') run['trained_model'].upload('model.pt')
neptune.download_artifact('model.pt') run['trained_model'].download()

Note

In the legacy API, artifacts mimicked a file system. What you uploaded was saved under the same name, with the extension, etc.

The idea behind the new Python API is more database-like: We have a field (with a path) and under it, we store some content - Float, String, series of String, or File. In this model, the extension is part of the content.

Example: If under the path 'model/last' you upload a .pt file, the file will be displayed as 'last.pt' in the app.

When it's unambiguous, we implicitly convert an object to type File and there is no need for explicit conversion.

Example: For Matplotlib charts, you can do

run["conf_matrix"].upload(plt_fig)

instead of

run["conf_matrix"].upload(File.as_image(plt_fig))

neptune-contrib#

We've integrated neptune-contrib file-related functionalities into the core library.

The conversion methods are available as File factory methods:

Interactive charts (Altair, Bokeh, Plotly, Matplotlib)#

Legacy New
from neptunecontrib.api import log_chart from neptune.new.types import File
log_chart('int_chart', chart) run['int_chart'].upload(File.as_html(chart))

pandas DataFrame#

Legacy New
from neptunecontrib.api import log_table from neptune.new.types import File
log_table('pred_df', df) run['pred_df'].upload(File.as_html(df))

Audio and video#

We've expanded the range of files that are natively supported in the Neptune app, so for audio and video files you no longer need to use conversion methods:

Legacy New

from neptunecontrib.api import log_audio

log_audio('sample.mp3')

run['sample'].upload('sample.mp3')

from neptunecontrib.api import log_video

log_audio('sample.mp4')

run['sample'].upload('sample.mp4')

Pickled objects#

Legacy New
from neptunecontrib.api import log_pickle from neptune.new.types import File
log_pickle('model.pkl', model) run['model'].upload(File.as_pickle(model))

HTML strings#

Legacy New
from neptunecontrib.api import log_html from neptune.new.types import File
log_html('custom_viz', html_string) run['custom_viz'].upload(File.from_content(model), extension='html')

Scores and metrics#

Logging metrics is quite similar, except that you can now organize them in a hierarchical structure:

Legacy API New API
neptune.log_metric('acc', 0.97) run['acc'].log(0.97)
neptune.log_metric('train_acc', 0.97) run['train/acc'].log(0.97)
neptune.log_metric('loss', 0.8) run['key'].log()

To log scores you don't need to use Series fields anymore as you can track single values anytime, anywhere:

Legacy API New API
neptune.log_metric('final_accuracy', 0.8) run['final_accuracy'] = 0.8

Text and image series#

Similar changes need to be applied for text and image series:

Legacy API New API
neptune.log_text('train_log', custom_log_msg) run['train/log'].log(custom_log_msg)
neptune.log_image('misclasified', filepath) run['misclasified'].log(File(filepath))
neptune.log_image('pred_dist', hist_chart) run['pred_dist'].log(hist_chart)

To add a single image that you want to view with the rest of the metrics you no longer need to use Series fields. As you control whether they are grouped in the same namespace you can upload it as a single File field.

Legacy API New API
neptune.log_image('train_hist', hist_chart) run['train/hist'].upload(hist_chart)

Integrations#

We've redesigned how our integrations work:

  • They're more tightly integrated with our base library and the API is more unified.
  • You can update the version of each integration separately in case of dependency conflicts.
  • There's a smaller number of dependencies if you're not using all integrations.

There is no longer a neptune-contrib library for the new Python API – instead, each integration has two parts:

  • Boilerplate code for ease of use in the main library:

    import neptune.new.integrations.framework_name

  • The actual integration that can be installed separately:

    pip install neptune-framework-name

    or as an extra together with neptune-client:

    pip install "neptune-client[framework-name]"

Existing integrations from neptune-contrib are still fully functional. You can use them both with projects using the previous structure and the new structure. However, integrations from neptune-contrib use the legacy Python API, while the new integrations are re-written to fully leverage the new Python API for better metadata organization.

Related

For detailed integration guides, see the Integrations tab.

Let's look at how this works like in the case of the Keras integration:

Installation

Legacy New
pip install neptune-contrib

pip install "neptune-client[tensorflow-keras]"

or

pip install neptune-tensorflow-keras

Usage

Legacy
import neptune

neptune.init()

from neptunecontrib.monitoring.keras import NeptuneMonitor

model.fit(
    x_train,
    y_train,
    epochs=5,
    batch_size=64,
    callbacks=[NeptuneMonitor()],
)
New
import neptune.new as neptune

run = neptune.init_run()

from neptune.new.integrations.tensorflow_keras import NeptuneCallback

model.fit(
    x_train,
    y_train,
    epochs=5,
    batch_size=64,
    callbacks=[NeptuneCallback(run=run)],
)

Tags#

Interaction with tags is quite similar: tags are stored as a StringSet field under the sys/tags field.

Legacy
import neptune

neptune.init()

neptune.create_experiment(params=PARAMS, tags=["maskRCNN"])

neptune.append_tag("prod_v2.0.1")
neptune.append_tags("finetune", "keras")
New
import neptune.new as neptune

run = neptune.init_run(tags=["maskRCNN"])

run["sys/tags"].add("prod_v2.0.1")
run["sys/tags"].add(["finetune", "keras"])

Logging to a project with legacy runs#

If you have a project with a set of runs (previously Experiments) that were created using the Legacy API and you would like to add runs using the new API, there are a few things to remember.

Metadata structure#

Each type of metadata from the previous version (parameters, properties, logs, artifacts, and monitoring) are mapped into a separate fixed namespace. If you want to be able to compare those, you need to make sure that the field path is backward compatible:

Legacy New
neptune.create_experiment(params=params_dict) run['parameters'] = params_dict
neptune.set_property('SEED', 2458) run['properties/SEED'] = str(2458)
neptune.log_metric('acc', 0.98) run['logs/acc'].log(0.98)
neptune.log_artifact('model.pt', 'model.pt') run['artifacts/model'].upload('model.pt')

Data types#

Neptune is type-sensitive. For example, a column in the runs table represents a field with a specific path and type. In particular, FloatSeries and Float are two different types and Neptune by default doesn't know how to compare them.

With the Legacy API, it wasn't possible to set Float fields outside of the create_experiment() function, and a common practice was to store those as FloatSeries with one entry through the use of log_metric() function. To achieve backward compatibility, you need to create a FloatSeries with the new API as well, by using the log() method.

Properties set through Legacy API are always String fields. To achieve backward compatibility, use assignment with explicit casting to str to ensure that the resulting field will be a string.