Skip to content

Connection modes reference#

The connection mode determines how the metadata tracking will work when you initialize a run or other object.

Available connection modes:

Mode Value Description
Asynchronous (default) async Neptune stores the logged data locally and periodically synchronizes it with the server in the background.
Debug debug No data is stored or sent anywhere.
Offline offline Data is not sent to Neptune but stored locally.
Read-only read-only No data can be logged, only fetched.
Synchronous sync Neptune immediately synchronizes the data each time a tracking call (such as log() or upload()) is made.

Setting the connection mode#

You can set the mode in the following ways:

The environment variable is used if the mode parameter is omitted from the init method.

Tip

This can be a handy way to debug or "switch off" Neptune without modifying your code.

export NEPTUNE_MODE='debug'
export NEPTUNE_MODE='debug'
set NEPTUNE_MODE='debug'

When initializing a run or other object, provide the mode parameter to the initialization method:

import neptune.new as neptune

run = neptune.init_run(
    name="My debug run",
    mode="debug",  # "async", "sync", "offline", "read-only", or "debug"
)

Available modes#

Asynchronous mode#

By default, metadata tracking works asynchronously: The tracked data is temporarily stored on the local disk and periodically synchronized by a separate thread in the background. All tracking calls (like log(), assign(), upload()) are non-blocking.

To be able to restore tracked data in case you experience connectivity issues or your machine restarts (for example, if you use spot instances), you should use Neptune with persistent disks.

Disk flushing#

Neptune triggers disk flushing:

  • Every 5 seconds (default).
    • You can set a different interval with the flush_period parameter.
  • When you call wait() on a Neptune object.
  • When the connection to the object stops (the object context is destroyed).

Error handling#

In asynchronous mode, metadata tracking calls do not throw exceptions related to connectivity or metadata consistency issues. Such issues are printed to stderr.

Connectivity issues#

In asynchronous mode, if there is a problem with the connection to Neptune servers (for example, due to internet connectivity issues), the Neptune client library will track your metadata locally and continuously try to re-establish a connection with the Neptune servers.

Once your run is finished, if the connection has not been re-established, Neptune will continue trying for another 5 minutes. After this period, the process will end. All the unsuccessful tracking calls are stored on the local disk.

The unsynchronized metadata can later be uploaded later via the Neptune Command Line Interface, using the neptune sync command.

Debug mode#

Debug mode essentially turns off all the Neptune-related logging. It can come in handy when you're debugging your code and don't want to pollute your Neptune project.

In contrast to offline mode, debug mode doesn't save any data locally.

import neptune.new as neptune

run = neptune.init_run(mode="debug")

Offline mode#

In this mode, no connection to Neptune servers is established. Instead, all the tracked metadata is stored on the local disk, in the .neptune folder of the current working directory.

Tip

The data can be uploaded to Neptune servers manually with the neptune sync command.

import neptune.new as neptune

run = neptune.init_run(mode="offline")

Read-only mode#

Read-only mode is useful when you only want to fetch metadata rather than record it. For example, when you're connecting to Neptune to perform specific analysis on your run metadata.

In read-only mode:

  • No write operation is possible. Any write-like operation will be ignored.
  • No metadata is sent to Neptune. In particular, stdout and stderr are not captured, and no hardware consumption is logged.
  • The run state does not switch to active and so does not count toward monitoring hours.

You can only initialize pre-existing objects in read-only mode.

import neptune.new as neptune

# Connecting to run with identifier NER-2
run = neptune.init_run(with_id="NER-2", mode="read-only")

Synchronous mode#

In synchronous mode, Neptune executes a tracking call only after all other calls have reached the Neptune servers.

In this mode, tracking methods throw exceptions in case of issues related to connectivity or the consistency of an object's local representation. Neptune will retry each request for 60 seconds before throwing the exception.

import neptune.new as neptune

run = neptune.init_run(name="My synchronous", mode="sync")

Tip

If there's an issue with your code but you're not sure where, synchronous mode can help you locate the problematic part.

Note that this mode may slow down your logging.