Skip to content

neptune#

You can use the global neptune object to initialize new Neptune objects or resume existing ones.

Import statement
import neptune

init_project()#

Starts a connection to an existing Neptune project. You can use it to fetch metadata of runs in the project.

You can also use the project object to manage metadata common to the whole project, such as information about datasets, links to documents, or key project metrics.

Parameters

Name      Type Default     Description
project str, optional None Name of a project in the form workspace-name/project-name. If None, the value of the NEPTUNE_PROJECT environment variable is used.
api_token str, optional None Your Neptune API token (or a service account's API token). If None, the value of the NEPTUNE_API_TOKEN environment variable is used.

âš  To keep your token secure, avoid placing it in source code. Instead, save it as an environment variable.

mode str, optional async Connection mode in which the logging will work. Possible values are async, sync, read-only, and debug.

If you leave it out, the value of the NEPTUNE_MODE environment variable is used. If that's not set, the default async is used.

flush_period float, optional 5 (seconds) In asynchronous (default) connection mode, how often Neptune should trigger disk flushing.
proxies dict, optional None Argument passed to HTTP calls made via the Requests library. For details on proxies, see the Requests documentation.
async_lag_callback NeptuneObjectCallback, optional None Custom callback function which is called if the lag between a queued operation and its synchronization with the server exceeds the duration defined by async_lag_threshold. The callback should take a Project object as the argument and can contain any custom code, such as calling stop() on the object.

Note: Instead of using this argument, you can use Neptune's default callback by setting the NEPTUNE_ENABLE_DEFAULT_ASYNC_LAG_CALLBACK environment variable to TRUE.

async_lag_threshold float, optional 1800.0 (seconds) Duration between the queueing and synchronization of an operation. If a lag callback (default callback enabled via environment variable or custom callback passed to the async_lag_callback argument) is enabled, the callback is called when this duration is exceeded.
async_no_progress_callback NeptuneObjectCallback, optional None Custom callback function which is called if there has been no synchronization progress whatsoever for the duration defined by async_no_progress_threshold. The callback should take a Project object as the argument and can contain any custom code, such as calling stop() on the object.

Note: Instead of using this argument, you can use Neptune's default callback by setting the NEPTUNE_ENABLE_DEFAULT_ASYNC_NO_PROGRESS_CALLBACK environment variable to TRUE.

async_no_progress_threshold float, optional 300.0 (seconds) For how long there has been no synchronization progress. If a no-progress callback (default callback enabled via environment variable or custom callback passed to the async_no_progress_callback argument) is enabled, the callback is called when this duration is exceeded.
Deprecated name parameter

The name parameter was changed to project in neptune-client 0.16.16.

Returns

Project object that can be used to interact with the project as a whole, like logging or fetching project-level metadata.

Example

Connect to the project "classification" in the workspace "ml-team":

import neptune

project = neptune.init_project(project="ml-team/classification")

project["general/data_analysis"].upload("data_analysis.ipynb")
Note on collaboration

The project object follows the same logic as other Neptune objects: If you assign a new value to an existing field, the new value overwrites the previous one.

In a given project, you always initialize and work with the same project object, so take care not to accidentally overwrite each other's entries if your team is collaborating on project metadata.

Tip: Recall that the append() method appends the logged value to a series. It works for text strings as well as numerical values.

Connect to a project in read-only mode:

project = neptune.init_project(
    project="ml-team/classification",
    mode="read-only",
)

init_run()#

Starts a new tracked run and adds it to the top of the experiments table.

Since all parameters are optional, the minimal invocation is:

import neptune

run = neptune.init_run()

See also: Defining a custom init_run() function, for keeping the logging consistent within your team.

Note for interactive sessions

If you initialize a run in an interactive session (Python interpreter or Jupyter Notebook), the following monitoring options are disabled by default:

  • capture_stdout
  • capture_stderr
  • capture_traceback
  • capture_hardware_metrics

To enable them, set each parameter to True when initializing the run. âš  Note: The monitoring will continue until you call run.stop() or the kernel stops.

Also note: Your source files can only be tracked if you pass the path(s) to the source_code argument. For help, see Logging source code.

Parameters

Name      Type Default     Description
project str, optional None Name of a project in the form workspace-name/project-name. If None, the value of the NEPTUNE_PROJECT environment variable is used.
api_token str, optional None Your Neptune API token (or a service account's API token). If None, the value of the NEPTUNE_API_TOKEN environment variable is used.

To keep your token secure, avoid placing it in source code. Instead, save it as an environment variable.

with_id str, optional None The Neptune identifier of an existing run to resume, such as "CLS-11". The identifier is stored in the object's sys/id field. If omitted or None is passed, a new tracked run is created.
custom_run_id str, optional None A unique identifier that can be used to log metadata to a single run from multiple locations. Max length: 36 characters. If None and the NEPTUNE_CUSTOM_RUN_ID environment variable is set, Neptune will use that as the custom_run_id value. For details, see Set custom run ID.
mode str, optional async Connection mode in which the logging will work. Possible values are async, sync, offline, read-only, and debug.

If you leave it out, the value of the NEPTUNE_MODE environment variable is used. If that's not set, the default async is used.

name str, optional Neptune ID Custom name for the run. You can use it as a human-readable ID and add it as a column in the experiments table (sys/name). If left empty, once the run is synchronized with the server, Neptune sets the auto-generated identifier (sys/id) as the name.
description str, optional "" Editable description of the run. You can add it as a column in the experiments table (sys/description).
tags list, optional [] Must be a list of str which represent the tags for the run. You can edit them after run is created, either in the run information or experiments table.
source_files list or str, optional None

List of source files to be uploaded. Must be list of str or a single str. Uploaded sources are displayed in the Source code section of the run.

If None is passed, the Python file from which the run was created will be uploaded. When resuming a run, no file will be uploaded by default. Pass an empty list ([]) to upload no files.

Unix style pathname pattern expansion is supported. For example, you can pass ".py" to upload all Python source files from the current directory. Paths of uploaded files are resolved relative to the calculated common root of all uploaded source files. For recursion lookup, use "**/.py" (for Python 3.5 and later). For details, see the glob library.

capture_stdout Boolean, optional True Whether to log the standard output stream. Is logged in the monitoring namespace.
capture_stderr Boolean, optional True Whether to log the standard error stream. Is logged in the monitoring namespace.
capture_hardware_metrics Boolean, optional True Whether to track hardware consumption (CPU, GPU, memory utilization). Logged in the monitoring namespace.
fail_on_exception Boolean, optional True If an uncaught exception occurs, whether to set run's Failed state to True.
monitoring_namespace str, optional "monitoring" Namespace inside which all monitoring logs will be stored.
flush_period float, optional 5 (seconds) In asynchronous (default) connection mode, how often Neptune should trigger disk flushing.
proxies dict, optional None Argument passed to HTTP calls made via the Requests library. For details on proxies, see the Requests documentation.
capture_traceback Boolean, optional True In case of an exception, whether to log the traceback of the run.
git_ref GitRef or Boolean None GitRef object containing information about the Git repository path.

If None, Neptune looks for a repository in the path of the script that is executed.

To specify a different location, set to GitRef(repository_path="path/to/repo").

To turn off Git tracking for the run, set to GitRef.DISABLED or False.

For examples, see Logging Git info.
dependencies str, optional None Tracks environment requirements. If you pass "infer" to this argument, Neptune logs dependencies installed in the current environment. You can also pass a path to your dependency file directly. If left empty, no dependency file is uploaded.
async_lag_callback NeptuneObjectCallback, optional None Custom callback function which is called if the lag between a queued operation and its synchronization with the server exceeds the duration defined by async_lag_threshold. The callback should take a Run object as the argument and can contain any custom code, such as calling stop() on the object.

Note: Instead of using this argument, you can use Neptune's default callback by setting the NEPTUNE_ENABLE_DEFAULT_ASYNC_LAG_CALLBACK environment variable to TRUE.

async_lag_threshold float, optional 1800.0 (seconds) Duration between the queueing and synchronization of an operation. If a lag callback (default callback enabled via environment variable or custom callback passed to the async_lag_callback argument) is enabled, the callback is called when this duration is exceeded.
async_no_progress_callback NeptuneObjectCallback, optional None Custom callback function which is called if there has been no synchronization progress whatsoever for the duration defined by async_no_progress_threshold. The callback should take a Run object as the argument and can contain any custom code, such as calling stop() on the object.

Note: Instead of using this argument, you can use Neptune's default callback by setting the NEPTUNE_ENABLE_DEFAULT_ASYNC_NO_PROGRESS_CALLBACK environment variable to TRUE.

async_no_progress_threshold float, optional 300.0 (seconds) For how long there has been no synchronization progress. If a no-progress callback (default callback enabled via environment variable or custom callback passed to the async_no_progress_callback argument) is enabled, the callback is called when this duration is exceeded.
Deprecated run parameter

The run parameter was changed to with_id in neptune-client 0.16.7.

Returns

Run object that is used to manage the tracked run and log metadata to it.

Examples

Create a new run:

import neptune

run = neptune.init_run()

Create a run with a custom name:

run = neptune.init_run(name="timid-bramble")

Create a run with:

  • a name and description
  • custom monitoring namespace (recommended)
  • no sources files or Git info uploaded
run = neptune.init_run(
    name="jolly-butter",
    description="neural net trained on MNIST",
    monitoring_namespace="monitoring",
    source_files=[],
    git_ref=False,
)

Resume an existing run, to continue logging to it:

import neptune

run = neptune.init_run(with_id="CLAS-134")

run["namespace/field"] = some_metadata
...
How do I find the ID?

The Neptune ID is a unique identifier for the run. The Experiments tab displays it in the leftmost column.

In the run structure, the ID is stored in the system namespace (sys).

  • If the run is active, you can obtain its ID with run["sys/id"].fetch(). For example:

    >>> run = neptune.init_run()
    ...
    >>> run["sys/id"].fetch()
    'CLS-26'
    
  • If you set a custom run ID, it's stored at sys/custom_run_id:

    >>> run["sys/custom_run_id"].fetch()
    'vigilant-puffin-20bt9'
    

Resume a run in read-only mode, to only read metadata from it:

run = neptune.init_run(with_id="CLAS-134", mode="read-only")

run["train/model_weights"].download()

Resume a run in order to delete data:

run = neptune.init_run(
    with_id="CLAS-134",
    capture_hardware_metrics=False,
    capture_stderr=False,
    capture_stdout=False,
    capture_traceback=False,
    git_ref=False,
    source_code=[],
)

del run["namespace"]

Start a new run and log all Python files in the current working directory (excluding hidden files with names beginning with a dot):

run = neptune.init_run(source_files="*.py")

Log all Python files from all subdirectories (excluding hidden files):

run = neptune.init_run(source_files="**/*.py")

Log all files and directories in the current working directory (excluding hidden files):

run = neptune.init_run(source_files="*")

Log all files and directories in the current working directory, including hidden files

run = neptune.init_run(source_files=["*", ".*"])

Larger example:

run = neptune.init_run(
    name="first-pytorch-ever",
    description="Write a longer description here.",
    tags=["pytorch", "test", "do-not-delete"],
    source_files=["training_with_pytorch.py", "net.py"],
    capture_hardware_metrics=False,
    dependencies="infer",
)

While it's not a recommended practice, you can also pass your Neptune credentials in the code when initializing Neptune.

run = neptune.init_run(
    project="ml-team/classification",  # your full project name here
    api_token="h0dHBzOi8aHR0cHM6Lkc78ghs74kl0jvYh3Kb8",  # your API token here
)
How do I save my credentials as environment variables?

Set your Neptune API token and full project name to the NEPTUNE_API_TOKEN and NEPTUNE_PROJECT environment variables, respectively.

export NEPTUNE_API_TOKEN="h0dHBzOi8aHR0cHM.4kl0jvYh3Kb8...ifQ=="
export NEPTUNE_PROJECT="ml-team/classification"
export NEPTUNE_API_TOKEN="h0dHBzOi8aHR0cHM.4kl0jvYh3Kb8...ifQ=="
export NEPTUNE_PROJECT="ml-team/classification"
setx NEPTUNE_API_TOKEN "h0dHBzOi8aHR0cHM.4kl0jvYh3Kb8...ifQ=="
setx NEPTUNE_PROJECT "ml-team/classification"

You can also navigate to SettingsEdit the system environment variables and add the variables there.

%env NEPTUNE_API_TOKEN="h0dHBzOi8aHR0cHM.4kl0jvYh3Kb8...ifQ=="
%env NEPTUNE_PROJECT="ml-team/classification"

To find your credentials:

  • API token: In the bottom-left corner of the Neptune app, expand your user menu and select Get your API token. If you need the token of a service account, go to the workspace or project settings and enter the Service accounts settings.
  • Project name: Your full project name has the form workspace-name/project-name. You can copy it from the project menu ( Details & privacy).

If you're working in Google Colab, you can set your credentials with the os and getpass libraries:

import os
from getpass import getpass
os.environ["NEPTUNE_API_TOKEN"] = getpass("Enter your Neptune API token: ")
os.environ["NEPTUNE_PROJECT"] = "workspace-name/project-name"

ANONYMOUS_API_TOKEN#

API token for anonymous logging.

You can use this value for the api_token argument of the init functions.

Example

Start a run as an anonymous user:

import neptune

run = neptune.init_run(
    api_token=neptune.ANONYMOUS_API_TOKEN,
    project="common/quickstarts",
)

If you're not using a function from the neptune package to initialize Neptune, you can also import and use the anonymous API token separately.

NeptuneLogger and PyTorch Lightning
from pytorch_lightning import Trainer
from pytorch_lightning.loggers import NeptuneLogger

from neptune import ANONYMOUS_API_TOKEN

neptune_logger = NeptuneLogger(
    api_key=ANONYMOUS_API_TOKEN,  
    project=...,
)

Deprecated#

This part of the API is deprecated.

get_last_run()#

Deprecated

This function is not available as of neptune 1.0.

Fetches the last created run object.

Returns

run object last created by the global neptune object.

Example

import neptune

# Crate a new tracked run
neptune.init_run(name="A new approach", source_files="**/*.py")

# Oops - we didn't assign it to a "run" variable.

# Not a problem! We've got you covered:
run = neptune.get_last_run()

get_project()#

Deprecated

This function is not available as of neptune 1.0.

Use neptune.init_project(mode="read-only") instead.


init()#

Deprecated

This function is not available as of neptune 1.0.

Use neptune.init_run() instead.


init_model()#

Deprecated

This function is deprecated.

For model management using runs, see Log model metadata.

Initializes a Model object from an existing or new model.

You can use this function to create a new model from code or to perform actions on existing models.

Parameters

Name      Type Default     Description
with_id str, optional None The Neptune identifier of an existing model to resume, in the form PROJECTKEY-MODELKEY. For example, "NLU-PRE1". The identifier is stored in the object's sys/id field. If omitted or None is passed, a new model is created.
name str, optional "Untitled" A custom name for the model. You can use it as a human-readable ID and add it to the models table as a column (sys/name).
key str, optional None Key for a model. Required when creating a new model.
  • Must be unique within the project and contain at least 1 capital letter (up to 10 capital letters or numbers).
  • Forms the full model identifier, together with the project key (PROJECTKEY-MODELKEY).
  • If you're initializing an existing model, use the with_id parameter instead.
project str, optional None Name of a project in the form workspace-name/project-name. If None, the value of the NEPTUNE_PROJECT environment variable is used.
api_token str, optional None Your Neptune API token (or a service account's API token). If None, the value of the NEPTUNE_API_TOKEN environment variable is used.

To keep your token secure, avoid placing it in source code. Instead, save it as an environment variable.

mode str, optional async Connection mode in which the logging will work. Possible values are async, sync, read-only, and debug.

If you leave it out, the value of the NEPTUNE_MODE environment variable is used. If that's not set, the default async is used.

flush_period float, optional 5 (seconds) In asynchronous (default) connection mode, how often Neptune should trigger disk flushing.
proxies dict, optional None Argument passed to HTTP calls made via the Requests library. For details on proxies, see the Requests documentation.
async_lag_callback NeptuneObjectCallback, optional None Custom callback function which is called if the lag between a queued operation and its synchronization with the server exceeds the duration defined by async_lag_threshold. The callback should take a Model object as the argument and can contain any custom code, such as calling stop() on the object.

Note: Instead of using this argument, you can use Neptune's default callback by setting the NEPTUNE_ENABLE_DEFAULT_ASYNC_LAG_CALLBACK environment variable to TRUE.

async_lag_threshold float, optional 1800.0 (seconds) Duration between the queueing and synchronization of an operation. If a lag callback (default callback enabled via environment variable or custom callback passed to the async_lag_callback argument) is enabled, the callback is called when this duration is exceeded.
async_no_progress_callback NeptuneObjectCallback, optional None Custom callback function which is called if there has been no synchronization progress whatsoever for the duration defined by async_no_progress_threshold. The callback should take a Model object as the argument and can contain any custom code, such as calling stop() on the object.

Note: Instead of using this argument, you can use Neptune's default callback by setting the NEPTUNE_ENABLE_DEFAULT_ASYNC_NO_PROGRESS_CALLBACK environment variable to TRUE.

async_no_progress_threshold float, optional 300.0 (seconds) For how long there has been no synchronization progress. If a no-progress callback (default callback enabled via environment variable or custom callback passed to the async_no_progress_callback argument) is enabled, the callback is called when this duration is exceeded.
Deprecated model parameter

The model parameter was changed to with_id in neptune-client 0.16.7.

Returns

Model object that is used to manage the model and log metadata to it.

Examples

Create a new model and log the signature:

import neptune

model = neptune.init_model(key="PRE")

model["signature"].upload("model_signature.json")

You can provide the project parameter as an environment variable or directly in the init_model() function:

model = neptune.init_model(key="PRE", project="workspace-name/project-name")

When creating a model, you can give it a name:

model = neptune.init_model(key="PRE", name="Pre-trained model")

Initialize existing model with identifier "CLS-PRE":

model = neptune.init_model(with_id="CLS-PRE")

To prevent modifications when connecting to an existing model, you can connect in read-only mode:

model = neptune.init_model(with_id="CLS-PRE", mode="read-only")

model["validation/data/v0.1"].download()

init_model_version()#

Deprecated

This function is deprecated.

For model management using runs, see Log model metadata.

Initializes a ModelVersion object from an existing or new model version.

You can use this function to create a new model version from code or to perform actions on existing model versions.

Parameters

Name      Type Default     Description
with_id str, optional None The Neptune identifier of an existing model version to resume, such as "CLS-PRE-3". The identifier is stored in the object's sys/id field. If omitted or None is passed, a new model version is created.
name str, optional "Untitled" A custom name for the model version. You can use it as a human-readable ID and add it to the model versions table as a column (sys/name).
model str, optional None Identifier of the model for which the new version should be created. Required when creating a new model version. The identifier is stored in the model's sys/id field.
project str, optional None Name of a project in the form workspace-name/project-name. If None, the value of the NEPTUNE_PROJECT environment variable is used.
api_token str, optional None Your Neptune API token (or a service account's API token). If None, the value of the NEPTUNE_API_TOKEN environment variable is used.

To keep your token secure, avoid placing it in source code. Instead, save it as an environment variable.

mode str, optional async Connection mode in which the logging will work. Possible values are async, sync, read-only, and debug.

If you leave it out, the value of the NEPTUNE_MODE environment variable is used. If that's not set, the default async is used.

flush_period float, optional 5 (seconds) In asynchronous (default) connection mode, how often Neptune should trigger disk flushing.
proxies dict, optional None Argument passed to HTTP calls made via the Requests library. For details on proxies, see the Requests documentation.
async_lag_callback NeptuneObjectCallback, optional None Custom callback function which is called if the lag between a queued operation and its synchronization with the server exceeds the duration defined by async_lag_threshold. The callback should take a ModelVersion object as the argument and can contain any custom code, such as calling stop() on the object.

Note: Instead of using this argument, you can use Neptune's default callback by setting the NEPTUNE_ENABLE_DEFAULT_ASYNC_LAG_CALLBACK environment variable to TRUE.

async_lag_threshold float, optional 1800.0 (seconds) Duration between the queueing and synchronization of an operation. If a lag callback (default callback enabled via environment variable or custom callback passed to the async_lag_callback argument) is enabled, the callback is called when this duration is exceeded.
async_no_progress_callback NeptuneObjectCallback, optional None Custom callback function which is called if there has been no synchronization progress whatsoever for the duration defined by async_no_progress_threshold. The callback should take a ModelVersion object as the argument and can contain any custom code, such as calling stop() on the object.

Note: Instead of using this argument, you can use Neptune's default callback by setting the NEPTUNE_ENABLE_DEFAULT_ASYNC_NO_PROGRESS_CALLBACK environment variable to TRUE.

async_no_progress_threshold float, optional 300.0 (seconds) For how long there has been no synchronization progress. If a no-progress callback (default callback enabled via environment variable or custom callback passed to the async_no_progress_callback argument) is enabled, the callback is called when this duration is exceeded.
Deprecated version parameter

The version parameter was changed to with_id in neptune-client 0.16.7.

Returns

ModelVersion object that is used to manage the model version and log metadata to it.

Examples

Create a new model version for a model with identifier "CLS-PRE":

import neptune

model_version = neptune.init_model_version(model="CLS-PRE")

You can provide the project parameter as an environment variable or directly in the init_model_version() function:

model_version = neptune.init_model_version(
    model="CLS-PRE",
    project="ml-team/classification",
)

Initialize an existing model version with identifier "CLS-PRE-12" and log additional artifact metadata:

model_version = neptune.init_model_version(with_id="CLS-PRE-12")

model_version["dataset"].track_files("path/to/more/files")

To prevent modifications when connecting to an existing model version, you can connect in read-only mode:

model_version = neptune.init_model_version(with_id="CLS-PRE-12", mode="read-only")

val_acc = model_version["validation/metrics/acc"].fetch()

ANONYMOUS#

Deprecated

This function is not available as of neptune 1.0.

Use ANONYMOUS_API_TOKEN instead.