`neptune`¶

neptune is global object that provides the convenience of doing most of the logging using a single global variable.

With global neptune object you can:

connect to Neptune and sets a global project context with neptune.init() (ref: init())
create experiments within the current global project context with neptune.create_experiment() (ref: create_experiment())

Note

You can also explicitly create projects by:

project = neptune.init()

and then explicitly create experiments under those projects by:

exp = project.create_experiment()

The following are the main classes in the Neptune client Python library:

Class	Description
Neptune	A global object that provides the convenience of doing most of the logging using a single neptune global variable, similar to Numpy’s `import numpy as np` statement. In Neptune, you write `import neptune`.
Project	This is the Neptune project to which you want to log things. You need to create it in the application. This is a place where you can create experiments. You can create new ones and update or download information from the existing one.
Experiment	This is an object to which you log any piece of information you consider to be important during your run. Interaction with the experiment feels similar to interacting with a singleton dictionary object. Neptune gives you all the freedom - you simply log metrics, images, text, and everything else to particular names and those objects are sent to the application. You can have one or multiple experiments in one script. You can re-instantiate the experiments you have created in the past and update them.
Notebook	Contains all the information about a Neptune Jupyter Notebook.
Git Info	Keeps information about the Git repository in an experiment.

Learn how to get started with logging and managing experiment data using Neptune here.

Subpackages¶

neptune.vendor
- neptune.vendor.pynvml

Submodules¶

Package Contents¶

Classes¶

`Project`(backend, internal_id, namespace, name)	A class for storing information and managing Neptune project.
`Session`(api_token=None, proxies=None, backend=None)	A class for running communication with Neptune.

Functions¶

`init`(project_qualified_name=None, api_token=None, proxies=None, backend=None)	Initialize Neptune client library to work with
`set_project`(project_qualified_name)	Setups Neptune client library to work with specific project.
`create_experiment`(name=None, description=None, params=None, properties=None, tags=None, upload_source_files=None, abort_callback=None, logger=None, upload_stdout=True, upload_stderr=True, send_hardware_metrics=True, run_monitoring_thread=True, handle_uncaught_exceptions=True, git_info=None, hostname=None, notebook_id=None)	Create and start Neptune experiment.
`get_experiment`()
`append_tag`(tag, *tags)	Append tag(s) to the experiment on the top of experiments view.
`append_tags`(tag, *tags)	Append tag(s) to the experiment on the top of experiments view.
`remove_tag`(tag)	Removes single tag from experiment.
`set_property`(key, value)	Set key-value pair as an experiment property.
`remove_property`(key)	Removes a property with given key.
`send_metric`(channel_name, x, y=None, timestamp=None)	Log metrics (numeric values) in Neptune.
`log_metric`(log_name, x, y=None, timestamp=None)	Log metrics (numeric values) in Neptune.
`send_text`(channel_name, x, y=None, timestamp=None)	Log text data in Neptune.
`log_text`(log_name, x, y=None, timestamp=None)	Log text data in Neptune.
`send_image`(channel_name, x, y=None, name=None, description=None, timestamp=None)	Log image data in Neptune.
`log_image`(log_name, x, y=None, image_name=None, description=None, timestamp=None)	Log image data in Neptune.
`send_artifact`(artifact, destination=None)	Save an artifact (file) in experiment storage.
`delete_artifacts`(path)	Delete an artifact (file/directory) from experiment storage.
`log_artifact`(artifact, destination=None)	Save an artifact (file) in experiment storage.
`stop`(traceback=None)	Marks experiment as finished (succeeded or failed).

class neptune.Project(backend, internal_id, namespace, name)[source]¶

Bases: object

A class for storing information and managing Neptune project.

Parameters

backend (Backend, required) – A Backend object.
internal_id (str, required) – UUID of the project.
namespace (str, required) – It can either be your workspace or user name.
name (str, required) – project name.

Note

namespace and name joined together with / form project_qualified_name.

full_id¶: Project qualified name as str, for example john/sandbox.

get_members(self)[source]¶

Retrieve a list of project members.

Returns: list of str - A list of usernames of project members.

Examples

project = session.get_projects('neptune-ai')['neptune-ai/Salt-Detection']
project.get_members()

get_experiments(self, id=None, state=None, owner=None, tag=None, min_running_time=None)[source]¶

Retrieve list of experiments matching the specified criteria.

All parameters are optional, each of them specifies a single criterion. Only experiments matching all of the criteria will be returned.

Parameters

id (str or list of str, optional, default is None) –

An experiment id like 'SAN-1' or list of ids like ['SAN-1', 'SAN-2'].

Matching any element of the list is sufficient to pass criterion.
state (str or list of str, optional, default is None) –

An experiment state like 'succeeded' or list of states like ['succeeded', 'running'].

Possible values: 'running', 'succeeded', 'failed', 'aborted'.

Matching any element of the list is sufficient to pass criterion.
owner (str or list of str, optional, default is None) –

Username of the experiment owner (User who created experiment is an owner) like 'josh' or list of owners like ['frederic', 'josh'].

Matching any element of the list is sufficient to pass criterion.
tag (str or list of str, optional, default is None) –

An experiment tag like 'lightGBM' or list of tags like ['pytorch', 'cycleLR'].

Only experiments that have all specified tags will match this criterion.
min_running_time (int, optional, default is None) – Minimum running time of an experiment in seconds, like 2000.

Returns

list of Experiment objects.

Examples

# Fetch a project
project = session.get_projects('neptune-ai')['neptune-ai/Salt-Detection']

# Get list of experiments
project.get_experiments(state=['aborted'], owner=['neyo'], min_running_time=100000)

# Example output:
# [Experiment(SAL-1609),
#  Experiment(SAL-1765),
#  Experiment(SAL-1941),
#  Experiment(SAL-1960),
#  Experiment(SAL-2025)]

get_leaderboard(self, id=None, state=None, owner=None, tag=None, min_running_time=None)[source]¶

Fetch Neptune experiments view as pandas DataFrame.

returned DataFrame

In the returned DataFrame each row is an experiment and columns represent all system properties,
numeric and text logs, parameters and properties in these experiments.
Note that, returned DataFrame does not contain all columns across the entire project.
Some columns may be empty, since experiments may define various logs, properties, etc.
For each log at most one (the last one) value is returned per experiment.
Text values are trimmed to 255 characters.

about parameters

All parameters are optional, each of them specifies a single criterion. Only experiments matching all of the criteria will be returned.

Parameters

id (str or list of str, optional, default is None) –

An experiment id like 'SAN-1' or list of ids like ['SAN-1', 'SAN-2'].

Matching any element of the list is sufficient to pass criterion.
state (str or list of str, optional, default is None) –

An experiment state like 'succeeded' or list of states like ['succeeded', 'running'].

Possible values: 'running', 'succeeded', 'failed', 'aborted'.

Matching any element of the list is sufficient to pass criterion.
owner (str or list of str, optional, default is None) –

Username of the experiment owner (User who created experiment is an owner) like 'josh' or list of owners like ['frederic', 'josh'].

Matching any element of the list is sufficient to pass criterion.
tag (str or list of str, optional, default is None) –

An experiment tag like 'lightGBM' or list of tags like ['pytorch', 'cycleLR'].

Only experiments that have all specified tags will match this criterion.
min_running_time (int, optional, default is None) – Minimum running time of an experiment in seconds, like 2000.

Returns

pandas.DataFrame - Fetched Neptune experiments view.

Examples

# Fetch a project.
project = session.get_projects('neptune-ai')['neptune-ai/Salt-Detection']

# Get DataFrame that resembles experiment view.
project.get_leaderboard(state=['aborted'], owner=['neyo'], min_running_time=100000)

create_experiment(self, name=None, description=None, params=None, properties=None, tags=None, upload_source_files=None, abort_callback=None, logger=None, upload_stdout=True, upload_stderr=True, send_hardware_metrics=True, run_monitoring_thread=True, handle_uncaught_exceptions=True, git_info=None, hostname=None, notebook_id=None, notebook_path=None)[source]¶

Create and start Neptune experiment.

Create experiment, set its status to running and append it to the top of the experiments view. All parameters are optional, hence minimal invocation: neptune.create_experiment().

Parameters

name (str, optional, default is 'Untitled') – Editable name of the experiment. Name is displayed in the experiment’s Details (Metadata section) and in experiments view as a column.
description (str, optional, default is '') – Editable description of the experiment. Description is displayed in the experiment’s Details (Metadata section) and can be displayed in the experiments view as a column.
params (dict, optional, default is {}) – Parameters of the experiment. After experiment creation params are read-only (see: get_parameters()). Parameters are displayed in the experiment’s Details (Parameters section) and each key-value pair can be viewed in experiments view as a column.
properties (dict, optional, default is {}) – Properties of the experiment. They are editable after experiment is created. Properties are displayed in the experiment’s Details (Properties section) and each key-value pair can be viewed in experiments view as a column.
tags (list, optional, default is []) – Must be list of str. Tags of the experiment. They are editable after experiment is created (see: append_tag() and remove_tag()). Tags are displayed in the experiment’s Details (Metadata section) and can be viewed in experiments view as a column.
upload_source_files (list or str, optional, default is None) –
List of source files to be uploaded. Must be list of str or single str. Uploaded sources are displayed in the experiment’s Source code tab.

If None is passed, Python file from which experiment was created will be uploaded.

Pass empty list ([]) to upload no files.

Unix style pathname pattern expansion is supported. For example, you can pass '*.py' to upload all python source files from the current directory. For Python 3.5 or later, paths of uploaded files on server are resolved as relative to the

calculated common root of all uploaded source files. For older Python versions, paths on server are

resolved always as relative to the current directory. For recursion lookup use '**/*.py' (for Python 3.5 and later). For more information see glob library.
abort_callback (callable, optional, default is None) –
Callback that defines how abort experiment action in the Web application should work. Actual behavior depends on your setup:
- (default) If abort_callback=None and psutil is installed, then current process and it’s children are aborted by sending SIGTERM. If, after grace period, processes are not terminated, SIGKILL is sent.
- If abort_callback=None and psutil is not installed, then abort experiment action just marks experiment as aborted in the Web application. No action is performed on the current process.
- If abort_callback=callable, then callable is executed when abort experiment action in the Web application is triggered.
logger (logging.Logger or None, optional, default is None) – If Python’s Logger is passed, new experiment’s text log (see: log_text()) with name “logger” is created. Each time Python logger logs new data, it is automatically sent to the “logger” in experiment. As a results all data from Python logger are in the Logs tab in the experiment.
upload_stdout (Boolean, optional, default is True) – Whether to send stdout to experiment’s Monitoring.
upload_stderr (Boolean, optional, default is True) – Whether to send stderr to experiment’s Monitoring.
send_hardware_metrics (Boolean, optional, default is True) – Whether to send hardware monitoring logs (CPU, GPU, Memory utilization) to experiment’s Monitoring.
run_monitoring_thread (Boolean, optional, default is True) – Whether to run thread that pings Neptune server in order to determine if experiment is responsive.
handle_uncaught_exceptions (Boolean, optional, default is True) –
Two options True and False are possible:
- If set to True and uncaught exception occurs, then Neptune automatically place Traceback in the experiment’s Details and change experiment status to Failed.
- If set to False and uncaught exception occurs, then no action is performed in the Web application. As a consequence, experiment’s status is running or not responding.
git_info (GitInfo, optional, default is None) –

Instance of the class GitInfo that provides information about the git repository from which experiment was started.

If None is passed, system attempts to automatically extract information about git repository in the following way:
- System looks for .git file in the current directory and, if not found, goes up recursively until .git file will be found (see: get_git_info()).
- If there is no git repository, then no information about git is displayed in experiment details in Neptune web application.
hostname (str, optional, default is None) – If None, neptune automatically get hostname information. User can also set hostname directly by passing str.

Returns

Experiment object that is used to manage experiment and log data to it.

Raises

ExperimentValidationError – When provided arguments are invalid.
ExperimentLimitReached – When experiment limit in the project has been reached.

Examples

# minimal invoke
neptune.create_experiment()

# explicitly return experiment object
experiment = neptune.create_experiment()

# create experiment with name and two parameters
neptune.create_experiment(name='first-pytorch-ever',
                          params={'lr': 0.0005,
                                  'dropout': 0.2})

# create experiment with name and description, and no sources files uploaded
neptune.create_experiment(name='neural-net-mnist',
                          description='neural net trained on MNIST',
                          upload_source_files=[])

# Send all py files in cwd (excluding hidden files with names beginning with a dot)
neptune.create_experiment(upload_source_files='*.py')

# Send all py files from all subdirectories (excluding hidden files with names beginning with a dot)
# Supported on Python 3.5 and later.
neptune.create_experiment(upload_source_files='**/*.py')

# Send all files and directories in cwd (excluding hidden files with names beginning with a dot)
neptune.create_experiment(upload_source_files='*')

# Send all files and directories in cwd including hidden files
neptune.create_experiment(upload_source_files=['*', '.*'])

# Send files with names being a single character followed by '.py' extension.
neptune.create_experiment(upload_source_files='?.py')

# larger example
neptune.create_experiment(name='first-pytorch-ever',
                          params={'lr': 0.0005,
                                  'dropout': 0.2},
                          properties={'key1': 'value1',
                                      'key2': 17,
                                      'key3': 'other-value'},
                          description='write longer description here',
                          tags=['list-of', 'tags', 'goes-here', 'as-list-of-strings'],
                          upload_source_files=['training_with_pytorch.py', 'net.py'])

_get_experiment_link(self, experiment)[source]¶

create_notebook(self)[source]¶

Create a new notebook object and return corresponding Notebook instance.

Returns: Notebook object.

Examples

# Instantiate a session and fetch a project
project = neptune.init()

# Create a notebook in Neptune
notebook = project.create_notebook()

get_notebook(self, notebook_id)[source]¶

Get a Notebook object with given notebook_id.

Returns: Notebook object.

Examples

# Instantiate a session and fetch a project
project = neptune.init()

# Get a notebook object
notebook = project.get_notebook('d1c1b494-0620-4e54-93d5-29f4e848a51a')

__str__(self)[source]¶: Return str(self).

__repr__(self)[source]¶: Return repr(self).

__eq__(self, o)[source]¶: Return self==value.

__ne__(self, o)[source]¶: Return self!=value.

_fetch_leaderboard(self, id, state, owner, tag, min_running_time)[source]¶

static _sort_leaderboard_columns(column_names)[source]¶

_get_current_experiment(self)[source]¶

_push_new_experiment(self, new_experiment)[source]¶

_remove_stopped_experiment(self, experiment)[source]¶

_shutdown_hook(self)[source]¶

class neptune.Session(api_token=None, proxies=None, backend=None)[source]¶

Bases: object

A class for running communication with Neptune.

In order to query Neptune experiments you need to instantiate this object first.

Parameters

backend (Backend, optional, default is None) –
By default, Neptune client library sends logs, metrics, images, etc to Neptune servers: either publicly available SaaS, or an on-premises installation.

You can pass the default backend instance explicitly to specify its parameters:
```
from neptune import Session, HostedNeptuneBackend
session = Session(backend=HostedNeptuneBackend(...))
```
Passing an instance of OfflineBackend makes your code run without communicating with Neptune servers.
```
from neptune import Session, OfflineBackend
session = Session(backend=OfflineBackend())
```
api_token (str, optional, default is None) –
User’s API token. If None, the value of NEPTUNE_API_TOKEN environment variable will be taken. Parameter is ignored if backend is passed.

Deprecated since version 0.4.4.

Instead, use:
```
from neptune import Session
session = Session.with_default_backend(api_token='...')
```
proxies (str, optional, default is None) –
Argument passed to HTTP calls made via the Requests library. For more information see their proxies section. Parameter is ignored if backend is passed.

Deprecated since version 0.4.4.

Instead, use:
```
from neptune import Session, HostedNeptuneBackend
session = Session(backend=HostedNeptuneBackend(proxies=...))
```

Examples

Create session, assuming you have created an environment variable NEPTUNE_API_TOKEN

from neptune import Session
session = Session.with_default_backend()

Create session and pass api_token

from neptune import Session
session = Session.with_default_backend(api_token='...')

Create an offline session

from neptune import Session, OfflineBackend
session = Session(backend=OfflineBackend())

classmethod with_default_backend(cls, api_token=None)[source]¶

The simplest way to instantiate a Session.

Parameters: api_token (str) – User’s API token. If None, the value of NEPTUNE_API_TOKEN environment variable will be taken.

Examples

from neptune import Session
session = Session.with_default_backend()

get_project(self, project_qualified_name)[source]¶

Get a project with given project_qualified_name.

In order to access experiments data one needs to get a Project object first. This method gives you the ability to do that.

Parameters: project_qualified_name (str) – Qualified name of a project in a form of namespace/project_name.
Returns: Project object.

Raise:: ProjectNotFound: When a project with given name does not exist.

Examples

# Create a Session instance
from neptune.sessions import Session
session = Session()

# Get a project by it's ``project_qualified_name``:
my_project = session.get_project('namespace/project_name')

get_projects(self, namespace)[source]¶

Get all projects that you have permissions to see in given workspace.

This method gets you all available projects names and their
corresponding Project objects.
Both private and public projects may be returned for the workspace.
If you have role in private project, it is included.
You can retrieve all the public projects that belong to any user or workspace,
as long as you know their username or workspace name.

Parameters

namespace (str) – It can either be name of the workspace or username.

Returns

OrderedDict: keys are project_qualified_name that is: ‘workspace/project_name’

values are corresponding Project objects.

Raises

NamespaceNotFound – When the given namespace does not exist.

Examples

# create Session
from neptune.sessions import Session
session = Session()

# Now, you can list all the projects available for a selected namespace.
# You can use `YOUR_NAMESPACE` which is your workspace or user name.
# You can also list public projects created in other workspaces.
# For example you can use the `neptune-ai` namespace.

session.get_projects('neptune-ai')

# Example output:
# OrderedDict([('neptune-ai/credit-default-prediction',
#               Project(neptune-ai/credit-default-prediction)),
#              ('neptune-ai/GStore-Customer-Revenue-Prediction',
#               Project(neptune-ai/GStore-Customer-Revenue-Prediction)),
#              ('neptune-ai/human-protein-atlas',
#               Project(neptune-ai/human-protein-atlas)),
#              ('neptune-ai/Ships',
#               Project(neptune-ai/Ships)),
#              ('neptune-ai/Mapping-Challenge',
#               Project(neptune-ai/Mapping-Challenge))
#              ])

neptune.__version__¶

neptune.session¶

neptune.project¶

neptune.__lock¶

neptune._logger¶: Access Neptune as an anonymous user. You can pass this value as api_token during init() call, either by an environment variable or passing it directly

neptune.ANONYMOUS¶: Anonymous user API token. You can pass this value as api_token during init() call, either by an environment variable or passing it directly

neptune.ANONYMOUS_API_TOKEN¶

neptune.init(project_qualified_name=None, api_token=None, proxies=None, backend=None)[source]¶

Initialize Neptune client library to work with specific project.

Authorize user, sets value of global variable project to Project object that can be used to create or list experiments, notebooks, etc.

Parameters

project_qualified_name (str, optional, default is None) – Qualified name of a project in a form of namespace/project_name. If None, the value of NEPTUNE_PROJECT environment variable will be taken.
api_token (str, optional, default is None) –
User’s API token. If None, the value of NEPTUNE_API_TOKEN environment variable will be taken.

Note

It is strongly recommended to use NEPTUNE_API_TOKEN environment variable rather than placing your API token in plain text in your source code.
proxies (dict, optional, default is None) –
Argument passed to HTTP calls made via the Requests library. For more information see their proxies section.

Note

Only http and https keys are supported by all features.

Deprecated since version 0.4.4.

Instead, use:
```
from neptune import HostedNeptuneBackend
neptune.init(backend=HostedNeptuneBackend(proxies=...))
```
backend (Backend, optional, default is None) –
By default, Neptune client library sends logs, metrics, images, etc to Neptune servers: either publicly available SaaS, or an on-premises installation.

You can also pass the default backend instance explicitly to specify its parameters:
```
from neptune import HostedNeptuneBackend
neptune.init(backend=HostedNeptuneBackend(...))
```
Passing an instance of OfflineBackend makes your code run without communicating with Neptune servers.
```
from neptune import OfflineBackend
neptune.init(backend=OfflineBackend())
```
Note

Instead of passing a neptune.OfflineBackend instance as backend, you can set an environment variable NEPTUNE_BACKEND=offline to override the default behaviour.

Returns

Project object that is used to create or list experiments, notebooks, etc.

Raises

NeptuneMissingApiTokenException – When api_token is None and NEPTUNE_API_TOKEN environment variable was not set.
NeptuneMissingProjectQualifiedNameException – When project_qualified_name is None and NEPTUNE_PROJECT environment variable was not set.
InvalidApiKey – When given api_token is malformed.
Unauthorized – When given api_token is invalid.

Examples

# minimal invoke
neptune.init()

# specifying project name
neptune.init('jack/sandbox')

# running offline
neptune.init(backend=neptune.OfflineBackend())

neptune.set_project(project_qualified_name)[source]¶

Setups Neptune client library to work with specific project.

Sets value of global variable project to Project object that can be used to create or list experiments, notebooks, etc.

If Neptune client library was not previously initialized via init() call it will be initialized with API token taken from NEPTUNE_API_TOKEN environment variable.

Parameters: project_qualified_name (str) – Qualified name of a project in a form of namespace/project_name.
Returns: Project object that is used to create or list experiments, notebooks, etc.
Raises: NeptuneMissingApiTokenException – When library was not initialized previously by init call and NEPTUNE_API_TOKEN environment variable is not set.

Examples

# minimal invoke
neptune.set_project('jack/sandbox')

neptune.create_experiment(name=None, description=None, params=None, properties=None, tags=None, upload_source_files=None, abort_callback=None, logger=None, upload_stdout=True, upload_stderr=True, send_hardware_metrics=True, run_monitoring_thread=True, handle_uncaught_exceptions=True, git_info=None, hostname=None, notebook_id=None)[source]¶

Create and start Neptune experiment.

Alias for: create_experiment()

neptune.get_experiment()[source]¶

neptune.append_tag(tag, *tags)[source]¶

Append tag(s) to the experiment on the top of experiments view.

Alias for: append_tag()

neptune.append_tags(tag, *tags)[source]¶

Append tag(s) to the experiment on the top of experiments view.

Alias for: append_tags()

neptune.remove_tag(tag)[source]¶

Removes single tag from experiment.

Alias for: remove_tag()

neptune.set_property(key, value)[source]¶

Set key-value pair as an experiment property.

If property with given key does not exist, it adds a new one.

Alias for: set_property()

neptune.remove_property(key)[source]¶

Removes a property with given key.

Alias for: remove_property()

neptune.send_metric(channel_name, x, y=None, timestamp=None)[source]¶

Log metrics (numeric values) in Neptune.

Alias for log_metric()

neptune.log_metric(log_name, x, y=None, timestamp=None)[source]¶

Log metrics (numeric values) in Neptune.

Alias for log_metric()

neptune.send_text(channel_name, x, y=None, timestamp=None)[source]¶

Log text data in Neptune.

Alias for log_text()

neptune.log_text(log_name, x, y=None, timestamp=None)[source]¶

Log text data in Neptune.

Alias for log_text()

neptune.send_image(channel_name, x, y=None, name=None, description=None, timestamp=None)[source]¶

Log image data in Neptune.

Alias for log_image()

neptune.log_image(log_name, x, y=None, image_name=None, description=None, timestamp=None)[source]¶

Log image data in Neptune.

Alias for log_image()

neptune.send_artifact(artifact, destination=None)[source]¶

Save an artifact (file) in experiment storage.

Alias for log_artifact()

neptune.delete_artifacts(path)[source]¶

Delete an artifact (file/directory) from experiment storage.

Alias for delete_artifacts()

neptune.log_artifact(artifact, destination=None)[source]¶

Save an artifact (file) in experiment storage.

Alias for log_artifact()

neptune.stop(traceback=None)[source]¶

Marks experiment as finished (succeeded or failed).

Alias for stop()

neptune¶

Subpackages¶

Submodules¶

Package Contents¶

Classes¶

Functions¶

`neptune`¶