neptune
¶
neptune
is global object that provides the convenience of doing most of the logging using a single global variable.
With global neptune
object you can:
connect to Neptune and sets a global project context with
neptune.init()
(ref:init()
)create experiments within the current global project context with
neptune.create_experiment()
(ref:create_experiment()
)

Note
You can also explicitly create projects by:
project = neptune.init()
and then explicitly create experiments under those projects by:
exp = project.create_experiment()
The following are the main classes in the Neptune client Python library:
Class |
Description |
---|---|
A global object that provides the convenience of doing most of the logging using a single neptune global variable, similar to Numpy’s |
|
This is the Neptune project to which you want to log things. You need to create it in the application. This is a place where you can create experiments. You can create new ones and update or download information from the existing one. |
|
This is an object to which you log any piece of information you consider to be important during your run. Interaction with the experiment feels similar to interacting with a singleton dictionary object. Neptune gives you all the freedom - you simply log metrics, images, text, and everything else to particular names and those objects are sent to the application. You can have one or multiple experiments in one script. You can re-instantiate the experiments you have created in the past and update them. |
|
Contains all the information about a Neptune Jupyter Notebook. |
|
Keeps information about the Git repository in an experiment. |
Learn how to get started with logging and managing experiment data using Neptune here.
Subpackages¶
Package Contents¶
Classes¶
|
A class for storing information and managing Neptune project. |
|
A class for running communication with Neptune. |
Functions¶
|
Initialize Neptune client library to work with |
|
Setups Neptune client library to work with specific project. |
|
Create and start Neptune experiment. |
|
Append tag(s) to the experiment on the top of experiments view. |
|
Append tag(s) to the experiment on the top of experiments view. |
|
Removes single tag from experiment. |
|
Set key-value pair as an experiment property. |
|
Removes a property with given key. |
|
Log metrics (numeric values) in Neptune. |
|
Log metrics (numeric values) in Neptune. |
|
Log text data in Neptune. |
|
Log text data in Neptune. |
|
Log image data in Neptune. |
|
Log image data in Neptune. |
|
Save an artifact (file) in experiment storage. |
|
Delete an artifact (file/directory) from experiment storage. |
|
Save an artifact (file) in experiment storage. |
|
Marks experiment as finished (succeeded or failed). |
-
class
neptune.
Project
(backend, internal_id, namespace, name)[source]¶ Bases:
object
A class for storing information and managing Neptune project.
- Parameters
backend (
Backend
, required) – A Backend object.internal_id (
str
, required) – UUID of the project.namespace (
str
, required) – It can either be your workspace or user name.name (
str
, required) – project name.
Note
namespace
andname
joined together with/
formproject_qualified_name
.-
full_id
¶ Project qualified name as
str
, for example john/sandbox.
-
get_members
(self)[source]¶ Retrieve a list of project members.
- Returns
list
ofstr
- A list of usernames of project members.
Examples
project = session.get_projects('neptune-ai')['neptune-ai/Salt-Detection'] project.get_members()
-
get_experiments
(self, id=None, state=None, owner=None, tag=None, min_running_time=None)[source]¶ Retrieve list of experiments matching the specified criteria.
All parameters are optional, each of them specifies a single criterion. Only experiments matching all of the criteria will be returned.
- Parameters
id (
str
orlist
ofstr
, optional, default isNone
) –An experiment id like'SAN-1'
or list of ids like['SAN-1', 'SAN-2']
.Matching any element of the list is sufficient to pass criterion.state (
str
orlist
ofstr
, optional, default isNone
) –An experiment state like'succeeded'
or list of states like['succeeded', 'running']
.Possible values:'running'
,'succeeded'
,'failed'
,'aborted'
.Matching any element of the list is sufficient to pass criterion.owner (
str
orlist
ofstr
, optional, default isNone
) –Username of the experiment owner (User who created experiment is an owner) like'josh'
or list of owners like['frederic', 'josh']
.Matching any element of the list is sufficient to pass criterion.tag (
str
orlist
ofstr
, optional, default isNone
) –An experiment tag like'lightGBM'
or list of tags like['pytorch', 'cycleLR']
.Only experiments that have all specified tags will match this criterion.min_running_time (
int
, optional, default isNone
) – Minimum running time of an experiment in seconds, like2000
.
- Returns
list
ofExperiment
objects.
Examples
# Fetch a project project = session.get_projects('neptune-ai')['neptune-ai/Salt-Detection'] # Get list of experiments project.get_experiments(state=['aborted'], owner=['neyo'], min_running_time=100000) # Example output: # [Experiment(SAL-1609), # Experiment(SAL-1765), # Experiment(SAL-1941), # Experiment(SAL-1960), # Experiment(SAL-2025)]
-
get_leaderboard
(self, id=None, state=None, owner=None, tag=None, min_running_time=None)[source]¶ Fetch Neptune experiments view as pandas
DataFrame
.returned DataFrame
In the returnedDataFrame
each row is an experiment and columns represent all system properties, numeric and text logs, parameters and properties in these experiments.Note that, returnedDataFrame
does not contain all columns across the entire project.Some columns may be empty, since experiments may define various logs, properties, etc.For each log at most one (the last one) value is returned per experiment.Text values are trimmed to 255 characters.about parameters
All parameters are optional, each of them specifies a single criterion. Only experiments matching all of the criteria will be returned.
- Parameters
id (
str
orlist
ofstr
, optional, default isNone
) –An experiment id like'SAN-1'
or list of ids like['SAN-1', 'SAN-2']
.Matching any element of the list is sufficient to pass criterion.state (
str
orlist
ofstr
, optional, default isNone
) –An experiment state like'succeeded'
or list of states like['succeeded', 'running']
.Possible values:'running'
,'succeeded'
,'failed'
,'aborted'
.Matching any element of the list is sufficient to pass criterion.owner (
str
orlist
ofstr
, optional, default isNone
) –Username of the experiment owner (User who created experiment is an owner) like'josh'
or list of owners like['frederic', 'josh']
.Matching any element of the list is sufficient to pass criterion.tag (
str
orlist
ofstr
, optional, default isNone
) –An experiment tag like'lightGBM'
or list of tags like['pytorch', 'cycleLR']
.Only experiments that have all specified tags will match this criterion.min_running_time (
int
, optional, default isNone
) – Minimum running time of an experiment in seconds, like2000
.
- Returns
pandas.DataFrame
- Fetched Neptune experiments view.
Examples
# Fetch a project. project = session.get_projects('neptune-ai')['neptune-ai/Salt-Detection'] # Get DataFrame that resembles experiment view. project.get_leaderboard(state=['aborted'], owner=['neyo'], min_running_time=100000)
-
create_experiment
(self, name=None, description=None, params=None, properties=None, tags=None, upload_source_files=None, abort_callback=None, logger=None, upload_stdout=True, upload_stderr=True, send_hardware_metrics=True, run_monitoring_thread=True, handle_uncaught_exceptions=True, git_info=None, hostname=None, notebook_id=None, notebook_path=None)[source]¶ Create and start Neptune experiment.
Create experiment, set its status to running and append it to the top of the experiments view. All parameters are optional, hence minimal invocation:
neptune.create_experiment()
.- Parameters
name (
str
, optional, default is'Untitled'
) – Editable name of the experiment. Name is displayed in the experiment’s Details (Metadata section) and in experiments view as a column.description (
str
, optional, default is''
) – Editable description of the experiment. Description is displayed in the experiment’s Details (Metadata section) and can be displayed in the experiments view as a column.params (
dict
, optional, default is{}
) – Parameters of the experiment. After experiment creationparams
are read-only (see:get_parameters()
). Parameters are displayed in the experiment’s Details (Parameters section) and each key-value pair can be viewed in experiments view as a column.properties (
dict
, optional, default is{}
) – Properties of the experiment. They are editable after experiment is created. Properties are displayed in the experiment’s Details (Properties section) and each key-value pair can be viewed in experiments view as a column.tags (
list
, optional, default is[]
) – Must be list ofstr
. Tags of the experiment. They are editable after experiment is created (see:append_tag()
andremove_tag()
). Tags are displayed in the experiment’s Details (Metadata section) and can be viewed in experiments view as a column.upload_source_files (
list
orstr
, optional, default isNone
) –List of source files to be uploaded. Must be list of
str
or singlestr
. Uploaded sources are displayed in the experiment’s Source code tab.IfNone
is passed, Python file from which experiment was created will be uploaded.Pass empty list ([]
) to upload no files.Unix style pathname pattern expansion is supported. For example, you can pass'*.py'
to upload all python source files from the current directory. For Python 3.5 or later, paths of uploaded files on server are resolved as relative to thecalculated common root of all uploaded source files. For older Python versions, paths on server areresolved always as relative to the current directory. For recursion lookup use'**/*.py'
(for Python 3.5 and later). For more information see glob library.abort_callback (
callable
, optional, default isNone
) –Callback that defines how abort experiment action in the Web application should work. Actual behavior depends on your setup:
(default) If
abort_callback=None
and psutil is installed, then current process and it’s children are aborted by sending SIGTERM. If, after grace period, processes are not terminated, SIGKILL is sent.If
abort_callback=None
and psutil is not installed, then abort experiment action just marks experiment as aborted in the Web application. No action is performed on the current process.If
abort_callback=callable
, thencallable
is executed when abort experiment action in the Web application is triggered.
logger (
logging.Logger
or None, optional, default isNone
) – If Python’s Logger is passed, new experiment’s text log (see:log_text()
) with name “logger” is created. Each time Python logger logs new data, it is automatically sent to the “logger” in experiment. As a results all data from Python logger are in the Logs tab in the experiment.upload_stdout (
Boolean
, optional, default isTrue
) – Whether to send stdout to experiment’s Monitoring.upload_stderr (
Boolean
, optional, default isTrue
) – Whether to send stderr to experiment’s Monitoring.send_hardware_metrics (
Boolean
, optional, default isTrue
) – Whether to send hardware monitoring logs (CPU, GPU, Memory utilization) to experiment’s Monitoring.run_monitoring_thread (
Boolean
, optional, default isTrue
) – Whether to run thread that pings Neptune server in order to determine if experiment is responsive.handle_uncaught_exceptions (
Boolean
, optional, default isTrue
) –Two options
True
andFalse
are possible:If set to
True
and uncaught exception occurs, then Neptune automatically place Traceback in the experiment’s Details and change experiment status to Failed.If set to
False
and uncaught exception occurs, then no action is performed in the Web application. As a consequence, experiment’s status is running or not responding.
git_info (
GitInfo
, optional, default isNone
) –Instance of the classGitInfo
that provides information about the git repository from which experiment was started.IfNone
is passed, system attempts to automatically extract information about git repository in the following way:System looks for .git file in the current directory and, if not found, goes up recursively until .git file will be found (see:
get_git_info()
).If there is no git repository, then no information about git is displayed in experiment details in Neptune web application.
hostname (
str
, optional, default isNone
) – IfNone
, neptune automatically get hostname information. User can also set hostname directly by passingstr
.
- Returns
Experiment
object that is used to manage experiment and log data to it.- Raises
ExperimentValidationError – When provided arguments are invalid.
ExperimentLimitReached – When experiment limit in the project has been reached.
Examples
# minimal invoke neptune.create_experiment() # explicitly return experiment object experiment = neptune.create_experiment() # create experiment with name and two parameters neptune.create_experiment(name='first-pytorch-ever', params={'lr': 0.0005, 'dropout': 0.2}) # create experiment with name and description, and no sources files uploaded neptune.create_experiment(name='neural-net-mnist', description='neural net trained on MNIST', upload_source_files=[]) # Send all py files in cwd (excluding hidden files with names beginning with a dot) neptune.create_experiment(upload_source_files='*.py') # Send all py files from all subdirectories (excluding hidden files with names beginning with a dot) # Supported on Python 3.5 and later. neptune.create_experiment(upload_source_files='**/*.py') # Send all files and directories in cwd (excluding hidden files with names beginning with a dot) neptune.create_experiment(upload_source_files='*') # Send all files and directories in cwd including hidden files neptune.create_experiment(upload_source_files=['*', '.*']) # Send files with names being a single character followed by '.py' extension. neptune.create_experiment(upload_source_files='?.py') # larger example neptune.create_experiment(name='first-pytorch-ever', params={'lr': 0.0005, 'dropout': 0.2}, properties={'key1': 'value1', 'key2': 17, 'key3': 'other-value'}, description='write longer description here', tags=['list-of', 'tags', 'goes-here', 'as-list-of-strings'], upload_source_files=['training_with_pytorch.py', 'net.py'])
-
create_notebook
(self)[source]¶ Create a new notebook object and return corresponding
Notebook
instance.- Returns
Notebook
object.
Examples
# Instantiate a session and fetch a project project = neptune.init() # Create a notebook in Neptune notebook = project.create_notebook()
-
class
neptune.
Session
(api_token=None, proxies=None, backend=None)[source]¶ Bases:
object
A class for running communication with Neptune.
In order to query Neptune experiments you need to instantiate this object first.
- Parameters
backend (
Backend
, optional, default isNone
) –By default, Neptune client library sends logs, metrics, images, etc to Neptune servers: either publicly available SaaS, or an on-premises installation.
You can pass the default backend instance explicitly to specify its parameters:
from neptune import Session, HostedNeptuneBackend session = Session(backend=HostedNeptuneBackend(...))
Passing an instance of
OfflineBackend
makes your code run without communicating with Neptune servers.from neptune import Session, OfflineBackend session = Session(backend=OfflineBackend())
api_token (
str
, optional, default isNone
) –User’s API token. If
None
, the value ofNEPTUNE_API_TOKEN
environment variable will be taken. Parameter is ignored ifbackend
is passed.Deprecated since version 0.4.4.
Instead, use:
from neptune import Session session = Session.with_default_backend(api_token='...')
proxies (
str
, optional, default isNone
) –Argument passed to HTTP calls made via the Requests library. For more information see their proxies section. Parameter is ignored if
backend
is passed.Deprecated since version 0.4.4.
Instead, use:
from neptune import Session, HostedNeptuneBackend session = Session(backend=HostedNeptuneBackend(proxies=...))
Examples
Create session, assuming you have created an environment variable
NEPTUNE_API_TOKEN
from neptune import Session session = Session.with_default_backend()
Create session and pass
api_token
from neptune import Session session = Session.with_default_backend(api_token='...')
Create an offline session
from neptune import Session, OfflineBackend session = Session(backend=OfflineBackend())
-
classmethod
with_default_backend
(cls, api_token=None)[source]¶ The simplest way to instantiate a
Session
.- Parameters
api_token (
str
) – User’s API token. IfNone
, the value ofNEPTUNE_API_TOKEN
environment variable will be taken.
Examples
from neptune import Session session = Session.with_default_backend()
-
get_project
(self, project_qualified_name)[source]¶ Get a project with given
project_qualified_name
.In order to access experiments data one needs to get a
Project
object first. This method gives you the ability to do that.- Parameters
project_qualified_name (
str
) – Qualified name of a project in a form ofnamespace/project_name
.- Returns
Project
object.
- Raise:
ProjectNotFound
: When a project with given name does not exist.
Examples
# Create a Session instance from neptune.sessions import Session session = Session() # Get a project by it's ``project_qualified_name``: my_project = session.get_project('namespace/project_name')
-
get_projects
(self, namespace)[source]¶ Get all projects that you have permissions to see in given workspace.
This method gets you all available projects names and their correspondingProject
objects.Both private and public projects may be returned for the workspace. If you have role in private project, it is included.You can retrieve all the public projects that belong to any user or workspace, as long as you know their username or workspace name.- Parameters
namespace (
str
) – It can either be name of the workspace or username.- Returns
OrderedDict
- keys are
project_qualified_name
that is: ‘workspace/project_name’values are correspondingProject
objects.
- Raises
NamespaceNotFound – When the given namespace does not exist.
Examples
# create Session from neptune.sessions import Session session = Session() # Now, you can list all the projects available for a selected namespace. # You can use `YOUR_NAMESPACE` which is your workspace or user name. # You can also list public projects created in other workspaces. # For example you can use the `neptune-ai` namespace. session.get_projects('neptune-ai') # Example output: # OrderedDict([('neptune-ai/credit-default-prediction', # Project(neptune-ai/credit-default-prediction)), # ('neptune-ai/GStore-Customer-Revenue-Prediction', # Project(neptune-ai/GStore-Customer-Revenue-Prediction)), # ('neptune-ai/human-protein-atlas', # Project(neptune-ai/human-protein-atlas)), # ('neptune-ai/Ships', # Project(neptune-ai/Ships)), # ('neptune-ai/Mapping-Challenge', # Project(neptune-ai/Mapping-Challenge)) # ])
-
neptune.
__version__
¶
-
neptune.
session
¶
-
neptune.
project
¶
-
neptune.
__lock
¶
-
neptune.
_logger
¶ Access Neptune as an anonymous user. You can pass this value as api_token during init() call, either by an environment variable or passing it directly
-
neptune.
ANONYMOUS
¶ Anonymous user API token. You can pass this value as api_token during init() call, either by an environment variable or passing it directly
-
neptune.
ANONYMOUS_API_TOKEN
¶
-
neptune.
init
(project_qualified_name=None, api_token=None, proxies=None, backend=None)[source]¶ Initialize Neptune client library to work with specific project.
Authorize user, sets value of global variable
project
toProject
object that can be used to create or list experiments, notebooks, etc.- Parameters
project_qualified_name (
str
, optional, default isNone
) – Qualified name of a project in a form ofnamespace/project_name
. IfNone
, the value ofNEPTUNE_PROJECT
environment variable will be taken.api_token (
str
, optional, default isNone
) –User’s API token. If
None
, the value ofNEPTUNE_API_TOKEN
environment variable will be taken.Note
It is strongly recommended to use
NEPTUNE_API_TOKEN
environment variable rather than placing your API token in plain text in your source code.proxies (
dict
, optional, default isNone
) –Argument passed to HTTP calls made via the Requests library. For more information see their proxies section.
Note
Only http and https keys are supported by all features.
Deprecated since version 0.4.4.
Instead, use:
from neptune import HostedNeptuneBackend neptune.init(backend=HostedNeptuneBackend(proxies=...))
backend (
Backend
, optional, default isNone
) –By default, Neptune client library sends logs, metrics, images, etc to Neptune servers: either publicly available SaaS, or an on-premises installation.
You can also pass the default backend instance explicitly to specify its parameters:
from neptune import HostedNeptuneBackend neptune.init(backend=HostedNeptuneBackend(...))
Passing an instance of
OfflineBackend
makes your code run without communicating with Neptune servers.from neptune import OfflineBackend neptune.init(backend=OfflineBackend())
Note
Instead of passing a
neptune.OfflineBackend
instance asbackend
, you can set an environment variableNEPTUNE_BACKEND=offline
to override the default behaviour.
- Returns
Project
object that is used to create or list experiments, notebooks, etc.- Raises
NeptuneMissingApiTokenException – When
api_token
is None andNEPTUNE_API_TOKEN
environment variable was not set.NeptuneMissingProjectQualifiedNameException – When
project_qualified_name
is None andNEPTUNE_PROJECT
environment variable was not set.InvalidApiKey – When given
api_token
is malformed.Unauthorized – When given
api_token
is invalid.
Examples
# minimal invoke neptune.init() # specifying project name neptune.init('jack/sandbox') # running offline neptune.init(backend=neptune.OfflineBackend())
-
neptune.
set_project
(project_qualified_name)[source]¶ Setups Neptune client library to work with specific project.
Sets value of global variableproject
toProject
object that can be used to create or list experiments, notebooks, etc.If Neptune client library was not previously initialized viainit()
call it will be initialized with API token taken fromNEPTUNE_API_TOKEN
environment variable.- Parameters
project_qualified_name (
str
) – Qualified name of a project in a form ofnamespace/project_name
.- Returns
Project
object that is used to create or list experiments, notebooks, etc.- Raises
NeptuneMissingApiTokenException – When library was not initialized previously by
init
call andNEPTUNE_API_TOKEN
environment variable is not set.
Examples
# minimal invoke neptune.set_project('jack/sandbox')
-
neptune.
create_experiment
(name=None, description=None, params=None, properties=None, tags=None, upload_source_files=None, abort_callback=None, logger=None, upload_stdout=True, upload_stderr=True, send_hardware_metrics=True, run_monitoring_thread=True, handle_uncaught_exceptions=True, git_info=None, hostname=None, notebook_id=None)[source]¶ Create and start Neptune experiment.
Alias for:
create_experiment()
-
neptune.
append_tag
(tag, *tags)[source]¶ Append tag(s) to the experiment on the top of experiments view.
Alias for:
append_tag()
Append tag(s) to the experiment on the top of experiments view.
Alias for:
append_tags()
-
neptune.
remove_tag
(tag)[source]¶ Removes single tag from experiment.
Alias for:
remove_tag()
-
neptune.
set_property
(key, value)[source]¶ Set key-value pair as an experiment property.
If property with given
key
does not exist, it adds a new one.Alias for:
set_property()
-
neptune.
remove_property
(key)[source]¶ Removes a property with given key.
Alias for:
remove_property()
-
neptune.
send_metric
(channel_name, x, y=None, timestamp=None)[source]¶ Log metrics (numeric values) in Neptune.
Alias for
log_metric()
-
neptune.
log_metric
(log_name, x, y=None, timestamp=None)[source]¶ Log metrics (numeric values) in Neptune.
Alias for
log_metric()
-
neptune.
send_text
(channel_name, x, y=None, timestamp=None)[source]¶ Log text data in Neptune.
Alias for
log_text()
-
neptune.
log_text
(log_name, x, y=None, timestamp=None)[source]¶ Log text data in Neptune.
Alias for
log_text()
-
neptune.
send_image
(channel_name, x, y=None, name=None, description=None, timestamp=None)[source]¶ Log image data in Neptune.
Alias for
log_image()
-
neptune.
log_image
(log_name, x, y=None, image_name=None, description=None, timestamp=None)[source]¶ Log image data in Neptune.
Alias for
log_image()
-
neptune.
send_artifact
(artifact, destination=None)[source]¶ Save an artifact (file) in experiment storage.
Alias for
log_artifact()
-
neptune.
delete_artifacts
(path)[source]¶ Delete an artifact (file/directory) from experiment storage.
Alias for
delete_artifacts()
-
neptune.
log_artifact
(artifact, destination=None)[source]¶ Save an artifact (file) in experiment storage.
Alias for
log_artifact()