Universal methods and parameters

This reference lists methods and parameters that are common to all Neptune objects (or classes): You can use them whether you're working with runs, models, or project metadata.

Related

For more detailed usage and examples, as well as class-specific methods, see the class references:

• Run
• Model
• ModelVersion
• Project

Note: Most metadata logging methods apply to fields within the object rather than the object as a whole. For the full reference and methods, see:

• Field types

Methods

Field lookup: []

You can access the field of an object through a dict-like field lookup: object[field_path].

This way, you can:

1. Track metadata:

   run["params/max_epochs"] = 20
   for epoch in range(20):
       run["epoch/acc"].append(epoch_accuracy)
   

2. Fetch already tracked metadata, such as retrieving parameters when resuming a run or downloading metadata for a custom analysis:

   learning_rate = run["params/max_epochs"].fetch()
   acc_df = run["epoch/acc"].fetch_values()
   

Returns

The returned type depends on the field type and whether a field is stored under the given path.

Path	Example	Returns
Field exists	-	The returned type matches the type of the field
Field does not exist	-	Handler object
Path is namespace and has field	Path: "train" Field "train/acc" exists	Namespace handler object

Examples

import neptune
run = neptune.init_run()

# Create new Float field
run["parameters/momentum"] = 0.9

# Create new FloatSeries field
run["train/acc"].append(0.01)

# Continue logging to existing Series field
run["train/acc"].append(0.02)

# If you access a namespace handler, you can interact with it as with the run object
params_ns = run["parameters"]
params_ns["learning_rate"] = 0.3  # Stores 0.3 under path "parameters/learning_rate"

Assignment: =

Convenience alias for assign().

assign()

Assign values to multiple fields from a dictionary.

Parameters

Name	Type	Default	Description
value	dict	None	A dictionary with values to assign, where keys (str) become the paths of the fields. The dictionary can be nested, in which case the path will be a combination of all keys.
wait	Boolean, optional	False	By default, tracked metadata is sent to the server in the background. With this option set to True, Neptune first sends all data before executing the call.

Example

import neptune
run = neptune.init_run()

# Assign a single value with the Python "=" operator
run["parameters/learning_rate"] = 0.8
# or the assign() method
run["parameters/learning_rate"].assign(0.8)

# Assign a dictionary with the Python "=" operator
run["parameters"] = {"max_epochs": 10, "optimizer": "Adam", "learning_rate": 0.8}
# or the assign() method
run.assign(
    {"parameters": {"max_epochs": 10, "optimizer": "Adam", "learning_rate": 0.8}}
)

del

Completely removes the field or namespace and all associated metadata stored under the path.

If you need to free up storage or reduce the number of fields, use del to remove unneeded metadata that takes up space.

See also: pop().

Examples

import neptune

model_version = neptune.init_model_version(with_id="CLS-TREE-5")

# Delete field with path "model/binary"
del model_version["model/binary"]

# You can also delete the whole namespace
del model_version["model"]

exists()

Checks if there is a field or namespace under the specified path.

Info

This method checks the local representation of the object. If the field was created by another process or the metadata has not reached the Neptune servers, it may not be possible to fetch. In this case you can:

• Call sync() on the object to synchronize the local representation with the server.
• Call wait() on the object to wait for all tracking calls to finish.

Parameters

Name	Type	Default	Description
path	str	-	Path to check for the existence of a field or namespace.

Examples

import neptune

# Resume run with identifier CLS-34
run = neptune.init_run(with_id="CLS-34")

# If the training is complete remove training checkpoints if they are there
if run.exists("model/checkpoints") and run["train/finished"].fetch() == True:
    del run["model/checkpoints"]

Using wait() to ensure that previous tracking calls reach the server:

import neptune

run = neptune.init_run()

run["learning_rate"] = 0.1

# The path exists in the local representation
if run.exists("learning_rate"):
    # However, the tracking call may have not reached Neptune servers yet
    run["learning_rate"].fetch()  # Error: the field does not exist

run.wait()

fetch()

Fetches values of all non-File Atom fields as a dictionary.

The result preserves the hierarchical structure of the object metadata, but contains only Atom fields that are not of the type File.

Related

To learn more about Atom and other types, see the field types reference.

Returns

dict containing the values of all non-File Atom fields.

Example

import neptune

# Resume version 12 of model CLS-FOREST
model_version = neptune.init_model_version(
    model="CLS-FOREST-12",
)

# Fetch the model parameters
params = model_version["params"].fetch()

get_structure()

Returns the metadata structure of an object in the form of a dictionary.

This method can be used to traverse the metadata structure programmatically when using Neptune in automated workflows.

See also: print_structure().

Warning

The returned object is a shallow copy of the object's internal structure. Any modifications to it may result in tracking malfunction.

Returns

dict with the metadata structure of the object.

Example

>>> run.get_structure()
{'data': {'val': <neptune.attributes.atoms.artifact.Artifact object at 0x0000020374B822C0>}, 'f1_score': <neptune.attributes.atoms.float.Float object at 0x0000020374B81A50>, 'monitoring': {'cpu': <neptune.attributes.series.float_series.FloatSeries object at 0x0000020374B81A80>, ... }}

get_url()

Returns a direct link to the object.

The same link is printed in the console once the object has been initialized.

Returns

str with the URL of the object in Neptune.

Example

>>> import neptune
>>> # Initialize existing model with ID "CLS-TREE"
>>> model = neptune.init_model(with_id="CLS-TREE")
>>> model.get_url()
https://app.neptune.ai/ml-team/classification/m/CLS-TREE

pop()

Completely removes the field or namespace and all associated metadata stored under the path.

See also del.

Parameters

Name	Type	Default	Description
path	str	-	Path of the field or namespace to be removed.
wait	Boolean, optional	False	By default, tracked metadata is sent to the server in the background. With this option set to True, Neptune first sends all data before executing the call.

Examples

import neptune

run = neptune.init_run()

run["parameters/learninggg_rata"] = 0.3
# Let's correct the misspelled field

# Delete the field along with its data
run.pop("parameters/learninggg_rata")

# Reassign to correct field
run["parameters/learning_rate"] = 0.3

print_structure()

Pretty-prints the structure of the object metadata.

Paths are ordered lexicographically and the structure is colored.

See also: get_structure()

Example

>>> import neptune
>>> model_version = neptune.init_model(with_id="CLS-TREE-5")
>>> model_version.print_structure()
'model':
    'binary': File
    'parameters':
        'eta': Float
        'gamma': Float
        'max_depth': Integer
'sys':
    'creation_time': Datetime
    'id': String
    'model_id': String
    'modification_time': Datetime
    'monitoring_time': Integer
    'name': String
    'owner': String
    'ping_time': Datetime
    'running_time': Float
    'size': Float
    'stage': String
    'state': RunState
    'tags': StringSet
    'trashed': Boolean
'train':
    'acc': Float
    'dataset': Artifact
'val':
    'acc': Float
    'dataset': Artifact

stop()

Stops the connection to Neptune and synchronizes all data.

Info

Every active object keeps a connection open with Neptune. If you're performing multiple training jobs from one script, one after the other, stop each object once tracking is no longer needed.

Neptune calls stop() automatically in the following scenarios:

• If you're running a script, the connection stops when the script finishes executing.
• When using context managers, the connection stops when exiting the object context.

However, it's a best practice to explicitly call stop() when the connection is no longer needed.

Warning

Always call stop() in interactive environments, such as a Python interpreter or Jupyter notebook. The connection to Neptune is not stopped when the cell has finished executing, but rather when the entire notebook stops.

Parameters

Name	Type	Default	Description
seconds	int or float, optional	None	Wait for the specified time for all tracking calls to finish before stopping the connection. If None, wait for all tracking calls to finish.

Examples

import neptune

run = neptune.init_run()
model = neptune.init_model(key="FOREST")

[...]  # Your training or monitoring code

# stop() is automatically called at the end for every Neptune object

Stopping multiple objects manually at the end of a training job:

for config in configs:
  run = neptune.init_run()
  model = neptune.init_model(key="FOREST")
  [...]  # Your training or monitoring code
  run.stop()
  model.stop()

Using with statement and context manager:

for config in configs:
  with neptune.init_run() as run:
    [...]  # Your training or monitoring code
    # stop() is automatically called
    # when code execution exits the with statement

sync()

Synchronizes the local representation of the object with the representation on the Neptune servers.

Parameters

Name	Type	Default	Description
wait	Boolean, optional	False	By default, tracked metadata is sent to the server in the background. With this option set to True, Neptune first sends all queued data before executing the call.

Info

some_neptune_object.sync(wait=True) is equivalent to:

some_neptune_object.wait()
some_neptune_object.sync()

Example

import neptune

# Connect to a run from Worker #3
worker_id = 3
run = neptune.init_run(
    with_id="DIST-43",
    monitoring_namespace=f"monitoring/{worker_id}",
)

# Try to access logs that were created in the meantime by Worker #2
worker_2_status = run["status/2"].fetch()
# Error if this field was created after this script starts

run.sync() # Synchronize local representation with Neptune servers
worker_2_status = run["status/2"].fetch() # No error

wait()

Wait for all the tracking calls (queued operations) to reach the Neptune servers.

Parameters

Name	Type	Default	Description
disk_only	Boolean, optional	False	If True, the process will wait only for the data to be saved locally from memory, but will not wait for it to reach Neptune servers.

Related

Connection modes reference

Initialization functions

To work with any Neptune object in your code, you initialize it with its respective init function.

Object type (class)	Function	ID format	Example
Run	init_run()	PROJKEY-#	New run: run = neptune.init_run() Existing run: run = neptune.init_run(with_id="CLS-7")
Model	init_model()	PROJKEY-MODKEY	New model: model = neptune.init_model(key="FOREST") Existing model: model = neptune.init_model(with_id="CLS-FOREST")
ModelVersion	init_model_version()	PROJKEY-MODKEY-#	New model version: model_version = neptune.init_model_version(model="CLS-FOREST") Existing model version: model_version = neptune.init_model_version(with_id="CLS-FOREST-11")
Project	init_project()	workspace-name/project-name	project = neptune.init_project(project="ml-team/classification")

The general rule is:

• If you supply the ID of an existing object through the with_id parameter, that object is resumed and you can continue logging to it. You can add, change, or delete fields as you normally would.
• If you don't supply an ID of an existing object, a new object is created.

For detailed usage, see API reference ≫ neptune

Initialization parameters

The table below lists parameters which are common to all Neptune init functions.

For object-specific initialization functions and examples, see neptune.

Parameters

Name	Type	Default	Description
api_token	str, optional	None	User's API token. If None, the value of the NEPTUNE_API_TOKEN environment variable is used.
flush_period	float, optional	5	In asynchronous mode, how often Neptune should trigger disk flushing (in seconds).
mode	str, optional	async	Connection mode in which the tracking will work.
name*	str, optional	Untitled	Custom name for the object.
project*	str, optional	None	Name of a project in the form workspace-name/project-name.*
proxies	dict, optional	None	Argument passed to HTTP calls made via the Requests library.
with_id*	str, optional	None	The Neptune identifier of an existing object to resume, such as "CLS-11". The identifier is stored in the object's "sys/id" field. If omitted or None is passed, a new object is created.

* Does not apply to init_project(). See the parameter-specific sections for details.

api_token

Name	Type	Default
api_token	str, optional	None

User's API token. If None, the value of the NEPTUNE_API_TOKEN environment variable is used.

Warning

To keep your token secure, avoid placing it in source code. Instead, save it as an environment variable.

Set to neptune.ANONYMOUS_API_TOKEN to log metadata anonymously.

flush_period

Name	Type	Default
flush_period	float, optional	5 (seconds)

In asynchronous (default) connection mode, how often Neptune should trigger disk flushing.

mode

Name	Type	Default
mode	str, optional	async

Connection mode in which the tracking will work. Possible values are async, sync, offline, read-only, and debug.

Related

To learn more, see Using neptune in different modes.

name

Name	Type	Default
name	str, optional	Untitled

A custom name for the run, model, or model version.

Neptune stores the name in the sys/name field.

Note

Does not apply to init_project(), which is used to initialize an existing project. In this case, the name argument specifies the name of the project to be initialized.

Example

# Create a new version of model "CLS-FOREST"
model_version = neptune.init_model_version(
    model="CLS-FOREST",
    name="New forest-based classification model for emotion detection",
)

# Initialize project "classification" to log metadata on project-level
project = neptune.init_project(project="ml-team/classification")

project

Name	Type	Default
project	str, optional	None

Name of a project in the form workspace-name/project-name. If None, the value of the NEPTUNE_PROJECT environment variable is used.

Note

When the object being initialized is itself a project (with init_project()), you supply the project name with the name argument instead of project.

Example

# Create a new run in the project "classification" in the workspace "ml-team"
run = neptune.init_run(project="ml-team/classification")

proxies

Name	Type	Default
proxies	dict, optional	None

Argument passed to HTTP calls made via the Requests library. For details on proxies, see the Requests documentation.

with_id

Name	Type	Default
with_id	str, optional	None

The Neptune identifier of an existing object to resume, such as "CLS-11" in the case of a run.

If omitted, a new object is created.

The identifier is stored in the object's sys/id field and displayed in the leftmost column of any table view.

Note

Does not apply to init_project(), as project objects don't have Neptune IDs. When you intialize an existing project, you specify the project with the name argument.

Example

# Resume logging to an existing run
run = neptune.init_run(with_id="CLS-11")
run["namespace/field"] = "new metadata"
...

# Resume a model in order to fetch metadata
model = neptune.init_model(with_id="CLS-TREE", mode="read_only")
model_sig = model["signature"].download()
...

How do I find the ID?

The Neptune ID is a unique identifier for the object. It's displayed in the Details view and in the leftmost column of the table views.

The ID is stored in the system namespace. You can obtain it with object["sys/id"].fetch(). For example:

>>> run = neptune.init_run(project="ml-team/classification")
>>> run["sys/id"].fetch()
'CLS-26'