Skip to content

Upcoming changes with the 1.0 release#

Info

This guide concerns the Neptune Python client library (neptune-client ).

The 1.0 release is scheduled for the second half of February this year.

This page lists all the changes that are coming to the Neptune client library with the 1.0 major release. Here you'll find all the information you need to update your code.

Renamed library and modules#

As we formally promote the "new" Neptune API to the main one, the library and package names will change as follows:

Before       After       Comment
neptune neptune.legacy The legacy API is moved to a different module (and continues to be deprecated).
neptune.new neptune The neptune.new module becomes the main API (neptune.new will work, but this use is deprecated).
neptune-client neptune The Python package name is changed (neptune-client will still work).

Installing

No

pip install neptune-client

Yes

pip install neptune

Importing

No

import neptune.new as neptune

Yes

import neptune

Deprecated parts to be removed#

The following were already deprecated previously and will be removed with the 1.0 release:

Deprecated methods#

Before          After           Comment
get_run_url() get_url() Replaced with universal method for fetching the URL of any Neptune object type.
neptune.get_project() neptune.init_project() Using the mode="read-only" parameter makes init_project() behave just like get_project().
neptune.init() neptune.init_run() Now that we have model and project objects to work with as well, the function names are clearer when they express the object that's being initialized.
neptune.get_last_run() This was a mainly a helper method for transitioning from the legacy API to the new one.

Replacement for log()

Instead of log(), you can use append() and extend() to log series of values or files.

However, we won't deprecate log() for the time being.

Deprecated parameters and constants#

Before          After             Comment
(in init methods) run, model, version with_id When initializing an existing Neptune object, use the universal parameter with_id to pass the object's Neptune ID.
(for existing project name) name project Whenever you pass a project name as the argument, the argument name is project. The only exception is when you actually create the project, in which case the name parameter is used to pass the name of the new project.
(for existing workspace name) name workspace Whenever you pass a workspace name as the argument, the argument name is workspace.
neptune.ANONYMOUS neptune.ANONYMOUS_API_TOKEN

Deprecated CLI arguments#

The following Neptune Command Line Interface argument is removed:

Argument Command Description
--run neptune sync Used to specify which runs or other Neptune objects should be synchronized.

No

neptune sync --run jackie/sandbox/SAN-42

Yes

neptune sync --object jackie/sandbox/SAN-42

Changes in behavior#

Some methods will require keyword arguments#

It has been possible to use many arguments positionally ‐ that is, passing them to Neptune methods without using the names of the arguments.

With 1.0, we'll change module-level methods (neptune and management) to generally require named arguments.

No

run = neptune.init_run("workspace-name/project-name")

from neptune import management
management.trash_objects(project_name, runs_to_trash)

Yes

run = neptune.init_run(project="workspace-name/project-name")

from neptune import management
management.trash_objects(project=project_name, ids=runs_to_trash)

The methods you typically use to actually log metadata will not be affected. For example, you will not have to change your upload("data_sample.csv") calls to upload(value="data_sample.csv").

System metrics not tracked in interactive sessions#

By default, monitoring of system metrics will be turned off for interactive Python kernels, such as Jupyter notebooks. This includes logging of hardware consumption and standard streams (stdout and stderr).

We're changing this to avoid unintentionally logging things in the background when trying things out in interactive sessions.

To turn it on, you'll need to explicitly set the related initialization parameters to a value of True:

>>> import neptune
>>> run = neptune.init_run(
...     capture_hardware_metrics=True,
...     capture_stderr=True,
...     capture_stdout=True,
... )

In scripts, monitoring will still be turned on by default.

Better support for parallel or distributed jobs#

Monitoring will work with multiple processes out of the box: Instead of every process trying to log metrics to the same namespace ("monitoring"), we'll split the metrics under unique sub-namespaces per process.

We'll also add three new fields under each monitoring namespace:

  1. "monitoring/<unique hash>/pid" – process ID
  2. "monitoring/<unique hash>/tid" – thread ID
  3. "monitoring/<unique hash>/hostname" – host name

Before

run/
|-- monitoring/
    |-- cpu         <- Each process tries to log CPU metrics under this field
    |-- stdout      <- Each process tries to log stdout under this field
    |-- ...

After

run/
|-- monitoring/<unique hash>
    |-- cpu
    |-- stdout
    |-- hostname    <- New field: Host name
    |-- pid         <- New field: Process ID
    |-- tid         <- New field: Thread ID
    |-- ...
|-- monitoring/<unique hash>
    |-- cpu
    |-- stdout
    |-- hostname
    |-- ...

As before, you can also provide your custom monitoring namespace name by passing it to the monitoring_namespace argument at initialization:

Example
import neptune
run = neptune.init_run(
    monitoring_namespace="some monitoring namespace name that's unique for each run"
)

This custom name would replace the monitoring/<unique hash> pattern.

Unified run states#

The run states "running" and "idle" have been shown as "Active" and "Inactive" in the web application. We're unifying the labels so that when queried through the API, the states are returned as "Active" or "Inactive".

When querying runs based on state, use active and inactive instead of running and idle.

Example: Fetching inactive runs from a project

No

project = neptune.init_project(project="workspace/project")
runs_table_df = project.fetch_runs_table(state="idle").to_pandas()

Yes

project = neptune.init_project(project="workspace/project")
runs_table_df = project.fetch_runs_table(state="inactive").to_pandas()

No more implicit casting to String#

We'll no longer cast unsupported types to String when using assign() (a.k.a. =), extend(), or log().

What does this mean?

Previously, if you attempted to log a value that is not among the supported neptune.types (such as a tuple) Neptune silently logged it as a string. As of release 1.0, you're expected to use str(value) if you want an object to be logged as a string; otherwise an exception is thrown in the case of unsupported types.

  • Before

    Works

    >>> run["tuple"] = ("hi", 1)
    >>> run["tuple"]
    <String field at "tuple">
    

    Works

    >>> run["tuple"] = str(("hi", 1))
    >>> run["tuple"]
    <String field at "tuple">
    
  • After

    Does not work

    >>> run["tuple"] = ("hi", 1)
    <error>
    >>> # tuple isn't supported
    

    Works

    >>> run["tuple"] = str(("hi", 1))
    >>> run["tuple"]
    <String field at "tuple">
    

For dictionaries that include unsupported types, you can use the stringify_unsupported() method to ensure that all nested values of an unsupported type are stored as strings.

Works

>>> run["complex_dict"] = {"tuple": ("hi", 1), "metric": 0.87}
>>> run["complex_dict/tuple"]
<String field at "complex_dict/tuple">

Works

>>> from neptune.new.utils import stringify_unsupported
>>> run["complex_dict"] = stringify_unsupported({"tuple": ("hi", 1), "metric": 0.87})
>>> run["complex_dict/tuple"]
<String field at "complex_dict/tuple">

Does not work

>>> run["complex_dict"] = {"tuple": ("hi", 1), "metric": 0.87}
<error>
>>> # Doesn't work because the tuple isn't a supported type

Works

>>> from neptune.utils import stringify_unsupported
>>> run["complex_dict"] = stringify_unsupported({"tuple": ("hi", 1), "metric": 0.87})
>>> run["complex_dict/tuple"]
<String field at "complex_dict/tuple">

What do I need to change in my code?#

Before the release#

You can already change the following in your code:

  • If you're still using deprecated methods or parameters, make sure to update those.
    • In particular, when passing the name of your existing Neptune project or workspace to a function, use the project or workspace argument instead of name.
  • Instead of log(), use append() (or extend(), if you're logging multiple values at a time).
  • If you're providing arguments positionally in your function calls, check if you should change them to include the argument names.
  • If you're using Neptune in interactive Python sessions and want to monitor hardware usage or system metrics, explicitly set the related init parameters to True.
  • If you have been relying on unsupported types being logged as strings, either explicitly log those as strings or look for a different way to log them with available Neptune methods.

Once you upgrade to 1.0#

In addition to the above:

  • Change the Neptune import statement to just import neptune.1
    • If you're importing other things from neptune.new, also change that part to just neptune.
  • When upgrading the client library, you can use pip install -U neptune.
  • If you're querying and filtering runs based on state, change the following:
    • idle to inactive
    • running to active

  1. If you're still using the legacy API, you'll need to change your import statement to: import neptune.legacy as neptune