Best practices
By following our best practices, you ensure smooth and secure working with Neptune.

Configuring your credentials

Best practice: Save your Neptune API token and project name as environment variables.
While it is possible to pass your API token and project name as arguments to the init() method in the code, it's both more secure and more convenient to save them as environment variables in your system.
This way, Neptune picks your credentials up automatically and you avoid storing them in the source code of your training scripts.

Setting your Neptune API token

To save your API token, assign it to the NEPTUNE_API_TOKEN environment variable:
Append to your .bashrc file:
Open Command Prompt and enter:
Append to your .zshrc or .bashrc file:
from getpass import getpass
api_token = getpass("Enter your Neptune API token: ")
For detailed instructions, see Installation and setup.

Setting your project name

To save your project name, assign it to the NEPTUNE_PROJECT environment variable:
Append to your .bashrc file:
Open Command Prompt and enter:
Append to your .zshrc or .bashrc file:
project = input("Enter your Neptune project name: ")
where WORKSPACE_NAME/PROJECT_NAME is the full project name. For example, john.smith/sample-project.
For detailed instructions, see Installation and setup.

Stopping runs and other objects

Best practice: When you're done logging to a Neptune object, stop it with the stop() method.
When a script finishes executing, runs are stopped automatically. However, to avoid logging metadata for longer than intended, it's best to stop the run explicitly with the stop() method.
Stopping a run
import as neptune
run = neptune.init()
When working in an interactive notebook environment, you should always call stop(). This is because the connection to Neptune is not stopped when the cell has finished executing, but rather only when you stop the notebook.
Other reasons to use stop() include:
  • Creating multiple runs in a single script
  • Continuous training flows
  • Model registry flows
While model and project objects are synchronous and do not monitor things in the background as runs do, it's also best to stop them when the connection to Neptune is not needed.

Maximum number of fields

Best practice: Make sure you create no more than 9000 fields in a single Neptune object.
To ensure optimal performance, the number of user-defined fields is limited to 9000. This limit concerns any one Neptune object that can contain metadata fields, such as a Run or Model object.

Workaround for exceeding the limit

If you exceed this limit while logging, Neptune stops the synchronization to the server and stores the data locally. To continue uploading the metadata, you need to delete some excess fields from the object in question, then sync the data manually.
Counting fields
Deleting fields
Syncing data manually
You can count the fields of a Neptune object using the get_structure() method and some scripting:
>>> import as neptune
>>> def count_fields(d):
... return sum([count_fields(v) if isinstance(v, dict)
... else 1 for v in d.values()])
>>> run = neptune.init_run(project="ml-team/classification",
... run="CLS-34", mode="read-only")
Remember to stop your run once you’ve finished logging your metadata...
>>> count_fields(run.get_structure())
You can delete fields or namespaces with the del command. For example, to delete the training/checkpoints namespace from a run with identifier "ACC-47":
import as neptune
# Resume an existing run
run = neptune.init(run="ACC-47")
# Delete
del run["training/checkpoints"]
Once you're done deleting fields, synchronize the data manually:
  1. 1.
    Navigate to the folder from which you executed the script. It should contain a .neptune folder.
  2. 2.
    In a terminal, enter the following command:
neptune sync
For further details, see the API reference:

Reserved namespaces

Best practice: Do not create fields in the sys or monitoring namespaces.
  • sys - The system namespace is reserved, that is, you cannot create new fields in this namespace.
    • Note: You can manually edit several of the sys field values, such as description, stage, and tags.
  • monitoring - The monitoring namespace is intended specifically for system monitoring, not monitoring related to model training. Examples of system monitoring include hardware consumption and standard streams (stdout and stderr).