Best practices

Configuring your credentials
Best practice: Save your Neptune API token and project name as environment variables.
While it is possible to pass your API token and project name as arguments to the init() method in the code, it's both more secure and more convenient to save them as environment variables in your system.
This way, Neptune picks your credentials up automatically and you avoid storing them in the source code of your training scripts.
Setting your Neptune API token
To save your API token, assign it to the NEPTUNE_API_TOKEN environment variable:
Linux
Windows
macOS
Notebook
Append to your .bashrc file:
export NEPTUNE_API_TOKEN="YOUR_API_TOKEN"
Open Command Prompt and enter:
setx NEPTUNE_API_TOKEN "YOUR_API_TOKEN"
Append to your .zshrc or .bashrc file:
export NEPTUNE_API_TOKEN="YOUR_API_TOKEN"
from getpass import getpass
api_token = getpass("Enter your Neptune API token: ")
For detailed instructions, see Installation and setup.
Setting your project name
To save your project name, assign it to the NEPTUNE_PROJECT environment variable:
Linux
Windows
macOS
Notebook
Append to your .bashrc file:
export NEPTUNE_PROJECT="WORKSPACE_NAME/PROJECT_NAME"
Open Command Prompt and enter:
setx NEPTUNE_PROJECT "WORKSPACE_NAME/PROJECT_NAME"
Append to your .zshrc or .bashrc file:
export NEPTUNE_PROJECT="WORKSPACE_NAME/PROJECT_NAME"
project = input("Enter your Neptune project name: ")
where WORKSPACE_NAME/PROJECT_NAME is the full project name. For example, john.smith/sample-project.
For detailed instructions, see Installation and setup.
Stopping runs and other objects
Best practice: When you're done logging to a Neptune object, stop it with the stop() method.
When a script finishes executing, runs are stopped automatically. However, to avoid logging metadata for longer than intended, it's best to stop the run explicitly with the stop() method.
Stopping a run
import neptune.new as neptune
​
run = neptune.init()
​
...
​
run.stop()
When working in an interactive notebook environment, you should always call stop(). This is because the connection to Neptune is not stopped when the cell has finished executing, but rather only when you stop the notebook.
Other reasons to use stop() include:
Creating multiple runs in a single script
Continuous training flows
Model registry flows 
While model and project objects are synchronous and do not monitor things in the background as runs do, it's also best to stop them when the connection to Neptune is not needed.
Maximum number of fields
Best practice: Make sure you create no more than 9000 fields in a single Neptune object.
To ensure optimal performance, the number of user-defined fields is limited to 9000. This limit concerns any one Neptune object that can contain metadata fields, such as a Run or Model object.
Workaround for exceeding the limit
If you exceed this limit while logging, Neptune stops the synchronization to the server and stores the data locally. To continue uploading the metadata, you need to delete some excess fields from the object in question, then sync the data manually.
Counting fields
Deleting fields
Syncing data manually
You can count the fields of a Neptune object using the get_structure() method and some scripting:
>>> import neptune.new as neptune
>>> def count_fields(d):
...     return sum([count_fields(v) if isinstance(v, dict)
...     else 1 for v in d.values()])
...
>>> run = neptune.init_run(project="ml-team/classification",
...                        run="CLS-34", mode="read-only")
https://app.neptune.ai/ml-team/classification/e/CLS-34
Remember to stop your run once you’ve finished logging your metadata...
>>> count_fields(run.get_structure())
571
You can delete fields or namespaces with the del command. For example, to delete the training/checkpoints namespace from a run with identifier "ACC-47":
import neptune.new as neptune
​
# Resume an existing run
run = neptune.init(run="ACC-47")
​
# Delete 
del run["training/checkpoints"]
Once you're done deleting fields, synchronize the data manually:
1.Navigate to the folder from which you executed the script. It should contain a .neptune folder.
2.In a terminal, enter the following command:
neptune sync
For further details, see the API reference:
​Run -> del and pop()
​Command Line Interface -> neptune sync
Reserved namespaces
Best practice: Do not create fields in the sys or monitoring namespaces.
sys - The system namespace is reserved, that is, you cannot create new fields in this namespace.
Note: You can manually edit several of the sys field values, such as description, stage, and tags.
monitoring - The monitoring namespace is intended specifically for system monitoring, not monitoring related to model training. Examples of system monitoring include hardware consumption and standard streams (stdout and stderr).

You should know - Previous

What can you log and display?

Next - You should know

Comparing runs

Last modified 1mo ago

Export as PDF

Copy link

Outline

Configuring your credentials

Setting your Neptune API token

Setting your project name

Stopping runs and other objects

Maximum number of fields

Reserved namespaces