Skip to content

Field count limit exceeded#


NeptuneFieldCountLimitExceedException: There are too many fields (more than 9000) in the <Neptune object>.

Issue: You're creating too many individual fields in a single Neptune object (run, model, etc.). The limit for a single run or other Neptune object is 9000 unique fields.

For example, when logging run["params/lr"] = 0.01, then "lr" is the field.



The field limit is not about metadata limits per se.

For example, rather than assigning 100 values to 100 fields (run["field1"] = 0, run["field2"] = 1, ..., run["field100"] = 99) you could construct a series of those 100 values and log them under a single field:

for i in range(100):

You can also do this with strings and images. For more, see Creating a series

If you exceed the field limit while logging, Neptune stops the synchronization to the server and stores the data locally. To continue uploading the metadata, you need to delete some excess fields from the object in question, then sync the data manually.

Sorting runs by number of fields#

You can query the API for runs with the largest number of fields logged.

We provide a Jupyter notebook that demonstrates how to fetch this information.

Notebook: Get runs with the most fields 

Counting fields#

To count the fields of a Neptune object, use the get_structure() method and some scripting:

>>> import neptune
>>> def count_fields(d):
...     return sum([count_fields(v) if isinstance(v, dict)
...     else 1 for v in d.values()])
>>> run = neptune.init_run(
...     project="ml-team/classification",
...     with_id="CLS-34",
...     mode="read-only",
... )
>>> count_fields(run.get_structure())

Deleting fields#

You can delete fields or namespaces with the del command. For example, to delete the "training/checkpoints" namespace from a run with identifier "ACC-47":

import neptune

# Resume an existing run
run = neptune.init_run(with_id="CLS-47")

# Delete data
del run["training/checkpoints"]
How do I find the ID?

The Neptune ID is a unique identifier for the run. In the table view, it's displayed in the leftmost column.

The ID is stored in the system namespace (sys/id).

If the run is active, you can obtain its ID with run["sys/id"].fetch(). For example:

>>> run = neptune.init_run(project="ml-team/classification")
>>> run["sys/id"].fetch()

Syncing data manually#

Once you're done deleting fields, synchronize the data manually:

  1. Navigate to the folder from which you executed the script. It should contain a .neptune folder.
  2. In a terminal, enter the following command:

    neptune sync

For further details, see the API reference:

Getting help