Skip to content

Field count limit exceeded#

Error

NeptuneFieldCountLimitExceedException: There are too many fields (more than 9000) in the <Neptune object>.

Issue: You're creating too many individual fields in a single Neptune object (run, model, etc.). The limit for a single run or other Neptune object is 9000 unique fields.

For example, when logging run["params/lr"] = 0.01, then "lr" is the field.

Workarounds

If you exceed the field limit while logging, Neptune stops the synchronization to the server and stores the data locally. To continue uploading the metadata, you need to delete some excess fields from the object in question, then sync the data manually.

Note that the field limit is not about metadata limits per se.

For example, rather than assigning 100 values to 100 fields (run["field1"] = 0, run["field2"] = 1, ..., run["field100"] = 99) you could construct a series of those 100 values and log them under a single field:

for i in range(100):
    run["field"].append(i)

You can also do this with strings and images. For more, see Creating a series

Sorting runs by number of fields#

You can query the API for runs with the largest number of fields logged.

We provide a Jupyter notebook that demonstrates how to fetch this information.

Notebook: Get runs with the most fields 

Counting fields#

To count the fields of a Neptune object, use the get_structure() method and some scripting:

>>> import neptune
>>> def count_fields(d):
...     return sum([count_fields(v) if isinstance(v, dict)
...     else 1 for v in d.values()])
...
>>> run = neptune.init_run(
...     project="ml-team/classification",
...     with_id="CLS-34",
...     mode="read-only",
... )
https://app.neptune.ai/ml-team/classification/e/CLS-34/metadata
...
>>> count_fields(run.get_structure())
571

Deleting fields#

You can delete fields or namespaces with the del command.

To avoid extra logging of new metadata, turn off system monitoring and other automatic logging options:

  • capture_hardware_metrics=False
  • capture_stderr=False
  • capture_stdout=False
  • capture_traceback=False
  • git_ref=False
  • source_files=[]

For example, to delete the training/checkpoints namespace from a run with the identifier "CLS-47":

import neptune

# Resume an existing run with auto-logging turned off
run = neptune.init_run(
    with_id="CLS-47",
    capture_hardware_metrics=False,
    capture_stderr=False,
    capture_stdout=False,
    capture_traceback=False,
    git_ref=False,
    source_files=[],
)

# Delete data
del run["training/checkpoints"]
How do I find the ID?

The Neptune ID is a unique identifier for the run. The Experiments tab displays it in the leftmost column.

In the run structure, the ID is stored in the system namespace (sys).

  • If the run is active, you can obtain its ID with run["sys/id"].fetch(). For example:

    >>> run = neptune.init_run()
...
>>> run["sys/id"].fetch()
'CLS-26'

  • If you set a custom run ID, it's stored at sys/custom_run_id:

    >>> run["sys/custom_run_id"].fetch()
'vigilant-puffin-20bt9'

Syncing data manually#

Once you're done deleting fields, synchronize the data manually:

  1. Navigate to the folder from which you executed the script. It should contain a .neptune folder.

  2. In a terminal, enter the following command:

    neptune sync

For further details, see the API reference:

Getting help