Skip to content

Resume a run#

To access or update an existing run, call neptune.init_run() and specify the run in one of the following ways:

  • Pass the run's auto-generated Neptune ID to the with_id argument.
  • If you set a custom run ID when the run was created (with the custom_run_id argument or the NEPTUNE_CUSTOM_RUN_ID environment variable), pass it to the custom_run_id argument.

When you've resumed the run, you can:

  • Add new data (such as visualizations or evaluation metrics) to a previously closed run
  • Do multi-stage training more easily
  • Overwrite a field with a new value
  • Delete data from the run
  • Fetch metadata in read-only mode

Good to know

  • There is no limit to the number of times you can resume a run.
  • By default, the Python file from which a run is resumed does not overwrite the record of the original entry-point file. However, you can log the file path manually by providing it to the source_files argument of the init_run() call. For details, see Logging source code.

To resume a run:

  1. Obtain the run identifier.

    The auto-generated identifier is stored in the run's sys/id field.

    IDs highlighted in the Neptune app

    A custom identifier is stored the run's sys/custom_run_id field.

  2. Initialize the run with the ID:

    run = neptune.init_run(with_id="CLS-123")

    run = neptune.init_run(custom_run_id="assertive-haddock-13c6gj2")

  3. Interact with the run as you normally would.

    You can overwrite existing fields, delete fields, or create new ones.

Related

Continue logging to an existing run#

You can update an existing run with new metadata by creating new namespace and fields.

Resume the run first, then use the run object to continue logging metadata as needed.

Log some score you forgot about
run["f1_score"] = f1_score
Resume logging from a checkpoint
# 450 is the epoch from where you want to resume training process
checkpoint = 450

# Continue training as usual
for epoch in range(checkpoint, 1000):
    run["train/accuracy"].append(0.75)
    ...

Editing existing metadata#

To edit the value of an existing field, you overwrite it with different data of the same type.

Previously logged
import neptune

run = neptune.init_run()  # run ID becomes CLS-44 (for this example)
run["learning_rate"] = 0.01
Assign a new value to the existing field
import neptune

run = neptune.init_run(with_id="CLS-44")
run["learning_rate"] = 0.02

The value of the "learning_rate" field has been changed to 0.02 instead of 0.01.

Related

Learn more: Overwriting logged metadata

Deleting metadata from a run#

See Trash and delete data.

Accessing a run in read only mode#

If you're just fetching metadata and not logging anything new, you can reinitialize the run in read only mode. This ensures that the run's metadata won't be modified.

Initialize in read-only mode
import neptune

run = neptune.init_run(with_id="CLS-123", mode="read-only")
Fetch some logged metadata
dataset_hash = run["data_version"].fetch_hash()
Download snapshot of logged model weights
run["train/model_weights"].download()
Query some basic metadata from the system namespace
owner = run["sys/owner"].fetch()

Related