Skip to main content
App version: 3.4.10

Log from different processes

You can log metadata to the same run from multiple separate processes at once.

Use the custom run ID to access the same run in each process:

Process A
run = Run(
    run_id="seagull-s68kj78",
)

for step in epoch:
    # some training loop
    run.log_metrics(
        data={"train/loss": 0.14},
        step=step,
    )
Process B
run = Run(
    run_id="seagull-s68kj78",
)

for step in epoch:
    # some validation loop
    run.log_metrics(
        data={"val/loss": 0.16},
        step=step,
    )

In the above example, the two processes are logging metrics separately, to different attribute paths: train/loss and val/loss, respectively.

You can log the same metric across processes, as long as steps aren't logged out of order within the same attribute. For details, see Log metrics: Setting step values.

Calling add_tags() extends the existing tagset with the provided tags.

caution

If you call log_configs() on an existing attribute, the new value overwrites the previous one. For details, see Edit run configs.

Troubleshooting​

If you initialize an existing run without setting resume=True, Neptune prints the NeptuneRunDuplicate warning. When deliberately accessing the same run in multiple places, you can safely ignore this warning.

Details on using the resume parameter

If using resume=True with log_metrics(), Neptune checks that a run identified by the run_id argument already exists. If such a run doesn't exist, the NeptuneRunNotFound error is raised when Neptune attempts to synchronize the data with the server, and no data is logged.

If you omit the resume parameter or set it to False, and Neptune can't find a run with the specified ID in the project, a new run with the specified ID is created. In this case, no data is lost, but it ends up distributed across multiple runs.

Related