Reproducing a run#
When building ML models for research or production, it's important to be able to reproduce a run to validate its results and performance. With Neptune, you can reproduce any run by retrieving its metadata, such as hyperparameters and datasets, and logging it into a newly created run.
In this guide, we'll show you how to:
- Reopen an old run in order to fetch the metadata needed to reproduce it.
- Use the fetched metadata to parametrize a new run with the same training loop.
To demonstrate this process, we've prepared an example run with some metadata already logged.
The metadata structure of the example run is as follows:
Before you start#
You have Neptune installed and your Neptune credentials are saved as environment variables.
For details, see Install Neptune.
You have an existing run in a Neptune project that you have access to.
Fetching metadata from an existing run#
Get the ID of an old run#
To query the metadata we need from a run to be reproduced, we need to know its Neptune ID. The Neptune ID is a unique identifier of an object, for example
You can grab the ID manually from the web app. In the runs table, it's displayed in the leftmost column.
You can also obtain the ID programmatically from the system namespace using the following code:
Resume the old run#
Using the ID obtained in the previous step, we can re-open the existing run.
We're not logging new data, so we can resume the run in read-only mode.
You can do this whenever you're initializing an existing Neptune object that you only want to query metadata from.
- For details, see Connection modes: Read-only mode
Fetch the metadata from the old run#
To re-run the training, we first need to get the required hyperparameters and dataset files. That way you'll be able to instantiate a model and dataset object with the same configuration. Use the
fetch() method to retrieve the relevant data.
Creating a new run and logging the metadata to it#
We can now create a run that will use the metadata fetched in the previous step.
Adding the "reproduce" and "new-run" tags is optional, but will make it easier to find the run in the future.
Log hyperparameters and dataset details to the new run#
Now that a new run exists, we can start logging the metadata from the old run to it.
When you're finished logging the metadata to the new run, remember to stop tracking both runs using the
You can also see the full code examples on our GitHub examples repo.