Detectron2 integration guide#
With the Neptune integration, you can keep track of your metadata when training models with Detectron2. This guide covers how to add Neptune logging to your training loop, including:
- Saving checkpoints during model training.
- Logging and visualizing the prediction of the trained model.
The following is logged automatically:
- Model configuration
- Training code and Git information
- System metrics and hardware consumption
See example in Neptune  Code examples 
Before you start#
- Sign up at neptune.ai/register.
- Create a project for storing your metadata.
- Have Detectron2 installed.
Installing the integration#
To use your pre-installed version of Neptune together with the integration:
To install both Neptune and the integration:
Passing your Neptune credentials
Once you've registered and created a project, set your Neptune API token and full project name to the NEPTUNE_API_TOKEN
and NEPTUNE_PROJECT
environment variables, respectively.
To find your API token: In the bottom-left corner of the Neptune app, expand the user menu and select Get my API token.
To find your project: Your full project name has the form workspace-name/project-name
. To copy the name, click the menu in the top-right corner and select Properties.
While it's not recommended especially for the API token, you can also pass your credentials in the code when initializing Neptune.
run = neptune.init_run(
project="ml-team/classification", # your full project name here
api_token="h0dHBzOi8aHR0cHM6Lkc78ghs74kl0jvYh...3Kb8", # your API token here
)
For more help, see Setting Neptune credentials.
If you'd rather follow the guide without any setup, you can run the example in Colab .
Logging example#
-
Create a Neptune run:
-
If you haven't set up your credentials, you can log anonymously:
For more run customization options, see
neptune.init_run()
. -
-
Set up your training data and configure your model.
For an example Detectron2 script, see neptune-ai/examples on GitHub.
-
Create a hook using the Neptune integration.
from neptune_detectron2 import NeptuneHook hook = NeptuneHook(run=run, log_model=True, metrics_update_freq=10)
In this case, metrics are logged every 10th epoch.
You can also log checkpoints by passing the argument
log_checkpoints=True
. -
Train the model with the hook.
-
Prepare the model for prediction.
-
Log the predictions to Neptune.
from detectron2.utils.visualizer import ColorMode from neptune.types import File dataset_dicts = get_balloon_dicts("balloon/val") for idx, d in enumerate(random.sample(dataset_dicts, 3)): im = cv2.imread(d["file_name"]) outputs = predictor( im ) # (1)! v = Visualizer( im[:, :, ::-1], metadata=balloon_metadata, scale=0.5, instance_mode=ColorMode.IMAGE, # (2)! ) out = v.draw_instance_predictions(outputs["instances"].to("cpu")) image = out.get_image()[:, :, ::-1] img_rgb = cv2.cvtColor(image, cv2.COLOR_BGR2RGB) run["training/prediction_visualization"][f"{idx}"].upload( File.as_image(img_rgb / 255.0) )
- The format is documented in the Detectron2 docs .
- Remove the colors of unsegmented pixels. This option is only available for segmentation models.
-
Run your script as you normally would.
To open the run and watch your model training live, click the Neptune link that appears in the console output.
Example link: https://app.neptune.ai/o/common/org/detectron2-integration/e/DET-156/metadata
If Neptune can't find your project name or API token
As a best practice, you should save your Neptune API token and project name as environment variables. However, you can also pass them as arguments when you're using a function that takes
api_token
andproject
as parameters:api_token="Your Neptune API token here"
- Find and copy your API token by expanding your user menu and selecting Get my API token.
project="workspace-name/project-name""
- Find and copy your project name in the top-right menu: → Properties.
For example:
-
To stop the connection to Neptune and sync all data, call the
stop()
method:
See example in Neptune  Code examples 
Related
- API reference ≫ Detectron2 integration
- What you can log and display
- neptune-detectron2 repo on GitHub
- Detectron2 repo on GitHub