Skip to content

Detectron2 integration guide#

Open in Colab

Custom dashboard displaying metadata logged with Detectron2

With the Neptune integration, you can keep track of your metadata when training models with Detectron2. This guide covers how to add Neptune logging to your training loop, including:

  • Saving checkpoints during model training.
  • Logging and visualizing the prediction of the trained model.

The following is logged automatically:

  • Model configuration
  • Training code and Git information
  • System metrics and hardware consumption

See example in Neptune  Code examples 

Before you start#

  • Sign up at neptune.ai/register.
  • Create a project for storing your metadata.

  • Have Detectron2, torch, and torchvision installed. See the Detectron2 documentation for details.

    pip install -U torch torchvision

    pip install -U 'git+https://github.com/facebookresearch/detectron2.git' --no-build-isolation

Installing the integration#

To use your preinstalled version of Neptune together with the integration:

pip
pip install -U neptune-detectron2

To install both Neptune and the integration:

pip
pip install -U "neptune[detectron2]"
Passing your Neptune credentials

Once you've registered and created a project, set your Neptune API token and full project name to the NEPTUNE_API_TOKEN and NEPTUNE_PROJECT environment variables, respectively.

export NEPTUNE_API_TOKEN="h0dHBzOi8aHR0cHM.4kl0jvYh3Kb8...6Lc"

To find your API token: In the bottom-left corner of the Neptune app, expand the user menu and select Get my API token.

export NEPTUNE_PROJECT="ml-team/classification"

Your full project name has the form workspace-name/project-name. You can copy it from the project settings: Click the menu in the top-right → Details & privacy.

On Windows, navigate to Settings → Edit the system environment variables, or enter the following in Command Prompt: setx SOME_NEPTUNE_VARIABLE 'some-value'

While it's not recommended especially for the API token, you can also pass your credentials in the code when initializing Neptune.

run = neptune.init_run(
    project="ml-team/classification",  # your full project name here
    api_token="h0dHBzOi8aHR0cHM6Lkc78ghs74kl0jvYh...3Kb8",  # your API token here
)

For more help, see Set Neptune credentials.

If you'd rather follow the guide without any setup, you can run the example in Colab .

Logging example#

  1. Create a Neptune run:

    import neptune

run = neptune.init_run() # (1)!

    1. If you haven't set up your credentials, you can log anonymously:

      neptune.init_run(
    api_token=neptune.ANONYMOUS_API_TOKEN,
    project="common/detectron2-integration",
)

    For more run customization options, see neptune.init_run().

  2. Set up your training data and configure your model.

    For an example Detectron2 script, see neptune-ai/examples on GitHub.

  3. Create a hook using the Neptune integration.

    from neptune_detectron2 import NeptuneHook

hook = NeptuneHook(run=run, log_model=True, metrics_update_freq=10)

    In this case, metrics are logged every 10th epoch.

    You can also log checkpoints by passing the argument log_checkpoints=True.

  4. Train the model with the hook.

    trainer = ...
trainer.register_hooks([hook])
trainer.train()

  5. Prepare the model for prediction.

    predictor = DefaultPredictor(cfg)

  6. Log the predictions to Neptune.

    from detectron2.utils.visualizer import ColorMode
from neptune.types import File

dataset_dicts = get_balloon_dicts("balloon/val")
for idx, d in enumerate(random.sample(dataset_dicts, 3)):
    im = cv2.imread(d["file_name"])
    outputs = predictor(
        im
    ) # (1)!
    v = Visualizer(
        im[:, :, ::-1],
        metadata=balloon_metadata,
        scale=0.5,
        instance_mode=ColorMode.IMAGE, # (2)!
    )
    out = v.draw_instance_predictions(outputs["instances"].to("cpu"))
    image = out.get_image()[:, :, ::-1]
    img_rgb = cv2.cvtColor(image, cv2.COLOR_BGR2RGB)
    run["training/prediction_visualization"][f"{idx}"].upload(
        File.as_image(img_rgb)
    )
    1. The format is documented in the Detectron2 docs .
    2. Remove the colors of unsegmented pixels. This option is only available for segmentation models.

  7. To stop the connection to Neptune and sync all data, call the stop() method:

    run.stop()

  8. Run your script as you normally would.

    To open the run and watch your model training live, click the Neptune link that appears in the console output.

    Example link: https://app.neptune.ai/o/common/org/detectron2-integration/e/DET-156

    If Neptune can't find your project name or API token

    As a best practice, you should save your Neptune API token and project name as environment variables:

    export NEPTUNE_API_TOKEN="h0dHBzOi8aHR0cHM6Lkc78ghs74kl0jv...Yh3Kb8"

    export NEPTUNE_PROJECT="ml-team/classification"

    Alternatively, you can pass the information when using a function that takes api_token and project as arguments:

    run = neptune.init_run(
    api_token="h0dHBzOi8aHR0cHM6Lkc78ghs74kl0jv...Yh3Kb8", # (1)!
    project="ml-team/classification", # (2)!
)
    1. In the bottom-left corner, expand the user menu and select Get my API token.
    2. You can copy the path from the project details ( → Details & privacy).

    If you haven't registered, you can log anonymously to a public project:

    api_token=neptune.ANONYMOUS_API_TOKEN
project="common/quickstarts"

    Make sure not to publish sensitive data through your code!

See example in Neptune  Code examples 

Related