Skip to content

Detectron2 integration guide#

Open in Colab

Custom dashboard displaying metadata logged with Detectron2

With the Neptune integration, you can keep track of your metadata when training models with Detectron2. This guide covers how to add Neptune logging to your training loop, including:

  • Saving checkpoints during model training.
  • Logging and visualizing the prediction of the trained model.

The following is logged automatically:

  • Model configuration
  • Training code and Git information
  • System metrics and hardware consumption

See example in Neptune  Code examples 

Before you start#

Installing the integration#

To use your preinstalled version of Neptune together with the integration:

pip
pip install -U neptune-detectron2
conda
conda install -c conda-forge neptune-detectron2

To install both Neptune and the integration:

pip
pip install -U "neptune[detectron2]"
conda
conda install -c conda-forge neptune neptune-detectron2
Passing your Neptune credentials

Once you've registered and created a project, set your Neptune API token and full project name to the NEPTUNE_API_TOKEN and NEPTUNE_PROJECT environment variables, respectively.

export NEPTUNE_API_TOKEN="h0dHBzOi8aHR0cHM.4kl0jvYh3Kb8...6Lc"

To find your API token: In the bottom-left corner of the Neptune app, expand the user menu and select Get my API token.

export NEPTUNE_PROJECT="ml-team/classification"

To find your project: Your full project name has the form workspace-name/project-name. To copy the name, click the menu in the top-right corner and select Edit project details.

While it's not recommended especially for the API token, you can also pass your credentials in the code when initializing Neptune.

run = neptune.init_run(
    project="ml-team/classification",  # your full project name here
    api_token="h0dHBzOi8aHR0cHM6Lkc78ghs74kl0jvYh...3Kb8",  # your API token here
)

For more help, see Set Neptune credentials.

If you'd rather follow the guide without any setup, you can run the example in Colab .

Logging example#

  1. Create a Neptune run:

    import neptune

run = neptune.init_run() # (1)!

    1. If you haven't set up your credentials, you can log anonymously:

      neptune.init_run(
    api_token=neptune.ANONYMOUS_API_TOKEN,
    project="common/detectron2-integration",
)

    For more run customization options, see neptune.init_run().

  2. Set up your training data and configure your model.

    For an example Detectron2 script, see neptune-ai/examples on GitHub.

  3. Create a hook using the Neptune integration.

    from neptune_detectron2 import NeptuneHook

hook = NeptuneHook(run=run, log_model=True, metrics_update_freq=10)

    In this case, metrics are logged every 10th epoch.

    You can also log checkpoints by passing the argument log_checkpoints=True.

  4. Train the model with the hook.

    trainer = ...
trainer.register_hooks([hook])
trainer.train()

  5. Prepare the model for prediction.

    ...
predictor = DefaultPredictor(cfg)

  6. Log the predictions to Neptune.

    from detectron2.utils.visualizer import ColorMode
from neptune.types import File

dataset_dicts = get_balloon_dicts("balloon/val")
for idx, d in enumerate(random.sample(dataset_dicts, 3)):
    im = cv2.imread(d["file_name"])
    outputs = predictor(
        im
    ) # (1)!
    v = Visualizer(
        im[:, :, ::-1],
        metadata=balloon_metadata,
        scale=0.5,
        instance_mode=ColorMode.IMAGE, # (2)!
    )
    out = v.draw_instance_predictions(outputs["instances"].to("cpu"))
    image = out.get_image()[:, :, ::-1]
    img_rgb = cv2.cvtColor(image, cv2.COLOR_BGR2RGB)
    run["training/prediction_visualization"][f"{idx}"].upload(
        File.as_image(img_rgb / 255.0)
    )
    1. The format is documented in the Detectron2 docs .
    2. Remove the colors of unsegmented pixels. This option is only available for segmentation models.

  7. Run your script as you normally would.

    To open the run and watch your model training live, click the Neptune link that appears in the console output.

    Example link: https://app.neptune.ai/o/common/org/detectron2-integration/e/DET-156/metadata

    If Neptune can't find your project name or API token

    As a best practice, you should save your Neptune API token and project name as environment variables. However, you can also pass them as arguments when you're using a function that takes api_token and project as parameters:

    • api_token="Your Neptune API token here"
      • Find and copy your API token by expanding your user menu and selecting Get my API token.
    • project="workspace-name/project-name""
      • Find and copy your project name in the top-right menu: → Edit project details.

    For example:

    Neptune client library
    model_version = neptune.init_model_version(
    api_token="h0dHBzOi8aHR0cHM6Lkc78ghs74kl0jvYh3Kb8",
    project="ml-team/named-entity-recognition",
    model= ...,
)
    Neptune integration
    neptune_logger = dl.NeptuneLogger(
    api_token="h0dHBzOi8aHR0cHM6Lkc78ghs74kl0jvYh3Kb8",
    project="ml-team/named-entity-recognition",
)

  8. To stop the connection to Neptune and sync all data, call the stop() method:

    run.stop()

See example in Neptune  Code examples 

Related