Skip to content

Detectron2 integration guide#

Open in Colab

Custom dashboard displaying metadata logged with Detectron2

With the Neptune integration, you can keep track of your metadata when training models with Detectron2. This guide covers how to add Neptune logging to your training loop, including:

  • Saving checkpoints during model training.
  • Logging and visualizing the prediction of the trained model.

The following is logged automatically:

  • Model configuration
  • Training code and Git information
  • System metrics and hardware consumption

See example in Neptune  Code examples 

Before you start#

  • Sign up at neptune.ai/register.
  • Create a project for storing your metadata.
  • Have Detectron2, torch, and torchvision installed. See the Detectron2 documentation for details.

    pip install -U torch torchvision
    
    pip install -U 'git+https://github.com/facebookresearch/detectron2.git' --no-build-isolation
    

Installing the integration#

To use your preinstalled version of Neptune together with the integration:

pip
pip install -U neptune-detectron2
conda
conda install -c conda-forge neptune-detectron2

To install both Neptune and the integration:

pip
pip install -U "neptune[detectron2]"
conda
conda install -c conda-forge neptune neptune-detectron2
Passing your Neptune credentials

Once you've registered and created a project, set your Neptune API token and full project name to the NEPTUNE_API_TOKEN and NEPTUNE_PROJECT environment variables, respectively.

export NEPTUNE_API_TOKEN="h0dHBzOi8aHR0cHM.4kl0jvYh3Kb8...6Lc"

To find your API token: In the bottom-left corner of the Neptune app, expand the user menu and select Get my API token.

export NEPTUNE_PROJECT="ml-team/classification"

Your full project name has the form workspace-name/project-name. You can copy it from the project settings: Click the menu in the top-right → Edit project details.

On Windows, navigate to SettingsEdit the system environment variables, or enter the following in Command Prompt: setx SOME_NEPTUNE_VARIABLE 'some-value'


While it's not recommended especially for the API token, you can also pass your credentials in the code when initializing Neptune.

run = neptune.init_run(
    project="ml-team/classification",  # your full project name here
    api_token="h0dHBzOi8aHR0cHM6Lkc78ghs74kl0jvYh...3Kb8",  # your API token here
)

For more help, see Set Neptune credentials.

If you'd rather follow the guide without any setup, you can run the example in Colab .

Logging example#

  1. Create a Neptune run:

    import neptune
    
    run = neptune.init_run() # (1)!
    
    1. If you haven't set up your credentials, you can log anonymously:

      neptune.init_run(
          api_token=neptune.ANONYMOUS_API_TOKEN,
          project="common/detectron2-integration",
      )
      

    For more run customization options, see neptune.init_run().

  2. Set up your training data and configure your model.

    For an example Detectron2 script, see neptune-ai/examples on GitHub.

  3. Create a hook using the Neptune integration.

    from neptune_detectron2 import NeptuneHook
    
    hook = NeptuneHook(run=run, log_model=True, metrics_update_freq=10)
    

    In this case, metrics are logged every 10th epoch.

    You can also log checkpoints by passing the argument log_checkpoints=True.

  4. Train the model with the hook.

    trainer = ...
    trainer.register_hooks([hook])
    trainer.train()
    
  5. Prepare the model for prediction.

    predictor = DefaultPredictor(cfg)
    
  6. Log the predictions to Neptune.

    from detectron2.utils.visualizer import ColorMode
    from neptune.types import File
    
    dataset_dicts = get_balloon_dicts("balloon/val")
    for idx, d in enumerate(random.sample(dataset_dicts, 3)):
        im = cv2.imread(d["file_name"])
        outputs = predictor(
            im
        ) # (1)!
        v = Visualizer(
            im[:, :, ::-1],
            metadata=balloon_metadata,
            scale=0.5,
            instance_mode=ColorMode.IMAGE, # (2)!
        )
        out = v.draw_instance_predictions(outputs["instances"].to("cpu"))
        image = out.get_image()[:, :, ::-1]
        img_rgb = cv2.cvtColor(image, cv2.COLOR_BGR2RGB)
        run["training/prediction_visualization"][f"{idx}"].upload(
            File.as_image(img_rgb)
        )
    
    1. The format is documented in the Detectron2 docs .
    2. Remove the colors of unsegmented pixels. This option is only available for segmentation models.
  7. To stop the connection to Neptune and sync all data, call the stop() method:

    run.stop()
    
  8. Run your script as you normally would.

    To open the run and watch your model training live, click the Neptune link that appears in the console output.

    Example link: https://app.neptune.ai/o/common/org/detectron2-integration/e/DET-156

    If Neptune can't find your project name or API token

    As a best practice, you should save your Neptune API token and project name as environment variables:

    export NEPTUNE_API_TOKEN="h0dHBzOi8aHR0cHM6Lkc78ghs74kl0jv...Yh3Kb8"
    
    export NEPTUNE_PROJECT="ml-team/classification"
    

    Alternatively, you can pass the information when using a function that takes api_token and project as arguments:

    run = neptune.init_run( # (1)!
        api_token="h0dHBzOi8aHR0cHM6Lkc78ghs74kl0jv...Yh3Kb8",  # your token here
        project="ml-team/classification",  # your full project name here
    )
    
    1. Also works for init_model(), init_model_version(), init_project(), and integrations that create Neptune runs underneath the hood, such as NeptuneLogger or NeptuneCallback.

    2. API token: In the bottom-left corner, expand the user menu and select Get my API token.

    3. Project name: You can copy the path from the project details ( Edit project details).

    If you haven't registered, you can log anonymously to a public project:

    api_token=neptune.ANONYMOUS_API_TOKEN
    project="common/quickstarts"
    

    Make sure not to publish sensitive data through your code!

See example in Neptune  Code examples 

Related