Working with Catalyst#

Custom dashboard displaying metadata logged with Catalyst

Neptune compatibility note

This integration has not yet been updated for neptune 1.x and requires using neptune-client <1.0.0.

Catalyst is a PyTorch framework for Deep Learning R&D. It focuses on reproducibility, rapid experimentation, and codebase reuse.

Neptune support is implemented as a logger in Catalyst. With the Neptune logger, you can automatically track:

Metrics
Hyperparameters
Images
Artifacts (such as videos, audio, model checkpoints, and files)
Hardware consumption metrics
stdout and stderr streams
Training code and Git information

See in Neptune Code examples

API reference ≫ Catalyst
NeptuneLogger reference in Catalyst API docs
Catalyst on GitHub

Quickstart#

Tip

This section is for Catalyst users who are familiar with loggers, like CSV logger or TensorBoard logger.

NeptuneLogger is part of the Catalyst library. To start logging, create a Neptune logger and pass it to the runner:

Create the logger:
```
import neptune  # (1)!
from catalyst import dl

neptune_logger = dl.NeptuneLogger(
    api_token=neptune.ANONYMOUS_API_TOKEN,  # (2)!
    project="common/catalyst-integration",  # (3)!
    tags=["pretraining", "retina"],  # optional
)
```
1. Only needed to access the anonymous API token. If you're logging to your own project, you can omit the import.
2. The api_token argument is included to enable anonymous logging. Once you register, you should leave the token out of your script and instead save it as an environment variable.
3. Projects in the common workspace are public and can be used for testing. To log to your own workspace, pass the full name of your Neptune project: workspace-name/project-name. For example, "ml-team/classification". To copy it, navigate to the project settings → Properties.
There are further ways to customize the behavior of the logger. For details, see the Catalyst API reference .

Pass the logger to the runner:

Pass to SupervisedRunner

my_runner = dl.SupervisedRunner()

my_runner.train(
    loggers={"neptune": neptune_logger},
    ...
)

Pass to custom Runner

class CustomRunner(dl.IRunner):
    ...
    def get_loggers(self):
        return {
            "console": dl.ConsoleLogger(),
            "neptune": neptune_logger
        }

runner = CustomRunner().run()

The Neptune logger setup is complete and you can run your scripts without additional changes.

Your metadata will be logged in the Neptune project for further analysis, comparison, and collaboration.

Catalyst logging example#

This guide walks you through connecting NeptuneLogger to your machine-learning scripts and using it in your experimentation.

Before you start#

Tip

If you'd rather follow the guide without any setup, you can run the example in Colab .

Set up Neptune. Instructions:
Install Catalyst and dependencies:
```
pip install -U "catalyst[neptune]"
```

Adding NeptuneLogger to the Catalyst script#

Catalyst has a unified way of logging metadata, by using loggers.

You can learn more about Catalyst Loggers in the Catalyst docs .

To start logging, create a Neptune logger and pass it to the runner:

Create a NeptuneLogger instance.
```
from catalyst import dl

# Create NeptuneLogger instance
neptune_logger = dl.NeptuneLogger()  # (1)!
```
1. If you haven't set up your credentials, you can log anonymously: dl.NeptuneLogger(api_token=neptune.ANONYMOUS_API_TOKEN, project="common/catalyst-integration")
There are further ways to customize the behavior of the logger. For details, see the Catalyst API reference .

Pass neptune_logger to the runner.

The example below uses SupervisedRunner as an example:

from catalyst import dl

# Create runner
my_runner = dl.SupervisedRunner()

my_runner.train(loggers={"neptune": neptune_logger}, ...)

The Neptune logger is now ready.

Run your script:
```
python main.py
```

If Neptune can't find your project name or API token

As a best practice, you should save your Neptune API token and project name as environment variables. However, you can also pass them as arguments when you're using a function that takes api_token and project as parameters:

api_token="Your Neptune API token here"
- Find and copy your API token by clicking your avatar and selecting Get my API token.
project="workspace-name/project-name""
- Find and copy your project name in the project settings → Properties.

For example:

Neptune client library

model_version = neptune.init_model_version(
    api_token="h0dHBzOi8aHR0cHM6Lkc78ghs74kl0jvYh3Kb8",
    project="ml-team/named-entity-recognition",
    model= ...
)

Neptune integration

neptune_logger = dl.NeptuneLogger(
    api_token="h0dHBzOi8aHR0cHM6Lkc78ghs74kl0jvYh3Kb8",
    project="ml-team/named-entity-recognition",
)

Analyzing the logged metadata in Neptune#

Your metadata will be logged in the given Neptune project for analysis, comparison, and collaboration.

To open the run, click the Neptune link that appears in the console output.

Example link: https://app.neptune.ai/o/common/org/catalyst-integration/e/CATALYST-38823

You can also open the project and look for your run in the Runs tab.

Manually logging metadata#

If you have other types of metadata that are not covered in this guide, you can still log them using the Neptune client library.

When you initialize the run, you get a run object, to which you can assign different types of metadata in a structure of your own choosing.

import neptune

# Create a new Neptune run
run = neptune.init_run()

# Log metrics or other values inside loops
for epoch in range(n_epochs):
    ...  # Your training loop

    run["train/epoch/loss"].append(loss)  # Each append() appends a value
    run["train/epoch/accuracy"].append(acc)

# Upload files
run["test/preds"].upload("path/to/test_preds.csv")

# Track and version artifacts
run["train/images"].track_files("./datasets/images")

# Record numbers or text
run["tokenizer"] = "regexp_tokenize"