Skip to content

API reference: MLflow integration#

You can use the neptune mlflow command to export existing run metadata from MLflow to Neptune.

Related

neptune mlflow#

Exports MLflow metadata to Neptune.

Note for large projects

When exporting runs from MLflow, Neptune checks whether each MLflow run already exists in the Neptune project.

Due to a technical limitation, a maximum of 10 000 Neptune runs can be checked for each export. Therefore, if your Neptune project contains more than 10 000 runs, it is possible that some MLflow runs will be copied over to Neptune even if they already exist in the Neptune project.

Command syntax: neptune mlflow [--api-token] [--project] [--mlflow-tracking-uri] [--exclude-artifacts] [--max-artifact-size]

Options         Type Default Description
-p, --project str - Neptune project name. If None, the value of the NEPTUNE_PROJECT environment variable is used.
-a, --api-token str - Neptune API token. If None, the value of the NEPTUNE_API_TOKEN environment variable is used.
-u, --mlflow-tracking-uri str - Your MLflow tracking URI. If not provided, it is left to the MLflow client to resolve it. For more, see the MLflow documentation .
-e, --exclude-artifacts bool False Whether to skip uploading artifacts to Neptune.
-m, --max-artifact-size int 50 Maximum size of the artifact to be uploaded to Neptune (in MB). For directories, this will be treated as the max size of the entire directory.
--help - - Show help message and exit.

Examples

If you've set your Neptune credentials as environment variables, you can use the following command to export data with default settings:

neptune mlflow
How do I save my credentials as environment variables?

Set your Neptune API token and full project name to the NEPTUNE_API_TOKEN and NEPTUNE_PROJECT environment variables, respectively.

For example:

export NEPTUNE_API_TOKEN="h0dHBzOi8aHR0cHM.4kl0jvYh3Kb8...6Lc" # (1)!
  1. On Windows, the command is set instead of export.
export NEPTUNE_PROJECT="ml-team/classification" # (1)!
  1. On Windows, the command is set instead of export.

Finding your credentials:

  • API token: In the bottom-left corner of the Neptune app, expand your user menu and select Get your API token.
  • Project: Your full project name has the form workspace-name/project-name. To copy the name, click the menu in the top-right corner and select Edit project details.

If you're working in Colab, you can set your credentials with the os and getpass libraries:

import os
from getpass import getpass
os.environ["NEPTUNE_API_TOKEN"] = getpass("Enter your Neptune API token: ")
os.environ["NEPTUNE_PROJECT"] = "workspace-name/project-name"

Otherwise, you can pass the credentials as options:

neptune mlflow \
    --api-token I2YzgMz1h0dHBzOi8aHR0cHM6Lkc78ghs74kl0jvh0dHBzOi8aHR0Yh3Kb8MifQ== \
    --project ml-team/llm-project \

Using the command with more options:

neptune mlflow \
    --project ml-team/llm-project-2 \
    --mlflow-tracking-uri file:///tmp/my_tracking \
    --max-artifact-size 100 \
    /

neptune-notebooks incompatibility

Currently, the CLI component of the integration does not work together with the Neptune-Jupyter extension (neptune-notebooks).

Until a fix is released, if you have neptune-notebooks installed, you must uninstall it to be able to use the neptune mlflow command.