Kedro API Reference

You can use Kedro-Neptune plugin to log metadata about the Kedro pipeline execution to Neptune.
You can find detailed information on how to install and use the integration in the user guide.​
kedro neptune
Command line interface (CLI) command for initializing Kedro-Neptune plugin.
Kedro-Neptune plugin lets you log metadata related to Kedro pipelines to Neptune so that you can monitor, visualize, and compare your pipelines and node outputs in the Neptune UI.
You can only use kedro neptune command in the Kedro project folder.
kedro neptune [OPTIONS] COMMAND [ARGS]...
Options
Text
--help
Shows CLI usage - options, commands and their arguments
kedro neptune init
Initialize the plugin, and configure Neptune credentials and logging options.
After initializing it, whenever you run $ kedro run, you will log:
parameters
pipeline execution configuration (run_params)
metadata about Kedro DataSet
hardware consumption and node execution time
configuration files from the conf/base directory
full Kedro run command
any additional metadata like metrics, charts, or images that you logged from inside of your node functions.
Options
Text
api-token
(str , optional, default is "$NEPTUNE_API_TOKEN") - Neptune API token or the environment variable name where it is stored. See How to find your API token.
project
(str, optional, default is "$NEPTUNE_PROJECT") - Neptune project name or the environment variable name where it is stored. See How to find your Neptune project name.
base-namespace
(str, optional, default is "kedro") - Namespace in Neptune where all the Kedro-related metadata is logged.
config
(str, optional, default is "base") - Name of the Subdirectory inside of the Kedro "conf" directory for configuration and catalog files.
Examples
1
kedro neptune init
Copied!
1
kedro neptune init --api-token $NEPTUNE_API_TOKEN --project common/kedro-integration
Copied!
NeptuneFileDataset
NeptuneFileDataSet is a Kedro Data Set that lets you log files to Neptune.

It can be any file on the POSIX compatible filesystem.
To log it, you need to define the NeptuneFileDataSet in any Kedro catalog, including catalog.yml.
Parameters
Text
filepath
(str) - Filepath in POSIX format to a text file prefixed with a protocol like s3://. Same as for Kedro TextDataset.
credentials
(dict, optional) - Credentials required to get access to the underlying filesystem. Same as for Kedro TextDataset.
fs_args
(dict, optional) - Extra arguments to pass into underlying filesystem class constructor. Same as for Kedro TextDataset.
Examples
catalog.yml
1
predictions:
2
  type: kedro.extras.datasets.json.JSONDataSet
3
  filepath: data/07_model_output/predictions.json
4
​
5
[email protected]:
6
  type: kedro_neptune.NeptuneFileDataSet
7
  filepath: data/07_model_output/predictions.json
Copied!

LightGBM API Reference

Python Logger

Last modified 5mo ago

Export as PDF

Copy link

Contents