Kedro

You can use Kedro-Neptune plugin to log metadata about the Kedro pipeline execution to Neptune.
You can find detailed information on how to install and use the integration in the user guide.​
kedro neptune
Command line interface (CLI) command for initializing Kedro-Neptune plugin.
Kedro-Neptune plugin lets you log metadata related to Kedro pipelines to Neptune so that you can monitor, visualize, and compare your pipelines and node outputs in the Neptune UI.
You can only use kedro neptune command in the Kedro project folder. 
kedro neptune [OPTIONS] COMMAND [ARGS]...
Options
​
--help
Shows CLI usage - options, commands and their arguments
kedro neptune init
Initialize the plugin, and configure Neptune credentials and logging options.
After initializing it, whenever you run $ kedro run, you will log:
parameters
pipeline execution configuration (run_params)
metadata about Kedro DataSet
hardware consumption and node execution time
configuration files from the conf/base directory
full Kedro run command
any additional metadata like metrics, charts, or images that you logged from inside of your node functions.
Options
​
api-token
(str , optional) Neptune API token or the environment variable name where it is stored. Default is '$NEPTUNE_API_TOKEN'. See How to find your API token.
project
(str, optional) Neptune project name or the environment variable name where it is stored. Default is '$NEPTUNE_PROJECT'. See How to find your Neptune project name.
base-namespace
(str, optional) Namespace in Neptune where all the Kedro-related metadata is logged. Default is 'kedro'.
config
(str, optional) Name of the Subdirectory inside of the Kedro 'conf' directory for configuration and catalog files. Default is 'base'.
Examples
kedro neptune init
kedro neptune init --api-token $NEPTUNE_API_TOKEN --project common/kedro-integration
NeptuneFileDataset
NeptuneFileDataSet is a Kedro Data Set that lets you log files to Neptune.

It can be any file on the POSIX compatible filesystem.
To log it, you need to define the NeptuneFileDataSet in any Kedro catalog, including catalog.yml.
Parameters
​
filepath
(str) Filepath in POSIX format to a text file prefixed with a protocol like s3://. Same as for Kedro TextDataset.
credentials
(dict, optional) Credentials required to get access to the underlying filesystem. Same as for Kedro TextDataset.
fs_args
(dict, optional) Extra arguments to pass into underlying filesystem class constructor. Same as for Kedro TextDataset.
Examples
catalog.yml
predictions:
  type: kedro.extras.datasets.json.JSONDataSet
  filepath: data/07_model_output/predictions.json
​
[email protected]:
  type: kedro_neptune.NeptuneFileDataSet
  filepath: data/07_model_output/predictions.json

LightGBM

Python Logger

Last updated 1 week ago

Options
--help	Shows CLI usage - options, commands and their arguments

Options
api-token	(`str` , optional) Neptune API token or the environment variable name where it is stored. Default is '$NEPTUNE_API_TOKEN'. See How to find your API token.
project	(`str`, optional) Neptune project name or the environment variable name where it is stored. Default is '$NEPTUNE_PROJECT'. See How to find your Neptune project name.
base-namespace	(`str`, optional) Namespace in Neptune where all the Kedro-related metadata is logged. Default is 'kedro'.
config	(`str`, optional) Name of the Subdirectory inside of the Kedro 'conf' directory for configuration and catalog files. Default is 'base'.

Parameters
filepath	(`str`) Filepath in POSIX format to a text file prefixed with a protocol like s3://. Same as for Kedro TextDataset.
credentials	(`dict`, optional) Credentials required to get access to the underlying filesystem. Same as for Kedro TextDataset.
fs_args	(`dict`, optional) Extra arguments to pass into underlying filesystem class constructor. Same as for Kedro TextDataset.