What you can log to experiments

There are various object types that you can log to Neptune. Some of them are logged automatically; some you need to specify explicitly. There is a place in the UI associated with every logging object type defined below.

  • Parameters: You can log your experiment hyper-parameters, such as learning rate, tree depth or regularization by passing them in as a dictionary during experiment creation. Note that Neptune parameters are immutable - you cannot change them later.

    For more information, see the create_experiment() method.

  • Metrics: You can log numerical values to Neptune. These could be machine learning metrics like accuracy or MSE, timing metrics like time of the forward pass or any other numerical value. You can log one value (like final validation loss) or multiple values (accuracy after every training epoch). If more than one value is sent to the same log, then charts are automatically created. Simply tell Neptune the name of the log and what value you want to send. There are specific methods for logging metrics, logging text, logging images, logging audio, logging interactive charts, logging custom html, and setting properties.

    For more information, see the Experiment class methods.

  • Text: You can send text values like warning messages, current parameter values or anything else. It can be one value (like “idea worked”) or multiple values (like parameters after each hyperparameter sweep iteration). Simply tell Neptune the name of the log and what value you want to send.

    For more information, see the log_text() method.

  • Images: You can send image data like ROC AUC charts, object detection predictions after every epoch, or anything else. It can be one image (like a test confusion matrix) or multiple images (like validation predictions after every epoch). Simply tell Neptune the name of the log and what images you want to send. You must first save the image file on disk and then send the file to Neptune. The following data types are supported:

    • PIL

    • Matplotlib

    • Numpy

    • Image files (PNG, JPG, and so on) on disk

    For more information, see the log_image() method.

  • Artifacts: You can send any data type as a file artifact in Neptune. It can be a model binary, validation prediction, model checkpoint or anything else. Simply tell Neptune which file you want to log. See this example.

  • Hardware consumption: Neptune automatically saves your hardware consumption data if the psutil library has been installed. Hardware types are:

    • GPU utilization, information from the nvidia-smi command - works both for single and multi-GPU setups.

    • GPU memory, information from the nvidia-smi command - works both for single and multi-GPU setups.

    • CPU utilization

    • Memory

    Here is an example.

  • Terminal outputs: Neptune automatically saves everything that is printed to your terminal and groups it into stdout (output) and stderr (error messages). See this example.

  • Properties: You can log your experiment information like status, data version, or anything else as a name:value (text) pair. Neptune properties are mutable - you can change them later. See the lower section in in this experiment.

  • Tags: You can attach tags (text) to every experiment to make the experiment organization easier. For more information, see Organize experiments.

  • Code: The following methods are available for versioning your code in Neptune. Whatever method you use, whenever you create an experiment, the code is versioned.

    • Git: Neptune automatically fetches your Git information, like commit id or commit message. If you have a Git repo (meaning, a .git in the directory from which you are starting an experiment), then Neptune automatically shows a Git reference in the experiment details. The same is true if the .git repo is above, in the directory tree from which you start an experiment. Example Git reference.

    • Code snapshots: You can specify files, directories or use regexp to choose files you want to snapshot and log directly to Neptune. Example.

    • Notebook snapshots: If you are running your experiments from Jupyter Notebooks and are using the Neptune extension, your .ipynb code is automatically snapshot whenever you create an experiment. For more information, see Using Jupyter Notebooks in Neptune.

  • Jupyter Notebook checkpoints: You can version any analysis you do in Jupyter Notebooks with the neptune-notebooks extension. The extension also lets you keep track of all your exploratory work by uploading Notebook checkpoints, naming them and adding descriptions for every piece of work you find important.

  • Integrations: We have created convenient integrations with many machine learning frameworks so that you don’t have to implement them from the atomic logging functions mentioned above.