How to use Neptune with pandas#
pandas is a popular open source data analysis and manipulation tool. With Neptune, you can log and visualize pandas DataFrames.
Before you start#
- Sign up at neptune.ai/register.
- Create a project for storing your metadata.
Have pandas and Neptune installed:
neptune-client already installed
Important: To smoothly upgrade to the
1.0 version of the Neptune client library, first uninstall the
neptune-client library and then install
pandas logging example#
Import Neptune and start a run:
If you haven't set up your credentials, you can log anonymously:
Create a pandas DataFrame object:
Upload the DataFrame to Neptune as HTML#
Upload the DataFrame to Neptune as CSV#
You can save the DataFrame as a CSV and then upload it to Neptune with the
upload() method. This lets you view and sort the data in Neptune's interactive table format.
If you want to avoid writing to disk, you can write the dataframe to an in-memory object, then upload it using the
(Optional) Log pandas profile report to Neptune#
You can log your dataset's Exploratory Data Analysis (EDA) report to Neptune, utilizing libraries that support pandas such as ydata-profiling .
View the DataFrame in Neptune#
To stop the connection to Neptune and sync all data, call the
Click the Neptune link in the console output to open the run.
The resulting dataframe is logged as an HTML or a CSV object.
You can view it in the All metadata section or create a custom dashboard and display the dataframe as a widget.