Using Neptune with pandas#
Tip
This guide shows how to log pandas DataFrames to Neptune.
For the best display experience, however, we recommend uploading tabular data as CSV:
This method lets you browse the data in interactive table format in the Neptune app. See example →
pandas is a popular open-source data analysis and manipulation tool. With Neptune, you can log and visualize pandas DataFrames.
Before you start#
- Sign up at neptune.ai/register.
- Create a project for storing your metadata.
-
Have pandas and Neptune installed:
Upgrading with neptune-client
already installed
Important: To smoothly upgrade to the 1.0
version of the Neptune client library, first uninstall the neptune-client
library and then install neptune
.
pandas logging example#
-
Import Neptune and start a run:
-
If you haven't set up your credentials, you can log anonymously:
``` py neptune.init_run( api_token=neptune.ANONYMOUS_API_TOKEN, project="common/quickstarts", )
-
-
Create a pandas DataFrame object:
-
Log the DataFrame to Neptune:
-
To stop the connection to Neptune and sync all data, call the
stop()
method: -
To open the run, click the Neptune link in the console output.
Example link: https://app.neptune.ai/o/common/org/showroom/e/SHOW-102/metadata
Result
The resulting dataframe is logged as an HTML object.
You can view it in the All metadata section.
Converting the DataFrame to CSV#
You can save the DataFrame as a CSV and then upload it to Neptune with the upload()
method. This lets you view and sort the data in Neptune's interactive table format.
You can also save the file to a CSV buffer, then upload the streaming buffer using the from_stream()
method: