Using Neptune with pandas#
This guide shows how to log pandas DataFrames to Neptune.
For the best display experience, however, we recommend uploading tabular data as CSV:
This method lets you browse the data in interactive table format in the Neptune app. See example →
pandas is a popular open-source data analysis and manipulation tool. With Neptune, you can log and visualize pandas DataFrames.
Before you start#
- Sign up at neptune.ai/register.
- Create a project for storing your metadata.
Have pandas and Neptune installed:
neptune-client already installed
Important: To smoothly upgrade to the
1.0 version of the Neptune client library, first uninstall the
neptune-client library and then install
pandas logging example#
Import Neptune and start a run:
If you haven't set up your credentials, you can log anonymously:
``` py neptune.init_run( api_token=neptune.ANONYMOUS_API_TOKEN, project="common/quickstarts", )
Create a pandas DataFrame object:
Log the DataFrame to Neptune:
To stop the connection to Neptune and sync all data, call the
To open the run, click the Neptune link in the console output.
Example link: https://app.neptune.ai/o/common/org/showroom/e/SHOW-102/metadata
The resulting dataframe is logged as an HTML object.
You can view it in the All metadata section.
Converting the DataFrame to CSV#
You can save the DataFrame as a CSV and then upload it to Neptune with the
upload() method. This lets you view and sort the data in Neptune's interactive table format.
You can also save the file to a CSV buffer, then upload the streaming buffer using the
from io import StringIO from neptune.types import File csv_buffer = StringIO() df.to_csv(csv_buffer, index=False) run["df_as_csv_from_buffer"].upload(File.from_stream(csv_buffer, extension="csv"))