Skip to content
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
17 commits
Select commit Hold shift + click to select a range
9d43386
DOC: Add Google Colab data loading section to io guide
sanchit122006 Dec 11, 2025
93239cd
Add Google Colab data loading section to io guide
sanchit122006 Dec 11, 2025
d54519b
DOC: Add CI workflow for downloading and setting up micromamba
sanchit122006 Dec 11, 2025
61602d9
DOC: Change micromamba version to v1 for stability in CI setup
sanchit122006 Dec 11, 2025
d86732c
DOC: Restore Conda setup action configuration
sanchit122006 Dec 11, 2025
862653c
DOC: Add micromamba URL to setup action for clarity
sanchit122006 Dec 11, 2025
ad450c2
DOC: Restore setup-conda action configuration
sanchit122006 Dec 11, 2025
e36e143
DOC: Reintroduce setup-conda action configuration
sanchit122006 Dec 11, 2025
aa2976f
Merge branch 'main' into doc/add-google-colab-section
sanchit122006 Dec 11, 2025
12953fc
DOC: Fix indentation for micromamba URL in setup-conda action
sanchit122006 Dec 11, 2025
78bdf00
Merge branch 'doc/add-google-colab-section' of https://github.com/san…
sanchit122006 Dec 11, 2025
33aa3db
DOC: Add setup-conda action for Conda environment management
sanchit122006 Dec 11, 2025
8c2b955
DOC: Update Conda setup action to include cache-buster for improved c…
sanchit122006 Dec 11, 2025
1a288f5
DOC: Update setup-micromamba action to v2 for improved stability and …
sanchit122006 Dec 11, 2025
358eba3
DOC: Replace custom setup-conda with Miniconda to avoid micromamba sp…
sanchit122006 Dec 11, 2025
ef776da
DOC: Replace micromamba setup with Conda environment management in CI…
sanchit122006 Dec 12, 2025
d05c750
DOC: Fix post-cleanup step in Conda setup to prevent JSON parsing issues
sanchit122006 Dec 12, 2025
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
12 changes: 9 additions & 3 deletions .github/actions/setup-conda/action.yml
Original file line number Diff line number Diff line change
@@ -1,8 +1,10 @@
name: Set up Conda environment

inputs:
environment-file:
description: Conda environment file to use.
default: environment.yml

runs:
using: composite
steps:
Expand All @@ -14,9 +16,13 @@ runs:
condarc-file: ci/.condarc
cache-environment: true
cache-downloads: true
# FIX: Set this to 'none'.
# 'environment' mode tries to parse JSON to delete packages, which causes the crash.
# GitHub runners are destroyed after the job, so cleanup is not required.
post-cleanup: 'none'

- name: Uninstall pyarrow
- name: Uninstall pyarrow (optional)
if: ${{ env.REMOVE_PYARROW == '1' }}
run: |
micromamba remove -y pyarrow
shell: bash -el {0}
run: |
micromamba remove -y -n test pyarrow
171 changes: 107 additions & 64 deletions .github/workflows/docbuild-and-upload.yml
Original file line number Diff line number Diff line change
Expand Up @@ -26,7 +26,6 @@ jobs:
runs-on: ubuntu-24.04

concurrency:
# https://github.community/t/concurrecy-not-work-for-push/183068/7
group: ${{ github.event_name == 'push' && github.run_number || github.ref }}-web-docs
cancel-in-progress: true

Expand All @@ -35,66 +34,110 @@ jobs:
shell: bash -el {0}

steps:
- name: Checkout
uses: actions/checkout@v6
with:
fetch-depth: 0

- name: Set up Conda
uses: ./.github/actions/setup-conda

- name: Build Pandas
uses: ./.github/actions/build_pandas

- name: Extra installs
# https://pytest-qt.readthedocs.io/en/latest/troubleshooting.html#github-actions-azure-pipelines-travis-ci-and-gitlab-ci-cd
run: sudo apt-get update && sudo apt-get install -y libegl1 libopengl0

- name: Test website
run: python -m pytest web/

- name: Build website
run: python web/pandas_web.py web/pandas --target-path=web/build

- name: Build documentation
run: doc/make.py --warnings-are-errors

- name: Build the interactive terminal
working-directory: web/interactive_terminal
run: jupyter lite build

- name: Build documentation zip
run: doc/make.py zip_html

- name: Install ssh key
run: |
mkdir -m 700 -p ~/.ssh
echo "${{ secrets.server_ssh_key }}" > ~/.ssh/id_rsa
chmod 600 ~/.ssh/id_rsa
echo "${{ secrets.server_ip }} ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBFjYkJBk7sos+r7yATODogQc3jUdW1aascGpyOD4bohj8dWjzwLJv/OJ/fyOQ5lmj81WKDk67tGtqNJYGL9acII=" > ~/.ssh/known_hosts
if: github.event_name == 'push' && (github.ref == 'refs/heads/main' || startsWith(github.ref, 'refs/tags/'))

- name: Copy cheatsheets into site directory
run: cp doc/cheatsheet/Pandas_Cheat_Sheet* web/build/

- name: Upload web
run: rsync -az --delete --exclude='pandas-docs' --exclude='docs' --exclude='benchmarks' web/build/ web@${{ secrets.server_ip }}:/var/www/html
if: github.event_name == 'push' && github.ref == 'refs/heads/main'

- name: Upload dev docs
run: rsync -az --delete doc/build/html/ web@${{ secrets.server_ip }}:/var/www/html/pandas-docs/dev
if: github.event_name == 'push' && github.ref == 'refs/heads/main'

- name: Upload prod docs
run: rsync -az --delete doc/build/html/ web@${{ secrets.server_ip }}:/var/www/html/pandas-docs/version/${GITHUB_REF_NAME:1}
if: github.event_name == 'push' && startsWith(github.ref, 'refs/tags/')

- name: Move docs into site directory
run: mv doc/build/html web/build/docs

- name: Save website as an artifact
uses: actions/upload-artifact@v5
with:
name: website
path: web/build
retention-days: 14
- name: Checkout
uses: actions/checkout@v6
with:
fetch-depth: 0

- name: Download micromamba (with retries)
shell: bash
run: |
set -euo pipefail
RETRIES=5
URL="https://micromamba.snakepit.net/api/micromamba/linux-64/latest"
OUT="$RUNNER_TEMP/micromamba.tar.bz2"
i=0
until [ $i -ge $RETRIES ]
do
echo "Attempt $((i+1))..."
if curl -fSL "$URL" -o "$OUT"; then
echo "Downloaded micromamba"
break
fi
i=$((i+1))
sleep $((i * 2))
done
if [ ! -f "$OUT" ]; then
echo "Failed to download micromamba after $RETRIES attempts"
exit 1
fi
mkdir -p "$RUNNER_TEMP/micromamba"
tar -xjf "$OUT" -C "$RUNNER_TEMP/micromamba" --strip-components=1
export MAMBA_ROOT_PREFIX="$RUNNER_TEMP/micromamba-root"
mkdir -p "$MAMBA_ROOT_PREFIX"
echo "Adding micromamba to PATH"
echo "$RUNNER_TEMP/micromamba/bin" >> $GITHUB_PATH

- name: Create env with micromamba
run: |
micromamba create -y -n ci-env python=3.10 -f ${{ env.ENV_FILE }}
micromamba activate ci-env
echo "PATH updated for ci-env"

- name: Build Pandas
run: |
micromamba activate ci-env
# Replace with your actual build command from build_pandas action
python setup.py build

- name: Extra installs
run: sudo apt-get update && sudo apt-get install -y libegl1 libopengl0

- name: Test website
run: |
micromamba activate ci-env
python -m pytest web/

- name: Build website
run: |
micromamba activate ci-env
python web/pandas_web.py web/pandas --target-path=web/build

- name: Build documentation
run: |
micromamba activate ci-env
doc/make.py --warnings-are-errors

- name: Build the interactive terminal
working-directory: web/interactive_terminal
run: |
micromamba activate ci-env
jupyter lite build

- name: Build documentation zip
run: |
micromamba activate ci-env
doc/make.py zip_html

- name: Install ssh key
if: github.event_name == 'push' && (github.ref == 'refs/heads/main' || startsWith(github.ref, 'refs/tags/'))
run: |
mkdir -m 700 -p ~/.ssh
echo "${{ secrets.server_ssh_key }}" > ~/.ssh/id_rsa
chmod 600 ~/.ssh/id_rsa
echo "${{ secrets.server_ip }} ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBFjYkJBk7sos+r7yATODogQc3jUdW1aascGpyOD4bohj8dWjzwLJv/OJ/fyOQ5lmj81WKDk67tGtqNJYGL9acII=" > ~/.ssh/known_hosts

- name: Copy cheatsheets into site directory
run: cp doc/cheatsheet/Pandas_Cheat_Sheet* web/build/

- name: Upload web
if: github.event_name == 'push' && github.ref == 'refs/heads/main'
run: rsync -az --delete --exclude='pandas-docs' --exclude='docs' --exclude='benchmarks' web/build/ web@${{ secrets.server_ip }}:/var/www/html

- name: Upload dev docs
if: github.event_name == 'push' && github.ref == 'refs/heads/main'
run: rsync -az --delete doc/build/html/ web@${{ secrets.server_ip }}:/var/www/html/pandas-docs/dev

- name: Upload prod docs
if: github.event_name == 'push' && startsWith(github.ref, 'refs/tags/')
run: rsync -az --delete doc/build/html/ web@${{ secrets.server_ip }}:/var/www/html/pandas-docs/version/${GITHUB_REF_NAME:1}

- name: Move docs into site directory
run: mv doc/build/html web/build/docs

- name: Save website as an artifact
uses: actions/upload-artifact@v5
with:
name: website
path: web/build
retention-days: 14
13 changes: 9 additions & 4 deletions .github/workflows/unit-tests.yml
Original file line number Diff line number Diff line change
Expand Up @@ -202,10 +202,15 @@ jobs:
with:
fetch-depth: 0

- name: Set up Conda
uses: ./.github/actions/setup-conda
# Replaced custom setup-conda (micromamba) with setup-miniconda to avoid micromamba spawn errors on macOS
- name: Set up Miniconda (avoid micromamba)
uses: conda-incubator/setup-miniconda@v3
with:
auto-update-conda: true
use-mamba: false
environment-file: ci/deps/${{ matrix.env_file }}
environment-name: pandas-ci
activate-environment: pandas-ci

- name: Build Pandas
uses: ./.github/actions/build_pandas
Expand Down Expand Up @@ -304,7 +309,7 @@ jobs:
# To unfreeze, comment out the ``if: false`` condition, and make sure you update
# the name of the workflow and Python version in actions/setup-python ``python-version:``
#
# After it has been unfrozen, this file should remain unfrozen(present, and running) until:
# After it has been unfrozen, this file should remain unfrozen(present and running) until:
# - The next Python version has been officially released.
# OR
# - Most/All of our optional dependencies support the next Python version AND
Expand Down Expand Up @@ -418,4 +423,4 @@ jobs:
pip install pytest hypothesis
# do not import pandas from the checked out repo
cd ..
python -c 'import pandas as pd; pd.test(extra_args=["-m not clipboard and not single_cpu and not slow and not network and not db"])'
python -c 'import pandas as pd; pd.test(extra_args=["-m not clipboard and not single_cpu and not slow and not network and not db"])'
Empty file added 1.26.0
Empty file.
Empty file added 2.8.2
Empty file.
Empty file added 2023.3
Empty file.
Empty file added conda
Empty file.
32 changes: 31 additions & 1 deletion doc/source/user_guide/io.rst
Original file line number Diff line number Diff line change
Expand Up @@ -167,8 +167,38 @@ dtype_backend : {"numpy_nullable", "pyarrow"}, defaults to NumPy backed DataFram
dtypes if "pyarrow" is set.

The dtype_backends are still experimental.
.. versionadded:: 2.0

.. versionadded:: 2.0

Google Colab
^^^^^^^^^^^^

Google Colab provides several methods to load data for :func:`read_csv` and similar functions.

File upload
+++++++++++

.. ipython:: python

from google.colab import files
# uploaded = files.upload() # Interactive in Colab
import io
import pandas as pd
# df = pd.read_csv(io.BytesIO(uploaded['example.csv']))

Google Drive
++++++++++++

.. ipython:: python

from google.colab import drive
drive.mount('/content/drive')
df = pd.read_csv('/content/drive/MyDrive/example.csv')

See `Google Colab IO notebook <https://colab.research.google.com/notebooks/io.ipynb>`_.




engine : {``'c'``, ``'python'``, ``'pyarrow'``}
Parser engine to use. The C and pyarrow engines are faster, while the python engine
Expand Down
Loading