Skip to content
Draft
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion .github/scripts/build-cpu.sh
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@ declare build_os

set -xeuo pipefail

pip install cmake==3.28.3
uv tool install cmake==3.28.3

if [ "${build_os:0:5}" == macos ] && [ "${build_arch}" == aarch64 ]; then
cmake -DCMAKE_OSX_ARCHITECTURES=arm64 -DCOMPUTE_BACKEND=cpu .
Expand Down
3 changes: 1 addition & 2 deletions .github/scripts/build-cuda.sh
Original file line number Diff line number Diff line change
Expand Up @@ -27,7 +27,7 @@ else
[[ "${cuda_version}" == 13.*.* ]] && build_capability="75;80;86;89;90;100;120"
fi

[[ "${build_os}" = windows-* ]] && python3 -m pip install ninja
uv tool install ninja cmake==3.28.3
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This will install on the host for both Windows and Linux, but on Linux we end up using a container with cmake installed via dnf. We only would need this on Windows, so I think it makes sense to move to line 44 below.


if [ "${build_os:0:6}" == ubuntu ]; then
# We'll use Rocky Linux 8 in order to maintain manylinux 2.24 compatibility.
Expand All @@ -41,7 +41,6 @@ if [ "${build_os:0:6}" == ubuntu ]; then
&& cmake -DCOMPUTE_BACKEND=cuda -DCOMPUTE_CAPABILITY=\"${build_capability}\" . \
&& cmake --build . --config Release"
else
pip install cmake==3.28.3
cmake -G Ninja -DCOMPUTE_BACKEND=cuda -DCOMPUTE_CAPABILITY="${build_capability}" -DCMAKE_BUILD_TYPE=Release -S .
cmake --build . --config Release
fi
Expand Down
2 changes: 1 addition & 1 deletion .github/scripts/build-rocm.sh
Original file line number Diff line number Diff line change
Expand Up @@ -18,7 +18,7 @@ if [ "${build_os:0:6}" == ubuntu ]; then
docker run --rm --platform "linux/$build_arch" -i \
-w /src -v "$PWD:/src" "$image" sh -c \
"apt-get update \
&& pip install cmake==3.31.6 \
&& uv tool install cmake==3.31.6 \
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The ROCm docker images likely won't have uv installed, and that's not covered by a setup-uv action running on the host. I recognize the advantage of using uv but in this case it's probably not worth changing since installing cmake is < 1% of the build runtime.

&& cmake -DCOMPUTE_BACKEND=hip -DCMAKE_BUILD_TYPE=MinSizeRel -DCMAKE_HIP_FLAGS=\"--offload-compress\" -DBNB_ROCM_ARCH=\"${bnb_rocm_arch}\" . \
&& cmake --build ."
fi
Expand Down
13 changes: 9 additions & 4 deletions .github/workflows/lint.yml
Original file line number Diff line number Diff line change
Expand Up @@ -10,10 +10,15 @@ jobs:
Lint:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- uses: actions/setup-python@v4
- uses: actions/checkout@v6
- uses: astral-sh/setup-uv@61cb8a9741eeb8a550a1b8544337180c0fc8476b # v7.2.0
with:
python-version: "3.12"
- uses: pre-commit/action@v3.0.0
enable-cache: true
cache-dependency-glob: '.pre-commit-config.yaml'
- uses: actions/cache@v5
with:
path: ~/.cache/pre-commit
key: pre-commit-3|${{ hashFiles('.pre-commit-config.yaml') }}
- run: uv run --python=3.14 --no-sync --with pre-commit-uv pre-commit run --show-diff-on-failure --color=always --all-files
env:
RUFF_OUTPUT_FORMAT: github
62 changes: 30 additions & 32 deletions .github/workflows/python-package.yml
Original file line number Diff line number Diff line change
Expand Up @@ -42,17 +42,17 @@ jobs:
arch: arm64
runs-on: ${{ matrix.os }}
steps:
- uses: actions/checkout@v4
- uses: actions/checkout@v6
- name: Setup MSVC
if: startsWith(matrix.os, 'windows')
uses: ilammy/msvc-dev-cmd@v1.13.0 # to use cl
uses: ilammy/msvc-dev-cmd@0b201ec74fa43914dc39ae48a89fd1d8cb592756 # v1.13.0, to use cl
- name: Build C++
run: bash .github/scripts/build-cpu.sh
env:
build_os: ${{ matrix.os }}
build_arch: ${{ matrix.arch }}
- name: Upload build artifact
uses: actions/upload-artifact@v4
uses: actions/upload-artifact@v6
with:
name: shared_library_${{ matrix.os }}_${{ matrix.arch }}
path: output/*
Expand All @@ -77,9 +77,9 @@ jobs:
["11.8.0", "12.0.1", "12.1.1", "12.2.2", "12.3.2", "12.4.1", "12.5.1", "12.6.3", "12.8.1", "12.9.1", "13.0.2"]
runs-on: ${{ matrix.os }}
steps:
- uses: actions/checkout@v4
- uses: actions/checkout@v6
# Windows: We install Cuda on the agent (slow)
- uses: Jimver/cuda-toolkit@6008063726ffe3309d1b22e413d9e88fed91a2f2 # v0.2.29
- uses: Jimver/cuda-toolkit@b6fc3a9f3f15256d9d94ffe1254f9c5a2565cde6 # v0.2.30
if: startsWith(matrix.os, 'windows')
id: cuda-toolkit
with:
Expand All @@ -92,15 +92,15 @@ jobs:
log-file-suffix: ${{matrix.os}}-${{matrix.cuda_version}}.txt
- name: Setup MSVC
if: startsWith(matrix.os, 'windows')
uses: ilammy/msvc-dev-cmd@v1.13.0 # to use cl
uses: ilammy/msvc-dev-cmd@0b201ec74fa43914dc39ae48a89fd1d8cb592756 # v1.13.0, to use cl
- name: Build C++
run: bash .github/scripts/build-cuda.sh
env:
build_os: ${{ matrix.os }}
build_arch: ${{ matrix.arch }}
cuda_version: ${{ matrix.cuda_version }}
- name: Upload build artifact
uses: actions/upload-artifact@v4
uses: actions/upload-artifact@v6
with:
name: shared_library_cuda_${{ matrix.os }}_${{ matrix.arch }}_${{ matrix.cuda_version }}
path: output/*
Expand All @@ -112,7 +112,7 @@ jobs:
os: [ubuntu-22.04, windows-2025]
runs-on: ${{ matrix.os }}
steps:
- uses: actions/checkout@v4
- uses: actions/checkout@v6
- name: Build C++ (Linux)
if: runner.os == 'Linux'
run: bash .github/scripts/build-xpu.sh
Expand All @@ -125,7 +125,7 @@ jobs:
env:
build_os: ${{ matrix.os }}
- name: Upload build artifact
uses: actions/upload-artifact@v4
uses: actions/upload-artifact@v6
with:
name: shared_library_xpu_${{ matrix.os }}_x86_64
path: output/*
Expand All @@ -139,7 +139,10 @@ jobs:
rocm_version: ["6.2.4", "6.3.4", "6.4.4", "7.0.2", "7.1"]
runs-on: ${{ matrix.os }}
steps:
- uses: actions/checkout@v4
- uses: actions/checkout@v6
- uses: astral-sh/setup-uv@61cb8a9741eeb8a550a1b8544337180c0fc8476b
with:
python-version: '3.14'
Comment on lines +143 to +145
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

See comment on build-rocm.sh; because we build in a container we shouldn't need to install python on the host.

- name: Clean up disk space
run: |
echo "Disk space before cleanup:"
Expand All @@ -163,7 +166,7 @@ jobs:
build_arch: ${{ matrix.arch }}
rocm_version: ${{ matrix.rocm_version }}
- name: Upload build artifact
uses: actions/upload-artifact@v4
uses: actions/upload-artifact@v6
with:
name: shared_library_rocm_${{ matrix.os }}_${{ matrix.arch }}_${{ matrix.rocm_version }}
path: output/*
Expand Down Expand Up @@ -196,9 +199,9 @@ jobs:
python-version: ["3.10"]
runs-on: ${{ matrix.os }}
steps:
- uses: actions/checkout@v4
- uses: actions/checkout@v6
- name: Download build artifacts
uses: actions/download-artifact@v4
uses: actions/download-artifact@v7
with:
merge-multiple: true
pattern: "shared_library*_${{ matrix.os }}_${{ matrix.arch }}*"
Expand All @@ -209,20 +212,19 @@ jobs:
ls -lR output/
cp output/${{ matrix.os }}/${{ matrix.arch }}/* bitsandbytes/
- name: Set up Python ${{ matrix.python-version }}
uses: actions/setup-python@v5
uses: astral-sh/setup-uv@61cb8a9741eeb8a550a1b8544337180c0fc8476b # v7.2.0
with:
python-version: ${{ matrix.python-version }}
cache: pip
- run: pip install build wheel
- run: python -m build .
enable-cache: 'true'
- run: uv build
- name: Determine and Set Platform Tag, then Tag Wheel
shell: bash
run: |
PLATFORM_TAG=$(python .github/scripts/set_platform_tag.py "${{ matrix.arch }}")
echo "PLATFORM_TAG=$PLATFORM_TAG"
wheel tags --remove --abi-tag=none --python-tag=py3 --platform-tag=$PLATFORM_TAG dist/bitsandbytes-*.whl
uvx wheel tags --remove --abi-tag=none --python-tag=py3 --platform-tag=$PLATFORM_TAG dist/bitsandbytes-*.whl
- name: Upload build artifact
uses: actions/upload-artifact@v4
uses: actions/upload-artifact@v6
with:
name: bdist_wheel_${{ matrix.os }}_${{ matrix.arch }}
path: dist/bitsandbytes-*.whl
Expand All @@ -238,7 +240,7 @@ jobs:
- build-wheels
steps:
- name: Download and rename artifacts
uses: actions/download-artifact@v4
uses: actions/download-artifact@v7
with:
path: tmp/
pattern: "bdist_wheel_*"
Expand Down Expand Up @@ -279,7 +281,7 @@ jobs:
- name: Inspect wheels directory after renaming files
run: ls -alFR wheels/

- uses: actions/checkout@v4
- uses: actions/checkout@v6
with:
path: repo

Expand Down Expand Up @@ -348,7 +350,7 @@ jobs:
cat body.md

- name: Create new pre-release and upload artifacts
uses: softprops/action-gh-release@v2.2.1
uses: softprops/action-gh-release@a06a81a03ee405af7f2048a818ed3f03bbf83c7b # v2.5.0
with:
files: wheels/*.whl
prerelease: true
Expand All @@ -372,18 +374,14 @@ jobs:
env:
PIP_DISABLE_PIP_VERSION_CHECK: 1
steps:
- uses: actions/checkout@v4
- uses: actions/checkout@v6
- name: Download wheel
uses: actions/download-artifact@v4
uses: actions/download-artifact@v7
with:
name: bdist_wheel_${{ matrix.os }}_${{ matrix.arch }}
path: wheels/
- name: Set up Python
uses: actions/setup-python@v5
with:
python-version: "3.12"
- run: pip install auditwheel
- run: python ./.github/scripts/auditwheel_show.py wheels/* | tee $GITHUB_STEP_SUMMARY
- uses: astral-sh/setup-uv@61cb8a9741eeb8a550a1b8544337180c0fc8476b # v7.2.0
- run: uv run --no-sync --with=auditwheel ./.github/scripts/auditwheel_show.py wheels/* | tee $GITHUB_STEP_SUMMARY

publish-wheels:
name: Publish wheels to PyPI
Expand All @@ -399,13 +397,13 @@ jobs:
id-token: write
steps:
- name: Download distribution artifacts
uses: actions/download-artifact@v4
uses: actions/download-artifact@v7
with:
path: dist/
pattern: "bdist_wheel_*"
merge-multiple: true

- name: Publish to PyPI
uses: pypa/gh-action-pypi-publish@release/v1
uses: pypa/gh-action-pypi-publish@ed0c53931b1dc9bd32cbe73a98c7f6766f8a527e # v1.13.0
with:
print-hash: true
31 changes: 15 additions & 16 deletions .github/workflows/test-runner.yml
Original file line number Diff line number Diff line change
Expand Up @@ -130,7 +130,7 @@ jobs:
echo "build_os=${BUILD_OS}" >> $GITHUB_OUTPUT
echo "arch=${ARCH}" >> $GITHUB_OUTPUT

- uses: actions/checkout@v4
- uses: actions/checkout@v6

- name: Set build environment variables
shell: bash
Expand All @@ -141,7 +141,7 @@ jobs:
# Windows + CUDA: Install CUDA Toolkit
- name: Install CUDA Toolkit
if: inputs.backend == 'cuda' && inputs.platform == 'windows'
uses: Jimver/cuda-toolkit@6008063726ffe3309d1b22e413d9e88fed91a2f2 # v0.2.29
uses: Jimver/cuda-toolkit@b6fc3a9f3f15256d9d94ffe1254f9c5a2565cde6 # v0.2.30
with:
cuda: ${{ inputs.cuda_version }}
method: "network"
Expand All @@ -151,7 +151,7 @@ jobs:
# Windows: Setup MSVC (needed for both CPU and CUDA builds)
- name: Setup MSVC
if: inputs.platform == 'windows'
uses: ilammy/msvc-dev-cmd@v1.13.0
uses: ilammy/msvc-dev-cmd@0b201ec74fa43914dc39ae48a89fd1d8cb592756 # v1.13.0

# Build CPU backend
- name: Build C++
Expand All @@ -167,7 +167,7 @@ jobs:
cuda_targets: "75;89"

- name: Upload build artifact
uses: actions/upload-artifact@v4
uses: actions/upload-artifact@v6
with:
name: ${{ steps.config.outputs.artifact_name }}
path: output/${{ steps.config.outputs.build_os }}/${{ steps.config.outputs.arch }}/*
Expand All @@ -184,42 +184,41 @@ jobs:
if: inputs.backend == 'cuda'
run: nvidia-smi

- uses: actions/checkout@v4
- uses: actions/checkout@v6

- name: Download build artifact
uses: actions/download-artifact@v4
uses: actions/download-artifact@v7
with:
name: ${{ needs.build.outputs.artifact_name }}
path: bitsandbytes/
merge-multiple: true

- name: Setup Python
uses: actions/setup-python@v5
- uses: astral-sh/setup-uv@61cb8a9741eeb8a550a1b8544337180c0fc8476b
with:
python-version: '3.10'
activate-environment: 'true'

# Windows: Setup MSVC for torch.compile
- name: Setup MSVC
if: inputs.platform == 'windows'
uses: ilammy/msvc-dev-cmd@v1.13.0
uses: ilammy/msvc-dev-cmd@0b201ec74fa43914dc39ae48a89fd1d8cb592756 # v1.13.0

- name: Install dependencies
run: |
pip install torch==${{ inputs.torch_version }} --index-url ${{ inputs.pypi_index }}
pip install -e ".[test]" -v
pip install pytest-cov
uv pip install torch==${{ inputs.torch_version }} --index-url ${{ inputs.pypi_index }}
uv pip install pytest-cov -e ".[test]"

# Windows: Downgrade NumPy for torch<2.4.1 compatibility
# See: https://github.com/pytorch/pytorch/issues/131668
- name: Downgrade NumPy
if: inputs.platform == 'windows' && startsWith(inputs.torch_version, '2.3.')
run: pip install "numpy<2"
run: uv pip install "numpy<2"

- name: Show installed packages
run: pip list
run: uv pip list

- name: Show environment information
run: python -m torch.utils.collect_env
run: uv run -m torch.utils.collect_env

- name: Run tests
run: pytest --durations=100
run: uv run pytest --durations=100
10 changes: 5 additions & 5 deletions .pre-commit-config.yaml
Original file line number Diff line number Diff line change
@@ -1,13 +1,13 @@
repos:
- repo: https://github.com/astral-sh/ruff-pre-commit
rev: v0.14.3
rev: 6ce4b4fc349b8881cbce40207dac3bf78824a88e # frozen: v0.14.11
hooks:
- id: ruff
- id: ruff-check
args:
- --fix
- id: ruff-format
- repo: https://github.com/pre-commit/pre-commit-hooks
rev: v5.0.0
rev: 3e8a8703264a2f4a69428a0aa4dcb512790b2c8c # frozen: v6.0.0
hooks:
- id: check-merge-conflict
- id: check-yaml
Expand All @@ -19,11 +19,11 @@ repos:
- --fix=lf
exclude: '\.bat$'
- repo: https://github.com/crate-ci/typos
rev: v1.26.0
rev: bb4666ad77b539a6b4ce4eda7ebb6de553704021 # frozen: v1.42.0
hooks:
- id: typos
- repo: https://github.com/pre-commit/mirrors-clang-format
rev: v20.1.6
rev: 75ca4ad908dc4a99f57921f29b7e6c1521e10b26 # frozen: v21.1.8
hooks:
- id: clang-format
types_or: [c++, c, cuda]
Expand Down
8 changes: 4 additions & 4 deletions bitsandbytes/functional.py
Original file line number Diff line number Diff line change
Expand Up @@ -241,13 +241,13 @@ def create_fp8_map(signed=True, exponent_bits=5, precision_bits=2, total_bits=8)
has_sign = 1 if signed else 0
assert e + p == total_bits - has_sign
# the exponent is biased to 2^(e-1) -1 == 0
evalues = []
e_values = []
for i, val in enumerate(range(-(2 ** (exponent_bits - has_sign)), 2 ** (exponent_bits - has_sign), 1)):
evalues.append(2**val)
e_values.append(2**val)

values = []
lst = list(itertools.product([0, 1], repeat=precision_bits))
# for ev in evalues:
# for ev in e_values:
bias = 2 ** (exponent_bits - 1)
for evalue in range(2 ** (exponent_bits)):
for bit_pattern in lst:
Expand Down Expand Up @@ -1501,7 +1501,7 @@ def check_matmul(A, B, out, transposed_A, transposed_B, expected_type=torch.int8

if not correct:
raise ValueError(
f"Tensor dimensions incorrect for matrix mulitiplication: A x B: {sA} x {sB} with transpose for A x B: {tA} x {tB}.",
f"Tensor dimensions incorrect for matrix multiplication: A x B: {sA} x {sB} with transpose for A x B: {tA} x {tB}.",
)

return sout
Expand Down
Loading
Loading