Skip to content

Releases: bitsandbytes-foundation/bitsandbytes

Release list

Latest `main` wheel

Latest `main` wheel Pre-release
Pre-release

Choose a tag to compare

View all tags
@github-actions github-actions released this 29 Jul 18:27
continuous-release_main
a2b90e6
This commit was created on GitHub.com and signed with GitHub’s verified signature.
GPG key ID: B5690EEEBB952194
Verified
Learn about vigilant mode.

Latest main pre-release wheel

This pre-release contains the latest development wheels for all supported platforms, rebuilt automatically on every commit to the main branch.

How to install:
Pick the correct command for your platform and run it in your terminal:

macOS 14+ (arm64)

pip install --force-reinstall https://github.com/bitsandbytes-foundation/bitsandbytes/releases/download/continuous-release_main/bitsandbytes-1.33.7.preview-py3-none-macosx_14_0_arm64.whl

Linux (aarch64)

pip install --force-reinstall https://github.com/bitsandbytes-foundation/bitsandbytes/releases/download/continuous-release_main/bitsandbytes-1.33.7.preview-py3-none-manylinux_2_24_aarch64.whl

Linux (x86_64)

pip install --force-reinstall https://github.com/bitsandbytes-foundation/bitsandbytes/releases/download/continuous-release_main/bitsandbytes-1.33.7.preview-py3-none-manylinux_2_24_x86_64.whl

Windows (x86_64)

pip install --force-reinstall https://github.com/bitsandbytes-foundation/bitsandbytes/releases/download/continuous-release_main/bitsandbytes-1.33.7.preview-py3-none-win_amd64.whl

Windows (arm64)

pip install --force-reinstall https://github.com/bitsandbytes-foundation/bitsandbytes/releases/download/continuous-release_main/bitsandbytes-1.33.7.preview-py3-none-win_arm64.whl

Custom PyTorch builds (Intel XPU, ROCm, etc.):
The --force-reinstall flag causes pip to re-resolve all dependencies from PyPI, which may replace your custom PyTorch build with the default CUDA variant. To avoid this, add --no-deps:

pip install --force-reinstall --no-deps https://github.com/bitsandbytes-foundation/bitsandbytes/releases/download/continuous-release_main/bitsandbytes-1.33.7.preview-py3-none-manylinux_2_24_x86_64.whl

Note:
These wheels are updated automatically with every commit to main and become available as soon as the python-package.yml workflow finishes.

The version number is replaced with 1.33.7-preview in order to keep the link stable, this however does not affect the installed version at all:

> pip install https://.../bitsandbytes-1.33.7-preview-py3-none-manylinux_2_24_x86_64.whl
Collecting bitsandbytes==1.33.7rc0
...
Successfully installed bitsandbytes-0.49.0.dev0
Loading

0.50.0: Faster 4-bit inference, CPU/ROCm/Apple Silicon improvements, and more!

0.50.0: Faster 4-bit inference, CPU/ROCm/Apple Silicon improvements, and more! Latest
Latest

Choose a tag to compare

View all tags
@matthewdouglas matthewdouglas released this 25 Jul 01:42
0.50.0
c5d481b

This release brings a new fused 4-bit GEMM for inference on CUDA and ROCm, faster CPU ops on x86-64 and ARM64, reduced host-side overhead, and a much improved Apple Silicon backend. We've also added Windows on ARM CPU support, ROCm builds for Windows, additional ROCm and CUDA build variants, and new optimizer support on CPU and Intel XPU.

Highlights

🚀 New 4-bit GEMM kernels for inference (CUDA)

New fused 4-bit dequantize + GEMM kernels replace the old GEMV and dequantize + F.linear paths for small-to-medium batch sizes. 4-bit inference is up to 4x faster at batch sizes of 2 through 64 across Turing through Blackwell, with wins at batch size 1 in many cases too. Nested (double) quantization and bias are fused in as well, so nested quant sees an additional benefit. Kernel selection happens automatically at runtime based on shape, GPU architecture, and SM count. See #1949 for benchmarks and details.

🚀 Fused 4-bit SIMT GEMM on ROCm

The SIMT version of the new 4-bit GEMM has been ported to ROCm and wired into the same dispatch for small inference batches, validated on gfx1100, gfx1201, and gfx1151 (#1979).

AMD ROCm improvements

Stability and performance improvements bring AMD ROCm support out of preview; it is now considered stable.

  • ROCm wheels for Windows are now published for ROCm 7.2 and 7.14 (#1915, #2007).
  • Added ROCm 7.14 and dropped the ROCm 6.2 and 6.3 builds (#2007).
  • Expanded GPU target coverage to add RDNA2 consumer GPUs, additional RDNA3 and RDNA3.5 GPUs, and CDNA1 (gfx908) data center GPUs (#1886, #2007).

Apple Silicon (MPS)

Apple Silicon support is improved. The MPS backend added optimized Metal kernels from the Hub (#1875) and was substantially improved (#1960, #1983, #1994) so that all 4-bit and LLM.int8() configurations now work on MPS. On macOS 26+, install the kernels package for the best performance, which enables the optimized Metal kernels; otherwise a naive fallback is used. The MPS path requires torch >= 2.9.

The last remaining feature for support parity is the 8bit optimizers, which will land in a future release.

CPU performance on x86-64 and ARM64

Blockwise quantization and dequantization on CPU are considerably faster, mostly from better SIMD usage plus some compile flag tuning. Improvements range from 1.1x to over 20x depending on op, dtype, and hardware, with the largest gains on fp16 and on x86-64 CPUs without AVX-512. The LLM.int8() matmul on CPU was improved as well. See #1968 for benchmarks.

Reduced host-side CPU overhead

Reduced Python dispatch overhead, especially on the CUDA/ROCm backend (#1953).

Windows on ARM CPU support

Windows ARM64 CPU wheels are now built with NEON-optimized kernels (#1959), with nightly test coverage added (#1962).

New optimizer support

  • CPU optimizers are now supported for both 32-bit and 8-bit blockwise (#1901).
  • Intel XPU gained paged optimizer support (#1898) and support for the LARS and LAMB optimizers (#1902).
  • Fixed Lion to use decoupled weight decay (#1993, #2001).
  • Other optimizer fixes (#1869, #1998, #1855).

NVIDIA CUDA

  • Added CUDA 13.2 build wheels and nightly test coverage (#1925).
  • Smaller wheels: reduced the CUDA build matrix and added a closest-version fallback for library loading (#1980).

Intel XPU

  • Added a oneAPI 2026 build; wheels now package both the 2025 and 2026 variants and pick the right one at runtime based on the PyTorch XPU version (#2002).
  • Fixed gemv_4bit bf16 correctness on Intel Arc A-series (Alchemist) GPUs (#1942).

🚨 Breaking Changes

  • Minimum PyTorch version is now 2.4 (#1926).
  • Removed deprecated APIs: the research module, non-blockwise (block_wise=False) optimizers, and legacy dynamic quantization functions, along with their CUDA/HIP kernels (#1871, #1880).
  • Removed legacy sparse functionality (spmm_coo, spmm_coo_very_sparse) and dropped the cusparse/hipsparse dependencies (#1881).

Deprecations

  • igemm, batched_igemm, and check_matmul are deprecated and now emit warnings (#2003).
  • Passing 4-bit weights quantized in transposed [in_features, out_features] orientation to matmul_4bit now emits a DeprecationWarning. Support is likely to be removed in the future. This is not a typical use case (#1949).

Other improvements and bug fixes

This release also includes a number of other improvements, bug fixes, and documentation updates. See the full changelog below.

What's Changed

Read more

Contributors

scw, eaglstun, and 23 other contributors
Assets 2
Loading

0.49.2

0.49.2

Choose a tag to compare

View all tags
@matthewdouglas matthewdouglas released this 16 Feb 21:29
0.49.2
f0e6ca3

Highlights

  • The default blocksize of 64 for 4bit quantization is now supported on ROCm. Previously the default was 128, which was a mismatch from the default for other devices.
  • ROCm 7.2 build is now included.

What's Changed

New Contributors

Full Changelog: 0.49.1...0.49.2

Contributors

TimDettmers, matthewdouglas, and 4 other contributors
Loading

0.49.1

0.49.1

Choose a tag to compare

View all tags
@matthewdouglas matthewdouglas released this 08 Jan 14:35
0.49.1
d87b7ec

What's Changed

Full Changelog: 0.49.0...0.49.1

Contributors

winglian and sstamenk
Loading

0.49.0

0.49.0

Choose a tag to compare

View all tags
@matthewdouglas matthewdouglas released this 11 Dec 20:51
0.49.0
26f0c7a

Highlights

x86-64 CPU Improvements

CPU performance for 4bit is significantly improved on x86-64, with optimized kernel paths for CPUs that have AVX512 or AVX512BF16 support.

AMD ROCm Experimental Wheels

  • Experimental support for AMD devices is now included in our PyPI wheels on Linux x86-64.
  • We've added additional GPU target devices as outlined in our docs.
  • Support for using the default blocksize of 64 for 4bit was added for RDNA GPUs in #1748.

macOS 14+ Wheels

  • We're now publishing wheels for macOS 14+!
  • The 4bit and 8bit quantization features are supported on MPS by slow implementations. We plan to enable Metal kernels with improved performance in the future.

🚨 Breaking Changes

  • Dropped support for Python 3.9.
  • Dropped compilation support for Maxwell GPUs in the CUDA backend.

What's Changed

New Contributors

Full Changelog: 0.48.2...0.49.0

Contributors

mfuntowicz, matthewdouglas, and 4 other contributors
Loading

0.48.2

0.48.2

Choose a tag to compare

View all tags
@matthewdouglas matthewdouglas released this 29 Oct 21:48
0.48.2
b48ecdb

What's Changed

Full Changelog: 0.48.1...0.48.2

Contributors

matthewdouglas
Loading

0.48.1

0.48.1

Choose a tag to compare

View all tags
@matthewdouglas matthewdouglas released this 02 Oct 17:47
0.48.1
7e16503

This release fixes a regression introduced in 0.48.0 related to LLM.int8(). This issue caused poor inference results with pre-quantized checkpoints in HF transformers.

What's Changed

Full Changelog: 0.48.0...0.48.1

Contributors

ccoulombe and matthewdouglas
Loading

0.48.0: Intel GPU & Gaudi support, CUDA 13, performance improvements, and more!

0.48.0: Intel GPU & Gaudi support, CUDA 13, performance improvements, and more!

Choose a tag to compare

View all tags
@matthewdouglas matthewdouglas released this 30 Sep 21:48
0.48.0
406fb6a

Highlights

🎉 Intel GPU Support

We now officially support Intel GPUs on Linux and Windows! Support is included for all major features (LLM.int8(), QLoRA, 8bit optimizers) with the exception of the paged optimizer feature.

This support includes the following hardware:

  • Intel® Arc™ B-Series Graphics
  • Intel® Arc™ A-Series Graphics
  • Intel® Data Center GPU Max Series

A compatible PyTorch version with Intel XPU support is required. The current minimum is PyTorch 2.6.0. It is recommended to use the latest stable release. See Getting Started on Intel GPU for guidance.

🎉 Intel Gaudi Support

We now officially support Intel Gaudi2 and Gaudi3 accelerators. This support includes LLM.int8() and QLoRA with the NF4 data type. At this time optimizers are not implemented.

A compatible PyTorch version with Intel Gaudi support is required. The current minimum is Gaudi v1.21 with PyTorch 2.6.0. It is recommended to use the latest stable release. See the Gaudi software installation guide for guidance.

NVIDIA CUDA

  • The 4bit dequantization kernel was improved by @Mhmd-Hisham in #1746. This change brings noticeable speed improvements for prefill, batch token generation, and training. The improvement is particularly prominent on A100, H100, and B200.
  • We've added CUDA 13.0 compatibility across Linux x86-64, Linux aarch64, and Windows x86-64 platforms.
    • Hardware support for CUDA 13.0 is limited to Turing generation and newer.
    • Support for Thor (SM110) is available in the Linux aarch64 build.

🚨 Breaking Changes

  • Dropped support for PyTorch 2.2. The new minimum requirement is 2.3.0.
  • Removed Maxwell GPU support for all CUDA builds.

What's Changed

New Contributors

Full Changelog: 0.47.0...0.48.0

Contributors

jasl, Egor-Krivov, and 9 other contributors
Loading

0.47.0

0.47.0

Choose a tag to compare

View all tags
@matthewdouglas matthewdouglas released this 11 Aug 18:59
0.47.0
c0dcdf2

Highlights:

  • FSDP2 compatibility for Params4bit (#1719)
  • Bugfix for 4bit quantization with large block sizes (#1721)
  • Further removal of previously deprecated code (#1669)
  • Improved CPU coverage (#1628)
  • Include NVIDIA Volta support in CUDA 12.8 and 12.9 builds (#1715)

What's Changed

New Contributors

Full Changelog: 0.46.0...0.47.0

Contributors

mgorny, Egor-Krivov, and 8 other contributors
Loading

0.46.1

0.46.1

Choose a tag to compare

View all tags
@matthewdouglas matthewdouglas released this 02 Jul 19:45
0.46.1
4bca844

What's Changed

New Contributors

Full Changelog: 0.46.0...0.46.1

Contributors

mgorny, mklabunde, and 3 other contributors
Loading