Skip to content

Conversation

@Abhra0404
Copy link

@Abhra0404 Abhra0404 commented Nov 5, 2025

Description

This PR fixes the wheel building issues for tokenizers and sentencepiece packages by:

  1. Adding specific versions for tokenizers and sentencepiece
  2. Adding necessary build dependencies (wheel, setuptools, ninja)

Changes Made

  • Added tokenizers==0.15.2
  • Added sentencepiece==0.1.99
  • Added build dependencies:
    • wheel
    • setuptools>=65.5.1
    • ninja

Testing Done

  • Verified that the packages can be installed without wheel building errors
  • Tested with a clean environment to ensure reproducibility

License

I confirm that I have the right to license this contribution under the MIT license and agree by submitting this PR that my contributions are licensed under the MIT license.

Code of Conduct

I have read and agree to follow the project's Code of Conduct.

Fixes #393

@kushvinth
Copy link
Contributor

kushvinth commented Dec 31, 2025

Hi @Abhra0404,
Could you please share the testing demos as a video or screenshots for evaluation?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

ERROR: Failed to build installable wheels for some pyproject.toml based projects (tokenizers, sentencepiece)

2 participants