The version of huggingface_hub in pixi.toml needs to be updated

[Gibyeng]

Bug description

When I import transformers in compare-bpe-tiktoken.ipynb of ch02, I find it leads to an error due to its ealry version. The error is gone after set the right version of huggingface_hub.

[feature.tests.dependencies]
huggingface_hub = ">=0.30.0,<1.0"

What operating system are you using?

None

Where do you run your code?

None

Environment

[d-kleine]

Hi @Gibyeng,

Thank you for reporting this issue. Unfortunately, @rasbt is currently recovering from an injury, but he’ll review your report as soon as he’s feeling better. In the meantime, I’ve started looking into your issue.

Could you please provide the huggingface_hub version you have used and the exact error message you encountered? That information will be super helpful for troubleshooting.

Tagging also @matthewfeickert – perhaps you might be able to help with this issue with the pixi env?

[matthewfeickert]

@d-kleine Sure. I'm in transit at the moment, but once I arrive later today I can look at this in more detail.

@Gibyeng a minimal reproducer of the exact commands and platform that you're using from a fresh clone would be useful.

[rasbt]

Thanks for the note. Looks like we manually patched the HF hub version in the workflow file. I think just using that version in the pixi.toml will do the trick. Will do that via #661

[matthewfeickert]

Thanks for getting to this before I could @rasbt.

When I import transformers in compare-bpe-tiktoken.ipynb of ch02

If this is actually needed maybe we should move it out of the test feature?