register the package?

[CarloLucibello]

this package is a good match to https://github.com/JuliaGenAI/HuggingFaceDatasets.jl
It would be useful to have it in the general registry.

[murrellb]

We also thought so: https://github.com/JuliaRegistries/General/tree/master/H/HuggingFaceTokenizers

We also set this up: https://github.com/MurrellGroup/HuggingFaceTransformers.jl but haven't registered it yet.

[CarloLucibello]

Ah sorry, I got confused by the Readme. It should be updated (and also the one for jlama)

[murrellb]

Ah thanks good point - done!