# tokenizers.js

this code fork from `https://github.com/huggingface/transformers`

# What changes?
- remove onnx dependencies
- remove env dependencies (esm/hf_repo_downloader)

# License
Apache-2.0