Instructions to use EvoLenTokenizer/base-100k with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Transformers
How to use EvoLenTokenizer/base-100k with Transformers:
# Use a pipeline as a high-level helper from transformers import pipeline pipe = pipeline("feature-extraction", model="EvoLenTokenizer/base-100k")# pip install -U transformers accelerate # Load model directly from transformers import AutoTokenizer, AutoModelForMaskedLM tokenizer = AutoTokenizer.from_pretrained("EvoLenTokenizer/base-100k") model = AutoModelForMaskedLM.from_pretrained("EvoLenTokenizer/base-100k", device_map="auto") - Notebooks
- Google Colab
- Kaggle
Download training_args.bin from EvoLenTokenizer/base-100k: direct link, hf CLI and curl.
- Browser
- Download file 5.07 kB
-
https://huggingface.co/EvoLenTokenizer/base-100k/resolve/main/training_args.bin
- Command line
-
hf download hf://EvoLenTokenizer/base-100k/training_args.bin
-
curl -L -o training_args.bin https://huggingface.co/EvoLenTokenizer/base-100k/resolve/main/training_args.bin
5.07 kB
- Xet hash:
- f21aaac8c294b31f4201a69df24d5ffb80484e8700751eee81fb00b9e185a4e0
- Size of remote file:
- 5.07 kB
- SHA256:
- 19a31779de64aed3cb2fe1b8aea0af1a6418068637cfa767c4c8c3f0b98980d2
·
Xet efficiently stores Large Files inside Git, intelligently splitting files into unique chunks and accelerating uploads and downloads. More info.