Instructions to use litert-community/LFM2.5-VL-3B with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- LiteRT-LM
How to use litert-community/LFM2.5-VL-3B with LiteRT-LM:
# LiteRT-LM runs on various platforms (Android, iOS, Windows, Linux, macOS, IoT, Web/WASM) # and supports many APIs (C++, Python, Kotlin, Swift, JavaScript, Flutter). # For platform-specific integration guides, please refer to the official developer website: # https://ai.google.dev/edge/litert-lm # To try LiteRT-LM, the easiest way is to use our CLI tool. # 1. Install the LiteRT-LM CLI tool: pip install -U litert-lm # 2. Download and run this model locally: # See: https://ai.google.dev/edge/litert-lm/cli # A single .litertlm file in the repo is picked automatically; otherwise the CLI asks which one to run # (or pass its name right after the repo id). litert-lm run \ --from-huggingface-repo=litert-community/LFM2.5-VL-3B \ --prompt="Write me a poem"
- LiteRT
How to use litert-community/LFM2.5-VL-3B with LiteRT:
# No code snippets available yet for this library. # To use this model, check the repository files and the library's documentation. # Want to help? PRs adding snippets are welcome at: # https://github.com/huggingface/huggingface.js
- Notebooks
- Google Colab
- Kaggle
LiteRT is Google's on-device runtime, the new name for TensorFlow Lite (Android: com.google.ai.edge.litert:litert), and litert-torch, the renamed ai-edge-torch, is its PyTorch converter: a PyTorch model converted unmodified with litert_torch.convert matched the original to 4e-7 on a Galaxy S26 (measured, LiteRT 2.2.0, Android 16, 2026-09-05).
Measured on device (edge-compat): Galaxy S26 · LiteRT-LM 0.16.0 · GPU · decode 20.1 tok/s · prefill 744 tok/s · TTFT 330 ms · all 937 ops delegated (2026-08-24); Galaxy S26 · LiteRT-LM 0.16.0 · CPU · decode 19.9 tok/s · prefill 277 tok/s · TTFT 790 ms (2026-09-05). Record: https://github.com/john-rocky/edge-compat/blob/main/cards/lfm2.5-vl-3b-int8/CARD.md
LFM2.5-VL-3B — LiteRT-LM
LiquidAI/LFM2.5-VL-3B converted to the LiteRT-LM (.litertlm) format for on-device inference with Google's LiteRT-LM runtime.
Text + image work end-to-end on the released litert-lm pip runtime (earliest verified runtime: 0.16.0 for LFM2.5-VL-3B_int8.litertlm, 0.19.0 for LFM2.5-VL-3B_int4.litertlm) — the file carries the vision encoder, the vision adapter and the LFM2 image-placeholder metadata, so litert-lm run … --attachment photo.png just works. To our knowledge this is the first LFM2.5-VL in LiteRT form.
Known issue: positional answers are wrong on the released runtime
On litert-lm 0.16.0 — and on current main — only the top quarter of the image reaches the model. Captioning, colour questions and OCR of a large dominant subject still work. Anything positional — locate, count, enumerate, "which one is at the bottom" — comes back wrong, with no error and well-formed output.
The cause is a shrink-factor assumption in the runtime's vision path, reported upstream as LiteRT-LM#3246. It affects every LFM2.5-VL file, not just this one: the family performs its 2×2 pixel-unshuffle in the vision adapter rather than the encoder, and the runtime assumes the opposite.
Quick check — a 512×512 image with 16 numbered horizontal bands, asked to list every number from top to bottom:
expected: 1 … 16 actual: 1 1 2 2 3 3 4 4 1
A repaired int4 build is in this repo: LFM2.5-VL-3B_int4_fixB.litertlm — the stock file with its two vision graphs re-exported so the 2×2 pixel-unshuffle happens inside the encoder (text weights, tokenizer and metadata unchanged; scripts: reexport_vision_unshuffle.py + repack_vision.py). On the ruler above it returns all 16 bands on litert-lm 0.16.1 and 0.17.0 (each number repeated, since the fixture prints it three times per band; measured 2026-09-16), where the stock file still stops at 4. On the 10-target phone screenshot from LiteRT-LM#3246 the same file scores 10/10 against the stock file's 2/10. The stock files stay published in case the runtime fix lands and makes them the right choice again.
LFM2.5-VL-3B pairs Liquid AI's hybrid text backbone (22 gated short-convolution blocks + 8 grouped-query attention layers, 128k vocab) with a SigLIP2 vision tower (27 layers, hidden 1152) and a pixel-unshuffle projector. On this runtime an image is processed at 512×512 into 256 soft tokens (single image per prompt; the runtime resizes for you) — but see the known issue above: on 0.16.0 those 256 tokens carry only the top quarter of the picture.
| File | Recipe | Size |
|---|---|---|
LFM2.5-VL-3B_int8.litertlm |
int8 dynamic (text linears + convs + embedding, vision tower) | 3.55 GB |
LFM2.5-VL-3B_int4.litertlm |
text int4 blockwise-32 OCTAV linears, int8 embedding + lm_head; vision tower int8 | 2.35 GB |
LFM2.5-VL-3B_int4_fixB.litertlm |
int4 as above, vision graphs re-exported with the pooling inside the encoder (full image visible on 0.16.x/0.17.0) | 2.35 GB |
| Context (KV cache) | 4096 max |
| Image input | 1 per prompt, resized to 512×512 → 256 tokens; PNG/JPEG via --attachment . On 0.16.0 those tokens cover only the top quarter — see the known issue |
| Backend | CPU and GPU (macOS verified by generation; Android OpenCL verified by generation on a Pixel 8a and a Galaxy S26 — device numbers, and the files they were measured on, below; iOS Metal fails at engine creation for this family, tracked upstream in LiteRT-LM#3129 — use CPU on iOS). Earliest verified runtime: litert-lm 0.16.0 for LFM2.5-VL-3B_int8.litertlm, 0.19.0 for LFM2.5-VL-3B_int4.litertlm. For the Android GPU path of LFM2.5-VL-3B_int4.litertlm and LFM2.5-VL-3B_int4_fixB.litertlm, see Known runtime issues |
| Template | in the file — ChatML-style; image placeholders are inserted by the runtime's LFM2 data processor (non-thinking model) |
| Base model | LiquidAI/LFM2.5-VL-3B (LFM Open License v1.0) |
Quality
Sanity checks on the 0.16.0 pip CLI (greedy, fresh engine per question, --cache no), the int4-b32 rows on an earlier int4 file of this repo. Vision: five deterministic synthetic fixtures (dominant color, large-text OCR, shape, counting three squares, largest word). Text: the 8-question check used across our LiteRT conversions.
| Configuration | text (8 questions) | image (5 questions) |
|---|---|---|
| PyTorch bf16 (reference) | 7/8 | 5/5 |
| LiteRT int8, CPU | 7/8 | 5/5 |
| LiteRT int8, GPU (macOS) | 7/8 | 5/5 |
| LiteRT int4-b32, CPU | 7/8 | 5/5 |
| LiteRT int4-b32, GPU (macOS) | 7/8 | 5/5 |
All five image answers from both LiteRT variants are verbatim identical to the bf16 reference on both backends ("Red." / "Hello." / "Circle." / "Three." / "CAT."). The single text miss (rhyme completion answered "Purple.") is shared with the source model's behavior at greedy decoding, not a conversion artifact. Zero degenerate outputs across all runs.
What the image check does not show. All five vision fixtures place their subject in the middle of the frame, below the cut described in the known issue above. They therefore cannot separate "the model resolved the image" from "the model answered from priors" — a file that sees only the top quarter still scores 5/5 on them. The band ruler above is the check that does separate the two. Concretely: circle.png puts its circle at y 128–384 of 512, entirely below the cut, and the 3B still answers "Circle." The 450M answers "Square." on the same fixture and flips to "Circle." once the file is repaired, with nothing else changed.
Usage
pip install litert-lm
litert-lm run ./LFM2.5-VL-3B_int4.litertlm --prompt "Describe this image." --attachment photo.png
Text-only prompts work the same way without --attachment. --vision-backend cpu|gpu selects the vision encoder backend independently of the text backend. With LiteRT-LM 0.19.0, --vision-backend gpu fails when the conversation is created: the GPU delegate does not take the vision encoder's RESIZE_BILINEAR op, whose input is a constant, and the runtime requires the vision encoder to be delegated in full. This was seen on macOS with LFM2.5-VL-3B_int4.litertlm and LFM2.5-VL-3B_int4_fixB.litertlm (2026-11-09) and on a Galaxy S26 with LFM2.5-VL-3B_int8.litertlm and the vision encoder set to the GPU (2026-10-30). With the vision encoder on the CPU the same files answered an image question. Later LiteRT-LM versions were not tried.
One Engine per conversation. On litert-lm 0.18.0 the runtime starts a new Conversation on an Engine that already ran another conversation from the earlier conversation's short-convolution state: it reuses the prompt tokens the two conversations share and does not reset the state (my open report). In a test that asks three questions (What is the capital of France? / What is 17 times 23? / What color is the sky on a clear day?) first in a pirate-persona conversation and then in a conversation told to answer in one word, on the same Engine against a fresh Engine in a new process (answers capped at 200 tokens; CPU backend; litert-lm 0.18.0 on a Mac), the _int4, _int4_fixB and _int8 files each gave the same three one-word answers on both Engines, in its one run. Those three short questions say nothing about longer conversations or image turns. Create a new Engine for each conversation. The first conversation on an Engine is unaffected. A fix landed on LiteRT-LM main on 2026-11-03 (e1f86b05). No release carries it yet (v0.19.1 does not); this note will be updated when one does.
Speed
litert-lm benchmark … --cache no, litert-lm 0.16.0 pip, Apple M4 Max (128 GB), text path (prefill/decode; image encoding is a separate one-shot vision-encoder call at prompt time; the int4 rows ran on an earlier int4 file of this repo):
CPU backend:
| Variant | Prefill (256) | Prefill (1024) | Decode | TTFT |
|---|---|---|---|---|
| int8 | 156 tok/s | 387 tok/s | 36.0 tok/s | 1.67 s |
| int4 | 141 tok/s | 181 tok/s | 40.1 tok/s | 1.84 s |
GPU backend (--backend gpu; both variants verified to generate correct text and answer the image check on GPU before quoting):
| Variant | Prefill (256) | Decode | TTFT |
|---|---|---|---|
| int8 | 1835 tok/s | 114.5 tok/s | 0.15 s |
| int4 | 1944 tok/s | 143.2 tok/s | 0.14 s |
On Android, an earlier int4 file of this repo ran GPU-accelerated on 2026-08-13. Pixel 8a (Tensor G3), litert_lm_main built from the v0.16.0 release tag, 292-token prompt, decode run to EOS (≈1.1k tokens sustained), --disable_cache (worst-case load; default caching makes subsequent loads much faster), that int4 file:
| Backend | Prefill (292 tok) | Decode | TTFT |
|---|---|---|---|
| GPU (OpenCL) | 91.6 tok/s | 10.8 tok/s | 3.3 s |
| CPU | 14.1 tok/s | 5.8 tok/s | 20.9 s |
In the GPU run the text graph delegated fully on Android OpenCL (937/937 nodes, zero rejected ops). One operational note: the GPU path writes multi-GB compile caches next to the model by default — on a nearly-full device the cache write can fail mid-initialization; run with --disable_cache or free storage first.
Galaxy S26 — GPU backend
This section describes the files this repo held on 2026-08-25: both ran on the Android GPU backend and generated. LFM2.5-VL-3B_int4.litertlm has been replaced since; LFM2.5-VL-3B_int8.litertlm has not. For the Android GPU path of the current LFM2.5-VL-3B_int4.litertlm, see Known runtime issues.
| file | GPU backend | delegation | peak |
|---|---|---|---|
LFM2.5-VL-3B_int4.litertlm |
ran | 11135 / 11135 ops across 12 subgraphs on LiteRT GPU |
1465 MiB |
LFM2.5-VL-3B_int8.litertlm |
ran | 11135 / 11135 ops across 12 subgraphs on LiteRT GPU |
1336 MiB |
Measured on a Samsung Galaxy S26 (SM-S942Q / SM8850, Android 16) with litert_lm_advanced_main from litert-lm 0.16.0, --backend=gpu --sampler_backend=cpu, prompt What is the capital of France?. Peak is the process high-water mark (VmHWM) sampled during that same run. Checked 2026-08-25.
The op counts above are the LiteRT GPU partitions. XNNPACK additionally takes 1 of the 4 nodes in main; the runtime accepts that split.
The test prompt carries no image, so this covers engine creation and the text path. The vision path on the GPU is not measured here.
No speed rows in this section, on purpose. A GPU throughput figure only means something beside a CPU row from the same handset. For LFM2.5-VL-3B_int8.litertlm both rows are in litertlm_manifest.json: the GPU row of 2026-08-25 and a CPU row measured on 2026-09-05 (LiteRT-LM 0.16.0), on the same file, which this repo still holds. In those rows the GPU was faster at prefill and decoded at 20.15–20.23 tok/s against 18.77–19.87 on the CPU. No Galaxy S26 CPU row exists for LFM2.5-VL-3B_int4.litertlm.
GPU wiring, including the Gallery import toggle: GPU guide.
Conversion
Converted with the open pipeline in hf-to-litertlm (lfm_work/convert_lfm25_vl.py, litert-torch 0.9.3 --task image_text_to_text): the exact recipe, the int4 post-processing (OCTAV int4-b32 + int8 embedder + zero-scale repair + executor metadata, all vision sections preserved) and the text and image check scripts are in the repo's REPRODUCE.md.
Known runtime issues
Measurements of LFM2.5-VL-3B_int4.litertlm and LFM2.5-VL-3B_int4_fixB.litertlm on this page made before 2026-11-09 used earlier files, not the current ones.
Android GPU with LiteRT-LM v0.18.0 and v0.18.1. LFM2.5-VL-3B_int4.litertlm and LFM2.5-VL-3B_int4_fixB.litertlm keep some of their weights outside the model graph (LiteRT's external weights), and on a Pixel 8a the GPU backend of the v0.18.0 and v0.18.1 Android builds crashes (SIGSEGV) on a Qwen3-0.6B file with external weights (LiteRT-LM#3986).
Long greedy replies on the GPU may vary between runs. On the GPU's default path, where the sampler writes the inputs of the next decode step, a long greedy reply of LFM2.5-2.6B differed from one process to the next on a Mac (WebGPU, LiteRT-LM 0.19.1 from PyPI and built from source) and on a Pixel 11 Pro GPU (PyPI litert-lm-api 0.19.0); with --sampler_handles_input=false (litert_lm_advanced_main built from source; the Python, Kotlin and Swift APIs of 0.19.1 have no such setting) it gave one reply. The files here run the same GPU path; this was not measured on them.
From the second turn of a conversation on, LiteRT-LM puts the previous answer's stop token into the KV cache with a zero embedding, because these files hold the embedding table in its own section and the decoder takes embeddings, so later answers in a conversation can differ from the answer the model gives to the same messages in one prompt (read in the source of LiteRT-LM 0.16.0 to 0.18.1; measured with 0.18.1 on WeVisOne-VL-2B and Qwen2.5-Coder-1.5B-Instruct, not on these files). One question per conversation avoids it, as does a new conversation that starts with the earlier messages and the new question.
- Downloads last month
- 1,531