himalaya-ai/devanagari-glue-ocr

Devanagari GLUE OCR

A synthetic OCR dataset of 130,000 single-line Devanagari text images paired with their transcriptions. The text comes from a Nepali web corpus and is rendered with 108 Noto Devanagari font files. Before rendering, each sentence was deliberately modified by a rule-based linguistic augmentation step that injects extra conjuncts, vowel signs (matras) and domain terms. The goal is more coverage of complex Devanagari glyph combinations than natural text provides.

Important: the transcription (ground_truth) is the augmented text, not the original sentence, so it is not clean, natural Nepali. The original sentence is preserved in the metadata field (see Recovering the original text).

Dataset at a glance

Rows	130,000 (13 shards of 10,000)
Split	train only
Modalities	Image + text
Image format	PNG, RGB, wide single-line strips: height 73-80 px, width 1,793-4,096 px (median 4,096)
Text length	100-512 characters (median 257)
Script / language	Devanagari, Nepali source text (84% Devanagari characters, 0.3% Latin)
Text source	100% kaggle:hsebarp/oscar-corpus-nepali
Fonts	3 families, 108 font files, roughly balanced
Size on Hub	2.4 GB, Parquet
License	See License

Image statistics were measured on 300 images from the first shard; all other figures cover all 130,000 rows.

Dataset structure

Data fields

Field	Type	Description
id	string	Zero-padded 8-digit row id (e.g. 00000000).
image	image	Rendered PNG strip containing one line of text.
ocr	string	Transcription of the image. Identical to ground_truth in all 130,000 rows.
ground_truth	string	Target transcription: the augmented text that was rendered.
source	string	Origin of the text. Always kaggle:hsebarp/oscar-corpus-nepali.
font	string	Font file used to render the row (108 distinct files, e.g. NotoSansDevanagari-CondensedMedium.ttf).
image_hash	string	32-character hash of the image. All 130,000 values are distinct.
metadata	string (JSON)	Per-row generation record, described below.

The metadata field

metadata is a JSON string with these keys:

Key	Content
image_path	Path on the build machine (/content/devanagari-ocr-datagen/...). Not usable; kept for traceability only.
ground_truth, source, font, image_hash	Copies of the top-level columns.
generation_params	Rendering settings (see below).
doc_id, url	Identifier of the source document and the source dataset URL.
linguistic_augmentation	original_text, the list of operations applied, and profile_before / profile_after character-class counts (consonants, matras, viramas, conjuncts, diacritics, etc.).

Generation parameters

In the first 2,000 rows inspected, every parameter had a single value:

Parameter	Value
font_size	42
ink_threshold / binarize_ink	200 / true
shirorekha_ref	max
vertical_margin, glyph_pad	6, 12
x_overlap_px, space_gap_min_px	1, 12
use_seamless_blend	false
poisson_clone_mode	NORMAL_CLONE
seamless_junction_columns / seamless_junction_band_rows	8 / 4
linguistic_rule_engine_enabled	true
conjunct_matra_injection_enabled	true
procedural_conjuncts, procedural_matras	false, false

These settings (glyph padding, overlap, shirorekha reference, junction parameters) suggest the line images are assembled from individual glyphs rather than rendered as a single text run.

Font families

Family	Rows
NotoSansDevanagariUI	43,474
NotoSerifDevanagari	43,273
NotoSansDevanagari	43,253

Each family is used in many width and weight variants (e.g. ExtraCondensedThin, SemiBold), about 1,260 rows each for the most common variants.

Example instance

{
  "id": "00000000",
  "image": "<PNG, 16,954 bytes>",
  "ocr": "बर्दिबास ग्रेनगरपालिकाको तेस्रो नगर श्रपरिषदोबाट पारिता आ.व.२०७३।७४ को संशोधित र २०७४।७५ को प्रस्तावित नीति, कार्यक्रम तथा बजेट छोटो उत्तर",
  "ground_truth": "बर्दिबास ग्रेनगरपालिकाको तेस्रो नगर श्रपरिषदोबाट पारिता आ.व.२०७३।७४ को संशोधित र २०७४।७५ को प्रस्तावित नीति, कार्यक्रम तथा बजेट छोटो उत्तर",
  "source": "kaggle:hsebarp/oscar-corpus-nepali",
  "font": "NotoSansDevanagari-CondensedMedium.ttf",
  "image_hash": "dcf4b17bc431a1e0bc462909f6a39409",
  "metadata": "{... see below ...}"
}

For this row, the augmentation recorded in metadata["linguistic_augmentation"] is:

{
  "original_text": "बर्दिबास नगरपालिकाको तेस्रो नगर परिषदबाट पारित आ.व.२०७३।७४ को संशोधित र २०७४।७५ को प्रस्तावित नीति, कार्यक्रम तथा बजेट",
  "operations": ["domain_term:छोटो उत्तर", "conjunct:श्र", "conjunct:ग्र", "matra:e", "matra:o", "matra:aa"]
}

Compare original_text with ground_truth: extra conjunct and matra characters were inserted into words and a domain term was appended, which changes the text.

Usage

import json
from datasets import load_dataset

ds = load_dataset("himalaya-ai/devanagari-glue-ocr", split="train", streaming=True)  # ~2.4 GB, streaming recommended
row = next(iter(ds))
row["image"]          # PIL image
row["ground_truth"]   # augmented transcription

Recovering the original text

meta = json.loads(row["metadata"])
original = meta["linguistic_augmentation"]["original_text"]
operations = meta["linguistic_augmentation"]["operations"]

Intended uses

• Training and fine-tuning text-recognition (line-level OCR) models for Devanagari, particularly for robustness to conjuncts and matras.
• Studying how recognition models handle rare glyph combinations.
• Per-font analysis, using the font field.

Dataset creation

Source data

All text comes from kaggle:hsebarp/oscar-corpus-nepali (https://www.kaggle.com/datasets/hsebarp/oscar-corpus-nepali), a Nepali web-text corpus. Each row references its source document through doc_id in metadata.

Annotations

There are no human annotations. Labels are produced by construction: the transcription is the text that was rendered.

Considerations

Limitations and biases

• Synthetic only. The images are rendered text; no scan noise, blur or perspective settings appear in the recorded generation parameters. Performance on real scans, photos or handwriting is not established by this data.
• Augmented text. ground_truth contains injected characters and appended terms, so it does not reflect natural Nepali spelling. Do not use it as a clean text corpus or for language modelling.
• Single text source. All rows derive from one web corpus and inherit its topics, register and any noise or biases.
• Limited visual variety. Three font families and uniform rendering settings (in the inspected rows).
• Possible width cap. Most images are exactly 4,096 px wide, which suggests a width limit. For the longest lines (up to 512 characters) the text may be scaled or cropped. Check this on your own sample before training on long lines.
• ocr is a duplicate of ground_truth and contains no independent OCR output.

Citation

@misc{himalayaai_devanagari_glue_ocr_2026,
  title        = {Devanagari GLUE OCR},
  author       = {{Himalaya AI Labs}},
  year         = {2026},
  publisher    = {Hugging Face},
  howpublished = {\url{https://huggingface.co/datasets/himalaya-ai/devanagari-glue-ocr}}
}

About Himalaya AI Labs

Himalaya AI Labs is a non-profit lab building open-source Nepali AI models and datasets. Website · GitHub · Hugging Face