machine learning#pytorch#ocr#fine-tuning#khmer nlp#computer vision#hugging face

fine-tuning troryong ocr: training a custom khmer vision model from scratch

august 2026·8 min read·chan ekmongkol

executive overview & impact

Enables accurate digital recognition of printed and handwritten Khmer script using a lightweight 5.5M parameter model that runs efficiently on low-cost devices without cloud subscription fees.

key takeaways for business & product teams:

·Preserves Khmer cultural text by digitizing documents and classroom materials accurately.
·Overcomes complex 2D stacked vowels and subscript consonants that break standard Latin OCR models.
·Runs 100% locally on standard CPUs and low-power hardware with zero data leakage.

why this matters (the big picture)

Digitizing text in the Khmer language is a uniquely difficult computer vision problem. Unlike English or other Latin scripts where letters sit in a single horizontal row, Khmer words feature vowels and subscript consonants (Cheung) that stack vertically above and below each root letter.

Standard OCR tools like Tesseract or generic cloud APIs often scramble these complex stacked characters. In this guide, we fine-tune TrorYong OCR, an efficient open-source vision-language model tailored specifically for Khmer, on 3,000 text samples using PyTorch and Hugging Face datasets.

step 1: install dependencies

We begin by installing the evaluation utilities and the specialized TrorYong library:

bash

pip install evaluate jiwer
pip install tror-yong-ocr

The tror-yong-ocr package includes the pre-trained TrorYongOCRModel weights along with a custom Khmer byte-pair tokenizer.

step 2: load the dataset

We load 3,000 samples from the public Hugging Face dataset seanghay/khmer-hanuman-100k, which contains paired text and line images of Khmer text:

python

from datasets import load_dataset

data = load_dataset(
    "seanghay/khmer-hanuman-100k",
    split="train[:3000]"
)

print(data)          # Dataset({ features: ['image', 'text'], num_rows: 3000 })
print(data[0])       # {'image': <PIL.Image>, 'text': 'ដោយក្រសួងក្នុង...'}

Each sample provides a clear line image of Khmer text. 3,000 samples is ideal for rapid experimentation on a single GPU.

step 3: tokenizer & character analysis

TrorYong uses a dedicated tokenizer with a vocabulary of 185 tokens. This compact vocabulary covers all 33 base consonants, stacked consonant forms, vowels, numbers, and punctuation marks:

python

from tror_yong_ocr import get_tokenizer

tokenizer = get_tokenizer()
print("Vocabulary Size:", len(tokenizer))  # 185

Token lengths across our 3,000 samples mostly range between 15 and 50 tokens, giving us a predictable sequence length for batch padding.

step 4: custom pytorch dataset

Images are resized to a fixed resolution of 32 x 128 pixels (height x width). Our custom KmDataset handles tokenization and standard ImageNet normalization:

python

import torch
from torchvision import transforms

class KmDataset(torch.utils.data.Dataset):
    def __init__(self, tokenizer, ds, transform):
        self.tokenizer = tokenizer
        self.ds = ds
        self.transform = transform

    def __getitem__(self, index):
        dp = self.ds[index]
        tok_ids = self.tokenizer.encode(dp['text'], add_special_tokens=True)
        return {
            'img_tensor': self.transform(dp['image']),
            'token_ids': tok_ids,
        }

    def __len__(self):
        return len(self.ds)

train_transform = transforms.Compose([
    transforms.Resize((32, 128)),
    transforms.ToImage(),
    transforms.ToDtype(torch.float32, scale=True),
    transforms.Normalize(mean=[0.485, 0.456, 0.406], std=[0.229, 0.224, 0.225]),
])

# Split into 80% training and 20% validation
data = data.train_test_split(0.2, seed=168)
train_dataset = KmDataset(tokenizer, data['train'], train_transform)
valid_dataset = KmDataset(tokenizer, data['test'], train_transform)

step 5: batch padding & teacher forcing

Because text lines vary in length, we use a custom DataCollatorWithPadding that creates shifted inputs and targets for teacher-forcing:

•input_ids: all tokens except the last token.
•target_ids: all tokens shifted right by one position.

python

from torch.nn.utils.rnn import pad_sequence

class DataCollatorWithPadding:
    def __call__(self, features):
        imgs, inp_tids, tgt_tids = [], [], []
        for f in features:
            imgs.append(f['img_tensor'])
            inp_tids.append(torch.as_tensor(f['token_ids'][:-1]))
            tgt_tids.append(torch.as_tensor(f['token_ids'][1:]))

        return {
            'img_tensor': torch.stack(imgs),
            'input_ids': pad_sequence(inp_tids, batch_first=True, padding_value=tokenizer.pad_id),
            'target_ids': pad_sequence(tgt_tids, batch_first=True, padding_value=tokenizer.pad_id),
        }

step 6: load the pre-trained troryong model

python

from tror_yong_ocr import TrorYongOCRModel

model = TrorYongOCRModel.from_pretrained('KrorngAI/TrorYongOCR')

Model Parameters:

•Total parameters: 5.506M
•Trainable parameters: 5.506M

At only 5.5 million parameters, the model is designed for lightning-fast inference on CPUs and low-cost edge devices.

step 7: patchify: how images become tokens

Instead of processing images through a heavy convolutional network, TrorYong slices the 32 x 128 image into fixed strips using torch.unfold:

python

def patchify(img):
    # Input: [Batch, 3, 32, 128]
    # Unfold height into 4px strips, width into 8px strips
    patches = img.unfold(2, 4, 4).unfold(3, 8, 8)
    patches = patches.permute(0, 2, 3, 1, 4, 5)
    # Output: [Batch, 128, 96] (128 patches of 3*4*8 = 96 features each)
    return patches.contiguous().view(img.size(0), -1, 96)

A 32 x 128 image becomes 128 patches, each containing 96 features. These patches feed directly into the transformer encoder.

step 8: training configuration

We set up training with OneCycleLR scheduling and bfloat16 automatic mixed precision to optimize convergence and memory:

python

from torch.optim.lr_scheduler import OneCycleLR

batch_size    = 16
learning_rate = 7e-3
epochs        = 8
grad_accum    = 4  # Effective batch size = 64

optimizer = torch.optim.AdamW(
    model.parameters(), lr=learning_rate,
    betas=(0.9, 0.95), weight_decay=0.01
)

scheduler = OneCycleLR(
    optimizer, max_lr=learning_rate,
    total_steps=304,
    pct_start=0.1, anneal_strategy='cos', final_div_factor=20,
)

step 9: the training loop

python

for epoch in range(epochs):
    model.train()
    for batch_idx, batch in enumerate(train_loader):
        imgs         = batch["img_tensor"].to(device)
        input_tokens = batch["input_ids"].long().to(device)
        target_tokens= batch["target_ids"].long().to(device)

        patches = patchify(imgs)

        with torch.amp.autocast('cuda', dtype=torch.bfloat16):
            output   = model(patches, input_tokens, target_tokens)
            raw_loss = output.loss
            loss     = raw_loss / grad_accum

        scaler.scale(loss).backward()

        if (batch_idx + 1) % grad_accum == 0:
            scaler.step(optimizer)
            scaler.update()
            scheduler.step()
            optimizer.zero_grad(set_to_none=True)

Training loss drops steadily from ~1.93 in epoch 1 as the transformer learns to associate image patches with correct Khmer character sequences.

step 10: character error rate (cer) evaluation

python

import evaluate

model.load_state_dict(torch.load('best_model.pt'))
model.eval()

cer_metrics = evaluate.load('cer')
result = cer_metrics.compute(references=refs, predictions=preds)
print("Validation CER:", result)

Qualitative evaluation shows the model successfully recognizes primary consonant shapes and learns the structure of complex stacked vowels.

key insights & takeaways

•Patch-based image tokenization is fast and effective: Slicing line images into 128 patches provides high spatial fidelity without the computational overhead of deep CNN backbones.
•Teacher forcing alignment is essential: Proper input/target token shifting ensures fast convergence during sequence decoding.
•OneCycleLR schedule speeds up training: Cosine learning rate warmup prevents early gradient explosion on small custom datasets.
•Next Steps: Scale training to the full 100K dataset with data augmentations (shadows, perspective shifts, blur) to reach production-grade accuracy across all handwriting styles.

questions or technical feedback?

open for research collaboration on github or direct email.

github email