VocabX Cuts Inference Cost with Domain-Specific Vocabularies
- Dharma AI tested VocabX on a Portuguese essay grader, keeping Qwen3-4B, the training data, and the training recipe while replacing its general-purpose vocabulary.
- The method selects both subword merges and phrase-level macro tokens according to estimated inference cost across prompts, user input, and model output, rather than token frequency alone.
- In a 500-request test using an RTX 5090 and vLLM, average responses fell from 350 tokens to 101; measured cost dropped 77%, while the benchmark score rose from 0.703 to 0.810.
- Across 15 comparisons spanning five public domains and three model families, pruned VocabX had lower estimated inference cost than the tested reference methods in every case.
Our take
Unlike FVT, MWT, and in-place BPE, VocabX weighs the serving cost of output tokens and combines subword changes with repeated-span macros. Teams adopting it should validate the tokenizer-model pairing and task quality: the reported grader results show that merging a decision, such as a grade, into a structural macro can reduce quality.
Sources
The summary and analysis were generated by AI. The original sources are authoritative. AI content policy