Tool
Token count for Korean text
The box is encoded with the published BPE ranks from js-tiktoken. No chat template is added. The number is the token count of the text you pasted.
The text stays in this tab. The tokenizer files load from this site into the browser. GearDel does not receive the paste.
How the count is made
The page lazy-loads js-tiktoken and one BPE rank file: cl100k_base or o200k_base. Other pages on GearDel do not download those files.
cl100k_base is the encoding js-tiktoken maps to GPT-3.5-turbo and GPT-4 (not GPT-4o). o200k_base is the encoding it maps to GPT-4o and later OpenAI names that use o200k.
encode() runs on the exact string in the box. Chat markup such as <|im_start|> is not inserted. Special-token strings are split as ordinary text unless they appear as the library’s special tokens and you have typed them as such.
The Hangul-to-token ratio on the result row is taken from that encode, not from a fixed Korean formula.
What this page does not do
- It does not implement Claude, Gemini, Llama, or Korean-only tokenizers. Those would be different numbers.
- It does not estimate dollar cost. Billing depends on the vendor, the model, and request overhead, none of which are counted here.
- It stops at 200,000 code points so the tab stays usable. Longer text needs splitting.