Skip to the tool

Tool

Token count for Korean text

encode() uses js-tiktoken ranks. o200k_base maps to GPT-4o, GPT-4.1, o1, o3, o4-mini, and GPT-5. cl100k_base maps to GPT-3.5-turbo, GPT-4, and GPT-4-turbo. GPT-6 ranks are not in the library.

The text stays in this tab. The tokenizer files load from this site into the browser. GearDel does not receive the paste.

How the count is made

The page lazy-loads js-tiktoken and one BPE rank file: o200k_base or cl100k_base. Other pages on GearDel do not download those files.

o200k_base is the encoding js-tiktoken’s getEncodingNameForModel returns for GPT-4o, GPT-4.1, o1, o3, o4-mini, and GPT-5. cl100k_base is the encoding it returns for GPT-3.5-turbo, GPT-4, and GPT-4-turbo.

encode() runs on the exact string in the box. Chat markup such as <|im_start|> is not inserted. Special-token strings are split as ordinary text unless they appear as the library’s special tokens and you have typed them as such.

The Hangul-to-token ratio on the result row is taken from that encode, not from a fixed Korean formula.

What this page does not do

  • GPT-6 is not in js-tiktoken’s model table. Claude, Gemini, Llama, Gemma, Qwen, and Korean-only tokenizers are also absent. Those would be different numbers, so this page does not guess.
  • It does not estimate dollar cost. Billing depends on the vendor, the model, and request overhead, none of which are counted here.
  • It stops at 200,000 code points so the tab stays usable. Longer text needs splitting.

Paste text

js-tiktoken maps GPT-4o, GPT-4.1, o1, o3, o4-mini, and GPT-5 to o200k_base. No chat template is added. gpt-4o · gpt-4.1 · o1 · o3 · o4-mini · gpt-5

GPT-6, Claude, Gemini, Llama, Gemma, and Qwen vocabularies are not in js-tiktoken. This page does not invent an approximation.

Tokens
—
Code points
0
Hangul syllables
0
Tokens / 100 code points
—

Tokens / 100 code points is this paste only. It is not a Korean-language average.

Questions

Does Korean use more tokens than English?

For the same number of characters, encode() often returns more tokens for Hangul because BPE splits it more finely. The ratio is whatever this paste produces, not a published average.

Why do GPT-4 and GPT-5 differ on this page?

js-tiktoken maps GPT-4 (and GPT-4-turbo) to cl100k_base, and GPT-5 (with GPT-4o, GPT-4.1, o1, o3, o4-mini) to o200k_base. Switch the button and count the same paste twice. GPT-6 has no rank file here.

Is this the same as the OpenAI API?

It uses the same published ranks through js-tiktoken for names in that table. It does not add a chat template or count HTTP overhead. If your client wraps messages, wrap them before pasting.

Related on GearDel

Pick a Korean dataset by task →

Korean LLM training rows →

Alpaca vs ShareGPT vs messages →

Eval sets vs training rows →

All browser tools →