tokforge playground

Text

Text to tokenize

Try

0tokens

Same text in	Tokens	Chars/token	Relative size

Vocabulary—

Round trip from your browser—

Type or paste text to see where the tokenizer cuts it.

Hover or tap a token to see its id and byte range.

Cuts prefer a blank line, then a line break, a sentence end, a word. Every chunk is re-encoded to prove it fits.

Set a token budget and press Split.