Hands-On AI Demos

Interactive experiments to understand how AI really works — no coding required.

✂️

Tokenizer

See how AI models break text into tokens

LLMs like ChatGPT don't read whole words — they break text into tokens (parts of words, punctuation, etc.). See how your text gets split:

Artificialintelligenceistransformingeducationworldwide.

Word tokenSub-word tokenPunctuation tokenTotal tokens: 10

Why it matters:AI models have a "context window" (token limit). GPT-4 can process ~128,000 tokens at once. More tokens = higher cost and more context the model can use.