Tokenization
by Anonymous
Cut words into chunks yourself and discover that chunk count comes from how common a word is, not how long it is.
About this app
Tokenization is an interactive AI literacy activity that shows how language models split words into tokens rather than reading every word as a whole. Learners cut words between letters, compare their result with the app’s tokenization, and explore why common words may stay whole while unfamiliar words are split into smaller pieces. Three guided tabs combine hands-on practice, comparisons of letter counts and token counts, and prediction quizzes.
Use cases
- Elementary students learning how language models process text
- Teachers introducing AI literacy and computer science concepts
- Parents exploring tokens with children through a visual activity
- Beginners comparing word length with token count
Features
- Interactive letter-boundary cutting with tap-to-join behavior
- AI tokenization comparisons using six example words
- Visual comparison of letter counts and token counts
- Four guided missions in each activity section
- Prediction quiz with immediate feedback and revealed token pieces
- Sound on/off control and reset or next-step navigation