Tokens are the minimals chunk of program that have meaning to the compiler – the smallest meaningful symbols in the language. Our code displays all 6 kinds of tokens, though the usual use of operators is not present here:
| Token type | Description/Purpose | Examples |
|---|---|---|
| Keywords | Words with special meaning to the compiler | int, double, for, auto |
| Identifiers | Names of things that are not built into the language | cout, std, x, myFunction |
| Literals | Basic constant values whose value is specified directly in the source code | "Hello, world!",24.3, 0, ’c’ |
| Operators | Mathematical or logical operations | +, -, &&, %, << |
| Punctuation/Separators | Punctuation defining the structure of a program | { } ( ) , ; |
| Whitespace | Spaces of various sorts; ignored by the compiler | Spaces, tabs, newlines, comments |