10.1.1 Tokens and identifiers

Tokens

A token in formal language theory is a unit of text which is individisible, and which carries a specific meaning. For example, in natural language, words are usually taken to be tokens, but so are "commas", "semicolons" and "periods".

In Amine's CGs, tokens include:

  • Open square bracket '['
  • Close square bracket ']'
  • Comma ','
  • Colon ':'
  • Semicolon ';'
  • Equals '='
  • The dash '-'
  • The arrows '<-' and '->'
  • The special keywords:
    • super
    • x_source
    • y_target
  • Identifiers

All these are treated as units that each has a special meaning.

But what is an identifier?

Identifiers

An identifier is a token that consists of:

  • Letters from the English alphabet: (a-z and A-Z; not foreign characters such as æ, ø, å).
  • Digits: (0-9)
  • Underscores: '_'

And further, the first two characters must be letters.

Examples include:

  • Process
  • Object
  • John
  • Female
  • He
  • She
  • FIFA_2004

However, the following all fail to be identifiers. Can you spot why, given the above definition?

  • C3PO
  • R2D2
  • X
  • _
  • _ABC
  • 2nd


PrevLite: 10.1 CG syntax
NextLite: 10.1.2 Concepts

Prev: 10.1 CG syntax
Up: 10.1 CG syntax
Next: 10.1.2 Concepts