.Token()

A token object is the smallest unit of text along with its definition that is meaningful to an application.

Constructor

Instances of ishml.Token are not intended to be created by calling the object's constructor. Instead, they are produced (directly or indirectly) as a result of calls to ishml.Lexicon.search(), ishml.Lexicon.tokenize(), or ishml.Parser.analyze().

Properties

.lexeme String

is the string of characters that identifies the token.

.definitions Array

is an Array of definitions retrieved from the Lexicon that give meaning to the lexeme. A definition may be a simple value or a complex object. It is an arbitrary payload. A definition typically holds one or more references to objects and functions defined elsewhere in the application.

See also parsing tutorial.