TokenStructure

std.lexer.TokenStructure

public struct TokenStructure(IdType, string extraFields = "")

The token that is returned by the lexer.

Parameters

IdType	The D type of the "type" token type field.
extraFields	A string containing D code for any extra fields that should be included in the token structure body. This string is passed directly to a mixin statement.

Examples

// No extra struct fields are desired in this example, so leave it blank.
alias Token = TokenStructure!(IdType, "");
Token minusToken = Token(tok!"-");

Variables

text	`string`	The _text of the token.
line	`size_t`	The _line number at which this token occurs.
column	`size_t`	The _column number at which this token occurs. This is measured in bytes and may not be correct when tab characters are involved.
index	`size_t`	The byte offset from the beginning of the input at which this token occurs.
type	`IdType`	The token type.

Functions

opEquals
this		Constructs a token from a token type.
this		Constructs a token.