Simple Tokens

Simple tokens, are tokens created by the Token Generator (Tokenizer).

The tokenizer has no deep knowledge of the VHDL language, thus it can only detect a limited number of distinct tokens. These token require only a context of up to two characters while parsing.

List of simple tokens:

Kind	Examples	pyVHDLParser Token Class
Single character	;, (	CharacterToken
Multiple characters	<=, :=, **	FusedCharacterToken
Whitespace (space, tab)		SpaceToken
Word	entity, xor	WordToken
Single-line comment	-- TODO	SingleLineCommentToken
Multi-line comment	/*comment*/	MultiLineCommentToken
Integer literal	42	IntegerLiteralToken
Real literal	1.25	RealLiteralToken
Character literal	'a', 'Z'	CharacterLiteralToken
String literal	"hello"	StringLiteralToken
Bit string literal	x"42"	BitStringLiteralToken
Extended identifiers	\$cell35\	ExtendedIdentifierToken
Tool directives		DirectiveToken
Linebreak	\n	LineBreakToken
indentation	\t	IndentationToken