Function tokenize

tokenize(source: string, options?: LexerOptions): Token[]
Tokenise a Wikidot markup source string in one call.

Shorthand for new Lexer(source, options).tokenize().
Parameters
- source: string
  Raw Wikidot markup
- Optionaloptions: LexerOptions
  Optional lexer configuration
Returns Token[]
A flat array of tokens, ending with an EOF token
- Defined in parser/src/lexer/lexer.ts:647