Class StandardTokenizer

Implements

Tokenizer

Index

Constructors

Methods

Constructors

constructor

new StandardTokenizer(removeStopWords?: boolean): StandardTokenizer
Parameters
- OptionalremoveStopWords: boolean
Returns StandardTokenizer
- Defined in search/dist/tokenizer.d.ts:14

Methods

tokenize

tokenize(text: string): SearchToken[]
Parameters
- text: string
Returns SearchToken[]
Implementation of Tokenizer.tokenize
- Defined in search/dist/tokenizer.d.ts:15