Skip to content

[source]

Word Tokenizer#

The Word tokenizer uses a regular expression to tokenize the words in a blob of text.

Parameters#

This tokenizer does not have any parameters.

Example#

use Rubix\ML\Tokenizers\Word;

$tokenizer = new Word();

Last update: 2021-03-27