Skip to content

[source]

Whitespace#

Tokens are delimited by a user-specified whitespace character.

Parameters#

# Name Default Type Description
1 delimiter ' ' string The whitespace character that delimits each token.

Example#

use Rubix\ML\Tokenizers\Whitespace;

$tokenizer = new Whitespace(',');

Last update: 2021-03-27