public function TokenizerTest::testNoTokenizer in Search API 8

Verifies that strings of non-CJK characters are not tokenized.

This is just a sanity check – it verifies that strings of letters are not tokenized.

File

tests/src/Unit/Processor/TokenizerTest.php, line 248

Class

TokenizerTest

Tests the "Tokenizer" processor.

Namespace

Drupal\Tests\search_api\Unit\Processor

Code

public function testNoTokenizer() {

  // Set the minimum word size to 1 (to split all CJK characters).
  $this->processor
    ->setConfiguration([
    'minimum_word_size' => 1,
  ]);
  $this
    ->invokeMethod('prepare');
  $letters = 'abcdefghijklmnopqrstuvwxyz';
  $out = $this
    ->invokeMethod('simplifyText', [
    $letters,
  ]);
  $this
    ->assertEquals($letters, $out, 'Latin letters are not CJK tokenized');
}