protected function AbstractLexer::scan in Zircon Profile 8

Same name and namespace in other branches

1. 8.0 vendor/doctrine/lexer/lib/Doctrine/Common/Lexer/AbstractLexer.php \Doctrine\Common\Lexer\AbstractLexer::scan()

Scans the input string for tokens.

Parameters

string $input A query string.:

Return value

void

1 call to AbstractLexer::scan()

AbstractLexer::setInput in vendor/doctrine/lexer/lib/Doctrine/Common/Lexer/AbstractLexer.php

Sets the input data to be tokenized.

File

vendor/doctrine/lexer/lib/Doctrine/Common/Lexer/AbstractLexer.php, line 245

Class

AbstractLexer

Base class for writing simple lexers, i.e. for creating small DSLs.

Namespace

Doctrine\Common\Lexer

Code

protected function scan($input) {
  static $regex;
  if (!isset($regex)) {
    $regex = sprintf('/(%s)|%s/%s', implode(')|(', $this
      ->getCatchablePatterns()), implode('|', $this
      ->getNonCatchablePatterns()), $this
      ->getModifiers());
  }
  $flags = PREG_SPLIT_NO_EMPTY | PREG_SPLIT_DELIM_CAPTURE | PREG_SPLIT_OFFSET_CAPTURE;
  $matches = preg_split($regex, $input, -1, $flags);
  foreach ($matches as $match) {

    // Must remain before 'value' assignment since it can change content
    $type = $this
      ->getType($match[0]);
    $this->tokens[] = array(
      'value' => $match[0],
      'type' => $type,
      'position' => $match[1],
    );
  }
}