class Tokenizer in Zircon Profile 8
Same name in this branch
- 8 vendor/symfony/css-selector/Parser/Tokenizer/Tokenizer.php \Symfony\Component\CssSelector\Parser\Tokenizer\Tokenizer
- 8 vendor/masterminds/html5/src/HTML5/Parser/Tokenizer.php \Masterminds\HTML5\Parser\Tokenizer
Same name and namespace in other branches
- 8.0 vendor/symfony/css-selector/Parser/Tokenizer/Tokenizer.php \Symfony\Component\CssSelector\Parser\Tokenizer\Tokenizer
CSS selector tokenizer.
This component is a port of the Python cssselect library, which is copyright Ian Bicking, @author Jean-François Simon <jeanfrancois.simon@sensiolabs.com>
Hierarchy
- class \Symfony\Component\CssSelector\Parser\Tokenizer\Tokenizer
Expanded class hierarchy of Tokenizer
See also
https://github.com/SimonSapin/cssselect.
1 file declares its use of Tokenizer
- Parser.php in vendor/
symfony/ css-selector/ Parser/ Parser.php
File
- vendor/
symfony/ css-selector/ Parser/ Tokenizer/ Tokenizer.php, line 27
Namespace
Symfony\Component\CssSelector\Parser\TokenizerView source
class Tokenizer {
/**
* @var Handler\HandlerInterface[]
*/
private $handlers;
/**
* Constructor.
*/
public function __construct() {
$patterns = new TokenizerPatterns();
$escaping = new TokenizerEscaping($patterns);
$this->handlers = array(
new Handler\WhitespaceHandler(),
new Handler\IdentifierHandler($patterns, $escaping),
new Handler\HashHandler($patterns, $escaping),
new Handler\StringHandler($patterns, $escaping),
new Handler\NumberHandler($patterns),
new Handler\CommentHandler(),
);
}
/**
* Tokenize selector source code.
*
* @param Reader $reader
*
* @return TokenStream
*/
public function tokenize(Reader $reader) {
$stream = new TokenStream();
while (!$reader
->isEOF()) {
foreach ($this->handlers as $handler) {
if ($handler
->handle($reader, $stream)) {
continue 2;
}
}
$stream
->push(new Token(Token::TYPE_DELIMITER, $reader
->getSubstring(1), $reader
->getPosition()));
$reader
->moveForward(1);
}
return $stream
->push(new Token(Token::TYPE_FILE_END, null, $reader
->getPosition()))
->freeze();
}
}
Members
Name | Modifiers | Type | Description | Overrides |
---|---|---|---|---|
Tokenizer:: |
private | property | ||
Tokenizer:: |
public | function | Tokenize selector source code. | |
Tokenizer:: |
public | function | Constructor. |