update v 1.0.7.5

This commit is contained in:
Sujit Prasad
2016-06-13 20:41:55 +05:30
parent aa9786d829
commit 283d97e3ea
5078 changed files with 339851 additions and 175995 deletions

View File

@@ -1,7 +1,7 @@
Introduction
============
This project is a PHP 5.2 to PHP 5.6 parser **written in PHP itself**.
This project is a PHP 5.2 to PHP 7.0 parser **written in PHP itself**.
What is this for?
-----------------
@@ -18,7 +18,7 @@ For example an AST abstracts away the fact that in PHP variables can be written
as `$$bar`, `${'foobar'}` or even `${!${''}=barfoo()}`. You don't have to worry about recognizing
all the different syntaxes from a stream of tokens.
Another questions is: Why would I want to have a PHP parser *written in PHP*? Well, PHP might not be
Another question is: Why would I want to have a PHP parser *written in PHP*? Well, PHP might not be
a language especially suited for fast parsing, but processing the AST is much easier in PHP than it
would be in other, faster languages like C. Furthermore the people most probably wanting to do
programmatic PHP code analysis are incidentally PHP developers, not C developers.
@@ -26,13 +26,12 @@ programmatic PHP code analysis are incidentally PHP developers, not C developers
What can it parse?
------------------
The parser uses a PHP 5.6 compliant grammar, which is backwards compatible with all PHP version from PHP 5.2
upwards (and maybe older).
The parser supports parsing PHP 5.2-5.6 and PHP 7.
As the parser is based on the tokens returned by `token_get_all` (which is only able to lex the PHP
version it runs on), additionally a wrapper for emulating new tokens from 5.3, 5.4, 5.5 and 5.6 is provided.
This allows to parse PHP 5.6 source code running on PHP 5.3, for example. This emulation is very hacky and not
perfect, but it should work well on any sane code.
version it runs on), additionally a wrapper for emulating new tokens from 5.5, 5.6 and 7.0 is
provided. This allows to parse PHP 7.0 source code running on PHP 5.4, for example. This emulation
is somewhat hacky and not perfect, but it should work well on any sane code.
What output does it produce?
----------------------------

View File

@@ -1,31 +0,0 @@
Installation
============
There are multiple ways to include the PHP parser into your project:
Installing via Composer
-----------------------
Run the following command inside your project:
php composer.phar require nikic/php-parser
If you haven't installed [Composer][1] yet, you can do so using:
curl -s http://getcomposer.org/installer | php
Installing as a Git Submodule
-----------------------------
Run the following command to install the parser into the `vendor/PHP-Parser` folder:
git submodule add git://github.com/nikic/PHP-Parser.git vendor/PHP-Parser
Installing from the Zip- or Tarball
-----------------------------------
Download the latest version from [the download page][2], unpack it and move the files somewhere into your project.
[1]: https://getcomposer.org/
[2]: https://github.com/nikic/PHP-Parser/tags

View File

@@ -6,55 +6,60 @@ This document explains how to use the parser, the pretty printer and the node tr
Bootstrapping
-------------
The library needs to register a class autoloader. You can either use the `vendor/autoload.php` file generated by
Composer or by including the bundled `lib/bootstrap.php` file:
To bootstrap the library, include the autoloader generated by composer:
```php
<?php
require 'path/to/PHP-Parser/lib/bootstrap.php';
// Or, if you're using Composer:
require 'path/to/vendor/autoload.php';
```
Additionally you may want to set the `xdebug.max_nesting_level` ini option to a higher value:
```php
<?php
ini_set('xdebug.max_nesting_level', 3000);
```
This ensures that there will be no errors when traversing highly nested node trees.
This ensures that there will be no errors when traversing highly nested node trees. However, it is
preferable to disable XDebug completely, as it can easily make this library more than five times
slower.
Parsing
-------
In order to parse some source code you first have to create a `PhpParser\Parser` object, which
needs to be passed a `PhpParser\Lexer` instance:
In order to parse code, you first have to create a parser instance:
```php
<?php
$parser = new PhpParser\Parser(new PhpParser\Lexer);
// or
$parser = new PhpParser\Parser(new PhpParser\Lexer\Emulative);
use PhpParser\ParserFactory;
$parser = (new ParserFactory)->create(ParserFactory::PREFER_PHP7);
```
Use of the emulative lexer is required if you want to parse PHP code from newer versions than the one
you're running on. For example it will allow you to parse PHP 5.6 code while running on PHP 5.3.
The factory accepts a kind argument, that determines how different PHP versions are treated:
Kind | Behavior
-----|---------
`ParserFactory::PREFER_PHP7` | Try to parse code as PHP 7. If this fails, try to parse it as PHP 5.
`ParserFactory::PREFER_PHP5` | Try to parse code as PHP 5. If this fails, try to parse it as PHP 7.
`ParserFactory::ONLY_PHP7` | Parse code as PHP 7.
`ParserFactory::ONLY_PHP5` | Parse code as PHP 5.
Unless you have strong reason to use something else, `PREFER_PHP7` is a reasonable default.
The `create()` method optionally accepts a `Lexer` instance as the second argument. Some use cases
that require customized lexers are discussed in the [lexer documentation](component/Lexer.markdown).
Subsequently you can pass PHP code (including the opening `<?php` tag) to the `parse` method in order to
create a syntax tree. If a syntax error is encountered, an `PhpParser\Error` exception will be thrown:
```php
<?php
$code = '<?php // some code';
use PhpParser\Error;
use PhpParser\ParserFactory;
$parser = new PhpParser\Parser(new PhpParser\Lexer\Emulative);
$code = '<?php // some code';
$parser = (new ParserFactory)->create(ParserFactory::PREFER_PHP7);
try {
$stmts = $parser->parse($code);
// $stmts is an array of statement nodes
} catch (PhpParser\Error $e) {
} catch (Error $e) {
echo 'Parse Error: ', $e->getMessage();
}
```
@@ -137,11 +142,14 @@ information the formatting is done using a specified scheme. Currently there is
namely `PhpParser\PrettyPrinter\Standard`.
```php
<?php
use PhpParser\Error;
use PhpParser\ParserFactory;
use PhpParser\PrettyPrinter;
$code = "<?php echo 'Hi ', hi\\getTarget();";
$parser = new PhpParser\Parser(new PhpParser\Lexer);
$prettyPrinter = new PhpParser\PrettyPrinter\Standard;
$parser = (new ParserFactory)->create(ParserFactory::PREFER_PHP7);
$prettyPrinter = new PrettyPrinter\Standard;
try {
// parse
@@ -158,7 +166,7 @@ try {
$code = $prettyPrinter->prettyPrint($stmts);
echo $code;
} catch (PhpParser\Error $e) {
} catch (Error $e) {
echo 'Parse Error: ', $e->getMessage();
}
```
@@ -188,11 +196,13 @@ For this purpose the parser provides a component for traversing and visiting the
structure of a program using this `PhpParser\NodeTraverser` looks like this:
```php
<?php
use PhpParser\NodeTraverser;
use PhpParser\ParserFactory;
use PhpParser\PrettyPrinter;
$parser = new PhpParser\Parser(new PhpParser\Lexer\Emulative);
$traverser = new PhpParser\NodeTraverser;
$prettyPrinter = new PhpParser\PrettyPrinter\Standard;
$parser = (new ParserFactory)->create(ParserFactory::PREFER_PHP7);
$traverser = new NodeTraverser;
$prettyPrinter = new PrettyPrinter\Standard;
// add your visitor
$traverser->addVisitor(new MyNodeVisitor);
@@ -218,10 +228,10 @@ try {
The corresponding node visitor might look like this:
```php
<?php
use PhpParser\Node;
use PhpParser\NodeVisitorAbstract;
class MyNodeVisitor extends PhpParser\NodeVisitorAbstract
class MyNodeVisitor extends NodeVisitorAbstract
{
public function leaveNode(Node $node) {
if ($node instanceof Node\Scalar\String_) {
@@ -236,10 +246,12 @@ The above node visitor would change all string literals in the program to `'foo'
All visitors must implement the `PhpParser\NodeVisitor` interface, which defines the following four
methods:
public function beforeTraverse(array $nodes);
public function enterNode(PhpParser\Node $node);
public function leaveNode(PhpParser\Node $node);
public function afterTraverse(array $nodes);
```php
public function beforeTraverse(array $nodes);
public function enterNode(\PhpParser\Node $node);
public function leaveNode(\PhpParser\Node $node);
public function afterTraverse(array $nodes);
```
The `beforeTraverse()` method is called once before the traversal begins and is passed the nodes the
traverser was called with. This method can be used for resetting values before traversation or
@@ -258,7 +270,7 @@ The `enterNode()` method can additionally return the value `NodeTraverser::DONT_
which instructs the traverser to skip all children of the current node.
The `leaveNode()` method can additionally return the value `NodeTraverser::REMOVE_NODE`, in which
case the current node will be removed from the parent array. Furthermove it is possible to return
case the current node will be removed from the parent array. Furthermore it is possible to return
an array of nodes, which will be merged into the parent array at the offset of the current node.
I.e. if in `array(A, B, C)` the node `B` should be replaced with `array(X, Y, Z)` the result will
be `array(A, X, Y, Z, C)`.
@@ -299,20 +311,24 @@ assume that no dynamic features are used.
We start off with the following base code:
```php
<?php
use PhpParser\ParserFactory;
use PhpParser\PrettyPrinter;
use PhpParser\NodeTraverser;
use PhpParser\NodeVisitor\NameResolver;
$inDir = '/some/path';
$outDir = '/some/other/path';
$parser = new PhpParser\Parser(new PhpParser\Lexer\Emulative);
$traverser = new PhpParser\NodeTraverser;
$prettyPrinter = new PhpParser\PrettyPrinter\Standard;
$parser = (new ParserFactory)->create(ParserFactory::PREFER_PHP7);
$traverser = new NodeTraverser;
$prettyPrinter = new PrettyPrinter\Standard;
$traverser->addVisitor(new PhpParser\NodeVisitor\NameResolver); // we will need resolved names
$traverser->addVisitor(new NodeVisitor\NamespaceConverter); // our own node visitor
$traverser->addVisitor(new NameResolver); // we will need resolved names
$traverser->addVisitor(new NamespaceConverter); // our own node visitor
// iterate over all .php files in the directory
$files = new RecursiveIteratorIterator(new RecursiveDirectoryIterator($inDir));
$files = new RegexIterator($files, '/\.php$/');
$files = new \RecursiveIteratorIterator(new \RecursiveDirectoryIterator($inDir));
$files = new \RegexIterator($files, '/\.php$/');
foreach ($files as $file) {
try {
@@ -343,9 +359,9 @@ Now lets start with the main code, the `NodeVisitor\NamespaceConverter`. One thi
is convert `A\\B` style names to `A_B` style ones.
```php
<?php
use PhpParser\Node;
class NodeVisitor_NamespaceConverter extends PhpParser\NodeVisitorAbstract
class NamespaceConverter extends \PhpParser\NodeVisitorAbstract
{
public function leaveNode(Node $node) {
if ($node instanceof Node\Name) {
@@ -362,14 +378,14 @@ create a name with backslashes either write `$node->toString()` or `(string) $no
a new name from the string and return it. Returning a new node replaces the old node.
Another thing we need to do is change the class/function/const declarations. Currently they contain
only the shortname (i.e. the last part of the name), but they need to contain the complete name inclduing
only the shortname (i.e. the last part of the name), but they need to contain the complete name including
the namespace prefix:
```php
<?php
use PhpParser\Node;
use PhpParser\Node\Stmt;
class NodeVisitor_NamespaceConverter extends PhpParser\NodeVisitorAbstract
class NodeVisitor_NamespaceConverter extends \PhpParser\NodeVisitorAbstract
{
public function leaveNode(Node $node) {
if ($node instanceof Node\Name) {
@@ -392,10 +408,10 @@ There is not much more to it than converting the namespaced name to string with
The last thing we need to do is remove the `namespace` and `use` statements:
```php
<?php
use PhpParser\Node;
use PhpParser\Node\Stmt;
class NodeVisitor_NamespaceConverter extends PhpParser\NodeVisitorAbstract
class NodeVisitor_NamespaceConverter extends \PhpParser\NodeVisitorAbstract
{
public function leaveNode(Node $node) {
if ($node instanceof Node\Name) {

View File

@@ -17,7 +17,6 @@ Furthermore it is possible to dump nodes into a human readable format using the
`PhpParser\NodeDumper`. This can be used for debugging.
```php
<?php
$code = <<<'CODE'
<?php
@@ -28,7 +27,7 @@ function printLine($msg) {
printLine('Hello World!!!');
CODE;
$parser = new PhpParser\Parser(new PhpParser\Lexer);
$parser = (new PhpParser\ParserFactory)->create(PhpParser\ParserFactory::PREFER_PHP7);
$nodeDumper = new PhpParser\NodeDumper;
try {
@@ -106,7 +105,7 @@ function printLine($msg) {
printLine('Hello World!!!');
CODE;
$parser = new PhpParser\Parser(new PhpParser\Lexer);
$parser = (new PhpParser\ParserFactory)->create(PhpParser\ParserFactory::PREFER_PHP7);
$serializer = new PhpParser\Serializer\XML;
try {

View File

@@ -14,8 +14,11 @@ the following syntactic elements:
Here is an example:
```php
<?php
$factory = new PhpParser\BuilderFactory;
use PhpParser\BuilderFactory;
use PhpParser\PrettyPrinter;
use PhpParser\Node;
$factory = new BuilderFactory;
$node = $factory->namespace('Name\Space')
->addStmt($factory->use('Some\Other\Thingy')->as('SomeOtherClass'))
->addStmt($factory->class('SomeClass')
@@ -26,6 +29,7 @@ $node = $factory->namespace('Name\Space')
->addStmt($factory->method('someMethod')
->makePublic()
->makeAbstract() // ->makeFinal()
->setReturnType('bool')
->addParam($factory->param('someParam')->setTypeHint('SomeClass'))
->setDocComment('/**
* This method does something.
@@ -38,7 +42,7 @@ $node = $factory->namespace('Name\Space')
->makeProtected() // ->makePublic() [default], ->makePrivate()
->addParam($factory->param('someParam')->setDefault('test'))
// it is possible to add manually created nodes
->addStmt(new PhpParser\Node\Expr\Print_(new PhpParser\Node\Expr\Variable('someParam')))
->addStmt(new Node\Expr\Print_(new Node\Expr\Variable('someParam')))
)
// properties will be correctly reordered above the methods
@@ -50,7 +54,7 @@ $node = $factory->namespace('Name\Space')
;
$stmts = array($node);
$prettyPrinter = new PhpParser\PrettyPrinter\Standard();
$prettyPrinter = new PrettyPrinter\Standard();
echo $prettyPrinter->prettyPrintFile($stmts);
```
@@ -71,7 +75,7 @@ abstract class SomeClass extends SomeOtherClass implements A\Few, \Interfaces
*
* @param SomeClass And takes a parameter
*/
public abstract function someMethod(SomeClass $someParam);
public abstract function someMethod(SomeClass $someParam) : bool;
protected function anotherMethod($someParam = 'test')
{
print $someParam;

View File

@@ -17,7 +17,7 @@ position attributes in the lexer need to be enabled:
$lexer = new PhpParser\Lexer(array(
'usedAttributes' => array('comments', 'startLine', 'endLine', 'startFilePos', 'endFilePos'),
));
$parser = new PhpParser\Parser($lexer);
$parser = (new PhpParser\ParserFactory)->create(PhpParser\ParserFactory::PREFER_PHP7, $lexer);
try {
$stmts = $parser->parse($code);
@@ -58,7 +58,7 @@ or `null` if recovery fails.
A usage example:
```php
$parser = new PhpParser\Parser(new PhpParser\Lexer, array(
$parser = (new PhpParser\ParserFactory)->create(PhpParser\ParserFactory::PREFER_PHP7, null, array(
'throwOnError' => false,
));

View File

@@ -72,7 +72,7 @@ $lexer = new PhpParser\Lexer(array(
'comments', 'startLine', 'endLine', 'startTokenPos', 'endTokenPos'
)
));
$parser = new PhpParser\Parser($lexer);
$parser = (new PhpParser\ParserFactory)->create(PhpParser\ParserFactory::PREFER_PHP7, $lexer);
$visitor = new MyNodeVisitor();
$traverser = new PhpParser\NodeTraverser();
@@ -127,14 +127,17 @@ information about the formatting of integers (like decimal vs. hexadecimal) or s
escape sequences). This can be remedied by storing the original value in an attribute:
```php
class KeepOriginalValueLexer extends PHPParser\Lexer // or PHPParser\Lexer\Emulative
use PhpParser\Lexer;
use PhpParser\Parser\Tokens;
class KeepOriginalValueLexer extends Lexer // or Lexer\Emulative
{
public function getNextToken(&$value = null, &$startAttributes = null, &$endAttributes = null) {
$tokenId = parent::getNextToken($value, $startAttributes, $endAttributes);
if ($tokenId == PHPParser\Parser::T_CONSTANT_ENCAPSED_STRING // non-interpolated string
|| $tokenId == PHPParser\Parser::T_LNUMBER // integer
|| $tokenId == PHPParser\Parser::T_DNUMBER // floating point number
if ($tokenId == Tokens::T_CONSTANT_ENCAPSED_STRING // non-interpolated string
|| $tokenId == Tokens::T_LNUMBER // integer
|| $tokenId == Tokens::T_DNUMBER // floating point number
) {
// could also use $startAttributes, doesn't really matter here
$endAttributes['originalValue'] = $value;