Release PHP-Parser 4.1.0

* Add new constant to be returned from enterNode() to not travers current node for subsequent visitors and skip children traversing

* Allow visitors to replace nodes in leaveNode() when DONT_TRAVERSE_CURRENT_AND_CHILDREN is used

.travis.yml

@@ -23,7 +23,7 @@ matrix:
 
 script:
   - if [ $TRAVIS_PHP_VERSION = '7.0' ]; then vendor/bin/phpunit --coverage-clover build/logs/clover.xml; else vendor/bin/phpunit; fi
-  - if [ $TRAVIS_PHP_VERSION = '7.1' ]; then test_old/run-php-src.sh; fi
+  - if [ $TRAVIS_PHP_VERSION = '7.2' ]; then test_old/run-php-src.sh; fi
 
 after_success:
   - if [ $TRAVIS_PHP_VERSION = '7.0' ]; then php vendor/bin/coveralls; fi

CHANGELOG.md

@@ -1,8 +1,84 @@
-Version 4.0.1-dev
+Version 4.1.1-dev
 -----------------
 
 Nothing yet.
 
+Version 4.1.0 (2018-10-10)
+--------------------------
+
+### Added
+
+* Added support for PHP 7.3 flexible heredoc/nowdoc strings, completing support for PHP 7.3. There
+  are two caveats for this feature:
+   * In some rare, pathological cases flexible heredoc/nowdoc strings change the interpretation of
+     existing doc strings. PHP-Parser will now use the new interpretation.
+   * Flexible heredoc/nowdoc strings require special support from the lexer. Because this is not
+     available on PHP versions before 7.3, support has to be emulated. This emulation is not perfect
+     and some cases which we do not expect to occur in practice (such as flexible doc strings being
+     nested within each other through abuse of variable-variable interpolation syntax) may not be
+     recognized correctly.
+* Added `DONT_TRAVERSER_CURRENT_AND_CHILDREN` to `NodeTraverser` to skip both traversal of child
+  nodes, and prevent subsequent visitors from visiting the current node.
+
+Version 4.0.4 (2018-09-18)
+--------------------------
+
+### Added
+
+* The following methods have been added to `BuilderFactory`:
+  * `useTrait()` (fluent builder)
+  * `traitUseAdaptation()` (fluent builder)
+  * `useFunction()` (fluent builder)
+  * `useConst()` (fluent builder)
+  * `var()`
+  * `propertyFetch()`
+  
+### Deprecated
+
+* `Builder\Param::setTypeHint()` has been deprecated in favor of the newly introduced
+  `Builder\Param::setType()`.
+
+Version 4.0.3 (2018-07-15)
+--------------------------
+
+### Fixed
+
+* Fixed possible undefined offset notice in formatting-preserving printer. (#513)
+
+### Added
+
+* Improved error recovery inside arrays.
+* Preserve trailing comment inside classes. **Note:** This change is possibly BC breaking if your
+  code validates that classes can only contain certain statement types. After this change, classes
+  can also contain Nop statements, while this was not previously possible. (#509)
+
+Version 4.0.2 (2018-06-03)
+--------------------------
+
+### Added
+
+* Improved error recovery inside classes.
+* Support error recovery for `foreach` without `as`.
+* Support error recovery for parameters without variable (`function (Type ) {}`).
+* Support error recovery for functions without body (`function ($foo)`).
+
+Version 4.0.1 (2018-03-25)
+--------------------------
+
+### Added
+
+* [PHP 7.3] Added support for trailing commas in function calls.
+* [PHP 7.3] Added support for by-reference array destructuring. 
+* Added checks to node traverser to prevent replacing a statement with an expression or vice versa.
+  This should prevent common mistakes in the implementation of node visitors.
+* Added the following methods to `BuilderFactory`, to simplify creation of expressions:
+  * `funcCall()`
+  * `methodCall()`
+  * `staticCall()`
+  * `new()`
+  * `constFetch()`
+  * `classConstFetch()`
+
 Version 4.0.0 (2018-02-28)
 --------------------------
 

README.md

@@ -6,9 +6,9 @@ PHP Parser
 This is a PHP 5.2 to PHP 7.2 parser written in PHP. Its purpose is to simplify static code analysis and
 manipulation.
 
-[**Documentation for version 4.x**][doc_master] (stable; for running on PHP >= 7.0; for parsing PHP 5.2 to PHP 7.2).
+[**Documentation for version 4.x**][doc_master] (stable; for running on PHP >= 7.0; for parsing PHP 5.2 to PHP 7.3).
 
-[Documentation for version 3.x][doc_3_x] (stable; for running on PHP >= 5.5; for parsing PHP 5.2 to PHP 7.2).
+[Documentation for version 3.x][doc_3_x] (unsupported; for running on PHP >= 5.5; for parsing PHP 5.2 to PHP 7.2).
 
 Features
 --------
@@ -200,7 +200,7 @@ Component documentation:
    * Converting AST back to PHP code
    * Customizing formatting
    * Formatting-preserving code transformations
- * [AST builders](component/AST_builders.markdown)
+ * [AST builders](doc/component/AST_builders.markdown)
    * Fluent builders for AST nodes
  * [Lexer](doc/component/Lexer.markdown)
    * Lexer options
@@ -209,16 +209,16 @@ Component documentation:
  * [Error handling](doc/component/Error_handling.markdown)
    * Column information for errors
    * Error recovery (parsing of syntactically incorrect code)
- * [Constant expression evaluation](component/Constant_expression_evaluation.markdown)
+ * [Constant expression evaluation](doc/component/Constant_expression_evaluation.markdown)
    * Evaluating constant/property/etc initializers
    * Handling errors and unsupported expressions
- * [JSON representation](component/JSON_representation.markdown)
+ * [JSON representation](doc/component/JSON_representation.markdown)
    * JSON encoding and decoding of ASTs
  * [Performance](doc/component/Performance.markdown)
    * Disabling XDebug
    * Reusing objects
    * Garbage collection impact
- * [Frequently asked questions](component/FAQ.markdown)
+ * [Frequently asked questions](doc/component/FAQ.markdown)
    * Parent and sibling references
 
  [doc_3_x]: https://github.com/nikic/PHP-Parser/tree/3.x/doc

UPGRADE-4.0.md

@@ -17,7 +17,7 @@ HHVM is no longer actively supported.
   comparisons or strict-mode may require adjustment. The following is an exhaustive list of all
   affected subnodes:
 
-   * `Const::$name`
+   * `Const_::$name`
    * `NullableType::$type` (for simple types)
    * `Param::$type` (for simple types)
    * `Expr\ClassConstFetch::$name`
@@ -29,8 +29,8 @@ HHVM is no longer actively supported.
    * `Stmt\Class_::$name`
    * `Stmt\ClassMethod::$name`
    * `Stmt\ClassMethod::$returnType` (for simple types)
-   * `Stmt\Function::$name`
-   * `Stmt\Function::$returnType` (for simple types)
+   * `Stmt\Function_::$name`
+   * `Stmt\Function_::$returnType` (for simple types)
    * `Stmt\Goto_::$name`
    * `Stmt\Interface_::$name`
    * `Stmt\Label::$name`
@@ -48,7 +48,7 @@ HHVM is no longer actively supported.
 * The `name` subnode of `StaticVar` has been renamed to `var` and now contains a `Variable` rather
   than a plain string.
 * The `var` subnode of `ClosureUse` now contains a `Variable` rather than a plain string.
-* The `var` subnode of `Catch` now contains a `Variable` rather than a plain string.
+* The `var` subnode of `Catch_` now contains a `Variable` rather than a plain string.
 * The `alias` subnode of `UseUse` is now `null` if no explicit alias is given. As such,
   `use Foo\Bar` and `use Foo\Bar as Bar` are now represented differently. The `getAlias()` method
   can be used to get the effective alias, even if it is not explicitly given.
@@ -62,7 +62,7 @@ HHVM is no longer actively supported.
 
 ### Removed functionality
 
-* Removed `type` subnode on `Class`, `ClassMethod` and `Property` nodes. Use `flags` instead.
+* Removed `type` subnode on `Class_`, `ClassMethod` and `Property` nodes. Use `flags` instead.
 * The `ClassConst::isStatic()` method has been removed. Constants cannot have a static modifier.
 * The `NodeTraverser` no longer accepts `false` as a return value from a `leaveNode()` method.
   `NodeTraverser::REMOVE_NODE` should be returned instead.
@@ -74,4 +74,4 @@ HHVM is no longer actively supported.
   `Unserializer\XML`, as well as the interfaces `Serializer` and `Unserializer` no longer exist.
 * The `BuilderAbstract` class has been removed. It's functionality is moved into `BuilderHelpers`.
   However, this is an internal class and should not be used directly.
-* The `Autoloader` class has been removed in favor of relying on the Composer autoloader.
+* The `Autoloader` class has been removed in favor of relying on the Composer autoloader.

composer.json

@@ -24,7 +24,7 @@
     "bin": ["bin/php-parse"],
     "extra": {
         "branch-alias": {
-            "dev-master": "4.0-dev"
+            "dev-master": "4.1-dev"
         }
     }
 }

doc/2_Usage_of_basic_components.markdown

@@ -409,7 +409,7 @@ $files = new \RegexIterator($files, '/\.php$/');
 foreach ($files as $file) {
     try {
         // read the file that should be converted
-        $code = file_get_contents($file);
+        $code = file_get_contents($file->getPathName());
 
         // parse
         $stmts = $parser->parse($code);

doc/component/AST_builders.markdown

@@ -28,16 +28,26 @@ use PhpParser\Node;
 
 $factory = new BuilderFactory;
 $node = $factory->namespace('Name\Space')
-    ->addStmt($factory->use('Some\Other\Thingy')->as('SomeOtherClass'))
+    ->addStmt($factory->use('Some\Other\Thingy')->as('SomeClass'))
+    ->addStmt($factory->useFunction('strlen'))
+    ->addStmt($factory->useConst('PHP_VERSION'))
     ->addStmt($factory->class('SomeOtherClass')
         ->extend('SomeClass')
         ->implement('A\Few', '\Interfaces')
         ->makeAbstract() // ->makeFinal()
 
+        ->addStmt($factory->useTrait('FirstTrait'))
+
+        ->addStmt($factory->useTrait('SecondTrait', 'ThirdTrait')
+            ->and('AnotherTrait')
+            ->with($factory->traitUseAdaptation('foo')->as('bar'))
+            ->with($factory->traitUseAdaptation('AnotherTrait', 'baz')->as('test'))
+            ->with($factory->traitUseAdaptation('AnotherTrait', 'func')->insteadof('SecondTrait')))
+
         ->addStmt($factory->method('someMethod')
             ->makePublic()
             ->makeAbstract() // ->makeFinal()
-            ->setReturnType('bool')
+            ->setReturnType('bool') // ->makeReturnByRef()
             ->addParam($factory->param('someParam')->setTypeHint('SomeClass'))
             ->setDocComment('/**
                               * This method does something.
@@ -74,8 +84,16 @@ This will produce the following output with the standard pretty printer:
 namespace Name\Space;
 
 use Some\Other\Thingy as SomeClass;
+use function strlen;
+use const PHP_VERSION;
 abstract class SomeOtherClass extends SomeClass implements A\Few, \Interfaces
 {
+    use FirstTrait;
+    use SecondTrait, ThirdTrait, AnotherTrait {
+        foo as bar;
+        AnotherTrait::baz as test;
+        AnotherTrait::func insteadof SecondTrait;
+    }
     protected $someProperty;
     private $anotherProperty = array(1, 2, 3);
     /**
@@ -98,9 +116,23 @@ The `BuilderFactory` also provides a number of additional helper methods, which
 nodes. The following methods are currently available:
 
  * `val($value)`: Creates an AST node for a literal value like `42` or `[1, 2, 3]`.
+ * `var($name)`: Creates variable node.
  * `args(array $args)`: Creates an array of function/method arguments, including the required `Arg`
    wrappers. Also converts literals to AST nodes.
+ * `funcCall($name, array $args = [])`: Create a function call node. Converts `$name` to a `Name`
+   node and normalizes arguments.
+ * `methodCall(Expr $var, $name, array $args = [])`: Create a method call node. Converts `$name` to
+   an `Identifier` node and normalizes arguments.
+ * `staticCall($class, $name, array $args = [])`: Create a static method call node. Converts
+   `$class` to a `Name` node, `$name` to an `Identifier` node and normalizes arguments.
+ * `new($class, array $args = [])`: Create a "new" (object creation) node. Converts `$class` to a
+   `Name` node.
+ * `constFetch($name)`: Create a constant fetch node. Converts `$name` to a `Name` node.
+ * `classConstFetch($class, $name)`: Create a class constant fetch node. Converts `$class` to a
+   `Name` node and `$name` to an `Identifier` node.
+ * `propertyFetch($var, $name)`: Creates a property fetch node. Converts `$name` to an `Identifier`
+   node.
  * `concat(...$exprs)`: Create a tree of `BinaryOp\Concat` nodes for the given expressions.
 
 These methods may be expanded on an as-needed basis. Please open an issue or PR if a common
-operation is missing.
+operation is missing.

doc/component/Performance.markdown

@@ -17,9 +17,9 @@ the `xdebug.default_enable=0` ini option does *not* disable XDebug. The *only* w
 XDebug is to not load the extension in the first place.
 
 If you are building a command-line utility for use by developers (who often have XDebug enabled),
-you may want to consider automatically restarting PHP with XDebug unloaded. See the composer
-[XdebugHandler](https://github.com/composer/composer/blob/master/src/Composer/XdebugHandler.php)
-for an implementation of such functionality.
+you may want to consider automatically restarting PHP with XDebug unloaded. The
+[composer/xdebug-handler](https://github.com/composer/xdebug-handler) package can be used to do
+this.
 
 If you do run with XDebug, you may need to increase the `xdebug.max_nesting_level` option to a
 higher level, such as 3000. While the parser itself is recursion free, most other code working on

grammar/php5.y

@@ -454,11 +454,17 @@ static_var:
     | plain_variable '=' static_scalar                      { $$ = Stmt\StaticVar[$1, $3]; }
 ;
 
-class_statement_list:
-      class_statement_list class_statement                  { push($1, $2); }
+class_statement_list_ex:
+      class_statement_list_ex class_statement               { if ($2 !== null) { push($1, $2); } }
     | /* empty */                                           { init(); }
 ;
 
+class_statement_list:
+      class_statement_list_ex
+          { makeNop($nop, $this->lookaheadStartAttributes, $this->endAttributes);
+            if ($nop !== null) { $1[] = $nop; } $$ = $1; }
+;
+
 class_statement:
       variable_modifiers property_declaration_list ';'
           { $$ = Stmt\Property[$1, $2]; $this->checkProperty($$, #1); }
@@ -785,11 +791,9 @@ common_scalar:
     | T_FUNC_C                                              { $$ = Scalar\MagicConst\Function_[]; }
     | T_NS_C                                                { $$ = Scalar\MagicConst\Namespace_[]; }
     | T_START_HEREDOC T_ENCAPSED_AND_WHITESPACE T_END_HEREDOC
-          { $attrs = attributes(); setDocStringAttrs($attrs, $1);
-            $$ = new Scalar\String_(Scalar\String_::parseDocString($1, $2, false), $attrs); }
+          { $$ = $this->parseDocString($1, $2, $3, attributes(), stackAttributes(#3), false); }
     | T_START_HEREDOC T_END_HEREDOC
-          { $attrs = attributes(); setDocStringAttrs($attrs, $1);
-            $$ = new Scalar\String_('', $attrs); }
+          { $$ = $this->parseDocString($1, '', $2, attributes(), stackAttributes(#2), false); }
 ;
 
 static_scalar:
@@ -850,8 +854,7 @@ scalar:
           { $attrs = attributes(); $attrs['kind'] = Scalar\String_::KIND_DOUBLE_QUOTED;
             parseEncapsed($2, '"', true); $$ = new Scalar\Encapsed($2, $attrs); }
     | T_START_HEREDOC encaps_list T_END_HEREDOC
-          { $attrs = attributes(); setDocStringAttrs($attrs, $1);
-            parseEncapsedDoc($2, true); $$ = new Scalar\Encapsed($2, $attrs); }
+          { $$ = $this->parseDocString($1, $2, $3, attributes(), stackAttributes(#3), true); }
 ;
 
 static_array_pair_list:

grammar/php7.y

@@ -240,6 +240,8 @@ non_empty_statement:
           { $$ = Stmt\Foreach_[$3, $5[0], ['keyVar' => null, 'byRef' => $5[1], 'stmts' => $7]]; }
     | T_FOREACH '(' expr T_AS variable T_DOUBLE_ARROW foreach_variable ')' foreach_statement
           { $$ = Stmt\Foreach_[$3, $7[0], ['keyVar' => $5, 'byRef' => $7[1], 'stmts' => $9]]; }
+    | T_FOREACH '(' expr error ')' foreach_statement
+          { $$ = Stmt\Foreach_[$3, new Expr\Error(stackAttributes(#4)), ['stmts' => $6]]; }
     | T_DECLARE '(' declare_list ')' declare_statement      { $$ = Stmt\Declare_[$3, $5]; }
     | T_TRY '{' inner_statement_list '}' catches optional_finally
           { $$ = Stmt\TryCatch[$3, $5, $6]; $this->checkTryCatch($$); }
@@ -277,7 +279,7 @@ optional_finally:
 ;
 
 variables_list:
-      non_empty_variables_list no_comma                     { $$ = $1; }
+      non_empty_variables_list optional_comma               { $$ = $1; }
 ;
 
 non_empty_variables_list:
@@ -295,9 +297,14 @@ optional_ellipsis:
     | T_ELLIPSIS                                            { $$ = true; }
 ;
 
+block_or_error:
+      '{' inner_statement_list '}'                          { $$ = $2; }
+    | error                                                 { $$ = []; }
+;
+
 function_declaration_statement:
-    T_FUNCTION optional_ref identifier '(' parameter_list ')' optional_return_type '{' inner_statement_list '}'
-        { $$ = Stmt\Function_[$3, ['byRef' => $2, 'params' => $5, 'returnType' => $7, 'stmts' => $9]]; }
+    T_FUNCTION optional_ref identifier '(' parameter_list ')' optional_return_type block_or_error
+        { $$ = Stmt\Function_[$3, ['byRef' => $2, 'params' => $5, 'returnType' => $7, 'stmts' => $8]]; }
 ;
 
 class_declaration_statement:
@@ -447,6 +454,8 @@ parameter:
           { $$ = Node\Param[$4, null, $1, $2, $3]; $this->checkParam($$); }
     | optional_param_type optional_ref optional_ellipsis plain_variable '=' expr
           { $$ = Node\Param[$4, $6, $1, $2, $3]; $this->checkParam($$); }
+    | optional_param_type optional_ref optional_ellipsis error
+          { $$ = Node\Param[Expr\Error[], null, $1, $2, $3]; }
 ;
 
 type_expr:
@@ -472,7 +481,7 @@ optional_return_type:
 
 argument_list:
       '(' ')'                                               { $$ = array(); }
-    | '(' non_empty_argument_list no_comma ')'              { $$ = $2; }
+    | '(' non_empty_argument_list optional_comma ')'        { $$ = $2; }
 ;
 
 non_empty_argument_list:
@@ -513,11 +522,17 @@ static_var:
     | plain_variable '=' expr                               { $$ = Stmt\StaticVar[$1, $3]; }
 ;
 
-class_statement_list:
-      class_statement_list class_statement                  { push($1, $2); }
+class_statement_list_ex:
+      class_statement_list_ex class_statement               { if ($2 !== null) { push($1, $2); } }
     | /* empty */                                           { init(); }
 ;
 
+class_statement_list:
+      class_statement_list_ex
+          { makeNop($nop, $this->lookaheadStartAttributes, $this->endAttributes);
+            if ($nop !== null) { $1[] = $nop; } $$ = $1; }
+;
+
 class_statement:
       variable_modifiers property_declaration_list ';'
           { $$ = Stmt\Property[$1, $2]; $this->checkProperty($$, #1); }
@@ -527,6 +542,7 @@ class_statement:
           { $$ = Stmt\ClassMethod[$4, ['type' => $1, 'byRef' => $3, 'params' => $6, 'returnType' => $8, 'stmts' => $9]];
             $this->checkClassMethod($$, #1); }
     | T_USE class_name_list trait_adaptations               { $$ = Stmt\TraitUse[$2, $3]; }
+    | error                                                 { $$ = null; /* will be skipped */ }
 ;
 
 trait_adaptations:
@@ -562,7 +578,7 @@ trait_method_reference:
 
 method_body:
       ';' /* abstract method */                             { $$ = null; }
-    | '{' inner_statement_list '}'                          { $$ = $2; }
+    | block_or_error                                        { $$ = $1; }
 ;
 
 variable_modifiers:
@@ -708,11 +724,11 @@ expr:
     | T_YIELD expr T_DOUBLE_ARROW expr                      { $$ = Expr\Yield_[$4, $2]; }
     | T_YIELD_FROM expr                                     { $$ = Expr\YieldFrom[$2]; }
     | T_FUNCTION optional_ref '(' parameter_list ')' lexical_vars optional_return_type
-      '{' inner_statement_list '}'
-          { $$ = Expr\Closure[['static' => false, 'byRef' => $2, 'params' => $4, 'uses' => $6, 'returnType' => $7, 'stmts' => $9]]; }
+      block_or_error
+          { $$ = Expr\Closure[['static' => false, 'byRef' => $2, 'params' => $4, 'uses' => $6, 'returnType' => $7, 'stmts' => $8]]; }
     | T_STATIC T_FUNCTION optional_ref '(' parameter_list ')' lexical_vars optional_return_type
-      '{' inner_statement_list '}'
-          { $$ = Expr\Closure[['static' => true, 'byRef' => $3, 'params' => $5, 'uses' => $7, 'returnType' => $8, 'stmts' => $10]]; }
+      block_or_error
+          { $$ = Expr\Closure[['static' => true, 'byRef' => $3, 'params' => $5, 'uses' => $7, 'returnType' => $8, 'stmts' => $9]]; }
 ;
 
 anonymous_class:
@@ -831,17 +847,14 @@ scalar:
     | dereferencable_scalar                                 { $$ = $1; }
     | constant                                              { $$ = $1; }
     | T_START_HEREDOC T_ENCAPSED_AND_WHITESPACE T_END_HEREDOC
-          { $attrs = attributes(); setDocStringAttrs($attrs, $1);
-            $$ = new Scalar\String_(Scalar\String_::parseDocString($1, $2), $attrs); }
+          { $$ = $this->parseDocString($1, $2, $3, attributes(), stackAttributes(#3), true); }
     | T_START_HEREDOC T_END_HEREDOC
-          { $attrs = attributes(); setDocStringAttrs($attrs, $1);
-            $$ = new Scalar\String_('', $attrs); }
+          { $$ = $this->parseDocString($1, '', $2, attributes(), stackAttributes(#2), true); }
     | '"' encaps_list '"'
           { $attrs = attributes(); $attrs['kind'] = Scalar\String_::KIND_DOUBLE_QUOTED;
             parseEncapsed($2, '"', true); $$ = new Scalar\Encapsed($2, $attrs); }
     | T_START_HEREDOC encaps_list T_END_HEREDOC
-          { $attrs = attributes(); setDocStringAttrs($attrs, $1);
-            parseEncapsedDoc($2, true); $$ = new Scalar\Encapsed($2, $attrs); }
+          { $$ = $this->parseDocString($1, $2, $3, attributes(), stackAttributes(#3), true); }
 ;
 
 optional_expr:
@@ -929,8 +942,10 @@ list_expr_elements:
 
 list_expr_element:
       variable                                              { $$ = Expr\ArrayItem[$1, null, false]; }
+    | '&' variable                                          { $$ = Expr\ArrayItem[$2, null, true]; }
     | list_expr                                             { $$ = Expr\ArrayItem[$1, null, false]; }
     | expr T_DOUBLE_ARROW variable                          { $$ = Expr\ArrayItem[$3, $1, false]; }
+    | expr T_DOUBLE_ARROW '&' variable                      { $$ = Expr\ArrayItem[$4, $1, true]; }
     | expr T_DOUBLE_ARROW list_expr                         { $$ = Expr\ArrayItem[$3, $1, false]; }
     | /* empty */                                           { $$ = null; }
 ;
@@ -940,8 +955,13 @@ array_pair_list:
           { $$ = $1; $end = count($$)-1; if ($$[$end] === null) array_pop($$); }
 ;
 
+comma_or_error:
+      ','
+    | error
+;
+
 inner_array_pair_list:
-      inner_array_pair_list ',' array_pair                  { push($1, $3); }
+      inner_array_pair_list comma_or_error array_pair       { push($1, $3); }
     | array_pair                                            { init($1); }
 ;
 

grammar/rebuildParsers.php

@@ -166,15 +166,6 @@ function resolveMacros($code) {
                      . ' $s->value = Node\Scalar\String_::parseEscapeSequences($s->value, ' . $args[1] . ', ' . $args[2] . '); } }';
             }
 
-            if ('parseEncapsedDoc' == $name) {
-                assertArgs(2, $args, $name);
-
-                return 'foreach (' . $args[0] . ' as $s) { if ($s instanceof Node\Scalar\EncapsedStringPart) {'
-                     . ' $s->value = Node\Scalar\String_::parseEscapeSequences($s->value, null, ' . $args[1] . '); } }'
-                     . ' $s->value = preg_replace(\'~(\r\n|\n|\r)\z~\', \'\', $s->value);'
-                     . ' if (\'\' === $s->value) array_pop(' . $args[0] . ');';
-            }
-
             if ('makeNop' == $name) {
                 assertArgs(3, $args, $name);
 
@@ -192,15 +183,6 @@ function resolveMacros($code) {
                      . '? Scalar\String_::KIND_SINGLE_QUOTED : Scalar\String_::KIND_DOUBLE_QUOTED)';
             }
 
-            if ('setDocStringAttrs' == $name) {
-                assertArgs(2, $args, $name);
-
-                return $args[0] . '[\'kind\'] = strpos(' . $args[1] . ', "\'") === false '
-                     . '? Scalar\String_::KIND_HEREDOC : Scalar\String_::KIND_NOWDOC; '
-                     . 'preg_match(\'/\A[bB]?<<<[ \t]*[\\\'"]?([a-zA-Z_\x7f-\xff][a-zA-Z0-9_\x7f-\xff]*)[\\\'"]?(?:\r\n|\n|\r)\z/\', ' . $args[1] . ', $matches); '
-                     . $args[0] . '[\'docLabel\'] = $matches[1];';
-            }
-
             if ('prependLeadingComments' == $name) {
                 assertArgs(1, $args, $name);
 

lib/PhpParser/Builder/Param.php

@@ -42,13 +42,13 @@ class Param implements PhpParser\Builder
     }
 
     /**
-     * Sets type hint for the parameter.
+     * Sets type for the parameter.
      *
-     * @param string|Node\Name|Node\NullableType $type Type hint to use
+     * @param string|Node\Name|Node\NullableType $type Parameter type
      *
      * @return $this The builder instance (for fluid interface)
      */
-    public function setTypeHint($type) {
+    public function setType($type) {
         $this->type = BuilderHelpers::normalizeType($type);
         if ($this->type == 'void') {
             throw new \LogicException('Parameter type cannot be void');
@@ -57,6 +57,19 @@ class Param implements PhpParser\Builder
         return $this;
     }
 
+    /**
+     * Sets type for the parameter.
+     *
+     * @param string|Node\Name|Node\NullableType $type Parameter type
+     *
+     * @return $this The builder instance (for fluid interface)
+     *
+     * @deprecated Use setType() instead
+     */
+    public function setTypeHint($type) {
+        return $this->setType($type);
+    }
+
     /**
      * Make the parameter accept the value by reference.
      *

lib/PhpParser/Builder/TraitUse.php

@@ -0,0 +1,64 @@
+<?php declare(strict_types=1);
+
+namespace PhpParser\Builder;
+
+use PhpParser\Builder;
+use PhpParser\BuilderHelpers;
+use PhpParser\Node;
+use PhpParser\Node\Stmt;
+
+class TraitUse implements Builder
+{
+    protected $traits = [];
+    protected $adaptations = [];
+
+    /**
+     * Creates a trait use builder.
+     *
+     * @param Node\Name|string ...$traits Names of used traits
+     */
+    public function __construct(...$traits) {
+        foreach ($traits as $trait) {
+            $this->and($trait);
+        }
+    }
+
+    /**
+     * Adds used trait.
+     *
+     * @param Node\Name|string $trait Trait name
+     *
+     * @return $this The builder instance (for fluid interface)
+     */
+    public function and($trait) {
+        $this->traits[] = BuilderHelpers::normalizeName($trait);
+        return $this;
+    }
+
+    /**
+     * Adds trait adaptation.
+     *
+     * @param Stmt\TraitUseAdaptation|Builder\TraitUseAdaptation $adaptation Trait adaptation
+     *
+     * @return $this The builder instance (for fluid interface)
+     */
+    public function with($adaptation) {
+        $adaptation = BuilderHelpers::normalizeNode($adaptation);
+
+        if (!$adaptation instanceof Stmt\TraitUseAdaptation) {
+            throw new \LogicException('Adaptation must have type TraitUseAdaptation');
+        }
+
+        $this->adaptations[] = $adaptation;
+        return $this;
+    }
+
+    /**
+     * Returns the built node.
+     *
+     * @return Node The built node
+     */
+    public function getNode() : Node {
+        return new Stmt\TraitUse($this->traits, $this->adaptations);
+    }
+}

lib/PhpParser/Builder/TraitUseAdaptation.php

@@ -0,0 +1,148 @@
+<?php declare(strict_types=1);
+
+namespace PhpParser\Builder;
+
+use PhpParser\Builder;
+use PhpParser\BuilderHelpers;
+use PhpParser\Node;
+use PhpParser\Node\Stmt;
+
+class TraitUseAdaptation implements Builder
+{
+    const TYPE_UNDEFINED  = 0;
+    const TYPE_ALIAS      = 1;
+    const TYPE_PRECEDENCE = 2;
+
+    /** @var int Type of building adaptation */
+    protected $type;
+
+    protected $trait;
+    protected $method;
+
+    protected $modifier = null;
+    protected $alias = null;
+
+    protected $insteadof = [];
+
+    /**
+     * Creates a trait use adaptation builder.
+     *
+     * @param Node\Name|string|null  $trait  Name of adaptated trait
+     * @param Node\Identifier|string $method Name of adaptated method
+     */
+    public function __construct($trait, $method) {
+        $this->type = self::TYPE_UNDEFINED;
+
+        $this->trait = is_null($trait)? null: BuilderHelpers::normalizeName($trait);
+        $this->method = BuilderHelpers::normalizeIdentifier($method);
+    }
+
+    /**
+     * Sets alias of method.
+     *
+     * @param Node\Identifier|string $alias Alias for adaptated method
+     *
+     * @return $this The builder instance (for fluid interface)
+     */
+    public function as($alias) {
+        if ($this->type === self::TYPE_UNDEFINED) {
+            $this->type = self::TYPE_ALIAS;
+        }
+
+        if ($this->type !== self::TYPE_ALIAS) {
+            throw new \LogicException('Cannot set alias for not alias adaptation buider');
+        }
+
+        $this->alias = $alias;
+        return $this;
+    }
+
+    /**
+     * Sets adaptated method public.
+     *
+     * @return $this The builder instance (for fluid interface)
+     */
+    public function makePublic() {
+        $this->setModifier(Stmt\Class_::MODIFIER_PUBLIC);
+        return $this;
+    }
+
+    /**
+     * Sets adaptated method protected.
+     *
+     * @return $this The builder instance (for fluid interface)
+     */
+    public function makeProtected() {
+        $this->setModifier(Stmt\Class_::MODIFIER_PROTECTED);
+        return $this;
+    }
+
+    /**
+     * Sets adaptated method private.
+     *
+     * @return $this The builder instance (for fluid interface)
+     */
+    public function makePrivate() {
+        $this->setModifier(Stmt\Class_::MODIFIER_PRIVATE);
+        return $this;
+    }
+
+    /**
+     * Adds overwritten traits.
+     *
+     * @param Node\Name|string ...$traits Traits for overwrite
+     *
+     * @return $this The builder instance (for fluid interface)
+     */
+    public function insteadof(...$traits) {
+        if ($this->type === self::TYPE_UNDEFINED) {
+            if (is_null($this->trait)) {
+                throw new \LogicException('Precedence adaptation must have trait');
+            }
+
+            $this->type = self::TYPE_PRECEDENCE;
+        }
+
+        if ($this->type !== self::TYPE_PRECEDENCE) {
+            throw new \LogicException('Cannot add overwritten traits for not precedence adaptation buider');
+        }
+
+        foreach ($traits as $trait) {
+            $this->insteadof[] = BuilderHelpers::normalizeName($trait);
+        }
+
+        return $this;
+    }
+
+    protected function setModifier(int $modifier) {
+        if ($this->type === self::TYPE_UNDEFINED) {
+            $this->type = self::TYPE_ALIAS;
+        }
+
+        if ($this->type !== self::TYPE_ALIAS) {
+            throw new \LogicException('Cannot set access modifier for not alias adaptation buider');
+        }
+
+        if (is_null($this->modifier)) {
+            $this->modifier = $modifier;
+        } else {
+            throw new \LogicException('Multiple access type modifiers are not allowed');
+        }
+    }
+
+    /**
+     * Returns the built node.
+     *
+     * @return Node The built node
+     */
+    public function getNode() : Node {
+        switch ($this->type) {
+            case self::TYPE_ALIAS:
+                return new Stmt\TraitUseAdaptation\Alias($this->trait, $this->method, $this->modifier, $this->alias);
+            case self::TYPE_PRECEDENCE:
+                return new Stmt\TraitUseAdaptation\Precedence($this->trait, $this->method, $this->insteadof);
+            default:
+                throw new \LogicException('Type of adaptation is not defined');
+        }
+    }
+}

lib/PhpParser/BuilderFactory.php

@@ -5,7 +5,10 @@ namespace PhpParser;
 use PhpParser\Node\Arg;
 use PhpParser\Node\Expr;
 use PhpParser\Node\Expr\BinaryOp\Concat;
+use PhpParser\Node\Identifier;
+use PhpParser\Node\Name;
 use PhpParser\Node\Scalar\String_;
+use PhpParser\Node\Stmt;
 use PhpParser\Node\Stmt\Use_;
 
 class BuilderFactory
@@ -54,6 +57,34 @@ class BuilderFactory
         return new Builder\Trait_($name);
     }
 
+    /**
+     * Creates a trait use builder.
+     *
+     * @param Node\Name|string ...$traits Trait names
+     *
+     * @return Builder\TraitUse The create trait use builder
+     */
+    public function useTrait(...$traits) : Builder\TraitUse {
+        return new Builder\TraitUse(...$traits);
+    }
+
+    /**
+     * Creates a trait use adaptation builder.
+     *
+     * @param Node\Name|string|null  $trait  Trait name
+     * @param Node\Identifier|string $method Method name
+     *
+     * @return Builder\TraitUseAdaptation The create trait use adaptation builder
+     */
+    public function traitUseAdaptation($trait, $method = null) : Builder\TraitUseAdaptation {
+        if (is_null($method)) {
+            $method = $trait;
+            $trait = null;
+        }
+
+        return new Builder\TraitUseAdaptation($trait, $method);
+    }
+
     /**
      * Creates a method builder.
      *
@@ -101,14 +132,36 @@ class BuilderFactory
     /**
      * Creates a namespace/class use builder.
      *
-     * @param string|Node\Name Name to alias
+     * @param Node\Name|string $name Name of the entity (namespace or class) to alias
      *
-     * @return Builder\Use_ The create use builder
+     * @return Builder\Use_ The created use builder
      */
     public function use($name) : Builder\Use_ {
         return new Builder\Use_($name, Use_::TYPE_NORMAL);
     }
 
+    /**
+     * Creates a function use builder.
+     *
+     * @param Node\Name|string $name Name of the function to alias
+     *
+     * @return Builder\Use_ The created use function builder
+     */
+    public function useFunction($name) : Builder\Use_ {
+        return new Builder\Use_($name, Use_::TYPE_FUNCTION);
+    }
+
+    /**
+     * Creates a constant use builder.
+     *
+     * @param Node\Name|string $name Name of the const to alias
+     *
+     * @return Builder\Use_ The created use const builder
+     */
+    public function useConst($name) : Builder\Use_ {
+        return new Builder\Use_($name, Use_::TYPE_CONSTANT);
+    }
+
     /**
      * Creates node a for a literal value.
      *
@@ -120,6 +173,21 @@ class BuilderFactory
         return BuilderHelpers::normalizeValue($value);
     }
 
+    /**
+     * Creates variable node.
+     *
+     * @param string|Expr $name Name
+     *
+     * @return Expr\Variable
+     */
+    public function var($name) : Expr\Variable {
+        if (!\is_string($name) && !$name instanceof Expr) {
+            throw new \LogicException('Variable name must be string or Expr');
+        }
+
+        return new Expr\Variable($name);
+    }
+
     /**
      * Normalizes an argument list.
      *
@@ -141,6 +209,108 @@ class BuilderFactory
         return $normalizedArgs;
     }
 
+    /**
+     * Creates a function call node.
+     *
+     * @param string|Name|Expr $name Function name
+     * @param array            $args Function arguments
+     *
+     * @return Expr\FuncCall
+     */
+    public function funcCall($name, array $args = []) : Expr\FuncCall {
+        return new Expr\FuncCall(
+            BuilderHelpers::normalizeNameOrExpr($name),
+            $this->args($args)
+        );
+    }
+
+    /**
+     * Creates a method call node.
+     *
+     * @param Expr                   $var  Variable the method is called on
+     * @param string|Identifier|Expr $name Method name
+     * @param array                  $args Method arguments
+     *
+     * @return Expr\MethodCall
+     */
+    public function methodCall(Expr $var, $name, array $args = []) : Expr\MethodCall {
+        return new Expr\MethodCall(
+            $var,
+            BuilderHelpers::normalizeIdentifierOrExpr($name),
+            $this->args($args)
+        );
+    }
+
+    /**
+     * Creates a static method call node.
+     *
+     * @param string|Name|Expr       $class Class name
+     * @param string|Identifier|Expr $name  Method name
+     * @param array                  $args  Method arguments
+     *
+     * @return Expr\StaticCall
+     */
+    public function staticCall($class, $name, array $args = []) : Expr\StaticCall {
+        return new Expr\StaticCall(
+            BuilderHelpers::normalizeNameOrExpr($class),
+            BuilderHelpers::normalizeIdentifierOrExpr($name),
+            $this->args($args)
+        );
+    }
+
+    /**
+     * Creates an object creation node.
+     *
+     * @param string|Name|Expr $class Class name
+     * @param array            $args  Constructor arguments
+     *
+     * @return Expr\New_
+     */
+    public function new($class, array $args = []) : Expr\New_ {
+        return new Expr\New_(
+            BuilderHelpers::normalizeNameOrExpr($class),
+            $this->args($args)
+        );
+    }
+
+    /**
+     * Creates a constant fetch node.
+     *
+     * @param string|Name $name Constant name
+     *
+     * @return Expr\ConstFetch
+     */
+    public function constFetch($name) : Expr\ConstFetch {
+        return new Expr\ConstFetch(BuilderHelpers::normalizeName($name));
+    }
+    
+    /**
+     * Creates a property fetch node.
+     *
+     * @param Expr                   $var  Variable holding object
+     * @param string|Identifier|Expr $name Property name
+     *
+     * @return Expr\PropertyFetch
+     */
+    public function propertyFetch(Expr $var, $name) : Expr\PropertyFetch {
+        return new Expr\PropertyFetch($var, BuilderHelpers::normalizeIdentifierOrExpr($name));
+    }
+
+    /**
+     * Creates a class constant fetch node.
+     *
+     * @param string|Name|Expr  $class Class name
+     * @param string|Identifier $name  Constant name
+     *
+     * @return Expr\ClassConstFetch
+     */
+    public function classConstFetch($class, $name): Expr\ClassConstFetch {
+        return new Expr\ClassConstFetch(
+            BuilderHelpers::normalizeNameOrExpr($class),
+            BuilderHelpers::normalizeIdentifier($name)
+        );
+    }
+
     /**
      * Creates nested Concat nodes from a list of expressions.
      *
@@ -161,12 +331,16 @@ class BuilderFactory
         return $lastConcat;
     }
 
-    private function normalizeStringExpr($expr) {
+    /**
+     * @param string|Expr $expr
+     * @return Expr
+     */
+    private function normalizeStringExpr($expr) : Expr {
         if ($expr instanceof Expr) {
             return $expr;
         }
 
-        if (is_string($expr)) {
+        if (\is_string($expr)) {
             return new String_($expr);
         }
 

lib/PhpParser/BuilderHelpers.php

@@ -56,13 +56,74 @@ final class BuilderHelpers
     }
 
     /**
-     * Normalizes a name: Converts plain string names to PhpParser\Node\Name.
+     * Normalizes strings to Identifier.
+     *
+     * @param string|Identifier $name The identifier to normalize
+     *
+     * @return Identifier The normalized identifier
+     */
+    public static function normalizeIdentifier($name) : Identifier {
+        if ($name instanceof Identifier) {
+            return $name;
+        }
+
+        if (\is_string($name)) {
+            return new Identifier($name);
+        }
+
+        throw new \LogicException('Expected string or instance of Node\Identifier');
+    }
+
+    /**
+     * Normalizes strings to Identifier, also allowing expressions.
+     *
+     * @param string|Identifier|Expr $name The identifier to normalize
+     *
+     * @return Identifier|Expr The normalized identifier or expression
+     */
+    public static function normalizeIdentifierOrExpr($name) {
+        if ($name instanceof Identifier || $name instanceof Expr) {
+            return $name;
+        }
+
+        if (\is_string($name)) {
+            return new Identifier($name);
+        }
+
+        throw new \LogicException('Expected string or instance of Node\Identifier or Node\Expr');
+    }
+
+    /**
+     * Normalizes a name: Converts string names to Name nodes.
      *
      * @param Name|string $name The name to normalize
      *
      * @return Name The normalized name
      */
     public static function normalizeName($name) : Name {
+        return self::normalizeNameCommon($name, false);
+    }
+
+    /**
+     * Normalizes a name: Converts string names to Name nodes, while also allowing expressions.
+     *
+     * @param Expr|Name|string $name The name to normalize
+     *
+     * @return Name|Expr The normalized name or expression
+     */
+    public static function normalizeNameOrExpr($name) {
+        return self::normalizeNameCommon($name, true);
+    }
+
+    /**
+     * Normalizes a name: Converts string names to Name nodes, optionally allowing expressions.
+     *
+     * @param Expr|Name|string $name      The name to normalize
+     * @param bool             $allowExpr Whether to also allow expressions
+     *
+     * @return Name|Expr The normalized name, or expression (if allowed)
+     */
+    private static function normalizeNameCommon($name, bool $allowExpr) {
         if ($name instanceof Name) {
             return $name;
         } elseif (is_string($name)) {
@@ -79,7 +140,16 @@ final class BuilderHelpers
             }
         }
 
-        throw new \LogicException('Name must be a string or an instance of PhpParser\Node\Name');
+        if ($allowExpr) {
+            if ($name instanceof Expr) {
+                return $name;
+            }
+            throw new \LogicException(
+                'Name must be a string or an instance of Node\Name or Node\Expr'
+            );
+        } else {
+            throw new \LogicException('Name must be a string or an instance of Node\Name');
+        }
     }
 
     /**

lib/PhpParser/Error.php

@@ -15,7 +15,7 @@ class Error extends \RuntimeException
      *                              (or start line of error -- deprecated)
      */
     public function __construct(string $message, $attributes = []) {
-        $this->rawMessage = (string) $message;
+        $this->rawMessage = $message;
         if (is_array($attributes)) {
             $this->attributes = $attributes;
         } else {
@@ -76,7 +76,7 @@ class Error extends \RuntimeException
      * @param string $message Error message
      */
     public function setRawMessage(string $message) {
-        $this->rawMessage = (string) $message;
+        $this->rawMessage = $message;
         $this->updateMessage();
     }
 
@@ -86,7 +86,7 @@ class Error extends \RuntimeException
      * @param int $line Error start line
      */
     public function setStartLine(int $line) {
-        $this->attributes['startLine'] = (int) $line;
+        $this->attributes['startLine'] = $line;
         $this->updateMessage();
     }
 

lib/PhpParser/Lexer/Emulative.php

@@ -2,7 +2,202 @@
 
 namespace PhpParser\Lexer;
 
+use PhpParser\Error;
+use PhpParser\ErrorHandler;
+
 class Emulative extends \PhpParser\Lexer
 {
-    /* No features requiring emulation have been added in PHP > 7.0 */
-}
+    const PHP_7_3 = '7.3.0dev';
+
+    /**
+     * @var array Patches used to reverse changes introduced in the code
+     */
+    private $patches;
+
+    public function startLexing(string $code, ErrorHandler $errorHandler = null) {
+        $this->patches = [];
+        $preparedCode = $this->prepareCode($code);
+        if (null === $preparedCode) {
+            // Nothing to emulate, yay
+            parent::startLexing($code, $errorHandler);
+            return;
+        }
+
+        $collector = new ErrorHandler\Collecting();
+        parent::startLexing($preparedCode, $collector);
+        $this->fixupTokens();
+
+        $errors = $collector->getErrors();
+        if (!empty($errors)) {
+            $this->fixupErrors($errors);
+            foreach ($errors as $error) {
+                $errorHandler->handleError($error);
+            }
+        }
+    }
+
+    /**
+     * Prepares code for emulation. If nothing has to be emulated null is returned.
+     *
+     * @param string $code
+     * @return null|string
+     */
+    private function prepareCode(string $code) {
+        if (version_compare(\PHP_VERSION, self::PHP_7_3, '>=')) {
+            return null;
+        }
+
+        if (strpos($code, '<<<') === false) {
+            // Definitely doesn't contain heredoc/nowdoc
+            return null;
+        }
+
+        $flexibleDocStringRegex = <<<'REGEX'
+/<<<[ \t]*(['"]?)([a-zA-Z_\x80-\xff][a-zA-Z0-9_\x80-\xff]*)\1\r?\n
+(?:.*\r?\n)*?
+(?<indentation>\h*)\2(?![a-zA-Z_\x80-\xff])(?<separator>(?:;?[\r\n])?)/x
+REGEX;
+        if (!preg_match_all($flexibleDocStringRegex, $code, $matches, PREG_SET_ORDER|PREG_OFFSET_CAPTURE)) {
+            // No heredoc/nowdoc found
+            return null;
+        }
+
+        // Keep track of how much we need to adjust string offsets due to the modifications we
+        // already made
+        $posDelta = 0;
+        foreach ($matches as $match) {
+            $indentation = $match['indentation'][0];
+            $indentationStart = $match['indentation'][1];
+
+            $separator = $match['separator'][0];
+            $separatorStart = $match['separator'][1];
+
+            if ($indentation === '' && $separator !== '') {
+                // Ordinary heredoc/nowdoc
+                continue;
+            }
+
+            if ($indentation !== '') {
+                // Remove indentation
+                $indentationLen = strlen($indentation);
+                $code = substr_replace($code, '', $indentationStart + $posDelta, $indentationLen);
+                $this->patches[] = [$indentationStart + $posDelta, 'add', $indentation];
+                $posDelta -= $indentationLen;
+            }
+
+            if ($separator === '') {
+                // Insert newline as separator
+                $code = substr_replace($code, "\n", $separatorStart + $posDelta, 0);
+                $this->patches[] = [$separatorStart + $posDelta, 'remove', "\n"];
+                $posDelta += 1;
+            }
+        }
+
+        if (empty($this->patches)) {
+            // We did not end up emulating anything
+            return null;
+        }
+
+        return $code;
+    }
+
+    private function fixupTokens() {
+        assert(count($this->patches) > 0);
+
+        // Load first patch
+        $patchIdx = 0;
+        list($patchPos, $patchType, $patchText) = $this->patches[$patchIdx];
+
+        // We use a manual loop over the tokens, because we modify the array on the fly
+        $pos = 0;
+        for ($i = 0, $c = \count($this->tokens); $i < $c; $i++) {
+            $token = $this->tokens[$i];
+            if (\is_string($token)) {
+                // We assume that patches don't apply to string tokens
+                $pos += \strlen($token);
+                continue;
+            }
+
+            $len = \strlen($token[1]);
+            $posDelta = 0;
+            while ($patchPos >= $pos && $patchPos < $pos + $len) {
+                $patchTextLen = \strlen($patchText);
+                if ($patchType === 'remove') {
+                    if ($patchPos === $pos && $patchTextLen === $len) {
+                        // Remove token entirely
+                        array_splice($this->tokens, $i, 1, []);
+                        $i--;
+                        $c--;
+                    } else {
+                        // Remove from token string
+                        $this->tokens[$i][1] = substr_replace(
+                            $token[1], '', $patchPos - $pos + $posDelta, $patchTextLen
+                        );
+                        $posDelta -= $patchTextLen;
+                    }
+                } elseif ($patchType === 'add') {
+                    // Insert into the token string
+                    $this->tokens[$i][1] = substr_replace(
+                        $token[1], $patchText, $patchPos - $pos + $posDelta, 0
+                    );
+                    $posDelta += $patchTextLen;
+                } else {
+                    assert(false);
+                }
+
+                // Fetch the next patch
+                $patchIdx++;
+                if ($patchIdx >= \count($this->patches)) {
+                    // No more patches, we're done
+                    return;
+                }
+
+                list($patchPos, $patchType, $patchText) = $this->patches[$patchIdx];
+
+                // Multiple patches may apply to the same token. Reload the current one to check
+                // If the new patch applies
+                $token = $this->tokens[$i];
+            }
+
+            $pos += $len;
+        }
+
+        // A patch did not apply
+        assert(false);
+    }
+
+    /**
+     * Fixup line and position information in errors.
+     *
+     * @param Error[] $errors
+     */
+    private function fixupErrors(array $errors) {
+        foreach ($errors as $error) {
+            $attrs = $error->getAttributes();
+
+            $posDelta = 0;
+            $lineDelta = 0;
+            foreach ($this->patches as $patch) {
+                list($patchPos, $patchType, $patchText) = $patch;
+                if ($patchPos >= $attrs['startFilePos']) {
+                    // No longer relevant
+                    break;
+                }
+
+                if ($patchType === 'add') {
+                    $posDelta += strlen($patchText);
+                    $lineDelta += substr_count($patchText, "\n");
+                } else {
+                    $posDelta -= strlen($patchText);
+                    $lineDelta -= substr_count($patchText, "\n");
+                }
+            }
+
+            $attrs['startFilePos'] += $posDelta;
+            $attrs['endFilePos'] += $posDelta;
+            $attrs['startLine'] += $lineDelta;
+            $attrs['endLine'] += $lineDelta;
+            $error->setAttributes($attrs);
+        }
+    }
+}

lib/PhpParser/Node/Expr/BinaryOp.php

@@ -12,7 +12,7 @@ abstract class BinaryOp extends Expr
     public $right;
 
     /**
-     * Constructs a bitwise and node.
+     * Constructs a binary operator node.
      *
      * @param Expr  $left       The left hand side expression
      * @param Expr  $right      The right hand side expression

lib/PhpParser/Node/Expr/FuncCall.php

@@ -16,8 +16,8 @@ class FuncCall extends Expr
      * Constructs a function call node.
      *
      * @param Node\Name|Expr $name       Function name
-     * @param Node\Arg[]                    $args       Arguments
-     * @param array                                   $attributes Additional attributes
+     * @param Node\Arg[]     $args       Arguments
+     * @param array          $attributes Additional attributes
      */
     public function __construct($name, array $args = [], array $attributes = []) {
         parent::__construct($attributes);

lib/PhpParser/Node/Expr/StaticCall.php

@@ -10,7 +10,7 @@ class StaticCall extends Expr
 {
     /** @var Node\Name|Expr Class name */
     public $class;
-    /** @var string|Identifier|Expr Method name */
+    /** @var Identifier|Expr Method name */
     public $name;
     /** @var Node\Arg[] Arguments */
     public $args;

lib/PhpParser/Node/Param.php

@@ -12,7 +12,7 @@ class Param extends NodeAbstract
     public $byRef;
     /** @var bool Whether this is a variadic argument */
     public $variadic;
-    /** @var Expr\Variable Parameter variable */
+    /** @var Expr\Variable|Expr\Error Parameter variable */
     public $var;
     /** @var null|Expr Default value */
     public $default;
@@ -20,7 +20,7 @@ class Param extends NodeAbstract
     /**
      * Constructs a parameter node.
      *
-     * @param Expr\Variable                 $var        Parameter variable
+     * @param Expr\Variable|Expr\Error      $var        Parameter variable
      * @param null|Expr                     $default    Default value
      * @param null|string|Name|NullableType $type       Typehint
      * @param bool                          $byRef      Whether is passed by reference
@@ -28,7 +28,7 @@ class Param extends NodeAbstract
      * @param array                         $attributes Additional attributes
      */
     public function __construct(
-        Expr\Variable $var, Expr $default = null, $type = null,
+        $var, Expr $default = null, $type = null,
         bool $byRef = false, bool $variadic = false, array $attributes = []
     ) {
         parent::__construct($attributes);

lib/PhpParser/Node/Scalar/String_.php

@@ -134,29 +134,6 @@ class String_ extends Scalar
         }
         throw new Error('Invalid UTF-8 codepoint escape sequence: Codepoint too large');
     }
-
-    /**
-     * @internal
-     *
-     * Parses a constant doc string.
-     *
-     * @param string $startToken Doc string start token content (<<<SMTHG)
-     * @param string $str        String token content
-     * @param bool $parseUnicodeEscape Whether to parse PHP 7 \u escapes
-     *
-     * @return string Parsed string
-     */
-    public static function parseDocString(string $startToken, string $str, bool $parseUnicodeEscape = true) : string {
-        // strip last newline (thanks tokenizer for sticking it into the string!)
-        $str = preg_replace('~(\r\n|\n|\r)\z~', '', $str);
-
-        // nowdoc string
-        if (false !== strpos($startToken, '\'')) {
-            return $str;
-        }
-
-        return self::parseEscapeSequences($str, null, $parseUnicodeEscape);
-    }
     
     public function getType() : string {
         return 'Scalar_String';

lib/PhpParser/NodeTraverser.php

@@ -30,17 +30,23 @@ class NodeTraverser implements NodeTraverserInterface
      */
     const REMOVE_NODE = 3;
 
+    /**
+     * If NodeVisitor::enterNode() returns DONT_TRAVERSE_CURRENT_AND_CHILDREN, child nodes
+     * of the current node will not be traversed for any visitors.
+     *
+     * For subsequent visitors enterNode() will not be called as well.
+     * leaveNode() will be invoked for visitors that has enterNode() method invoked.
+     */
+    const DONT_TRAVERSE_CURRENT_AND_CHILDREN = 4;
+
     /** @var NodeVisitor[] Visitors */
-    protected $visitors;
+    protected $visitors = [];
 
     /** @var bool Whether traversal should be stopped */
     protected $stopTraversal;
 
-    /**
-     * Constructs a node traverser.
-     */
     public function __construct() {
-        $this->visitors = [];
+        // for BC
     }
 
     /**
@@ -111,13 +117,20 @@ class NodeTraverser implements NodeTraverserInterface
                 }
             } elseif ($subNode instanceof Node) {
                 $traverseChildren = true;
-                foreach ($this->visitors as $visitor) {
+                $breakVisitorIndex = null;
+
+                foreach ($this->visitors as $visitorIndex => $visitor) {
                     $return = $visitor->enterNode($subNode);
                     if (null !== $return) {
                         if ($return instanceof Node) {
+                            $this->ensureReplacementReasonable($subNode, $return);
                             $subNode = $return;
                         } elseif (self::DONT_TRAVERSE_CHILDREN === $return) {
                             $traverseChildren = false;
+                        } elseif (self::DONT_TRAVERSE_CURRENT_AND_CHILDREN === $return) {
+                            $traverseChildren = false;
+                            $breakVisitorIndex = $visitorIndex;
+                            break;
                         } elseif (self::STOP_TRAVERSAL === $return) {
                             $this->stopTraversal = true;
                             break 2;
@@ -136,10 +149,12 @@ class NodeTraverser implements NodeTraverserInterface
                     }
                 }
 
-                foreach ($this->visitors as $visitor) {
+                foreach ($this->visitors as $visitorIndex => $visitor) {
                     $return = $visitor->leaveNode($subNode);
+
                     if (null !== $return) {
                         if ($return instanceof Node) {
+                            $this->ensureReplacementReasonable($subNode, $return);
                             $subNode = $return;
                         } elseif (self::STOP_TRAVERSAL === $return) {
                             $this->stopTraversal = true;
@@ -155,6 +170,10 @@ class NodeTraverser implements NodeTraverserInterface
                             );
                         }
                     }
+
+                    if ($breakVisitorIndex === $visitorIndex) {
+                        break;
+                    }
                 }
             }
         }
@@ -175,13 +194,20 @@ class NodeTraverser implements NodeTraverserInterface
         foreach ($nodes as $i => &$node) {
             if ($node instanceof Node) {
                 $traverseChildren = true;
-                foreach ($this->visitors as $visitor) {
+                $breakVisitorIndex = null;
+
+                foreach ($this->visitors as $visitorIndex => $visitor) {
                     $return = $visitor->enterNode($node);
                     if (null !== $return) {
                         if ($return instanceof Node) {
+                            $this->ensureReplacementReasonable($node, $return);
                             $node = $return;
                         } elseif (self::DONT_TRAVERSE_CHILDREN === $return) {
                             $traverseChildren = false;
+                        } elseif (self::DONT_TRAVERSE_CURRENT_AND_CHILDREN === $return) {
+                            $traverseChildren = false;
+                            $breakVisitorIndex = $visitorIndex;
+                            break;
                         } elseif (self::STOP_TRAVERSAL === $return) {
                             $this->stopTraversal = true;
                             break 2;
@@ -200,10 +226,12 @@ class NodeTraverser implements NodeTraverserInterface
                     }
                 }
 
-                foreach ($this->visitors as $visitor) {
+                foreach ($this->visitors as $visitorIndex => $visitor) {
                     $return = $visitor->leaveNode($node);
+
                     if (null !== $return) {
                         if ($return instanceof Node) {
+                            $this->ensureReplacementReasonable($node, $return);
                             $node = $return;
                         } elseif (\is_array($return)) {
                             $doNodes[] = [$i, $return];
@@ -225,6 +253,10 @@ class NodeTraverser implements NodeTraverserInterface
                             );
                         }
                     }
+
+                    if ($breakVisitorIndex === $visitorIndex) {
+                        break;
+                    }
                 }
             } elseif (\is_array($node)) {
                 throw new \LogicException('Invalid node structure: Contains nested arrays');
@@ -239,4 +271,21 @@ class NodeTraverser implements NodeTraverserInterface
 
         return $nodes;
     }
+
+    private function ensureReplacementReasonable($old, $new) {
+        if ($old instanceof Node\Stmt && $new instanceof Node\Expr) {
+            throw new \LogicException(
+                "Trying to replace statement ({$old->getType()}) " .
+                "with expression ({$new->getType()}). Are you missing a " .
+                "Stmt_Expression wrapper?"
+            );
+        }
+
+        if ($old instanceof Node\Expr && $new instanceof Node\Stmt) {
+            throw new \LogicException(
+                "Trying to replace expression ({$old->getType()}) " .
+                "with statement ({$new->getType()})"
+            );
+        }
+    }
 }

lib/PhpParser/ParserAbstract.php

@@ -9,6 +9,7 @@ namespace PhpParser;
 use PhpParser\Node\Expr;
 use PhpParser\Node\Name;
 use PhpParser\Node\Param;
+use PhpParser\Node\Scalar\Encapsed;
 use PhpParser\Node\Scalar\LNumber;
 use PhpParser\Node\Scalar\String_;
 use PhpParser\Node\Stmt\Class_;
@@ -109,8 +110,6 @@ abstract class ParserAbstract implements Parser
 
     /** @var ErrorHandler Error handler */
     protected $errorHandler;
-    /** @var Error[] Errors collected during last parse */
-    protected $errors;
     /** @var int Error state, used to avoid error floods */
     protected $errorState;
 
@@ -129,7 +128,6 @@ abstract class ParserAbstract implements Parser
      */
     public function __construct(Lexer $lexer, array $options = []) {
         $this->lexer = $lexer;
-        $this->errors = [];
 
         if (isset($options['throwOnError'])) {
             throw new \LogicException(
@@ -713,6 +711,119 @@ abstract class ParserAbstract implements Parser
         return new LNumber($num, $attributes);
     }
 
+    protected function stripIndentation(
+        string $string, int $indentLen, string $indentChar,
+        bool $newlineAtStart, bool $newlineAtEnd, array $attributes
+    ) {
+        if ($indentLen === 0) {
+            return $string;
+        }
+
+        $start = $newlineAtStart ? '(?:(?<=\n)|\A)' : '(?<=\n)';
+        $end = $newlineAtEnd ? '(?:(?=[\r\n])|\z)' : '(?=[\r\n])';
+        $regex = '/' . $start . '([ \t]*)(' . $end . ')?/';
+        return preg_replace_callback(
+            $regex,
+            function ($matches) use ($indentLen, $indentChar, $attributes) {
+                $prefix = substr($matches[1], 0, $indentLen);
+                if (false !== strpos($prefix, $indentChar === " " ? "\t" : " ")) {
+                    $this->emitError(new Error(
+                        'Invalid indentation - tabs and spaces cannot be mixed', $attributes
+                    ));
+                } elseif (strlen($prefix) < $indentLen && !isset($matches[2])) {
+                    $this->emitError(new Error(
+                        'Invalid body indentation level ' .
+                        '(expecting an indentation level of at least ' . $indentLen . ')',
+                        $attributes
+                    ));
+                }
+                return substr($matches[0], strlen($prefix));
+            },
+            $string
+        );
+    }
+
+    protected function parseDocString(
+        string $startToken, $contents, string $endToken,
+        array $attributes, array $endTokenAttributes, bool $parseUnicodeEscape
+    ) {
+        $kind = strpos($startToken, "'") === false
+            ? String_::KIND_HEREDOC : String_::KIND_NOWDOC;
+
+        $regex = '/\A[bB]?<<<[ \t]*[\'"]?([a-zA-Z_\x7f-\xff][a-zA-Z0-9_\x7f-\xff]*)[\'"]?(?:\r\n|\n|\r)\z/';
+        $result = preg_match($regex, $startToken, $matches);
+        assert($result === 1);
+        $label = $matches[1];
+
+        $result = preg_match('/\A[ \t]*/', $endToken, $matches);
+        assert($result === 1);
+        $indentation = $matches[0];
+
+        $attributes['kind'] = $kind;
+        $attributes['docLabel'] = $label;
+        $attributes['docIndentation'] = $indentation;
+
+        $indentHasSpaces = false !== strpos($indentation, " ");
+        $indentHasTabs = false !== strpos($indentation, "\t");
+        if ($indentHasSpaces && $indentHasTabs) {
+            $this->emitError(new Error(
+                'Invalid indentation - tabs and spaces cannot be mixed',
+                $endTokenAttributes
+            ));
+
+            // Proceed processing as if this doc string is not indented
+            $indentation = '';
+        }
+
+        $indentLen = \strlen($indentation);
+        $indentChar = $indentHasSpaces ? " " : "\t";
+
+        if (\is_string($contents)) {
+            if ($contents === '') {
+                return new String_('', $attributes);
+            }
+
+            $contents = $this->stripIndentation(
+                $contents, $indentLen, $indentChar, true, true, $attributes
+            );
+            $contents = preg_replace('~(\r\n|\n|\r)\z~', '', $contents);
+
+            if ($kind === String_::KIND_HEREDOC) {
+                $contents = String_::parseEscapeSequences($contents, null, $parseUnicodeEscape);
+            }
+
+            return new String_($contents, $attributes);
+        } else {
+            assert(count($contents) > 0);
+            if (!$contents[0] instanceof Node\Scalar\EncapsedStringPart) {
+                // If there is no leading encapsed string part, pretend there is an empty one
+                $this->stripIndentation(
+                    '', $indentLen, $indentChar, true, false, $contents[0]->getAttributes()
+                );
+            }
+
+            $newContents = [];
+            foreach ($contents as $i => $part) {
+                if ($part instanceof Node\Scalar\EncapsedStringPart) {
+                    $isLast = $i === \count($contents) - 1;
+                    $part->value = $this->stripIndentation(
+                        $part->value, $indentLen, $indentChar,
+                        $i === 0, $isLast, $part->getAttributes()
+                    );
+                    $part->value = String_::parseEscapeSequences($part->value, null, $parseUnicodeEscape);
+                    if ($isLast) {
+                        $part->value = preg_replace('~(\r\n|\n|\r)\z~', '', $part->value);
+                    }
+                    if ('' === $part->value) {
+                        continue;
+                    }
+                }
+                $newContents[] = $part;
+            }
+            return new Encapsed($newContents, $attributes);
+        }
+    }
+
     protected function checkModifier($a, $b, $modifierPos) {
         // Jumping through some hoops here because verifyModifier() is also used elsewhere
         try {

lib/PhpParser/PrettyPrinter/Standard.php

@@ -198,6 +198,10 @@ class Standard extends PrettyPrinterAbstract
         return preg_match('/^-?[0-9]+$/', $stringValue) ? $stringValue . '.0' : $stringValue;
     }
 
+    protected function pScalar_EncapsedStringPart(Scalar\EncapsedStringPart $node) {
+        throw new \LogicException('Cannot directly print EncapsedStringPart');
+    }
+
     // Assignments
 
     protected function pExpr_Assign(Expr\Assign $node) {

lib/PhpParser/PrettyPrinterAbstract.php

@@ -951,12 +951,15 @@ abstract class PrettyPrinterAbstract
      * @param string $append
      */
     protected function safeAppend(string &$str, string $append) {
-        // $append must not be empty in this function
         if ($str === "") {
             $str = $append;
             return;
         }
 
+        if ($append === "") {
+            return;
+        }
+
         if (!$this->labelCharMap[$append[0]]
                 || !$this->labelCharMap[$str[\strlen($str) - 1]]) {
             $str .= $append;

phpunit.xml.dist

@@ -1,14 +1,7 @@
 <?xml version="1.0" encoding="UTF-8"?>
 
 <phpunit backupGlobals="false"
-         backupStaticAttributes="false"
-         colors="false"
-         convertErrorsToExceptions="true"
-         convertNoticesToExceptions="true"
-         convertWarningsToExceptions="true"
-         processIsolation="false"
-         stopOnFailure="false"
-         syntaxCheck="false"
+         colors="true"
          beStrictAboutTestsThatDoNotTestAnything="false"
          bootstrap="./test/bootstrap.php">
     <testsuites>

test/PhpParser/Builder/ClassTest.php

@@ -123,39 +123,31 @@ DOC;
         );
     }
 
-    /**
-     * @expectedException \LogicException
-     * @expectedExceptionMessage Unexpected node of type "Stmt_Echo"
-     */
     public function testInvalidStmtError() {
+        $this->expectException(\LogicException::class);
+        $this->expectExceptionMessage('Unexpected node of type "Stmt_Echo"');
         $this->createClassBuilder('Test')
             ->addStmt(new Stmt\Echo_([]))
         ;
     }
 
-    /**
-     * @expectedException \LogicException
-     * @expectedExceptionMessage Doc comment must be a string or an instance of PhpParser\Comment\Doc
-     */
     public function testInvalidDocComment() {
+        $this->expectException(\LogicException::class);
+        $this->expectExceptionMessage('Doc comment must be a string or an instance of PhpParser\Comment\Doc');
         $this->createClassBuilder('Test')
             ->setDocComment(new Comment('Test'));
     }
 
-    /**
-     * @expectedException \LogicException
-     * @expectedExceptionMessage Name cannot be empty
-     */
     public function testEmptyName() {
+        $this->expectException(\LogicException::class);
+        $this->expectExceptionMessage('Name cannot be empty');
         $this->createClassBuilder('Test')
             ->extend('');
     }
 
-    /**
-     * @expectedException \LogicException
-     * @expectedExceptionMessage Name must be a string or an instance of PhpParser\Node\Name
-     */
     public function testInvalidName() {
+        $this->expectException(\LogicException::class);
+        $this->expectExceptionMessage('Name must be a string or an instance of Node\Name');
         $this->createClassBuilder('Test')
             ->extend(['Foo']);
     }

test/PhpParser/Builder/FunctionTest.php

@@ -92,29 +92,23 @@ class FunctionTest extends TestCase
         ], []), $node);
     }
 
-    /**
-     * @expectedException \LogicException
-     * @expectedExceptionMessage void type cannot be nullable
-     */
     public function testInvalidNullableVoidType() {
+        $this->expectException(\LogicException::class);
+        $this->expectExceptionMessage('void type cannot be nullable');
         $this->createFunctionBuilder('test')->setReturnType('?void');
     }
 
-    /**
-     * @expectedException \LogicException
-     * @expectedExceptionMessage Expected parameter node, got "Name"
-     */
     public function testInvalidParamError() {
+        $this->expectException(\LogicException::class);
+        $this->expectExceptionMessage('Expected parameter node, got "Name"');
         $this->createFunctionBuilder('test')
             ->addParam(new Node\Name('foo'))
         ;
     }
 
-    /**
-     * @expectedException \LogicException
-     * @expectedExceptionMessage Expected statement or expression node
-     */
     public function testAddNonStmt() {
+        $this->expectException(\LogicException::class);
+        $this->expectExceptionMessage('Expected statement or expression node');
         $this->createFunctionBuilder('test')
             ->addStmt(new Node\Name('Test'));
     }

test/PhpParser/Builder/InterfaceTest.php

@@ -79,11 +79,9 @@ class InterfaceTest extends TestCase
         ]), $node);
     }
 
-    /**
-     * @expectedException \LogicException
-     * @expectedExceptionMessage Unexpected node of type "Stmt_PropertyProperty"
-     */
     public function testInvalidStmtError() {
+        $this->expectException(\LogicException::class);
+        $this->expectExceptionMessage('Unexpected node of type "Stmt_PropertyProperty"');
         $this->builder->addStmt(new Stmt\PropertyProperty('invalid'));
     }
 

test/PhpParser/Builder/MethodTest.php

@@ -135,33 +135,27 @@ class MethodTest extends TestCase
         ], []), $node);
     }
 
-    /**
-     * @expectedException \LogicException
-     * @expectedExceptionMessage Cannot add statements to an abstract method
-     */
     public function testAddStmtToAbstractMethodError() {
+        $this->expectException(\LogicException::class);
+        $this->expectExceptionMessage('Cannot add statements to an abstract method');
         $this->createMethodBuilder('test')
             ->makeAbstract()
             ->addStmt(new Print_(new String_('test')))
         ;
     }
 
-    /**
-     * @expectedException \LogicException
-     * @expectedExceptionMessage Cannot make method with statements abstract
-     */
     public function testMakeMethodWithStmtsAbstractError() {
+        $this->expectException(\LogicException::class);
+        $this->expectExceptionMessage('Cannot make method with statements abstract');
         $this->createMethodBuilder('test')
             ->addStmt(new Print_(new String_('test')))
             ->makeAbstract()
         ;
     }
 
-    /**
-     * @expectedException \LogicException
-     * @expectedExceptionMessage Expected parameter node, got "Name"
-     */
     public function testInvalidParamError() {
+        $this->expectException(\LogicException::class);
+        $this->expectExceptionMessage('Expected parameter node, got "Name"');
         $this->createMethodBuilder('test')
             ->addParam(new Node\Name('foo'))
         ;

test/PhpParser/Builder/ParamTest.php

@@ -80,9 +80,9 @@ class ParamTest extends TestCase
     }
 
     /**
-     * @dataProvider provideTestTypeHints
+     * @dataProvider provideTestTypes
      */
-    public function testTypeHints($typeHint, $expectedType) {
+    public function testTypes($typeHint, $expectedType) {
         $node = $this->createParamBuilder('test')
             ->setTypeHint($typeHint)
             ->getNode()
@@ -100,7 +100,7 @@ class ParamTest extends TestCase
         $this->assertEquals($expectedType, $type);
     }
 
-    public function provideTestTypeHints() {
+    public function provideTestTypes() {
         return [
             ['array', new Node\Identifier('array')],
             ['callable', new Node\Identifier('callable')],
@@ -129,20 +129,16 @@ class ParamTest extends TestCase
         ];
     }
 
-    /**
-     * @expectedException \LogicException
-     * @expectedExceptionMessage Parameter type cannot be void
-     */
     public function testVoidTypeError() {
-        $this->createParamBuilder('test')->setTypeHint('void');
+        $this->expectException(\LogicException::class);
+        $this->expectExceptionMessage('Parameter type cannot be void');
+        $this->createParamBuilder('test')->setType('void');
     }
 
-    /**
-     * @expectedException \LogicException
-     * @expectedExceptionMessage Type must be a string, or an instance of Name, Identifier or NullableType
-     */
     public function testInvalidTypeError() {
-        $this->createParamBuilder('test')->setTypeHint(new \stdClass);
+        $this->expectException(\LogicException::class);
+        $this->expectExceptionMessage('Type must be a string, or an instance of Name, Identifier or NullableType');
+        $this->createParamBuilder('test')->setType(new \stdClass);
     }
 
     public function testByRef() {

test/PhpParser/Builder/TraitTest.php

@@ -37,11 +37,9 @@ class TraitTest extends TestCase
         ]), $trait);
     }
 
-    /**
-     * @expectedException \LogicException
-     * @expectedExceptionMessage Unexpected node of type "Stmt_Echo"
-     */
     public function testInvalidStmtError() {
+        $this->expectException(\LogicException::class);
+        $this->expectExceptionMessage('Unexpected node of type "Stmt_Echo"');
         $this->createTraitBuilder('Test')
             ->addStmt(new Stmt\Echo_([]))
         ;

test/PhpParser/Builder/TraitUseAdaptationTest.php

@@ -0,0 +1,109 @@
+<?php declare(strict_types=1);
+
+namespace PhpParser\Builder;
+
+use PhpParser\Comment;
+use PhpParser\Node;
+use PhpParser\Node\Name;
+use PhpParser\Node\Stmt;
+use PhpParser\Node\Stmt\Class_;
+use PHPUnit\Framework\TestCase;
+
+class TraitUseAdaptationTest extends TestCase
+{
+    protected function createTraitUseAdaptationBuilder($trait, $method) {
+        return new TraitUseAdaptation($trait, $method);
+    }
+
+    public function testAsMake() {
+        $builder = $this->createTraitUseAdaptationBuilder(null, 'foo');
+
+        $this->assertEquals(
+            new Stmt\TraitUseAdaptation\Alias(null, 'foo', null, 'bar'),
+            (clone $builder)->as('bar')->getNode()
+        );
+
+        $this->assertEquals(
+            new Stmt\TraitUseAdaptation\Alias(null, 'foo', Class_::MODIFIER_PUBLIC, null),
+            (clone $builder)->makePublic()->getNode()
+        );
+
+        $this->assertEquals(
+            new Stmt\TraitUseAdaptation\Alias(null, 'foo', Class_::MODIFIER_PROTECTED, null),
+            (clone $builder)->makeProtected()->getNode()
+        );
+
+        $this->assertEquals(
+            new Stmt\TraitUseAdaptation\Alias(null, 'foo', Class_::MODIFIER_PRIVATE, null),
+            (clone $builder)->makePrivate()->getNode()
+        );
+    }
+
+    public function testInsteadof() {
+        $node = $this->createTraitUseAdaptationBuilder('SomeTrait', 'foo')
+            ->insteadof('AnotherTrait')
+            ->getNode()
+        ;
+
+        $this->assertEquals(
+            new Stmt\TraitUseAdaptation\Precedence(
+                new Name('SomeTrait'),
+                'foo',
+                [new Name('AnotherTrait')]
+            ),
+            $node
+        );
+    }
+
+    public function testAsOnNotAlias() {
+        $this->expectException(\LogicException::class);
+        $this->expectExceptionMessage('Cannot set alias for not alias adaptation buider');
+        $this->createTraitUseAdaptationBuilder('Test', 'foo')
+            ->insteadof('AnotherTrait')
+            ->as('bar')
+        ;
+    }
+
+    public function testInsteadofOnNotPrecedence() {
+        $this->expectException(\LogicException::class);
+        $this->expectExceptionMessage('Cannot add overwritten traits for not precedence adaptation buider');
+        $this->createTraitUseAdaptationBuilder('Test', 'foo')
+            ->as('bar')
+            ->insteadof('AnotherTrait')
+        ;
+    }
+
+    public function testInsteadofWithoutTrait() {
+        $this->expectException(\LogicException::class);
+        $this->expectExceptionMessage('Precedence adaptation must have trait');
+        $this->createTraitUseAdaptationBuilder(null, 'foo')
+            ->insteadof('AnotherTrait')
+        ;
+    }
+
+    public function testMakeOnNotAlias() {
+        $this->expectException(\LogicException::class);
+        $this->expectExceptionMessage('Cannot set access modifier for not alias adaptation buider');
+        $this->createTraitUseAdaptationBuilder('Test', 'foo')
+            ->insteadof('AnotherTrait')
+            ->makePublic()
+        ;
+    }
+
+    public function testMultipleMake() {
+        $this->expectException(\LogicException::class);
+        $this->expectExceptionMessage('Multiple access type modifiers are not allowed');
+        $this->createTraitUseAdaptationBuilder(null, 'foo')
+            ->makePrivate()
+            ->makePublic()
+        ;
+    }
+
+    public function testUndefinedType() {
+        $this->expectException(\LogicException::class);
+        $this->expectExceptionMessage('Type of adaptation is not defined');
+        $this->createTraitUseAdaptationBuilder(null, 'foo')
+            ->getNode()
+        ;
+    }
+}

test/PhpParser/Builder/TraitUseTest.php

@@ -0,0 +1,55 @@
+<?php declare(strict_types=1);
+
+namespace PhpParser\Builder;
+
+use PhpParser\Comment;
+use PhpParser\Node;
+use PhpParser\Node\Name;
+use PhpParser\Node\Stmt;
+use PHPUnit\Framework\TestCase;
+
+class TraitUseTest extends TestCase
+{
+    protected function createTraitUseBuilder(...$traits) {
+        return new TraitUse(...$traits);
+    }
+
+    public function testAnd() {
+        $node = $this->createTraitUseBuilder('SomeTrait')
+            ->and('AnotherTrait')
+            ->getNode()
+        ;
+
+        $this->assertEquals(
+            new Stmt\TraitUse([
+                new Name('SomeTrait'),
+                new Name('AnotherTrait')
+            ]),
+            $node
+        );
+    }
+
+    public function testWith() {
+        $node = $this->createTraitUseBuilder('SomeTrait')
+            ->with(new Stmt\TraitUseAdaptation\Alias(null, 'foo', null, 'bar'))
+            ->with((new TraitUseAdaptation(null, 'test'))->as('baz'))
+            ->getNode()
+        ;
+
+        $this->assertEquals(
+            new Stmt\TraitUse([new Name('SomeTrait')], [
+                new Stmt\TraitUseAdaptation\Alias(null, 'foo', null, 'bar'),
+                new Stmt\TraitUseAdaptation\Alias(null, 'test', null, 'baz')
+            ]),
+            $node
+        );
+    }
+
+    public function testInvalidAdaptationNode() {
+        $this->expectException(\LogicException::class);
+        $this->expectExceptionMessage('Adaptation must have type TraitUseAdaptation');
+        $this->createTraitUseBuilder('Test')
+            ->with(new Stmt\Echo_([]))
+        ;
+    }
+}

test/PhpParser/Builder/UseTest.php

@@ -26,5 +26,10 @@ class UseTest extends TestCase
         $this->assertEquals(new Stmt\Use_([
             new Stmt\UseUse(new Name('foo\bar'), 'foo')
         ], Stmt\Use_::TYPE_FUNCTION), $node);
+
+        $node = $this->createUseBuilder('foo\BAR', Stmt\Use_::TYPE_CONSTANT)->as('FOO')->getNode();
+        $this->assertEquals(new Stmt\Use_([
+            new Stmt\UseUse(new Name('foo\BAR'), 'FOO')
+        ], Stmt\Use_::TYPE_CONSTANT), $node);
     }
 }

test/PhpParser/BuilderFactoryTest.php

@@ -6,8 +6,12 @@ use PhpParser\Builder;
 use PhpParser\Node\Arg;
 use PhpParser\Node\Expr;
 use PhpParser\Node\Expr\BinaryOp\Concat;
+use PhpParser\Node\Identifier;
+use PhpParser\Node\Name;
+use PhpParser\Node\Scalar\LNumber;
 use PhpParser\Node\Scalar\String_;
 use PHPUnit\Framework\TestCase;
+use Symfony\Component\Yaml\Tests\A;
 
 class BuilderFactoryTest extends TestCase
 {
@@ -21,15 +25,17 @@ class BuilderFactoryTest extends TestCase
 
     public function provideTestFactory() {
         return [
-            ['namespace', Builder\Namespace_::class],
-            ['class',     Builder\Class_::class],
-            ['interface', Builder\Interface_::class],
-            ['trait',     Builder\Trait_::class],
-            ['method',    Builder\Method::class],
-            ['function',  Builder\Function_::class],
-            ['property',  Builder\Property::class],
-            ['param',     Builder\Param::class],
-            ['use',       Builder\Use_::class],
+            ['namespace',   Builder\Namespace_::class],
+            ['class',       Builder\Class_::class],
+            ['interface',   Builder\Interface_::class],
+            ['trait',       Builder\Trait_::class],
+            ['method',      Builder\Method::class],
+            ['function',    Builder\Function_::class],
+            ['property',    Builder\Property::class],
+            ['param',       Builder\Param::class],
+            ['use',         Builder\Use_::class],
+            ['useFunction', Builder\Use_::class],
+            ['useConst',    Builder\Use_::class],
         ];
     }
 
@@ -63,19 +69,15 @@ class BuilderFactoryTest extends TestCase
         );
     }
 
-    /**
-     * @expectedException \LogicException
-     * @expectedExceptionMessage Expected at least two expressions
-     */
     public function testConcatOneError() {
+        $this->expectException(\LogicException::class);
+        $this->expectExceptionMessage('Expected at least two expressions');
         (new BuilderFactory())->concat("a");
     }
 
-    /**
-     * @expectedException \LogicException
-     * @expectedExceptionMessage Expected string or Expr
-     */
     public function testConcatInvalidExpr() {
+        $this->expectException(\LogicException::class);
+        $this->expectExceptionMessage('Expected string or Expr');
         (new BuilderFactory())->concat("a", 42);
     }
 
@@ -92,23 +94,177 @@ class BuilderFactoryTest extends TestCase
         );
     }
 
+    public function testCalls() {
+        $factory = new BuilderFactory();
+
+        // Simple function call
+        $this->assertEquals(
+            new Expr\FuncCall(
+                new Name('var_dump'),
+                [new Arg(new String_('str'))]
+            ),
+            $factory->funcCall('var_dump', ['str'])
+        );
+        // Dynamic function call
+        $this->assertEquals(
+            new Expr\FuncCall(new Expr\Variable('fn')),
+            $factory->funcCall(new Expr\Variable('fn'))
+        );
+
+        // Simple method call
+        $this->assertEquals(
+            new Expr\MethodCall(
+                new Expr\Variable('obj'),
+                new Identifier('method'),
+                [new Arg(new LNumber(42))]
+            ),
+            $factory->methodCall(new Expr\Variable('obj'), 'method', [42])
+        );
+        // Explicitly pass Identifier node
+        $this->assertEquals(
+            new Expr\MethodCall(
+                new Expr\Variable('obj'),
+                new Identifier('method')
+            ),
+            $factory->methodCall(new Expr\Variable('obj'), new Identifier('method'))
+        );
+        // Dynamic method call
+        $this->assertEquals(
+            new Expr\MethodCall(
+                new Expr\Variable('obj'),
+                new Expr\Variable('method')
+            ),
+            $factory->methodCall(new Expr\Variable('obj'), new Expr\Variable('method'))
+        );
+
+        // Simple static method call
+        $this->assertEquals(
+            new Expr\StaticCall(
+                new Name\FullyQualified('Foo'),
+                new Identifier('bar'),
+                [new Arg(new Expr\Variable('baz'))]
+            ),
+            $factory->staticCall('\Foo', 'bar', [new Expr\Variable('baz')])
+        );
+        // Dynamic static method call
+        $this->assertEquals(
+            new Expr\StaticCall(
+                new Expr\Variable('foo'),
+                new Expr\Variable('bar')
+            ),
+            $factory->staticCall(new Expr\Variable('foo'), new Expr\Variable('bar'))
+        );
+
+        // Simple new call
+        $this->assertEquals(
+            new Expr\New_(new Name\FullyQualified('stdClass')),
+            $factory->new('\stdClass')
+        );
+        // Dynamic new call
+        $this->assertEquals(
+            new Expr\New_(
+                new Expr\Variable('foo'),
+                [new Arg(new String_('bar'))]
+            ),
+            $factory->new(new Expr\Variable('foo'), ['bar'])
+        );
+    }
+
+    public function testConstFetches() {
+        $factory = new BuilderFactory();
+        $this->assertEquals(
+            new Expr\ConstFetch(new Name('FOO')),
+            $factory->constFetch('FOO')
+        );
+        $this->assertEquals(
+            new Expr\ClassConstFetch(new Name('Foo'), new Identifier('BAR')),
+            $factory->classConstFetch('Foo', 'BAR')
+        );
+        $this->assertEquals(
+            new Expr\ClassConstFetch(new Expr\Variable('foo'), new Identifier('BAR')),
+            $factory->classConstFetch(new Expr\Variable('foo'), 'BAR')
+        );
+    }
+
+    public function testVar() {
+        $factory = new BuilderFactory();
+        $this->assertEquals(
+            new Expr\Variable("foo"),
+            $factory->var("foo")
+        );
+        $this->assertEquals(
+            new Expr\Variable(new Expr\Variable("foo")),
+            $factory->var($factory->var("foo"))
+        );
+    }
+
+    public function testPropertyFetch() {
+        $f = new BuilderFactory();
+        $this->assertEquals(
+            new Expr\PropertyFetch(new Expr\Variable('foo'), 'bar'),
+            $f->propertyFetch($f->var('foo'), 'bar')
+        );
+        $this->assertEquals(
+            new Expr\PropertyFetch(new Expr\Variable('foo'), 'bar'),
+            $f->propertyFetch($f->var('foo'), new Identifier('bar'))
+        );
+        $this->assertEquals(
+            new Expr\PropertyFetch(new Expr\Variable('foo'), new Expr\Variable('bar')),
+            $f->propertyFetch($f->var('foo'), $f->var('bar'))
+        );
+    }
+
+    public function testInvalidIdentifier() {
+        $this->expectException(\LogicException::class);
+        $this->expectExceptionMessage('Expected string or instance of Node\Identifier');
+        (new BuilderFactory())->classConstFetch('Foo', new Expr\Variable('foo'));
+    }
+
+    public function testInvalidIdentifierOrExpr() {
+        $this->expectException(\LogicException::class);
+        $this->expectExceptionMessage('Expected string or instance of Node\Identifier or Node\Expr');
+        (new BuilderFactory())->staticCall('Foo', new Name('bar'));
+    }
+
+    public function testInvalidNameOrExpr() {
+        $this->expectException(\LogicException::class);
+        $this->expectExceptionMessage('Name must be a string or an instance of Node\Name or Node\Expr');
+        (new BuilderFactory())->funcCall(new Node\Stmt\Return_());
+    }
+
+    public function testInvalidVar() {
+        $this->expectException(\LogicException::class);
+        $this->expectExceptionMessage('Variable name must be string or Expr');
+        (new BuilderFactory())->var(new Node\Stmt\Return_());
+    }
+
     public function testIntegration() {
         $factory = new BuilderFactory;
         $node = $factory->namespace('Name\Space')
             ->addStmt($factory->use('Foo\Bar\SomeOtherClass'))
             ->addStmt($factory->use('Foo\Bar')->as('A'))
+            ->addStmt($factory->useFunction('strlen'))
+            ->addStmt($factory->useConst('PHP_VERSION'))
             ->addStmt($factory
                 ->class('SomeClass')
                 ->extend('SomeOtherClass')
                 ->implement('A\Few', '\Interfaces')
                 ->makeAbstract()
 
+                ->addStmt($factory->useTrait('FirstTrait'))
+
+                ->addStmt($factory->useTrait('SecondTrait', 'ThirdTrait')
+                    ->and('AnotherTrait')
+                    ->with($factory->traitUseAdaptation('foo')->as('bar'))
+                    ->with($factory->traitUseAdaptation('AnotherTrait', 'baz')->as('test'))
+                    ->with($factory->traitUseAdaptation('AnotherTrait', 'func')->insteadof('SecondTrait')))
+
                 ->addStmt($factory->method('firstMethod'))
 
                 ->addStmt($factory->method('someMethod')
                     ->makePublic()
                     ->makeAbstract()
-                    ->addParam($factory->param('someParam')->setTypeHint('SomeClass'))
+                    ->addParam($factory->param('someParam')->setType('SomeClass'))
                     ->setDocComment('/**
                                       * This method does something.
                                       *
@@ -134,8 +290,16 @@ namespace Name\Space;
 
 use Foo\Bar\SomeOtherClass;
 use Foo\Bar as A;
+use function strlen;
+use const PHP_VERSION;
 abstract class SomeClass extends SomeOtherClass implements A\Few, \Interfaces
 {
+    use FirstTrait;
+    use SecondTrait, ThirdTrait, AnotherTrait {
+        foo as bar;
+        AnotherTrait::baz as test;
+        AnotherTrait::func insteadof SecondTrait;
+    }
     protected $someProperty;
     private $anotherProperty = array(1, 2, 3);
     function firstMethod()

test/PhpParser/CodeParsingTest.php

@@ -52,7 +52,8 @@ class CodeParsingTest extends CodeTestAbstract
         ];
     }
 
-    private function getParseOutput(Parser $parser, $code, array $modes) {
+    // Must be public for updateTests.php
+    public function getParseOutput(Parser $parser, $code, array $modes) {
         $dumpPositions = isset($modes['positions']);
 
         $errors = new ErrorHandler\Collecting;

test/PhpParser/ConstExprEvaluatorTest.php

@@ -73,11 +73,9 @@ class ConstExprEvaluatorTest extends TestCase
         ];
     }
 
-    /**
-     * @expectedException \PhpParser\ConstExprEvaluationException
-     * @expectedExceptionMessage Expression of type Expr_Variable cannot be evaluated
-     */
     public function testEvaluateFails() {
+        $this->expectException(ConstExprEvaluationException::class);
+        $this->expectExceptionMessage('Expression of type Expr_Variable cannot be evaluated');
         $evaluator = new ConstExprEvaluator();
         $evaluator->evaluateDirectly(new Expr\Variable('a'));
     }

test/PhpParser/ErrorHandler/ThrowingTest.php

@@ -7,11 +7,9 @@ use PHPUnit\Framework\TestCase;
 
 class ThrowingTest extends TestCase
 {
-    /**
-     * @expectedException \PhpParser\Error
-     * @expectedExceptionMessage Test
-     */
     public function testHandleError() {
+        $this->expectException(Error::class);
+        $this->expectExceptionMessage('Test');
         $errorHandler = new Throwing();
         $errorHandler->handleError(new Error('Test'));
     }

test/PhpParser/ErrorTest.php

@@ -94,11 +94,9 @@ class ErrorTest extends TestCase
         }
     }
 
-    /**
-     * @expectedException \RuntimeException
-     * @expectedExceptionMessage Invalid position information
-     */
     public function testInvalidPosInfo() {
+        $this->expectException(\RuntimeException::class);
+        $this->expectExceptionMessage('Invalid position information');
         $error = new Error('Some error', [
             'startFilePos' => 10,
             'endFilePos' => 11,

test/PhpParser/Lexer/EmulativeTest.php

@@ -2,6 +2,7 @@
 
 namespace PhpParser\Lexer;
 
+use PhpParser\ErrorHandler;
 use PhpParser\LexerTest;
 use PhpParser\Parser\Tokens;
 
@@ -63,12 +64,11 @@ class EmulativeTest extends LexerTest
         $lexer = $this->getLexer();
         $lexer->startLexing('<?php ' . $code);
 
-        foreach ($expectedTokens as $expectedToken) {
-            list($expectedTokenType, $expectedTokenText) = $expectedToken;
-            $this->assertSame($expectedTokenType, $lexer->getNextToken($text));
-            $this->assertSame($expectedTokenText, $text);
+        $tokens = [];
+        while (0 !== $token = $lexer->getNextToken($text)) {
+            $tokens[] = [$token, $text];
         }
-        $this->assertSame(0, $lexer->getNextToken());
+        $this->assertSame($expectedTokens, $tokens);
     }
 
     /**
@@ -85,6 +85,29 @@ class EmulativeTest extends LexerTest
         $this->assertSame(0, $lexer->getNextToken());
     }
 
+    /**
+     * @dataProvider provideTestLexNewFeatures
+     */
+    public function testErrorAfterEmulation($code) {
+        $errorHandler = new ErrorHandler\Collecting;
+        $lexer = $this->getLexer([]);
+        $lexer->startLexing('<?php ' . $code . "\0", $errorHandler);
+
+        $errors = $errorHandler->getErrors();
+        $this->assertCount(1, $errors);
+
+        $error = $errors[0];
+        $this->assertSame('Unexpected null byte', $error->getRawMessage());
+
+        $attrs = $error->getAttributes();
+        $expPos = strlen('<?php ' . $code);
+        $expLine = 1 + substr_count('<?php ' . $code, "\n");
+        $this->assertSame($expPos, $attrs['startFilePos']);
+        $this->assertSame($expPos, $attrs['endFilePos']);
+        $this->assertSame($expLine, $attrs['startLine']);
+        $this->assertSame($expLine, $attrs['endLine']);
+    }
+
     public function provideTestLexNewFeatures() {
         return [
             ['yield from', [
@@ -128,6 +151,43 @@ class EmulativeTest extends LexerTest
                 [Tokens::T_END_HEREDOC, 'NOWDOC'],
                 [ord(';'), ';'],
             ]],
+
+            // Flexible heredoc/nowdoc
+            ["<<<LABEL\nLABEL,", [
+                [Tokens::T_START_HEREDOC, "<<<LABEL\n"],
+                [Tokens::T_END_HEREDOC, "LABEL"],
+                [ord(','), ','],
+            ]],
+            ["<<<LABEL\n    LABEL,", [
+                [Tokens::T_START_HEREDOC, "<<<LABEL\n"],
+                [Tokens::T_END_HEREDOC, "    LABEL"],
+                [ord(','), ','],
+            ]],
+            ["<<<LABEL\n    Foo\n  LABEL;", [
+                [Tokens::T_START_HEREDOC, "<<<LABEL\n"],
+                [Tokens::T_ENCAPSED_AND_WHITESPACE, "    Foo\n"],
+                [Tokens::T_END_HEREDOC, "  LABEL"],
+                [ord(';'), ';'],
+            ]],
+            ["<<<A\n A,<<<A\n A,", [
+                [Tokens::T_START_HEREDOC, "<<<A\n"],
+                [Tokens::T_END_HEREDOC, " A"],
+                [ord(','), ','],
+                [Tokens::T_START_HEREDOC, "<<<A\n"],
+                [Tokens::T_END_HEREDOC, " A"],
+                [ord(','), ','],
+            ]],
+            ["<<<LABEL\nLABELNOPE\nLABEL\n", [
+                [Tokens::T_START_HEREDOC, "<<<LABEL\n"],
+                [Tokens::T_ENCAPSED_AND_WHITESPACE, "LABELNOPE\n"],
+                [Tokens::T_END_HEREDOC, "LABEL"],
+            ]],
+            // Interpretation changed
+            ["<<<LABEL\n    LABEL\nLABEL\n", [
+                [Tokens::T_START_HEREDOC, "<<<LABEL\n"],
+                [Tokens::T_END_HEREDOC, "    LABEL"],
+                [Tokens::T_STRING, "LABEL"],
+            ]],
         ];
     }
 }

test/PhpParser/LexerTest.php

@@ -235,11 +235,9 @@ class LexerTest extends TestCase
         ];
     }
 
-    /**
-     * @expectedException \PhpParser\Error
-     * @expectedExceptionMessage __HALT_COMPILER must be followed by "();"
-     */
     public function testHandleHaltCompilerError() {
+        $this->expectException(Error::class);
+        $this->expectExceptionMessage('__HALT_COMPILER must be followed by "();"');
         $lexer = $this->getLexer();
         $lexer->startLexing('<?php ... __halt_compiler invalid ();');
 

test/PhpParser/Node/NameTest.php

@@ -51,35 +51,27 @@ class NameTest extends TestCase
         $this->assertNull($name->slice(-2, -2));
     }
 
-    /**
-     * @expectedException \OutOfBoundsException
-     * @expectedExceptionMessage Offset 4 is out of bounds
-     */
     public function testSliceOffsetTooLarge() {
+        $this->expectException(\OutOfBoundsException::class);
+        $this->expectExceptionMessage('Offset 4 is out of bounds');
         (new Name('foo\bar\baz'))->slice(4);
     }
 
-    /**
-     * @expectedException \OutOfBoundsException
-     * @expectedExceptionMessage Offset -4 is out of bounds
-     */
     public function testSliceOffsetTooSmall() {
+        $this->expectException(\OutOfBoundsException::class);
+        $this->expectExceptionMessage('Offset -4 is out of bounds');
         (new Name('foo\bar\baz'))->slice(-4);
     }
 
-    /**
-     * @expectedException \OutOfBoundsException
-     * @expectedExceptionMessage Length 4 is out of bounds
-     */
     public function testSliceLengthTooLarge() {
+        $this->expectException(\OutOfBoundsException::class);
+        $this->expectExceptionMessage('Length 4 is out of bounds');
         (new Name('foo\bar\baz'))->slice(0, 4);
     }
 
-    /**
-     * @expectedException \OutOfBoundsException
-     * @expectedExceptionMessage Length -4 is out of bounds
-     */
     public function testSliceLengthTooSmall() {
+        $this->expectException(\OutOfBoundsException::class);
+        $this->expectExceptionMessage('Length -4 is out of bounds');
         (new Name('foo\bar\baz'))->slice(0, -4);
     }
 
@@ -131,27 +123,21 @@ class NameTest extends TestCase
         $this->assertSame('namespace\foo', $name->toCodeString());
     }
 
-    /**
-     * @expectedException \InvalidArgumentException
-     * @expectedExceptionMessage Expected string, array of parts or Name instance
-     */
     public function testInvalidArg() {
+        $this->expectException(\InvalidArgumentException::class);
+        $this->expectExceptionMessage('Expected string, array of parts or Name instance');
         Name::concat('foo', new \stdClass);
     }
 
-    /**
-     * @expectedException \InvalidArgumentException
-     * @expectedExceptionMessage Name cannot be empty
-     */
     public function testInvalidEmptyString() {
+        $this->expectException(\InvalidArgumentException::class);
+        $this->expectExceptionMessage('Name cannot be empty');
         new Name('');
     }
 
-    /**
-     * @expectedException \InvalidArgumentException
-     * @expectedExceptionMessage Name cannot be empty
-     */
     public function testInvalidEmptyArray() {
+        $this->expectException(\InvalidArgumentException::class);
+        $this->expectExceptionMessage('Name cannot be empty');
         new Name([]);
     }
 

test/PhpParser/NodeDumperTest.php

@@ -98,11 +98,9 @@ OUT;
         $this->assertSame($this->canonicalize($expected), $this->canonicalize($dump));
     }
 
-    /**
-     * @expectedException        \InvalidArgumentException
-     * @expectedExceptionMessage Can only dump nodes and arrays.
-     */
     public function testError() {
+        $this->expectException(\InvalidArgumentException::class);
+        $this->expectExceptionMessage('Can only dump nodes and arrays.');
         $dumper = new NodeDumper;
         $dumper->dump(new \stdClass);
     }

test/PhpParser/NodeTraverserTest.php

@@ -117,11 +117,9 @@ class NodeTraverserTest extends TestCase
         );
     }
 
-    /**
-     * @expectedException \LogicException
-     * @expectedExceptionMessage Invalid node structure: Contains nested arrays
-     */
     public function testInvalidDeepArray() {
+        $this->expectException(\LogicException::class);
+        $this->expectExceptionMessage('Invalid node structure: Contains nested arrays');
         $strNode = new String_('Foo');
         $stmts = [[[$strNode]]];
 
@@ -167,6 +165,42 @@ class NodeTraverserTest extends TestCase
         $this->assertEquals($stmts, $traverser->traverse($stmts));
     }
 
+    public function testDontTraverseCurrentAndChildren() {
+        // print 'str'; -($foo * $foo);
+        $strNode = new String_('str');
+        $printNode = new Expr\Print_($strNode);
+        $varNode = new Expr\Variable('foo');
+        $mulNode = new Expr\BinaryOp\Mul($varNode, $varNode);
+        $divNode = new Expr\BinaryOp\Div($varNode, $varNode);
+        $negNode = new Expr\UnaryMinus($mulNode);
+        $stmts = [$printNode, $negNode];
+
+        $visitor1 = $this->getMockBuilder(NodeVisitor::class)->getMock();
+        $visitor2 = $this->getMockBuilder(NodeVisitor::class)->getMock();
+
+        $visitor1->expects($this->at(1))->method('enterNode')->with($printNode)
+            ->will($this->returnValue(NodeTraverser::DONT_TRAVERSE_CURRENT_AND_CHILDREN));
+        $visitor1->expects($this->at(2))->method('leaveNode')->with($printNode);
+
+        $visitor1->expects($this->at(3))->method('enterNode')->with($negNode);
+        $visitor2->expects($this->at(1))->method('enterNode')->with($negNode);
+
+        $visitor1->expects($this->at(4))->method('enterNode')->with($mulNode)
+            ->will($this->returnValue(NodeTraverser::DONT_TRAVERSE_CURRENT_AND_CHILDREN));
+        $visitor1->expects($this->at(5))->method('leaveNode')->with($mulNode)->willReturn($divNode);
+
+        $visitor1->expects($this->at(6))->method('leaveNode')->with($negNode);
+        $visitor2->expects($this->at(2))->method('leaveNode')->with($negNode);
+
+        $traverser = new NodeTraverser;
+        $traverser->addVisitor($visitor1);
+        $traverser->addVisitor($visitor2);
+
+        $resultStmts = $traverser->traverse($stmts);
+
+        $this->assertInstanceOf(Expr\BinaryOp\Div::class, $resultStmts[1]->expr);
+    }
+
     public function testStopTraversal() {
         $varNode1 = new Expr\Variable('a');
         $varNode2 = new Expr\Variable('b');
@@ -258,7 +292,7 @@ class NodeTraverserTest extends TestCase
         $this->expectException(\LogicException::class);
         $this->expectExceptionMessage($message);
 
-        $stmts = [new Node\Expr\UnaryMinus(new Node\Scalar\LNumber(42))];
+        $stmts = [new Node\Stmt\Expression(new Node\Scalar\LNumber(42))];
 
         $traverser = new NodeTraverser();
         $traverser->addVisitor($visitor);
@@ -268,19 +302,19 @@ class NodeTraverserTest extends TestCase
     public function provideTestInvalidReturn() {
         $visitor1 = $this->getMockBuilder(NodeVisitor::class)->getMock();
         $visitor1->expects($this->at(1))->method('enterNode')
-            ->will($this->returnValue('foobar'));
+            ->willReturn('foobar');
 
         $visitor2 = $this->getMockBuilder(NodeVisitor::class)->getMock();
         $visitor2->expects($this->at(2))->method('enterNode')
-            ->will($this->returnValue('foobar'));
+            ->willReturn('foobar');
 
         $visitor3 = $this->getMockBuilder(NodeVisitor::class)->getMock();
         $visitor3->expects($this->at(3))->method('leaveNode')
-            ->will($this->returnValue('foobar'));
+            ->willReturn('foobar');
 
         $visitor4 = $this->getMockBuilder(NodeVisitor::class)->getMock();
         $visitor4->expects($this->at(4))->method('leaveNode')
-            ->will($this->returnValue('foobar'));
+            ->willReturn('foobar');
 
         $visitor5 = $this->getMockBuilder(NodeVisitor::class)->getMock();
         $visitor5->expects($this->at(3))->method('leaveNode')
@@ -290,6 +324,14 @@ class NodeTraverserTest extends TestCase
         $visitor6->expects($this->at(4))->method('leaveNode')
             ->willReturn(false);
 
+        $visitor7 = $this->getMockBuilder(NodeVisitor::class)->getMock();
+        $visitor7->expects($this->at(1))->method('enterNode')
+            ->willReturn(new Node\Scalar\LNumber(42));
+
+        $visitor8 = $this->getMockBuilder(NodeVisitor::class)->getMock();
+        $visitor8->expects($this->at(2))->method('enterNode')
+            ->willReturn(new Node\Stmt\Return_());
+
         return [
             [$visitor1, 'enterNode() returned invalid value of type string'],
             [$visitor2, 'enterNode() returned invalid value of type string'],
@@ -297,6 +339,8 @@ class NodeTraverserTest extends TestCase
             [$visitor4, 'leaveNode() returned invalid value of type string'],
             [$visitor5, 'leaveNode() may only return an array if the parent structure is an array'],
             [$visitor6, 'bool(false) return from leaveNode() no longer supported. Return NodeTraverser::REMOVE_NODE instead'],
+            [$visitor7, 'Trying to replace statement (Stmt_Expression) with expression (Scalar_LNumber). Are you missing a Stmt_Expression wrapper?'],
+            [$visitor8, 'Trying to replace expression (Scalar_LNumber) with statement (Stmt_Return)'],
         ];
     }
 }

test/PhpParser/NodeVisitor/FirstFindingVisitorTest.php

@@ -34,6 +34,6 @@ class FirstFindingVisitorTest extends TestCase
         $stmts = [new Node\Stmt\Expression($assign)];
 
         $traverser->traverse($stmts);
-        $this->assertSame(null, $visitor->getFoundNode());
+        $this->assertNull($visitor->getFoundNode());
     }
 }

test/PhpParser/ParserTest.php

@@ -13,29 +13,23 @@ abstract class ParserTest extends TestCase
     /** @returns Parser */
     abstract protected function getParser(Lexer $lexer);
 
-    /**
-     * @expectedException \PhpParser\Error
-     * @expectedExceptionMessage Syntax error, unexpected EOF on line 1
-     */
     public function testParserThrowsSyntaxError() {
+        $this->expectException(Error::class);
+        $this->expectExceptionMessage('Syntax error, unexpected EOF on line 1');
         $parser = $this->getParser(new Lexer());
         $parser->parse('<?php foo');
     }
 
-    /**
-     * @expectedException \PhpParser\Error
-     * @expectedExceptionMessage Cannot use foo as self because 'self' is a special class name on line 1
-     */
     public function testParserThrowsSpecialError() {
+        $this->expectException(Error::class);
+        $this->expectExceptionMessage('Cannot use foo as self because \'self\' is a special class name on line 1');
         $parser = $this->getParser(new Lexer());
         $parser->parse('<?php use foo as self;');
     }
 
-    /**
-     * @expectedException \PhpParser\Error
-     * @expectedExceptionMessage Unterminated comment on line 1
-     */
     public function testParserThrowsLexerError() {
+        $this->expectException(Error::class);
+        $this->expectExceptionMessage('Unterminated comment on line 1');
         $parser = $this->getParser(new Lexer());
         $parser->parse('<?php /*');
     }
@@ -109,11 +103,9 @@ EOC;
         ], $var->getAttributes());
     }
 
-    /**
-     * @expectedException \RangeException
-     * @expectedExceptionMessage The lexer returned an invalid token (id=999, value=foobar)
-     */
     public function testInvalidToken() {
+        $this->expectException(\RangeException::class);
+        $this->expectExceptionMessage('The lexer returned an invalid token (id=999, value=foobar)');
         $lexer = new InvalidTokenLexer;
         $parser = $this->getParser($lexer);
         $parser->parse('dummy');
@@ -123,7 +115,7 @@ EOC;
      * @dataProvider provideTestExtraAttributes
      */
     public function testExtraAttributes($code, $expectedAttributes) {
-        $parser = $this->getParser(new Lexer);
+        $parser = $this->getParser(new Lexer\Emulative);
         $stmts = $parser->parse("<?php $code;");
         $node = $stmts[0] instanceof Stmt\Expression ? $stmts[0]->expr : $stmts[0];
         $attributes = $node->getAttributes();
@@ -152,17 +144,20 @@ EOC;
             ['"foo$bar"', ['kind' => String_::KIND_DOUBLE_QUOTED]],
             ['b"foo$bar"', ['kind' => String_::KIND_DOUBLE_QUOTED]],
             ['B"foo$bar"', ['kind' => String_::KIND_DOUBLE_QUOTED]],
-            ["<<<'STR'\nSTR\n", ['kind' => String_::KIND_NOWDOC, 'docLabel' => 'STR']],
-            ["<<<STR\nSTR\n", ['kind' => String_::KIND_HEREDOC, 'docLabel' => 'STR']],
-            ["<<<\"STR\"\nSTR\n", ['kind' => String_::KIND_HEREDOC, 'docLabel' => 'STR']],
-            ["b<<<'STR'\nSTR\n", ['kind' => String_::KIND_NOWDOC, 'docLabel' => 'STR']],
-            ["B<<<'STR'\nSTR\n", ['kind' => String_::KIND_NOWDOC, 'docLabel' => 'STR']],
-            ["<<< \t 'STR'\nSTR\n", ['kind' => String_::KIND_NOWDOC, 'docLabel' => 'STR']],
-            ["<<<'\xff'\n\xff\n", ['kind' => String_::KIND_NOWDOC, 'docLabel' => "\xff"]],
-            ["<<<\"STR\"\n\$a\nSTR\n", ['kind' => String_::KIND_HEREDOC, 'docLabel' => 'STR']],
-            ["b<<<\"STR\"\n\$a\nSTR\n", ['kind' => String_::KIND_HEREDOC, 'docLabel' => 'STR']],
-            ["B<<<\"STR\"\n\$a\nSTR\n", ['kind' => String_::KIND_HEREDOC, 'docLabel' => 'STR']],
-            ["<<< \t \"STR\"\n\$a\nSTR\n", ['kind' => String_::KIND_HEREDOC, 'docLabel' => 'STR']],
+            ["<<<'STR'\nSTR\n", ['kind' => String_::KIND_NOWDOC, 'docLabel' => 'STR', 'docIndentation' => '']],
+            ["<<<STR\nSTR\n", ['kind' => String_::KIND_HEREDOC, 'docLabel' => 'STR', 'docIndentation' => '']],
+            ["<<<\"STR\"\nSTR\n", ['kind' => String_::KIND_HEREDOC, 'docLabel' => 'STR', 'docIndentation' => '']],
+            ["b<<<'STR'\nSTR\n", ['kind' => String_::KIND_NOWDOC, 'docLabel' => 'STR', 'docIndentation' => '']],
+            ["B<<<'STR'\nSTR\n", ['kind' => String_::KIND_NOWDOC, 'docLabel' => 'STR', 'docIndentation' => '']],
+            ["<<< \t 'STR'\nSTR\n", ['kind' => String_::KIND_NOWDOC, 'docLabel' => 'STR', 'docIndentation' => '']],
+            ["<<<'\xff'\n\xff\n", ['kind' => String_::KIND_NOWDOC, 'docLabel' => "\xff", 'docIndentation' => '']],
+            ["<<<\"STR\"\n\$a\nSTR\n", ['kind' => String_::KIND_HEREDOC, 'docLabel' => 'STR', 'docIndentation' => '']],
+            ["b<<<\"STR\"\n\$a\nSTR\n", ['kind' => String_::KIND_HEREDOC, 'docLabel' => 'STR', 'docIndentation' => '']],
+            ["B<<<\"STR\"\n\$a\nSTR\n", ['kind' => String_::KIND_HEREDOC, 'docLabel' => 'STR', 'docIndentation' => '']],
+            ["<<< \t \"STR\"\n\$a\nSTR\n", ['kind' => String_::KIND_HEREDOC, 'docLabel' => 'STR', 'docIndentation' => '']],
+            ["<<<STR\n    STR\n", ['kind' => String_::KIND_HEREDOC, 'docLabel' => 'STR', 'docIndentation' => '    ']],
+            ["<<<STR\n\tSTR\n", ['kind' => String_::KIND_HEREDOC, 'docLabel' => 'STR', 'docIndentation' => "\t"]],
+            ["<<<'STR'\n    Foo\n  STR\n", ['kind' => String_::KIND_NOWDOC, 'docLabel' => 'STR', 'docIndentation' => '  ']],
             ["die", ['kind' => Expr\Exit_::KIND_DIE]],
             ["die('done')", ['kind' => Expr\Exit_::KIND_DIE]],
             ["exit", ['kind' => Expr\Exit_::KIND_EXIT]],

test/PhpParser/PrettyPrinterTest.php

@@ -184,11 +184,9 @@ class PrettyPrinterTest extends CodeTestAbstract
         ];
     }
 
-    /**
-     * @expectedException \LogicException
-     * @expectedExceptionMessage Cannot pretty-print AST with Error nodes
-     */
     public function testPrettyPrintWithError() {
+        $this->expectException(\LogicException::class);
+        $this->expectExceptionMessage('Cannot pretty-print AST with Error nodes');
         $stmts = [new Stmt\Expression(
             new Expr\PropertyFetch(new Expr\Variable('a'), new Expr\Error())
         )];
@@ -196,11 +194,9 @@ class PrettyPrinterTest extends CodeTestAbstract
         $prettyPrinter->prettyPrint($stmts);
     }
 
-    /**
-     * @expectedException \LogicException
-     * @expectedExceptionMessage Cannot pretty-print AST with Error nodes
-     */
     public function testPrettyPrintWithErrorInClassConstFetch() {
+        $this->expectException(\LogicException::class);
+        $this->expectExceptionMessage('Cannot pretty-print AST with Error nodes');
         $stmts = [new Stmt\Expression(
             new Expr\ClassConstFetch(new Name('Foo'), new Expr\Error())
         )];
@@ -208,6 +204,14 @@ class PrettyPrinterTest extends CodeTestAbstract
         $prettyPrinter->prettyPrint($stmts);
     }
 
+    public function testPrettyPrintEncapsedStringPart() {
+        $this->expectException(\LogicException::class);
+        $this->expectExceptionMessage('Cannot directly print EncapsedStringPart');
+        $expr = new Node\Scalar\EncapsedStringPart('foo');
+        $prettyPrinter = new PrettyPrinter\Standard;
+        $prettyPrinter->prettyPrintExpr($expr);
+    }
+
     /**
      * @dataProvider provideTestFormatPreservingPrint
      * @covers \PhpParser\PrettyPrinter\Standard<extended>

test/code/formatPreservation/nopCommentAtEnd.test

@@ -0,0 +1,11 @@
+Nop statement with comment at end (#513)
+-----
+<?php
+$foo;
+$bar;
+-----
+$stmts[1] = new Stmt\Nop(['comments' => [new Comment('//Some comment here')]]);
+-----
+<?php
+$foo;
+//Some comment here

test/code/parser/commentAtEndOfClass.test

@@ -0,0 +1,37 @@
+Comment at end of class (#509)
+-----
+<?php
+class MyClass {
+    protected $a;
+    // my comment
+}
+-----
+array(
+    0: Stmt_Class(
+        flags: 0
+        name: Identifier(
+            name: MyClass
+        )
+        extends: null
+        implements: array(
+        )
+        stmts: array(
+            0: Stmt_Property(
+                flags: MODIFIER_PROTECTED (2)
+                props: array(
+                    0: Stmt_PropertyProperty(
+                        name: VarLikeIdentifier(
+                            name: a
+                        )
+                        default: null
+                    )
+                )
+            )
+            1: Stmt_Nop(
+                comments: array(
+                    0: // my comment
+                )
+            )
+        )
+    )
+)

test/code/parser/errorHandling/recovery.test

@@ -613,11 +613,8 @@ A trailing comma is not allowed here from 11:25 to 11:25
 A trailing comma is not allowed here from 13:17 to 13:17
 A trailing comma is not allowed here from 14:14 to 14:14
 A trailing comma is not allowed here from 16:22 to 16:22
-A trailing comma is not allowed here from 18:9 to 18:9
-A trailing comma is not allowed here from 19:9 to 19:9
 A trailing comma is not allowed here from 21:13 to 21:13
 A trailing comma is not allowed here from 23:16 to 23:16
-A trailing comma is not allowed here from 24:7 to 24:7
 A trailing comma is not allowed here from 25:10 to 25:10
 A trailing comma is not allowed here from 26:10 to 26:10
 A trailing comma is not allowed here from 27:8 to 27:8
@@ -955,4 +952,425 @@ array(
             )
         )
     )
+)
+-----
+<?php
+
+class Foo {
+    publi $foo;
+    public $bar;
+}
+-----
+!!php7
+Syntax error, unexpected T_STRING from 4:5 to 4:9
+array(
+    0: Stmt_Class(
+        flags: 0
+        name: Identifier(
+            name: Foo
+        )
+        extends: null
+        implements: array(
+        )
+        stmts: array(
+            0: Stmt_Property(
+                flags: MODIFIER_PUBLIC (1)
+                props: array(
+                    0: Stmt_PropertyProperty(
+                        name: VarLikeIdentifier(
+                            name: bar
+                        )
+                        default: null
+                    )
+                )
+            )
+        )
+    )
+)
+-----
+<?php
+
+foreach ($foo) { $bar; }
+foreach ($foo as ) { $bar; }
+-----
+!!php7
+Syntax error, unexpected ')' from 3:14 to 3:14
+Syntax error, unexpected ')' from 4:18 to 4:18
+array(
+    0: Stmt_Foreach(
+        expr: Expr_Variable(
+            name: foo
+        )
+        keyVar: null
+        byRef: false
+        valueVar: Expr_Error(
+        )
+        stmts: array(
+            0: Stmt_Expression(
+                expr: Expr_Variable(
+                    name: bar
+                )
+            )
+        )
+    )
+    1: Stmt_Foreach(
+        expr: Expr_Variable(
+            name: foo
+        )
+        keyVar: null
+        byRef: false
+        valueVar: Expr_Error(
+        )
+        stmts: array(
+            0: Stmt_Expression(
+                expr: Expr_Variable(
+                    name: bar
+                )
+            )
+        )
+    )
+)
+-----
+<?php
+
+function foo(Type) {
+    $foo;
+}
+
+function foo(Type1 $foo, Type2) {
+    $bar;
+}
+
+function foo(...) {
+    $baz;
+}
+
+function foo(&) {
+    $qux;
+}
+
+function foo(Bar)
+
+class Bar {
+    function foo(Baz)
+}
+
+function(Foo);
+-----
+!!php7
+Syntax error, unexpected ')', expecting T_VARIABLE from 3:18 to 3:18
+Syntax error, unexpected ')', expecting T_VARIABLE from 7:31 to 7:31
+Syntax error, unexpected ')', expecting T_VARIABLE from 11:17 to 11:17
+Syntax error, unexpected ')', expecting T_VARIABLE from 15:15 to 15:15
+Syntax error, unexpected ')', expecting T_VARIABLE from 19:17 to 19:17
+Syntax error, unexpected ')', expecting T_VARIABLE from 22:21 to 22:21
+Syntax error, unexpected ')', expecting T_VARIABLE from 25:13 to 25:13
+array(
+    0: Stmt_Function(
+        byRef: false
+        name: Identifier(
+            name: foo
+        )
+        params: array(
+            0: Param(
+                type: Name(
+                    parts: array(
+                        0: Type
+                    )
+                )
+                byRef: false
+                variadic: false
+                var: Expr_Error(
+                )
+                default: null
+            )
+        )
+        returnType: null
+        stmts: array(
+            0: Stmt_Expression(
+                expr: Expr_Variable(
+                    name: foo
+                )
+            )
+        )
+    )
+    1: Stmt_Function(
+        byRef: false
+        name: Identifier(
+            name: foo
+        )
+        params: array(
+            0: Param(
+                type: Name(
+                    parts: array(
+                        0: Type1
+                    )
+                )
+                byRef: false
+                variadic: false
+                var: Expr_Variable(
+                    name: foo
+                )
+                default: null
+            )
+            1: Param(
+                type: Name(
+                    parts: array(
+                        0: Type2
+                    )
+                )
+                byRef: false
+                variadic: false
+                var: Expr_Error(
+                )
+                default: null
+            )
+        )
+        returnType: null
+        stmts: array(
+            0: Stmt_Expression(
+                expr: Expr_Variable(
+                    name: bar
+                )
+            )
+        )
+    )
+    2: Stmt_Function(
+        byRef: false
+        name: Identifier(
+            name: foo
+        )
+        params: array(
+            0: Param(
+                type: null
+                byRef: false
+                variadic: true
+                var: Expr_Error(
+                )
+                default: null
+            )
+        )
+        returnType: null
+        stmts: array(
+            0: Stmt_Expression(
+                expr: Expr_Variable(
+                    name: baz
+                )
+            )
+        )
+    )
+    3: Stmt_Function(
+        byRef: false
+        name: Identifier(
+            name: foo
+        )
+        params: array(
+            0: Param(
+                type: null
+                byRef: true
+                variadic: false
+                var: Expr_Error(
+                )
+                default: null
+            )
+        )
+        returnType: null
+        stmts: array(
+            0: Stmt_Expression(
+                expr: Expr_Variable(
+                    name: qux
+                )
+            )
+        )
+    )
+    4: Stmt_Function(
+        byRef: false
+        name: Identifier(
+            name: foo
+        )
+        params: array(
+            0: Param(
+                type: Name(
+                    parts: array(
+                        0: Bar
+                    )
+                )
+                byRef: false
+                variadic: false
+                var: Expr_Error(
+                )
+                default: null
+            )
+        )
+        returnType: null
+        stmts: array(
+        )
+    )
+    5: Stmt_Class(
+        flags: 0
+        name: Identifier(
+            name: Bar
+        )
+        extends: null
+        implements: array(
+        )
+        stmts: array(
+            0: Stmt_ClassMethod(
+                flags: 0
+                byRef: false
+                name: Identifier(
+                    name: foo
+                )
+                params: array(
+                    0: Param(
+                        type: Name(
+                            parts: array(
+                                0: Baz
+                            )
+                        )
+                        byRef: false
+                        variadic: false
+                        var: Expr_Error(
+                        )
+                        default: null
+                    )
+                )
+                returnType: null
+                stmts: array(
+                )
+            )
+        )
+    )
+    6: Stmt_Expression(
+        expr: Expr_Closure(
+            static: false
+            byRef: false
+            params: array(
+                0: Param(
+                    type: Name(
+                        parts: array(
+                            0: Foo
+                        )
+                    )
+                    byRef: false
+                    variadic: false
+                    var: Expr_Error(
+                    )
+                    default: null
+                )
+            )
+            uses: array(
+            )
+            returnType: null
+            stmts: array(
+            )
+        )
+    )
+)
+-----
+<?php
+$array = [
+    $this->value $oopsAnotherValue->get()
+];
+$array = [
+    $value $oopsAnotherValue
+];
+$array = [
+    'key' => $value $oopsAnotherValue
+];
+-----
+!!php7
+Syntax error, unexpected T_VARIABLE, expecting ',' or ')' or ']' from 3:18 to 3:34
+Syntax error, unexpected T_VARIABLE, expecting ',' or ')' or ']' from 6:12 to 6:28
+Syntax error, unexpected T_VARIABLE, expecting ',' or ')' or ']' from 9:21 to 9:37
+array(
+    0: Stmt_Expression(
+        expr: Expr_Assign(
+            var: Expr_Variable(
+                name: array
+            )
+            expr: Expr_Array(
+                items: array(
+                    0: Expr_ArrayItem(
+                        key: null
+                        value: Expr_PropertyFetch(
+                            var: Expr_Variable(
+                                name: this
+                            )
+                            name: Identifier(
+                                name: value
+                            )
+                        )
+                        byRef: false
+                    )
+                    1: Expr_ArrayItem(
+                        key: null
+                        value: Expr_MethodCall(
+                            var: Expr_Variable(
+                                name: oopsAnotherValue
+                            )
+                            name: Identifier(
+                                name: get
+                            )
+                            args: array(
+                            )
+                        )
+                        byRef: false
+                    )
+                )
+            )
+        )
+    )
+    1: Stmt_Expression(
+        expr: Expr_Assign(
+            var: Expr_Variable(
+                name: array
+            )
+            expr: Expr_Array(
+                items: array(
+                    0: Expr_ArrayItem(
+                        key: null
+                        value: Expr_Variable(
+                            name: value
+                        )
+                        byRef: false
+                    )
+                    1: Expr_ArrayItem(
+                        key: null
+                        value: Expr_Variable(
+                            name: oopsAnotherValue
+                        )
+                        byRef: false
+                    )
+                )
+            )
+        )
+    )
+    2: Stmt_Expression(
+        expr: Expr_Assign(
+            var: Expr_Variable(
+                name: array
+            )
+            expr: Expr_Array(
+                items: array(
+                    0: Expr_ArrayItem(
+                        key: Scalar_String(
+                            value: key
+                        )
+                        value: Expr_Variable(
+                            name: value
+                        )
+                        byRef: false
+                    )
+                    1: Expr_ArrayItem(
+                        key: null
+                        value: Expr_Variable(
+                            name: oopsAnotherValue
+                        )
+                        byRef: false
+                    )
+                )
+            )
+        )
+    )
 )

test/code/parser/expr/listReferences.test

@@ -0,0 +1,88 @@
+List reference assignments (PHP 7.3)
+-----
+<?php
+
+list(&$v) = $x;
+list('k' => &$v) = $x;
+[&$v] = $x;
+['k' => &$v] = $x;
+-----
+!!php7
+array(
+    0: Stmt_Expression(
+        expr: Expr_Assign(
+            var: Expr_List(
+                items: array(
+                    0: Expr_ArrayItem(
+                        key: null
+                        value: Expr_Variable(
+                            name: v
+                        )
+                        byRef: true
+                    )
+                )
+            )
+            expr: Expr_Variable(
+                name: x
+            )
+        )
+    )
+    1: Stmt_Expression(
+        expr: Expr_Assign(
+            var: Expr_List(
+                items: array(
+                    0: Expr_ArrayItem(
+                        key: Scalar_String(
+                            value: k
+                        )
+                        value: Expr_Variable(
+                            name: v
+                        )
+                        byRef: true
+                    )
+                )
+            )
+            expr: Expr_Variable(
+                name: x
+            )
+        )
+    )
+    2: Stmt_Expression(
+        expr: Expr_Assign(
+            var: Expr_Array(
+                items: array(
+                    0: Expr_ArrayItem(
+                        key: null
+                        value: Expr_Variable(
+                            name: v
+                        )
+                        byRef: true
+                    )
+                )
+            )
+            expr: Expr_Variable(
+                name: x
+            )
+        )
+    )
+    3: Stmt_Expression(
+        expr: Expr_Assign(
+            var: Expr_Array(
+                items: array(
+                    0: Expr_ArrayItem(
+                        key: Scalar_String(
+                            value: k
+                        )
+                        value: Expr_Variable(
+                            name: v
+                        )
+                        byRef: true
+                    )
+                )
+            )
+            expr: Expr_Variable(
+                name: x
+            )
+        )
+    )
+)

test/code/parser/expr/trailingCommas.test

@@ -0,0 +1,140 @@
+PHP 7.3 trailing comma additions
+-----
+<?php
+
+foo($a, $b, );
+$foo->bar($a, $b, );
+Foo::bar($a, $b, );
+new Foo($a, $b, );
+unset($a, $b, );
+isset($a, $b, );
+-----
+!!php7
+array(
+    0: Stmt_Expression(
+        expr: Expr_FuncCall(
+            name: Name(
+                parts: array(
+                    0: foo
+                )
+            )
+            args: array(
+                0: Arg(
+                    value: Expr_Variable(
+                        name: a
+                    )
+                    byRef: false
+                    unpack: false
+                )
+                1: Arg(
+                    value: Expr_Variable(
+                        name: b
+                    )
+                    byRef: false
+                    unpack: false
+                )
+            )
+        )
+    )
+    1: Stmt_Expression(
+        expr: Expr_MethodCall(
+            var: Expr_Variable(
+                name: foo
+            )
+            name: Identifier(
+                name: bar
+            )
+            args: array(
+                0: Arg(
+                    value: Expr_Variable(
+                        name: a
+                    )
+                    byRef: false
+                    unpack: false
+                )
+                1: Arg(
+                    value: Expr_Variable(
+                        name: b
+                    )
+                    byRef: false
+                    unpack: false
+                )
+            )
+        )
+    )
+    2: Stmt_Expression(
+        expr: Expr_StaticCall(
+            class: Name(
+                parts: array(
+                    0: Foo
+                )
+            )
+            name: Identifier(
+                name: bar
+            )
+            args: array(
+                0: Arg(
+                    value: Expr_Variable(
+                        name: a
+                    )
+                    byRef: false
+                    unpack: false
+                )
+                1: Arg(
+                    value: Expr_Variable(
+                        name: b
+                    )
+                    byRef: false
+                    unpack: false
+                )
+            )
+        )
+    )
+    3: Stmt_Expression(
+        expr: Expr_New(
+            class: Name(
+                parts: array(
+                    0: Foo
+                )
+            )
+            args: array(
+                0: Arg(
+                    value: Expr_Variable(
+                        name: a
+                    )
+                    byRef: false
+                    unpack: false
+                )
+                1: Arg(
+                    value: Expr_Variable(
+                        name: b
+                    )
+                    byRef: false
+                    unpack: false
+                )
+            )
+        )
+    )
+    4: Stmt_Unset(
+        vars: array(
+            0: Expr_Variable(
+                name: a
+            )
+            1: Expr_Variable(
+                name: b
+            )
+        )
+    )
+    5: Stmt_Expression(
+        expr: Expr_Isset(
+            vars: array(
+                0: Expr_Variable(
+                    name: a
+                )
+                1: Expr_Variable(
+                    name: b
+                )
+            )
+        )
+    )
+)

test/code/parser/scalar/flexibleDocString.test

@@ -0,0 +1,359 @@
+Flexible heredoc/nowdoc (PHP 7.3)
+-----
+<?php
+
+$ary = [
+    <<<FOO
+Test
+FOO,
+    <<<'BAR'
+    Test
+    BAR,
+];
+
+<<<'END'
+ END;
+
+<<<END
+
+  END;
+
+<<<END
+@@{ " " }@@
+  END;
+
+<<<'END'
+     a
+    b
+
+   c
+
+  d
+ e
+ END;
+
+<<<END
+	    a
+	   b
+	  $test
+	 d
+	e
+	END;
+
+<<<'END'
+
+    a
+
+   b
+
+  c
+
+ d
+
+e
+
+END;
+
+<<<END
+	a\r\n
+\ta\n
+   b\r\n
+  $test\n
+ d\r\n
+e\n
+END;
+
+<<<BAR
+ $one-
+ BAR;
+
+<<<BAR
+ $two -
+ BAR;
+
+<<<BAR
+ $three	-
+ BAR;
+
+<<<BAR
+ $four-$four
+ BAR;
+
+<<<BAR
+ $five-$five-
+ BAR;
+
+<<<BAR
+ $six-$six-$six
+ BAR;
+
+<<<BAR
+ $seven
+ -
+ BAR;
+
+<<<BAR
+ $eight
+  -
+ BAR;
+
+<<<BAR
+$nine
+BAR;
+
+<<<BAR
+ -
+ BAR;
+
+<<<BAR
+  -
+ BAR;
+-----
+array(
+    0: Stmt_Expression(
+        expr: Expr_Assign(
+            var: Expr_Variable(
+                name: ary
+            )
+            expr: Expr_Array(
+                items: array(
+                    0: Expr_ArrayItem(
+                        key: null
+                        value: Scalar_String(
+                            value: Test
+                        )
+                        byRef: false
+                    )
+                    1: Expr_ArrayItem(
+                        key: null
+                        value: Scalar_String(
+                            value: Test
+                        )
+                        byRef: false
+                    )
+                )
+            )
+        )
+    )
+    1: Stmt_Expression(
+        expr: Scalar_String(
+            value:
+        )
+    )
+    2: Stmt_Expression(
+        expr: Scalar_String(
+            value:
+        )
+    )
+    3: Stmt_Expression(
+        expr: Scalar_String(
+            value:
+        )
+    )
+    4: Stmt_Expression(
+        expr: Scalar_String(
+            value:     a
+           b
+
+          c
+
+         d
+        e
+        )
+    )
+    5: Stmt_Expression(
+        expr: Scalar_Encapsed(
+            parts: array(
+                0: Scalar_EncapsedStringPart(
+                    value:     a
+                   b
+
+                )
+                1: Expr_Variable(
+                    name: test
+                )
+                2: Scalar_EncapsedStringPart(
+                    value:
+                 d
+                e
+                )
+            )
+        )
+    )
+    6: Stmt_Expression(
+        expr: Scalar_String(
+            value:
+            a
+
+           b
+
+          c
+
+         d
+
+        e
+
+        )
+    )
+    7: Stmt_Expression(
+        expr: Scalar_Encapsed(
+            parts: array(
+                0: Scalar_EncapsedStringPart(
+                    value: 	a
+
+                @@{ "\t" }@@a
+
+                   b
+
+
+                )
+                1: Expr_Variable(
+                    name: test
+                )
+                2: Scalar_EncapsedStringPart(
+                    value:
+
+                 d
+
+                e
+
+                )
+            )
+        )
+    )
+    8: Stmt_Expression(
+        expr: Scalar_Encapsed(
+            parts: array(
+                0: Expr_Variable(
+                    name: one
+                )
+                1: Scalar_EncapsedStringPart(
+                    value: -
+                )
+            )
+        )
+    )
+    9: Stmt_Expression(
+        expr: Scalar_Encapsed(
+            parts: array(
+                0: Expr_Variable(
+                    name: two
+                )
+                1: Scalar_EncapsedStringPart(
+                    value:  -
+                )
+            )
+        )
+    )
+    10: Stmt_Expression(
+        expr: Scalar_Encapsed(
+            parts: array(
+                0: Expr_Variable(
+                    name: three
+                )
+                1: Scalar_EncapsedStringPart(
+                    value: 	-
+                )
+            )
+        )
+    )
+    11: Stmt_Expression(
+        expr: Scalar_Encapsed(
+            parts: array(
+                0: Expr_Variable(
+                    name: four
+                )
+                1: Scalar_EncapsedStringPart(
+                    value: -
+                )
+                2: Expr_Variable(
+                    name: four
+                )
+            )
+        )
+    )
+    12: Stmt_Expression(
+        expr: Scalar_Encapsed(
+            parts: array(
+                0: Expr_Variable(
+                    name: five
+                )
+                1: Scalar_EncapsedStringPart(
+                    value: -
+                )
+                2: Expr_Variable(
+                    name: five
+                )
+                3: Scalar_EncapsedStringPart(
+                    value: -
+                )
+            )
+        )
+    )
+    13: Stmt_Expression(
+        expr: Scalar_Encapsed(
+            parts: array(
+                0: Expr_Variable(
+                    name: six
+                )
+                1: Scalar_EncapsedStringPart(
+                    value: -
+                )
+                2: Expr_Variable(
+                    name: six
+                )
+                3: Scalar_EncapsedStringPart(
+                    value: -
+                )
+                4: Expr_Variable(
+                    name: six
+                )
+            )
+        )
+    )
+    14: Stmt_Expression(
+        expr: Scalar_Encapsed(
+            parts: array(
+                0: Expr_Variable(
+                    name: seven
+                )
+                1: Scalar_EncapsedStringPart(
+                    value:
+                -
+                )
+            )
+        )
+    )
+    15: Stmt_Expression(
+        expr: Scalar_Encapsed(
+            parts: array(
+                0: Expr_Variable(
+                    name: eight
+                )
+                1: Scalar_EncapsedStringPart(
+                    value:
+                 -
+                )
+            )
+        )
+    )
+    16: Stmt_Expression(
+        expr: Scalar_Encapsed(
+            parts: array(
+                0: Expr_Variable(
+                    name: nine
+                )
+            )
+        )
+    )
+    17: Stmt_Expression(
+        expr: Scalar_String(
+            value: -
+        )
+    )
+    18: Stmt_Expression(
+        expr: Scalar_String(
+            value:  -
+        )
+    )
+)

test/code/parser/scalar/flexibleDocStringErrors.test

@@ -0,0 +1,117 @@
+Error conditions for flexible doc strings
+-----
+<?php
+
+<<<A
+ @@{ "\t" }@@A;
+
+<<<A
+    FooBar
+ @@{ "\t" }@@A;
+
+echo <<<END
+@@{ "\t" }@@   X
+@@{ "\t\t" }@@END;
+
+echo <<<END
+      a
+     b
+    c
+     END;
+
+<<<END
+\ta
+@@{ "\t" }@@END;
+
+<<<TEST
+  Foo
+$var
+  TEST;
+
+<<<TEST
+$var
+ TEST;
+
+echo <<<END
+ a
+$a
+ END;
+-----
+Invalid indentation - tabs and spaces cannot be mixed from 4:1 to 4:3
+Invalid indentation - tabs and spaces cannot be mixed from 8:1 to 8:3
+Invalid indentation - tabs and spaces cannot be mixed from 10:6 to 12:5
+Invalid body indentation level (expecting an indentation level of at least 5) from 14:6 to 18:8
+Invalid body indentation level (expecting an indentation level of at least 1) from 20:1 to 22:4
+Invalid body indentation level (expecting an indentation level of at least 2) from 25:1 to 26:0
+Invalid body indentation level (expecting an indentation level of at least 1) from 30:1 to 30:4
+Invalid body indentation level (expecting an indentation level of at least 1) from 34:1 to 35:0
+array(
+    0: Stmt_Expression(
+        expr: Scalar_String(
+            value:
+        )
+    )
+    1: Stmt_Expression(
+        expr: Scalar_String(
+            value:     FooBar
+        )
+    )
+    2: Stmt_Echo(
+        exprs: array(
+            0: Scalar_String(
+                value:   X
+            )
+        )
+    )
+    3: Stmt_Echo(
+        exprs: array(
+            0: Scalar_String(
+                value:  a
+            b
+            c
+            )
+        )
+    )
+    4: Stmt_Expression(
+        expr: Scalar_String(
+            value: 	a
+        )
+    )
+    5: Stmt_Expression(
+        expr: Scalar_Encapsed(
+            parts: array(
+                0: Scalar_EncapsedStringPart(
+                    value: Foo
+
+                )
+                1: Expr_Variable(
+                    name: var
+                )
+            )
+        )
+    )
+    6: Stmt_Expression(
+        expr: Scalar_Encapsed(
+            parts: array(
+                0: Expr_Variable(
+                    name: var
+                )
+            )
+        )
+    )
+    7: Stmt_Echo(
+        exprs: array(
+            0: Scalar_Encapsed(
+                parts: array(
+                    0: Scalar_EncapsedStringPart(
+                        value: a
+
+                    )
+                    1: Expr_Variable(
+                        name: a
+                    )
+                )
+            )
+        )
+    )
+)

test/code/parser/semiReserved.test

@@ -222,6 +222,11 @@ array(
                     )
                 )
             )
+            8: Stmt_Nop(
+                comments: array(
+                    0: // __halt_compiler does not work
+                )
+            )
         )
     )
     1: Stmt_Expression(

test/updateTests.php

@@ -21,7 +21,7 @@ foreach (filesInDir($dir, 'test') as $fileName => $code) {
     foreach ($tests as list($modeLine, list($input, $expected))) {
         $modes = null !== $modeLine ? array_fill_keys(explode(',', $modeLine), true) : [];
         list($parser5, $parser7) = $codeParsingTest->createParsers($modes);
-        $output = isset($modes['php5'])
+        list(, $output) = isset($modes['php5'])
             ? $codeParsingTest->getParseOutput($parser5, $input, $modes)
             : $codeParsingTest->getParseOutput($parser7, $input, $modes);
         $newTests[] = [$modeLine, [$input, $output]];

test_old/run-php-src.sh

@@ -1,4 +1,4 @@
-wget -q https://github.com/php/php-src/archive/php-7.1.0.tar.gz
+wget -q https://github.com/php/php-src/archive/php-7.3.0RC1.tar.gz
 mkdir -p ./data/php-src
-tar -xzf ./php-7.1.0.tar.gz -C ./data/php-src --strip-components=1
+tar -xzf ./php-7.3.0RC1.tar.gz -C ./data/php-src --strip-components=1
 php -n test_old/run.php --verbose --no-progress PHP7 ./data/php-src

test_old/run.php

@@ -59,9 +59,24 @@ $dir = $arguments[1];
 
 switch ($testType) {
     case 'Symfony':
-        $version = 'Php5';
+        $version = 'Php7';
         $fileFilter = function($path) {
-            return preg_match('~\.php(?:\.cache)?$~', $path) && false === strpos($path, 'skeleton');
+            if (!preg_match('~\.php$~', $path)) {
+                return false;
+            }
+
+            if (preg_match('~(?:
+# invalid php code
+  dependency-injection.Tests.Fixtures.xml.xml_with_wrong_ext
+# difference in nop statement
+| framework-bundle.Resources.views.Form.choice_widget_options\.html
+# difference due to INF
+| yaml.Tests.InlineTest
+)\.php$~x', $path)) {
+                return false;
+            }
+
+            return true;
         };
         $codeExtractor = function($file, $code) {
             return $code;
@@ -77,26 +92,31 @@ switch ($testType) {
             if (preg_match('~(?:
 # skeleton files
   ext.gmp.tests.001
-| ext.skeleton.tests.001
+| ext.skeleton.tests.00\d
 # multibyte encoded files
 | ext.mbstring.tests.zend_multibyte-01
 | Zend.tests.multibyte.multibyte_encoding_001
 | Zend.tests.multibyte.multibyte_encoding_004
 | Zend.tests.multibyte.multibyte_encoding_005
+# invalid code due to missing WS after opening tag
+| tests.run-test.bug75042-3
 # pretty print difference due to INF vs 1e1000
 | ext.standard.tests.general_functions.bug27678
 | tests.lang.bug24640
+| Zend.tests.bug74947
 # pretty print differences due to negative LNumbers
 | Zend.tests.neg_num_string
 | Zend.tests.bug72918
 # pretty print difference due to nop statements
 | ext.mbstring.tests.htmlent
 | ext.standard.tests.file.fread_basic
+# its too hard to emulate these on old PHP versions
+| Zend.tests.flexible-heredoc-complex-test[1-4]
 )\.phpt$~x', $file)) {
                 return null;
             }
 
-            if (!preg_match('~--FILE--\s*(.*?)--[A-Z]+--~s', $code, $matches)) {
+            if (!preg_match('~--FILE--\s*(.*?)\n--[A-Z]+--~s', $code, $matches)) {
                 return null;
             }
             if (preg_match('~--EXPECT(?:F|REGEX)?--\s*(?:Parse|Fatal) error~', $code)) {