Release PHP-Parser 4.0 Alpha 1

A bit annoying that we have to keep $this->nl always synchronized.

.travis.yml

@@ -1,20 +1,28 @@
 language: php
-
+dist: trusty
 sudo: false
 
-php:
-  - 5.4
-  - 5.5
-  - 5.6
-  - 7.0
-  - nightly
-  - hhvm
+cache:
+  directories:
+    - $HOME/.composer/cache
 
-install: composer install --prefer-source
+php:
+  - 7.0
+  - 7.1
+  - nightly
+
+install:
+  - if [ $TRAVIS_PHP_VERSION = '7.0' ]; then composer require satooshi/php-coveralls '~1.0'; fi
+  - composer install --prefer-dist
 
 matrix:
   allow_failures:
     - php: nightly
   fast_finish: true
 
-script: vendor/bin/phpunit
+script:
+  - if [ $TRAVIS_PHP_VERSION = '7.0' ]; then vendor/bin/phpunit --coverage-clover build/logs/clover.xml; else vendor/bin/phpunit; fi
+  - if [ $TRAVIS_PHP_VERSION = '7.1' ]; then test_old/run-php-src.sh; fi
+
+after_success:
+  - if [ $TRAVIS_PHP_VERSION = '7.0' ]; then php vendor/bin/coveralls; fi

CHANGELOG.md

@@ -1,8 +1,390 @@
-Version 2.0.1-dev
+Version 4.0.0-dev
 -----------------
 
 Nothing yet.
 
+Version 4.0.0-alpha1 (2017-10-18)
+---------------------------------
+
+### Added
+
+* Added experimental support for format-preserving pretty-printing. In this mode formatting will be
+  preserved for parts of the code which have not been modified.
+* Added `replaceNodes` option to `NameResolver`, defaulting to true. If this option is disabled,
+  resolved names will be added as `resolvedName` attributes, instead of replacing the original
+  names.
+* Added `NodeFinder` class, which can be used to find nodes based on a callback or class name. This
+  is a utility to avoid custom node visitor implementations for simple search operations.
+* Added `ClassMethod::isMagic()` method.
+* Added `BuilderFactory` methods: `val()` method for creating an AST for a simple value, `concat()`
+  for creating concatenation trees, `args()` for preparing function arguments.
+* Added `NameContext` class, which encapsulates the `NameResolver` logic independently of the actual
+  AST traversal. This facilitates use in other context, such as class names in doc comments.
+  Additionally it provides an API for getting the shortest representation of a name.
+* Added `Node::setAttributes()` method.
+* Added `JsonDecoder`. This allows convertion JSON back into an AST.
+* Added `Name` methods `toLowerString()` and `isSpecialClassName()`.
+* Added `Identifier` and `VarLikeIdentifier` nodes, which are used in place of simple strings in
+  many places.
+* Added `getComments()`, `getStartLine()`, `getEndLine()`, `getStartTokenPos()`, `getEndTokenPos()`,
+  `getStartFilePos()` and `getEndFilePos()` methods to `Node`. These provide a more obvious access
+  point for the already existing attributes of the same name.
+* Added `ConstExprEvaluator` to evaluate constant expressions to PHP values.
+* Added `Expr\BinaryOp::getOperatorSigil()`, returning `+` for `Expr\BinaryOp\Plus`, etc.
+
+### Changed
+
+* Many subnodes that previously held simple strings now use `Identifier` (or `VarLikeIdentifier`)
+  nodes. Please see the UPGRADE-4.0 file for an exhaustive list of affected nodes and some notes on
+  possible impact.
+* Expression statements (`expr;`) are now represented using a `Stmt\Expression` node. Previously
+  these statements were directly represented as their constituent expression.
+* The `name` subnode of `Param` has been renamed to `var` and now contains a `Variable` rather than
+  a plain string.
+* The `name` subnode of `StaticVar` has been renamed to `var` and now contains a `Variable` rather
+  than a plain string.
+* The `var` subnode of `ClosureUse` now contains a `Variable` rather than a plain string.
+* The `var` subnode of `Catch` now contains a `Variable` rather than a plain string.
+* The `alias` subnode of `UseUse` is now `null` if no explicit alias is given. As such,
+  `use Foo\Bar` and `use Foo\Bar as Bar` are now represented differently. The `getAlias()` method
+  can be used to get the effective alias, even if it is not explicitly given.
+
+### Removed
+
+* Support for running on PHP 5 and HHVM has been removed. You can however still parse code of old
+  PHP versions (such as PHP 5.2), while running on PHP 7.
+* Removed `type` subnode on `Class`, `ClassMethod` and `Property` nodes. Use `flags` instead.
+* The `ClassConst::isStatic()` method has been removed. Constants cannot have a static modifier.
+* The `NodeTraverser` no longer accepts `false` as a return value from a `leaveNode()` method.
+  `NodeTraverser::REMOVE_NODE` should be returned instead.
+* The `Node::setLine()` method has been removed. If you really need to, you can use `setAttribute()`
+  instead.
+* The misspelled `Class_::VISIBILITY_MODIFER_MASK` constant has been dropped in favor of
+  `Class_::VISIBILITY_MODIFIER_MASK`.
+* The XML serializer has been removed. As such, the classes `Serializer\XML`, and
+  `Unserializer\XML`, as well as the interfaces `Serializer` and `Unserializer` no longer exist.
+* The `BuilderAbstract` class has been removed. It's functionality is moved into `BuilderHelpers`.
+  However, this is an internal class and should not be used directly.
+
+Version 3.1.2-dev
+-----------------
+
+### Fixed
+
+* Comments on empty blocks are now preserved on a `Stmt\Nop` node. (#382)
+
+### Added
+
+* Added `kind` attribute for `Stmt\Namespace_` node, which is one of `KIND_SEMICOLON` or
+  `KIND_BRACED`. (#417)
+
+Version 3.1.1 (2017-09-02)
+--------------------------
+
+### Fixed
+
+* Fixed syntax error on comment after brace-style namespace declaration. (#412)
+* Added support for TraitUse statements in trait builder. (#413)
+
+Version 3.1.0 (2017-07-28)
+--------------------------
+
+### Added
+
+* [PHP 7.2] Added support for trailing comma in group use statements.
+* [PHP 7.2] Added support for `object` type. This means `object` types will now be represented as a
+  builtin type (a simple `"object"` string), rather than a class `Name`.
+  
+### Fixed
+
+* Floating-point numbers are now printed correctly if the LC_NUMERIC locale uses a comma as decimal
+  separator.
+
+### Changed
+
+* `Name::$parts` is no longer deprecated.
+
+Version 3.0.6 (2017-06-28)
+--------------------------
+
+### Fixed
+
+* Fixed the spelling of `Class_::VISIBILITY_MODIFIER_MASK`. The previous spelling of
+  `Class_::VISIBILITY_MODIFER_MASK` is preserved for backwards compatibility.
+* The pretty printing will now preserve comments inside array literals and function calls by
+  printing the array items / function arguments on separate lines. Array literals and functions that
+  do not contain comments are not affected.
+
+### Added
+
+* Added `Builder\Param::makeVariadic()`.
+
+### Deprecated
+
+* The `Node::setLine()` method has been deprecated.
+
+Version 3.0.5 (2017-03-05)
+--------------------------
+
+### Fixed
+
+* Name resolution of `NullableType`s is now performed earlier, so that a fully resolved signature is
+  available when a function is entered. (#360)
+* `Error` nodes are now considered empty, while previously they extended until the token where the
+  error occurred. This made some nodes larger than expected. (#359)
+* Fixed notices being thrown during error recovery in some situations. (#362)
+
+Version 3.0.4 (2017-02-10)
+--------------------------
+
+### Fixed
+
+* Fixed some extensibility issues in pretty printer (`pUseType()` is now public and `pPrec()` calls
+  into `p()`, instead of directly dispatching to the type-specific printing method).
+* Fixed notice in `bin/php-parse` script.
+
+### Added
+
+* Error recovery from missing semicolons is now supported in more cases.
+* Error recovery from trailing commas in positions where PHP does not support them is now supported.
+
+Version 3.0.3 (2017-02-03)
+--------------------------
+
+### Fixed
+
+* In `"$foo[0]"` the `0` is now parsed as an `LNumber` rather than `String`. (#325)
+* Ensure integers and floats are always pretty printed preserving semantics, even if the particular
+  value can only be manually constructed.
+* Throw a `LogicException` when trying to pretty-print an `Error` node. Previously this resulted in
+  an undefined method exception or fatal error.
+
+### Added
+
+* [PHP 7.1] Added support for negative interpolated offsets: `"$foo[-1]"`
+* Added `preserveOriginalNames` option to `NameResolver`. If this option is enabled, an
+  `originalName` attribute, containing the unresolved name, will be added to each resolved name.
+* Added `php-parse --with-positions` option, which dumps nodes with position information.
+
+### Deprecated
+
+* The XML serializer has been deprecated. In particular, the classes `Serializer\XML`,
+  `Unserializer\XML`, as well as the interfaces `Serializer` and `Unserializer` are deprecated.
+
+Version 3.0.2 (2016-12-06)
+--------------------------
+
+### Fixed
+
+* Fixed name resolution of nullable types. (#324)
+* Fixed pretty-printing of nullable types.
+
+Version 3.0.1 (2016-12-01)
+--------------------------
+
+### Fixed
+
+* Fixed handling of nested `list()`s: If the nested list was unkeyed, it was directly included in
+  the list items. If it was keyed, it was wrapped in `ArrayItem`. Now nested `List_` nodes are
+  always wrapped in `ArrayItem`s. (#321)
+
+Version 3.0.0 (2016-11-30)
+--------------------------
+
+### Added
+
+* Added support for dumping node positions in the NodeDumper through the `dumpPositions` option.
+* Added error recovery support for `$`, `new`, `Foo::`.
+
+Version 3.0.0-beta2 (2016-10-29)
+--------------------------------
+
+This release primarily improves our support for error recovery.
+
+### Added
+
+* Added `Node::setDocComment()` method.
+* Added `Error::getMessageWithColumnInfo()` method.
+* Added support for recovery from lexer errors.
+* Added support for recovering from "special" errors (i.e. non-syntax parse errors).
+* Added precise location information for lexer errors.
+* Added `ErrorHandler` interface, and `ErrorHandler\Throwing` and `ErrorHandler\Collecting` as
+  specific implementations. These provide a general mechanism for handling error recovery.
+* Added optional `ErrorHandler` argument to `Parser::parse()`, `Lexer::startLexing()` and
+  `NameResolver::__construct()`.
+* The `NameResolver` now adds a `namespacedName` attribute on name nodes that cannot be statically
+  resolved (unqualified unaliased function or constant names in namespaces).
+  
+### Fixed
+
+* Fixed attribute assignment for `GroupUse` prefix and variables in interpolated strings.
+
+### Changed
+
+* The constants on `NameTraverserInterface` have been moved into the `NameTraverser` class.
+* Due to the error handling changes, the `Parser` interface and `Lexer` API have changed.
+* The emulative lexer now directly postprocesses tokens, instead of using `~__EMU__~` sequences.
+  This changes the protected API of the lexer.
+* The `Name::slice()` method now returns `null` for empty slices, previously `new Name([])` was
+  used. `Name::concat()` now also supports concatenation with `null`.
+
+### Removed
+
+* Removed `Name::append()` and `Name::prepend()`. These mutable methods have been superseded by
+  the immutable `Name::concat()`.
+* Removed `Error::getRawLine()` and `Error::setRawLine()`. These methods have been superseded by
+  `Error::getStartLine()` and `Error::setStartLine()`.
+* Removed support for node cloning in the `NodeTraverser`.
+* Removed `$separator` argument from `Name::toString()`.
+* Removed `throw_on_error` parser option and `Parser::getErrors()` method. Use the `ErrorHandler`
+  mechanism instead.
+
+Version 3.0.0-beta1 (2016-09-16)
+--------------------------------
+
+### Added
+
+* [7.1] Function/method and parameter builders now support PHP 7.1 type hints (void, iterable and
+  nullable types).
+* Nodes and Comments now implement `JsonSerializable`. The node kind is stored in a `nodeType`
+  property.
+* The `InlineHTML` node now has an `hasLeadingNewline` attribute, that specifies whether the
+  preceding closing tag contained a newline. The pretty printer honors this attribute.
+* Partial parsing of `$obj->` (with missing property name) is now supported in error recovery mode.
+* The error recovery mode is now exposed in the `php-parse` script through the `--with-recovery`
+  or `-r` flags.
+
+The following changes are also part of PHP-Parser 2.1.1:
+
+* The PHP 7 parser will now generate a parse error for `$var =& new Obj` assignments.
+* Comments on free-standing code blocks will now be retained as comments on the first statement in
+  the code block.
+
+Version 3.0.0-alpha1 (2016-07-25)
+---------------------------------
+
+### Added
+
+* [7.1] Added support for `void` and `iterable` types. These will now be represented as strings
+  (instead of `Name` instances) similar to other builtin types.
+* [7.1] Added support for class constant visibility. The `ClassConst` node now has a `flags` subnode
+  holding the visibility modifier, as well as `isPublic()`, `isProtected()` and `isPrivate()`
+  methods. The constructor changed to accept the additional subnode.
+* [7.1] Added support for nullable types. These are represented using a new `NullableType` node
+  with a single `type` subnode.
+* [7.1] Added support for short array destructuring syntax. This means that `Array` nodes may now
+  appear as the left-hand-side of assignments and foreach value targets. Additionally the array
+  items may now contain `null` values if elements are skipped.
+* [7.1] Added support for keys in list() destructuring. The `List` subnode `vars` has been renamed
+  to `items` and now contains `ArrayItem`s instead of plain variables.
+* [7.1] Added support for multi-catch. The `Catch` subnode `type` has been renamed to `types` and
+  is now an array of `Name`s.
+* `Name::slice()` now supports lengths and negative offsets. This brings it in line with
+  `array_slice()` functionality.
+
+### Changed
+
+Due to PHP 7.1 support additions described above, the node structure changed as follows:
+
+* `void` and `iterable` types are now stored as strings if the PHP 7 parser is used.
+* The `ClassConst` constructor changed to accept an additional `flags` subnode.
+* The `Array` subnode `items` may now contain `null` elements (destructuring).
+* The `List` subnode `vars` has been renamed to `items` and now contains `ArrayItem`s instead of
+  plain variables.
+* The `Catch` subnode `type` has been renamed to `types` and is now an array of `Name`s.
+
+Additionally the following changes were made:
+
+* The `type` subnode on `Class`, `ClassMethod` and `Property` has been renamed to `flags`. The
+  `type` subnode has retained for backwards compatibility and is populated to the same value as
+  `flags`. However, writes to `type` will not update `flags`.
+* The `TryCatch` subnode `finallyStmts` has been replaced with a `finally` subnode that holds an
+  explicit `Finally` node. This allows for more accurate attribute assignment.
+* The `Trait` constructor now has the same form as the `Class` and `Interface` constructors: It
+  takes an array of subnodes. Unlike classes/interfaces, traits can only have a `stmts` subnode.
+* The `NodeDumper` now prints class/method/property/constant modifiers, as well as the include and
+  use type in a textual representation, instead of only showing the number.
+* All methods on `PrettyPrinter\Standard` are now protected. Previoulsy most of them were public.
+
+### Removed
+
+* Removed support for running on PHP 5.4. It is however still possible to parse PHP 5.2-5.4 code
+  while running on a newer version.
+* The deprecated `Comment::setLine()` and `Comment::setText()` methods have been removed.
+* The deprecated `Name::set()`, `Name::setFirst()` and `Name::setLast()` methods have been removed.
+
+Version 2.1.1 (2016-09-16)
+--------------------------
+
+### Changed
+
+* The pretty printer will now escape all control characters in the range `\x00-\x1F` inside double
+  quoted strings. If no special escape sequence is available, an octal escape will be used.
+* The quality of the error recovery has been improved. In particular unterminated expressions should
+  be handled more gracefully.
+* The PHP 7 parser will now generate a parse error for `$var =& new Obj` assignments.
+* Comments on free-standing code blocks will no be retained as comments on the first statement in
+  the code block.
+
+Version 2.1.0 (2016-04-19)
+--------------------------
+
+### Fixed
+
+* Properly support `B""` strings (with uppercase `B`) in a number of places.
+* Fixed reformatting of indented parts in a certain non-standard comment style.
+
+### Added
+
+* Added `dumpComments` option to node dumper, to enable dumping of comments associated with nodes.
+* Added `Stmt\Nop` node, that is used to collect comments located at the end of a block or at the
+  end of a file (without a following node with which they could otherwise be associated).
+* Added `kind` attribute to `Expr\Exit` to distinguish between `exit` and `die`.
+* Added `kind` attribute to `Scalar\LNumber` to distinguish between decimal, binary, octal and
+  hexadecimal numbers.
+* Added `kind` attribtue to `Expr\Array` to distinguish between `array()` and `[]`.
+* Added `kind` attribute to `Scalar\String` and `Scalar\Encapsed` to distinguish between
+  single-quoted, double-quoted, heredoc and nowdoc string.
+* Added `docLabel` attribute to `Scalar\String` and `Scalar\Encapsed`, if it is a heredoc or
+  nowdoc string.
+* Added start file offset information to `Comment` nodes.
+* Added `setReturnType()` method to function and method builders.
+* Added `-h` and `--help` options to `php-parse` script.
+
+### Changed
+
+* Invalid octal literals now throw a parse error in PHP 7 mode.
+* The pretty printer takes all the new attributes mentioned in the previous section into account.
+* The protected `AbstractPrettyPrinter::pComments()` method no longer returns a trailing newline.
+* The bundled autoloader supports library files being stored in a different directory than
+  `PhpParser` for easier downstream distribution.
+
+### Deprecated
+
+* The `Comment::setLine()` and `Comment::setText()` methods have been deprecated. Construct new
+  objects instead.
+
+### Removed
+
+* The internal (but public) method `Scalar\LNumber::parse()` has been removed. A non-internal
+  `LNumber::fromString()` method has been added instead.
+
+Version 2.0.1 (2016-02-28)
+--------------------------
+
+### Fixed
+
+* `declare() {}` and `declare();` are not semantically equivalent and will now result in different
+  ASTs. The format case will have an empty `stmts` array, while the latter will set `stmts` to
+  `null`.
+* Magic constants are now supported as semi-reserved keywords.
+* A shebang line like `#!/usr/bin/env php` is now allowed at the start of a namespaced file.
+  Previously this generated an exception.
+* The `prettyPrintFile()` method will not strip a trailing `?>` from the raw data that follows a
+  `__halt_compiler()` statement.
+* The `prettyPrintFile()` method will not strip an opening `<?php` if the file starts with a
+  comment followed by InlineHTML.
+
 Version 2.0.0 (2015-12-04)
 --------------------------
 
@@ -40,7 +422,7 @@ A more detailed description of backwards incompatible changes can be found in th
 
 ### Removed
 
-* Removed support for running on PHP 5.4. It is however still possible to parse PHP 5.2 and PHP 5.3
+* Removed support for running on PHP 5.3. It is however still possible to parse PHP 5.2 and PHP 5.3
   code while running on a newer version.
 * Removed legacy class name aliases. This includes the old non-namespaced class names and the old
   names for classes that were renamed for PHP 7 compatibility.
@@ -72,242 +454,8 @@ A more detailed description of backwards incompatible changes can be found in th
 * The constructor for `Scalar` nodes no longer has a default value. E.g. `new LNumber()` should now
   be written as `new LNumber(0)`.
 
-Version 1.4.0 (2015-07-14)
---------------------------
-
-### Added
-
-* Added interface `PhpParser\Node\FunctionLike`, which is implemented by `Stmt\ClassMethod`,
-  `Stmt\Function_` and `Expr\Closure` nodes. This interface provides getters for their common
-  subnodes.
-* Added `Node\Stmt\ClassLike::getMethod()` to look up a specific method on a class/interface/trait.
-
-### Fixed
-
-* Fixed `isPublic()` return value for implicitly public properties and methods that define and
-  additional modifier like `static` or `abstract`.
-* Properties are now accepted by the trait builder.
-* Fixed `__HALT_COMPILER_OFFSET__` support on HHVM.
-
-Version 1.3.0 (2015-05-02)
---------------------------
-
-### Added
-
-* Errors can now store the attributes of the node/token where the error occurred. Previously only the start line was
-  stored.
-* If file positions are enabled in the lexer, errors can now provide column information if it is available. See
-  [documentation](https://github.com/nikic/PHP-Parser/blob/master/doc/component/Error.markdown#column-information).
-* The parser now provides an experimental error recovery mode, which can be enabled by disabling the `throwOnError`
-  parser option. In this mode the parser will try to construct a partial AST even if the code is not valid PHP. See
-  [documentation](https://github.com/nikic/PHP-Parser/blob/master/doc/component/Error.markdown#error-recovery).
-* Added support for PHP 7 `yield from` expression. It is represented by `Expr\YieldFrom`.
-* Added support for PHP 7 anonymous classes. These are represented by ordinary `Stmt\Class_` nodes with the name set to
-  `null`. Furthermore this implies that `Expr\New_` can now contain a `Stmt\Class_` in its `class` subnode.
-
-### Fixed
-
-* Fixed registration of PHP 7 aliases, for the case where the old name was used before the new name.
-* Fixed handling of precedence when pretty-printing `print` expressions.
-* Floating point numbers are now pretty-printed with a higher precision.
-* Checks for special class names like `self` are now case-insensitive.
-
-Version 1.2.2 (2015-04-03)
---------------------------
-
-* The `NameResolver` now resolves parameter type hints when entering the function/method/closure node. As such other
-  visitors running after it will be able to make use of the resolved names at that point already.
-* The autoloader no longer sets the `unserialize_callback_func` ini option on registration - this is not necessary and
-  may cause issues when running PhpUnit tests with process isolation.
-
-Version 1.2.1 (2015-03-24)
---------------------------
-
-* Fixed registration of the aliases introduced in 1.2.0. Previously the old class names could not be used in
-  `instanceof` checks under some circumstances.
-
-Version 1.2.0 (2015-03-22)
---------------------------
-
-### Changed
-
-* To ensure compatibility with PHP 7, the following node classes have been renamed:
-
-        OLD                             => NEW
-        PhpParser\Node\Expr\Cast\Bool   => PhpParser\Node\Expr\Cast\Bool_
-        PhpParser\Node\Expr\Cast\Int    => PhpParser\Node\Expr\Cast\Int_
-        PhpParser\Node\Expr\Cast\Object => PhpParser\Node\Expr\Cast\Object_
-        PhpParser\Node\Expr\Cast\String => PhpParser\Node\Expr\Cast\String_
-        PhpParser\Node\Scalar\String    => PhpParser\Node\Scalar\String_
-
-  **The previous class names are still supported as aliases.** However it is strongly encouraged to use the new names
-  in order to make your code compatible with PHP 7.
-
-* Subnodes are now stored using real properties instead of an array. This improves performance and memory usage of the
-  initial parse and subsequent node tree operations. The `NodeAbstract` class still supports the old way of specifying
-  subnodes, however this is *deprecated*. In any case properties that are assigned to a node after creation will no
-  longer be considered as subnodes.
-
-* Methods and property declarations will no longer set the `Stmt\Class_::MODIFIER_PUBLIC` flag if no visibility is
-  explicitly given. However the `isPublic()` method will continue to return true. This allows you to distinguish whether
-  a method/property is explicitly or implicitly public and control the pretty printer output more precisely.
-
-* The `Stmt\Class_`, `Stmt\Interface_` and `Stmt\Trait_` nodes now inherit from `Stmt\ClassLike`, which provides a
-  `getMethods()` method. Previously this method was only available on `Stmt\Class_`.
-
-* Support including the `bootstrap.php` file multiple times.
-
-* Make documentation and tests part of the release tarball again.
-
-* Improve support for HHVM and PHP 7.
-
-### Added
-
-* Added support for PHP 7 return type declarations. This adds an additional `returnType` subnode to `Stmt\Function_`,
-  `Stmt\ClassMethod` and `Expr\Closure`.
-
-* Added support for the PHP 7 null coalesce operator `??`. The operator is represented by `Expr\BinaryOp\Coalesce`.
-
-* Added support for the PHP 7 spaceship operator `<=>`. The operator is represented by `Expr\BinaryOp\Spaceship`.
-
-* Added use builder.
-
-* Added global namespace support to the namespace builder.
-
-* Added a constructor flag to `NodeTraverser`, which disables cloning of nodes.
-
-Version 1.1.0 (2015-01-18)
---------------------------
-
-* Methods that do not specify an explicit visibility (e.g. `function method()`) will now have the `MODIFIER_PUBLIC`
-  flag set. This also means that their `isPublic()` method will return true.
-
-* Declaring a property as abstract or final is now an error.
-
-* The `Lexer` and `Lexer\Emulative` classes now accept an `$options` array in their constructors. Currently only the
-  `usedAttributes` option is supported, which determines which attributes will be added to AST nodes. In particular
-  it is now possible to add information on the token and file positions corresponding to a node. For more details see
-  the [Lexer component](https://github.com/nikic/PHP-Parser/blob/master/doc/component/Lexer.markdown) documentation.
-
-* Node visitors can now return `NodeTraverser::DONT_TRAVERSE_CHILDREN` from `enterNode()` in order to skip all children
-  of the current node, for all visitors.
-
-* Added builders for traits and namespaces.
-
-* The class, interface, trait, function, method and property builders now support adding doc comments using the
-  `setDocComment()` method.
-
-* Added support for fully-qualified and namespace-relative names in builders. No longer allow use of name component
-  arrays.
-
-* Do not add documentation and tests to distribution archive files.
-
-Version 1.0.2 (2014-11-04)
---------------------------
-
-* The `NameResolver` visitor now also resolves names in trait adaptations (aliases and precedence declarations).
-
-* Remove stray whitespace when pretty-printing trait adaptations that only change visibility.
-
-Version 1.0.1 (2014-10-14)
---------------------------
-
-* Disallow `new` expressions without a class name. Previously `new;` was accidentally considered to be valid code.
-
-* Support T_ONUMBER token used by HHVM.
-
-* Add ability to directly pass code to the `php-parse.php` script.
-
-* Prevent truncation of `var_dump()` output in the `php-parse.php` script if XDebug is used.
-
-Version 1.0.0 (2014-09-12)
---------------------------
-
-* [BC] Removed deprecated `Template` and `TemplateLoader` classes.
-
-* Fixed XML unserializer to properly work with new namespaced node names.
-
-Version 1.0.0-beta2 (2014-08-31)
---------------------------------
-
-* [PHP 5.6] Updated support for constant scalar expressions to comply with latest changes. This means that arrays
-  and array dimension fetches are now supported as well.
-
-* [PHP 5.6] Direct array dereferencing of constants is supported now, i.e. both `FOO[0]` and `Foo::BAR[0]` are valid
-  now.
-
-* Fixed handling of special class names (`self`, `parent` and `static`) in the name resolver to be case insensitive.
-  Additionally the name resolver now enforces that special class names are only used as unqualified names, e.g. `\self`
-  is considered invalid.
-
-* The case of references to the `static` class name is now preserved. Previously `static` was always lowercased,
-  regardless of the case used in the source code.
-
-* The autoloader now only requires a file if it exists. This allows usages like
-  `class_exists('PhpParser\NotExistingClass')`.
-
-* Added experimental `bin/php-parse.php` script, which is intended to help exploring and debugging the node tree.
-
-* Separated the parser implemention (in `lib/PhpParser/ParserAbstract.php`) and the generated data (in
-  `lib/PhpParser/Parser.php`). Furthermore the parser now uses meaningful variable names and contains comments
-  explaining their usage.
-
-Version 1.0.0-beta1 (2014-03-27)
---------------------------------
-
-* [BC] PHP-Parser now requires PHP 5.3 or newer to run. It is however still possible to *parse* PHP 5.2 source code,
-  while running on a newer version.
-
-* [BC] The library has been moved to use namespaces with the `PhpParser` vendor prefix. However, the old names using
-  underscores are still available as aliases, as such most code should continue running on the new version without
-  further changes.
-
-  However, code performing dispatch operations on `Node::getType()` may be affected by some of the name changes. For
-  example a `+` node will now return type `Expr_BinaryOp_Plus` instead of `Expr_Plus`. In particular this may affect
-  custom pretty printers.
-
-  Due to conflicts with reserved keywords, some class names now end with an underscore, e.g. `PHPParser_Node_Stmt_Class`
-  is now `PhpParser\Node\Stmt\Class_`. (But as usual, the old name is still available)
-
-* [PHP 5.6] Added support for the power operator `**` (node `Expr\BinaryOp\Pow`) and the compound power assignment
-  operator `**=` (node `Expr\AssignOp\Pow`).
-
-* [PHP 5.6] Added support for variadic functions: `Param` nodes now have `variadic` as a boolean subnode.
-
-* [PHP 5.6] Added support for argument unpacking: `Arg` nodes now have `unpack` as a boolean subnode.
-
-* [PHP 5.6] Added support for aliasing of functions and constants. `Stmt\Use_` nodes now have an integral `type`
-  subnode, which is one of `Stmt\Use_::TYPE_NORMAL` (`use`), `Stmt\Use_::TYPE_FUNCTION` (`use function`) or
-  `Stmt\Use_::TYPE_CONSTANT` (`use const`).
-
-  The `NameResolver` now also supports resolution of such aliases.
-
-* [PHP 5.6] Added support for constant scalar expressions. This means that certain expressions are now allowed as the
-  initializer for constants, properties, parameters, static variables, etc.
-
-* [BC] Improved pretty printing of empty statements lists, which are now printed as `{\n}` instead of `{\n    \n}`.
-  This changes the behavior of the protected `PrettyPrinterAbstract::pStmts()` method, so custom pretty printing code
-  making use it of may need to be adjusted.
-
-* Changed the order of some subnodes to be consistent with their order in the sour code. For example `Stmt\If->cond`
-  will now appear before `Stmt\If->stmts` etc.
-
-* Added `Scalar\MagicConstant->getName()`, which returns the name of the magic constant (e.g. `__CLASS__`).
-
-**The following changes are also included in 0.9.5**:
-
-* [BC] Deprecated `PHPParser_Template` and `PHPParser_TemplateLoader`. This functionality does not belong in the main project
-  and - as far as I know - nobody is using it.
-
-* Add `NodeTraverser::removeVisitor()` method, which removes a visitor from the node traverser. This also modifies the
-  corresponding `NodeTraverserInterface`.
-
-* Fix alias resolution in `NameResolver`: Class names are now correctly handled as case-insensitive.
-
-* The undefined variable error, which is used to the lexer to reset the error state, will no longer interfere with
-  custom error handlers.
-
 ---
 
-**This changelog only includes changes from the 1.0 and 2.0 series. For older changes see the
-[0.9 series changelog][https://github.com/nikic/PHP-Parser/blob/0.9/CHANGELOG.md].**
+**This changelog only includes changes from the 2.0 series. For older changes see the
+[1.x series changelog](https://github.com/nikic/PHP-Parser/blob/1.x/CHANGELOG.md) and the
+[0.9 series changelog](https://github.com/nikic/PHP-Parser/blob/0.9/CHANGELOG.md).**

README.md

@@ -1,82 +1,182 @@
 PHP Parser
 ==========
 
-This is a PHP 5.2 to PHP 7.0 parser written in PHP. Its purpose is to simplify static code analysis and
+[![Build Status](https://travis-ci.org/nikic/PHP-Parser.svg?branch=master)](https://travis-ci.org/nikic/PHP-Parser) [![Coverage Status](https://coveralls.io/repos/github/nikic/PHP-Parser/badge.svg?branch=master)](https://coveralls.io/github/nikic/PHP-Parser?branch=master)
+
+This is a PHP 5.2 to PHP 7.2 parser written in PHP. Its purpose is to simplify static code analysis and
 manipulation.
 
+[**Documentation for version 3.x**][doc_3_x] (stable; for running on PHP >= 5.5; for parsing PHP 5.2 to PHP 7.2).
 
-[**Documentation for version 2.x**][doc_master] (stable; for running on PHP >= 5.4; for parsing PHP 5.2 to PHP 7.0).
+[Documentation for version 4.x][doc_master] (development; for running on PHP >= 7.0; for parsing PHP 5.2 to PHP 7.2).
 
-[Documentation for version 1.x][doc_1_x] (stable; for running on PHP >= 5.3; for parsing PHP 5.2 to PHP 5.6).
+Features
+--------
 
-In a Nutshell
--------------
+The main features provided by this library are:
 
-The parser turns PHP source code into an abstract syntax tree. For example, if you pass the following code into the
-parser:
+ * Parsing PHP 5 and PHP 7 code into an abstract syntax tree (AST).
+   * Invalid code can be parsed into a partial AST.
+   * The AST contains accurate location information.
+ * Dumping the AST in human-readable form.
+ * Converting an AST back to PHP code.
+   * Experimental: Formatting can be preserved for partially changed ASTs.
+ * Infrastructure to traverse and modify ASTs.
+ * Resolution of namespaced names.
+ * Evaluation of constant expressions.
+ * Builders to simplify AST construction for code generation.
+ * Converting an AST into JSON and back.
+
+Quick Start
+-----------
+
+Install the library using [composer](https://getcomposer.org):
+
+    php composer.phar require nikic/php-parser
+
+Parse some PHP code into an AST and dump the result in human-readable form:
 
 ```php
 <?php
-echo 'Hi', 'World';
-hello\world('foo', 'bar' . 'baz');
+use PhpParser\Error;
+use PhpParser\NodeDumper;
+use PhpParser\ParserFactory;
+
+$code = <<<'CODE'
+<?php
+
+function test($foo)
+{
+    var_dump($foo);
+}
+CODE;
+
+$parser = (new ParserFactory)->create(ParserFactory::PREFER_PHP7);
+try {
+    $ast = $parser->parse($code);
+} catch (Error $error) {
+    echo "Parse error: {$error->getMessage()}\n";
+    return;
+}
+
+$dumper = new NodeDumper;
+echo $dumper->dump($ast) . "\n";
 ```
 
-You'll get a syntax tree looking roughly like this:
+This dumps an AST looking something like this:
 
-```php
+```
 array(
-    0: Stmt_Echo(
-        exprs: array(
-            0: Scalar_String(
-                value: Hi
-            )
-            1: Scalar_String(
-                value: World
+    0: Stmt_Function(
+        byRef: false
+        name: Identifier(
+            name: test
+        )
+        params: array(
+            0: Param(
+                type: null
+                byRef: false
+                variadic: false
+                var: Expr_Variable(
+                    name: foo
+                )
+                default: null
             )
         )
-    )
-    1: Expr_FuncCall(
-        name: Name(
-            parts: array(
-                0: hello
-                1: world
-            )
-        )
-        args: array(
-            0: Arg(
-                value: Scalar_String(
-                    value: foo
-                )
-                byRef: false
-            )
-            1: Arg(
-                value: Expr_Concat(
-                    left: Scalar_String(
-                        value: bar
+        returnType: null
+        stmts: array(
+            0: Stmt_Expression(
+                expr: Expr_FuncCall(
+                    name: Name(
+                        parts: array(
+                            0: var_dump
+                        )
                     )
-                    right: Scalar_String(
-                        value: baz
+                    args: array(
+                        0: Arg(
+                            value: Expr_Variable(
+                                name: foo
+                            )
+                            byRef: false
+                            unpack: false
+                        )
                     )
                 )
-                byRef: false
             )
         )
     )
 )
 ```
 
-You can then work with this syntax tree, for example to statically analyze the code (e.g. to find
-programming errors or security issues).
+Let's traverse the AST and perform some kind of modification. For example, drop all function bodies:
 
-Additionally, you can convert a syntax tree back to PHP code. This allows you to do code preprocessing
-(like automatedly porting code to older PHP versions).
+```php
+use PhpParser\Node;
+use PhpParser\Node\Stmt\Function_;
+use PhpParser\NodeTraverser;
+use PhpParser\NodeVisitorAbstract;
 
-Installation
-------------
+$traverser = new NodeTraverser();
+$traverser->addVisitor(new class extends NodeVisitorAbstract {
+    public function enterNode(Node $node) {
+        if ($node instanceof Function_) {
+            // Clean out the function body
+            $node->stmts = [];
+        }
+    }
+});
 
-The preferred installation method is [composer](https://getcomposer.org):
+$ast = $traverser->traverse($ast);
+echo $dumper->dump($ast) . "\n";
+```
 
-    php composer.phar require nikic/php-parser
+This gives us an AST where the `Function_::$stmts` are empty:
+
+```
+array(
+    0: Stmt_Function(
+        byRef: false
+        name: Identifier(
+            name: test
+        )
+        params: array(
+            0: Param(
+                type: null
+                byRef: false
+                variadic: false
+                var: Expr_Variable(
+                    name: foo
+                )
+                default: null
+            )
+        )
+        returnType: null
+        stmts: array(
+        )
+    )
+)
+```
+
+Finally, we can convert the new AST back to PHP code:
+
+```php
+use PhpParser\PrettyPrinter;
+
+$prettyPrinter = new PrettyPrinter\Standard;
+echo $prettyPrinter->prettyPrintFile($ast);
+```
+
+This gives us our original code, minus the `var_dump()` call inside the function:
+
+```php
+<?php
+
+function test($foo)
+{
+}
+```
+
+For a more comprehensive introduction, see the documentation.
 
 Documentation
 -------------
@@ -85,11 +185,14 @@ Documentation
  2. [Usage of basic components](doc/2_Usage_of_basic_components.markdown)
  3. [Other node tree representations](doc/3_Other_node_tree_representations.markdown)
  4. [Code generation](doc/4_Code_generation.markdown)
+ 5. [Frequently asked questions](doc/5_FAQ.markdown)
 
 Component documentation:
 
- 1. [Error](doc/component/Error.markdown)
- 2. [Lexer](doc/component/Lexer.markdown)
+ * [Name resolution](doc/component/Name_resolution.markdown)
+ * [Pretty printing](doc/component/Pretty_printing.markdown)
+ * [Lexer](doc/component/Lexer.markdown)
+ * [Error handling](doc/component/Error_handling.markdown)
 
- [doc_1_x]: https://github.com/nikic/PHP-Parser/tree/1.x/doc
+ [doc_3_x]: https://github.com/nikic/PHP-Parser/tree/3.x/doc
  [doc_master]: https://github.com/nikic/PHP-Parser/tree/master/doc

UPGRADE-3.0.md

@@ -0,0 +1,160 @@
+Upgrading from PHP-Parser 2.x to 3.0
+====================================
+
+The backwards-incompatible changes in this release may be summarized as follows:
+
+ * The specific details of the node representation have changed in some cases, primarily to
+   accomodate new PHP 7.1 features.
+ * There have been significant changes to the error recovery implementation. This may affect you,
+   if you used the error recovery mode or have a custom lexer implementation.
+ * A number of deprecated methods were removed.
+
+### PHP version requirements
+
+PHP-Parser now requires PHP 5.5 or newer to run. It is however still possible to *parse* PHP 5.2,
+5.3 and 5.4 source code, while running on a newer version.
+
+### Changes to the node structure
+
+The following changes are likely to require code changes if the respective nodes are used:
+
+ * The `List` subnode `vars` has been renamed to `items` and now contains `ArrayItem`s instead of
+   plain variables.
+ * The `Catch` subnode `type` has been renamed to `types` and is now an array of `Name`s.
+ * The `TryCatch` subnode `finallyStmts` has been replaced with a `finally` subnode that holds an
+   explicit `Finally` node.
+ * The `type` subnode on `Class`, `ClassMethod` and `Property` has been renamed to `flags`. The
+   `type` subnode has retained for backwards compatibility and is populated to the same value as
+   `flags`. However, writes to `type` will not update `flags` and use of `type` is discouraged.
+
+The following changes are unlikely to require code changes:
+
+ * The `ClassConst` constructor changed to accept an additional `flags` subnode.
+ * The `Trait` constructor now has the same form as the `Class` and `Interface` constructors: It
+   takes an array of subnodes. Unlike classes/interfaces, traits can only have a `stmts` subnode.
+ * The `Array` subnode `items` may now contain `null` elements (due to destructuring).
+ * `void` and `iterable` types are now stored as strings if the PHP 7 parser is used. Previously
+   these would have been represented as `Name` instances.
+
+### Changes to error recovery mode
+
+Previously, error recovery mode was enabled by setting the `throwOnError` option to `false` when
+creating the parser, while collected errors were retrieved using the `getErrors()` method:
+
+```php
+$lexer = ...;
+$parser = (new ParserFactory)->create(ParserFactor::ONLY_PHP7, $lexer, [
+    'throwOnError' => true,
+]);
+
+$stmts = $parser->parse($code);
+$errors = $parser->getErrors();
+if ($errors) {
+    handleErrors($errors);
+}
+processAst($stmts);
+```
+
+Both the `throwOnError` option and the `getErrors()` method have been removed in PHP-Parser 3.0.
+Instead an instance of `ErrorHandler\Collecting` should be passed to the `parse()` method:
+
+```php
+$lexer = ...;
+$parser = (new ParserFactory)->create(ParserFactor::ONLY_PHP7, $lexer);
+
+$errorHandler = new ErrorHandler\Collecting;
+$stmts = $parser->parse($code, $errorHandler);
+if ($errorHandler->hasErrors()) {
+    handleErrors($errorHandler->getErrors());
+}
+processAst($stmts);
+```
+
+#### Multiple parser fallback in error recovery mode
+
+As a result of this change, if a `Multiple` parser is used (e.g. through the `ParserFactory` using
+`PREFER_PHP7` or `PREFER_PHP5`), it will now return the result of the first *non-throwing* parse. As
+parsing never throws in error recovery mode, the result from the first parser will always be
+returned.
+
+The PHP 7 parser is a superset of the PHP 5 parser, with the exceptions that `=& new` and
+`global $$foo->bar` are not supported (other differences are in representation only). The PHP 7
+parser will be able to recover from the error in both cases. For this reason, this change will
+likely pass unnoticed if you do not specifically test for this syntax.
+
+It is possible to restore the precise previous behavior with the following code:
+
+```php
+$lexer = ...;
+$parser7 = new Parser\Php7($lexer);
+$parser5 = new Parser\Php5($lexer);
+
+$errors7 = new ErrorHandler\Collecting();
+$stmts7 = $parser7->parse($code, $errors7);
+if ($errors7->hasErrors()) {
+    $errors5 = new ErrorHandler\Collecting();
+    $stmts5 = $parser5->parse($code, $errors5);
+    if (!$errors5->hasErrors()) {
+        // If PHP 7 parse has errors but PHP 5 parse has no errors, use PHP 5 result
+        return [$stmts5, $errors5];
+    }
+}
+// If PHP 7 succeeds or both fail use PHP 7 result
+return [$stmts7, $errors7];
+```
+
+#### Error handling in the lexer
+
+In order to support recovery from lexer errors, the signature of the `startLexing()` method changed
+to optionally accept an `ErrorHandler`:
+
+```php
+// OLD
+public function startLexing($code);
+// NEW
+public function startLexing($code, ErrorHandler $errorHandler = null);
+```
+
+If you use a custom lexer with overriden `startLexing()` method, it needs to be changed to accept
+the extra parameter. The value should be passed on to the parent method.
+
+#### Error checks in node constructors
+
+The constructors of certain nodes used to contain additional checks for semantic errors, such as
+creating a try block without either catch or finally. These checks have been moved from the node
+constructors into the parser. This allows recovery from such errors, as well as representing the
+resulting (invalid) AST.
+
+This means that certain error conditions are no longer checked for manually constructed nodes.
+
+### Removed methods, arguments, options
+
+The following methods, arguments or options have been removed:
+
+ * `Comment::setLine()`, `Comment::setText()`: Create new `Comment` instances instead.
+ * `Name::set()`, `Name::setFirst()`, `Name::setLast()`, `Name::append()`, `Name::prepend()`:
+    Use `Name::concat()` in combination with `Name::slice()` instead.
+ * `Error::getRawLine()`, `Error::setRawLine()`. Use `Error::getStartLine()` and
+   `Error::setStartLine()` instead.
+ * `Parser::getErrors()`. Use `ErrorHandler\Collecting` instead.
+ * `$separator` argument of `Name::toString()`. Use `strtr()` instead, if you really need it.
+ * `$cloneNodes` argument of `NodeTraverser::__construct()`. Explicitly clone nodes in the visitor
+   instead.
+ * `throwOnError` parser option. Use `ErrorHandler\Collecting` instead.
+
+### Miscellaneous
+
+ * The `NameResolver` will now resolve unqualified function and constant names in the global
+   namespace into fully qualified names. For example `foo()` in the global namespace resolves to
+   `\foo()`. For names where no static resolution is possible, a `namespacedName` attribute is
+   added now, containing the namespaced variant of the name.
+ * All methods on `PrettyPrinter\Standard` are now protected. Previoulsy most of them were public.
+   The pretty printer should only be invoked using the `prettyPrint()`, `prettyPrintFile()` and
+   `prettyPrintExpr()` methods.
+ * The node dumper now prints numeric values that act as enums/flags in a string representation.
+   If node dumper results are used in tests, updates may be needed to account for this.
+ * The constants on `NameTraverserInterface` have been moved into the `NameTraverser` class.
+ * The emulative lexer now directly postprocesses tokens, instead of using `~__EMU__~` sequences.
+   This changes the protected API of the emulative lexer.
+ * The `Name::slice()` method now returns `null` for empty slices, previously `new Name([])` was
+   used. `Name::concat()` now also supports concatenation with `null`.

UPGRADE-4.0.md

@@ -0,0 +1,76 @@
+Upgrading from PHP-Parser 3.x to 4.0
+====================================
+
+### PHP version requirements
+
+PHP-Parser now requires PHP 7.0 or newer to run. It is however still possible to *parse* PHP 5.2-5.6
+source code, while running on a newer version.
+
+Because HHVM does not support PHP 7, HHVM is no longer supported.
+
+### Changes to the node structure
+
+* Many subnodes that previously held simple strings now store `Identifier` nodes instead (or
+  `VarLikeIdentifier` nodes if they have form `$ident`). The constructors of the affected nodes will
+  automatically convert strings to `Identifier`s and `Identifier`s implement `__toString()`. As such
+  some code continues to work without changes, but anything using `is_string()`, type-strict
+  comparisons or strict-mode may require adjustment. The following is an exhaustive list of all
+  affected subnodes:
+
+   * `Const::$name`
+   * `NullableType::$type` (for simple types)
+   * `Param::$type` (for simple types)
+   * `Expr\ClassConstFetch::$name`
+   * `Expr\Closure::$returnType` (for simple types)
+   * `Expr\MethodCall::$name`
+   * `Expr\PropertyFetch::$name`
+   * `Expr\StaticCall::$name`
+   * `Expr\StaticPropertyFetch::$name` (uses `VarLikeIdentifier`)
+   * `Stmt\Class_::$name`
+   * `Stmt\ClassMethod::$name`
+   * `Stmt\ClassMethod::$returnType` (for simple types)
+   * `Stmt\Function::$name`
+   * `Stmt\Function::$returnType` (for simple types)
+   * `Stmt\Goto_::$name`
+   * `Stmt\Interface_::$name`
+   * `Stmt\Label::$name`
+   * `Stmt\PropertyProperty::$name` (uses `VarLikeIdentifier`)
+   * `Stmt\TraitUseAdaptation\Alias::$method`
+   * `Stmt\TraitUseAdaptation\Alias::$newName`
+   * `Stmt\TraitUseAdaptation\Precedence::$method`
+   * `Stmt\Trait_::$name`
+   * `Stmt\UseUse::$alias`
+
+* Expression statements (`expr;`) are now represented using a `Stmt\Expression` node. Previously
+  these statements were directly represented as their constituent expression.
+* The `name` subnode of `Param` has been renamed to `var` and now contains a `Variable` rather than
+  a plain string.
+* The `name` subnode of `StaticVar` has been renamed to `var` and now contains a `Variable` rather
+  than a plain string.
+* The `var` subnode of `ClosureUse` now contains a `Variable` rather than a plain string.
+* The `var` subnode of `Catch` now contains a `Variable` rather than a plain string.
+* The `alias` subnode of `UseUse` is now `null` if no explicit alias is given. As such,
+  `use Foo\Bar` and `use Foo\Bar as Bar` are now represented differently. The `getAlias()` method
+  can be used to get the effective alias, even if it is not explicitly given.
+  
+### Miscellaneous
+
+* The indenentation handling in the pretty printer has been changed (this is only relevant if you
+  extend the pretty printer). Previously indentation was automatic, and parts were excluded using
+  `pNoindent()`. Now no-indent is the default and newlins that require indentation should use
+  `$this->nl`.
+
+### Removed functionality
+
+* Removed `type` subnode on `Class`, `ClassMethod` and `Property` nodes. Use `flags` instead.
+* The `ClassConst::isStatic()` method has been removed. Constants cannot have a static modifier.
+* The `NodeTraverser` no longer accepts `false` as a return value from a `leaveNode()` method.
+  `NodeTraverser::REMOVE_NODE` should be returned instead.
+* The `Node::setLine()` method has been removed. If you really need to, you can use `setAttribute()`
+  instead.
+* The misspelled `Class_::VISIBILITY_MODIFER_MASK` constant has been dropped in favor of
+  `Class_::VISIBILITY_MODIFIER_MASK`.
+* The XML serializer has been removed. As such, the classes `Serializer\XML`, and
+  `Unserializer\XML`, as well as the interfaces `Serializer` and `Unserializer` no longer exist.
+* The `BuilderAbstract` class has been removed. It's functionality is moved into `BuilderHelpers`.
+  However, this is an internal class and should not be used directly.

bin/php-parse

@@ -26,13 +26,18 @@ if (empty($files)) {
     showHelp("Must specify at least one file.");
 }
 
-$lexer = new PhpParser\Lexer\Emulative(array('usedAttributes' => array(
-    'startLine', 'endLine', 'startFilePos', 'endFilePos'
-)));
-$parser = (new PhpParser\ParserFactory)->create(PhpParser\ParserFactory::PREFER_PHP7, $lexer);
-$dumper = new PhpParser\NodeDumper;
+$lexer = new PhpParser\Lexer\Emulative(['usedAttributes' => [
+    'startLine', 'endLine', 'startFilePos', 'endFilePos', 'comments'
+]]);
+$parser = (new PhpParser\ParserFactory)->create(
+    PhpParser\ParserFactory::PREFER_PHP7,
+    $lexer
+);
+$dumper = new PhpParser\NodeDumper([
+    'dumpComments' => true,
+    'dumpPositions' => $attributes['with-positions'],
+]);
 $prettyPrinter = new PhpParser\PrettyPrinter\Standard;
-$serializer = new PhpParser\Serializer\XML;
 
 $traverser = new PhpParser\NodeTraverser();
 $traverser->addVisitor(new PhpParser\NodeVisitor\NameResolver);
@@ -50,32 +55,32 @@ foreach ($files as $file) {
         echo "====> File $file:\n";
     }
 
-    try {
-        $stmts = $parser->parse($code);
-    } catch (PhpParser\Error $e) {
-        if ($attributes['with-column-info'] && $e->hasColumnInfo()) {
-            $startLine = $e->getStartLine();
-            $endLine = $e->getEndLine();
-            $startColumn = $e->getStartColumn($code);
-            $endColumn   = $e->getEndColumn($code);
-            $message .= $e->getRawMessage() . " from $startLine:$startColumn to $endLine:$endColumn";
-        } else {
-            $message = $e->getMessage();
+    if ($attributes['with-recovery']) {
+        $errorHandler = new PhpParser\ErrorHandler\Collecting;
+        $stmts = $parser->parse($code, $errorHandler);
+        foreach ($errorHandler->getErrors() as $error) {
+            $message = formatErrorMessage($error, $code, $attributes['with-column-info']);
+            echo $message . "\n";
+        }
+        if (null === $stmts) {
+            continue;
+        }
+    } else {
+        try {
+            $stmts = $parser->parse($code);
+        } catch (PhpParser\Error $error) {
+            $message = formatErrorMessage($error, $code, $attributes['with-column-info']);
+            die($message . "\n");
         }
-
-        die($message . "\n");
     }
 
     foreach ($operations as $operation) {
         if ('dump' === $operation) {
             echo "==> Node dump:\n";
-            echo $dumper->dump($stmts), "\n";
+            echo $dumper->dump($stmts, $code), "\n";
         } elseif ('pretty-print' === $operation) {
             echo "==> Pretty print:\n";
             echo $prettyPrinter->prettyPrintFile($stmts), "\n";
-        } elseif ('serialize-xml' === $operation) {
-            echo "==> Serialized XML:\n";
-            echo $serializer->serialize($stmts), "\n";
         } elseif ('var-dump' === $operation) {
             echo "==> var_dump():\n";
             var_dump($stmts);
@@ -86,9 +91,19 @@ foreach ($files as $file) {
     }
 }
 
-function showHelp($error) {
-    die($error . "\n\n" .
-        <<<OUTPUT
+function formatErrorMessage(PhpParser\Error $e, $code, $withColumnInfo) {
+    if ($withColumnInfo && $e->hasColumnInfo()) {
+        return $e->getMessageWithColumnInfo($code);
+    } else {
+        return $e->getMessage();
+    }
+}
+
+function showHelp($error = '') {
+    if ($error) {
+        echo $error . "\n\n";
+    }
+    die(<<<OUTPUT
 Usage: php-parse [operations] file1.php [file2.php ...]
    or: php-parse [operations] "<?php code"
 Turn PHP source code into an abstract syntax tree.
@@ -97,10 +112,12 @@ Operations is a list of the following options (--dump by default):
 
     -d, --dump              Dump nodes using NodeDumper
     -p, --pretty-print      Pretty print file using PrettyPrinter\Standard
-        --serialize-xml     Serialize nodes using Serializer\XML
         --var-dump          var_dump() nodes (for exact structure)
     -N, --resolve-names     Resolve names using NodeVisitor\NameResolver
     -c, --with-column-info  Show column-numbers for errors (if available)
+    -P, --with-positions    Show positions in node dumps
+    -r, --with-recovery     Use parsing with error recovery
+    -h, --help              Display this page
 
 Example:
     php-parse -d -p -N -d file.php
@@ -113,11 +130,13 @@ OUTPUT
 }
 
 function parseArgs($args) {
-    $operations = array();
-    $files = array();
-    $attributes = array(
+    $operations = [];
+    $files = [];
+    $attributes = [
         'with-column-info' => false,
-    );
+        'with-positions' => false,
+        'with-recovery' => false,
+    ];
 
     array_shift($args);
     $parseOptions = true;
@@ -150,6 +169,18 @@ function parseArgs($args) {
             case '-c';
                 $attributes['with-column-info'] = true;
                 break;
+            case '--with-positions':
+            case '-P':
+                $attributes['with-positions'] = true;
+                break;
+            case '--with-recovery':
+            case '-r':
+                $attributes['with-recovery'] = true;
+                break;
+            case '--help':
+            case '-h';
+                showHelp();
+                break;
             case '--':
                 $parseOptions = false;
                 break;
@@ -162,5 +193,5 @@ function parseArgs($args) {
         }
     }
 
-    return array($operations, $files, $attributes);
+    return [$operations, $files, $attributes];
 }

composer.json

@@ -10,11 +10,11 @@
         }
     ],
     "require": {
-        "php": ">=5.4",
+        "php": ">=7.0",
         "ext-tokenizer": "*"
     },
     "require-dev": {
-        "phpunit/phpunit": "~4.0"
+        "phpunit/phpunit": "^6"
     },
     "autoload": {
         "psr-4": {
@@ -24,7 +24,7 @@
     "bin": ["bin/php-parse"],
     "extra": {
         "branch-alias": {
-            "dev-master": "2.0-dev"
+            "dev-master": "4.0-dev"
         }
     }
 }

doc/0_Introduction.markdown

@@ -1,7 +1,7 @@
 Introduction
 ============
 
-This project is a PHP 5.2 to PHP 7.0 parser **written in PHP itself**.
+This project is a PHP 5.2 to PHP 7.1 parser **written in PHP itself**.
 
 What is this for?
 -----------------
@@ -18,7 +18,7 @@ For example an AST abstracts away the fact that in PHP variables can be written
 as `$$bar`, `${'foobar'}` or even `${!${''}=barfoo()}`. You don't have to worry about recognizing
 all the different syntaxes from a stream of tokens.
 
-Another questions is: Why would I want to have a PHP parser *written in PHP*? Well, PHP might not be
+Another question is: Why would I want to have a PHP parser *written in PHP*? Well, PHP might not be
 a language especially suited for fast parsing, but processing the AST is much easier in PHP than it
 would be in other, faster languages like C. Furthermore the people most probably wanting to do
 programmatic PHP code analysis are incidentally PHP developers, not C developers.
@@ -29,9 +29,9 @@ What can it parse?
 The parser supports parsing PHP 5.2-5.6 and PHP 7.
 
 As the parser is based on the tokens returned by `token_get_all` (which is only able to lex the PHP
-version it runs on), additionally a wrapper for emulating new tokens from 5.5, 5.6 and 7.0 is
-provided. This allows to parse PHP 7.0 source code running on PHP 5.4, for example. This emulation
-is somewhat hacky and not perfect, but it should work well on any sane code.
+version it runs on), additionally a wrapper for emulating tokens from newer versions is provided.
+This allows to parse PHP 7.1 source code running on PHP 5.5, for example. This emulation is somewhat
+hacky and not perfect, but it should work well on any sane code.
 
 What output does it produce?
 ----------------------------
@@ -69,7 +69,7 @@ Apart from the parser itself this package also bundles support for some other, r
  * Support for pretty printing, which is the act of converting an AST into PHP code. Please note
    that "pretty printing" does not imply that the output is especially pretty. It's just how it's
    called ;)
- * Support for serializing and unserializing the node tree to XML
+ * Support for serializing and unserializing the node tree to JSON
  * Support for dumping the node tree in a human readable form (see the section above for an
    example of how the output looks like)
  * Infrastructure for traversing and changing the AST (node traverser and node visitors)

doc/2_Usage_of_basic_components.markdown

@@ -18,7 +18,9 @@ Additionally you may want to set the `xdebug.max_nesting_level` ini option to a
 ini_set('xdebug.max_nesting_level', 3000);
 ```
 
-This ensures that there will be no errors when traversing highly nested node trees.
+This ensures that there will be no errors when traversing highly nested node trees. However, it is
+preferable to disable XDebug completely, as it can easily make this library more than five times
+slower.
 
 Parsing
 -------
@@ -268,7 +270,7 @@ The `enterNode()` method can additionally return the value `NodeTraverser::DONT_
 which instructs the traverser to skip all children of the current node.
 
 The `leaveNode()` method can additionally return the value `NodeTraverser::REMOVE_NODE`, in which
-case the current node will be removed from the parent array. Furthermove it is possible to return
+case the current node will be removed from the parent array. Furthermore it is possible to return
 an array of nodes, which will be merged into the parent array at the offset of the current node.
 I.e. if in `array(A, B, C)` the node `B` should be replaced with `array(X, Y, Z)` the result will
 be `array(A, X, Y, Z, C)`.
@@ -363,7 +365,7 @@ class NamespaceConverter extends \PhpParser\NodeVisitorAbstract
 {
     public function leaveNode(Node $node) {
         if ($node instanceof Node\Name) {
-            return new Node\Name($node->toString('_'));
+            return new Node\Name(str_replace('\\', '_', $node->toString()));
         }
     }
 }
@@ -371,12 +373,12 @@ class NamespaceConverter extends \PhpParser\NodeVisitorAbstract
 
 The above code profits from the fact that the `NameResolver` already resolved all names as far as
 possible, so we don't need to do that. We only need to create a string with the name parts separated
-by underscores instead of backslashes. This is what `$node->toString('_')` does. (If you want to
+by underscores instead of backslashes. This is what `str_replace('\\', '_', $node->toString())` does. (If you want to
 create a name with backslashes either write `$node->toString()` or `(string) $node`.) Then we create
 a new name from the string and return it. Returning a new node replaces the old node.
 
 Another thing we need to do is change the class/function/const declarations. Currently they contain
-only the shortname (i.e. the last part of the name), but they need to contain the complete name inclduing
+only the shortname (i.e. the last part of the name), but they need to contain the complete name including
 the namespace prefix:
 
 ```php
@@ -387,14 +389,14 @@ class NodeVisitor_NamespaceConverter extends \PhpParser\NodeVisitorAbstract
 {
     public function leaveNode(Node $node) {
         if ($node instanceof Node\Name) {
-            return new Node\Name($node->toString('_'));
+            return new Node\Name(str_replace('\\', '_', $node->toString()));
         } elseif ($node instanceof Stmt\Class_
                   || $node instanceof Stmt\Interface_
                   || $node instanceof Stmt\Function_) {
-            $node->name = $node->namespacedName->toString('_');
+            $node->name = str_replace('\\', '_', $node->namespacedName->toString());
         } elseif ($node instanceof Stmt\Const_) {
             foreach ($node->consts as $const) {
-                $const->name = $const->namespacedName->toString('_');
+                $const->name = str_replace('\\', '_', $const->namespacedName->toString());
             }
         }
     }
@@ -408,26 +410,27 @@ The last thing we need to do is remove the `namespace` and `use` statements:
 ```php
 use PhpParser\Node;
 use PhpParser\Node\Stmt;
+use PhpParser\NodeTraverser;
 
 class NodeVisitor_NamespaceConverter extends \PhpParser\NodeVisitorAbstract
 {
     public function leaveNode(Node $node) {
         if ($node instanceof Node\Name) {
-            return new Node\Name($node->toString('_'));
+            return new Node\Name(str_replace('\\', '_', $node->toString()));
         } elseif ($node instanceof Stmt\Class_
                   || $node instanceof Stmt\Interface_
                   || $node instanceof Stmt\Function_) {
-            $node->name = $node->namespacedName->toString('_');
+            $node->name = str_replace('\\', '_', $node->namespacedName->toString();
         } elseif ($node instanceof Stmt\Const_) {
             foreach ($node->consts as $const) {
-                $const->name = $const->namespacedName->toString('_');
+                $const->name = str_replace('\\', '_', $const->namespacedName->toString());
             }
         } elseif ($node instanceof Stmt\Namespace_) {
             // returning an array merges is into the parent array
             return $node->stmts;
         } elseif ($node instanceof Stmt\Use_) {
-            // returning false removed the node altogether
-            return false;
+            // return use nodes altogether
+            return NodeTraverser::REMOVE_NODE;
         }
     }
 }

doc/3_Other_node_tree_representations.markdown

@@ -8,7 +8,7 @@ Simple serialization
 
 It is possible to serialize the node tree using `serialize()` and also unserialize it using
 `unserialize()`. The output is not human readable and not easily processable from anything
-but PHP, but it is compact and generates fast. The main application thus is in caching.
+but PHP, but it is compact and generates quickly. The main application thus is in caching.
 
 Human readable dumping
 ----------------------
@@ -86,15 +86,13 @@ array(
 )
 ```
 
-Serialization to XML
---------------------
+JSON encoding
+-------------
 
-It is also possible to serialize the node tree to XML using `PhpParser\Serializer\XML->serialize()`
-and to unserialize it using `PhpParser\Unserializer\XML->unserialize()`. This is useful for
-interfacing with other languages and applications or for doing transformation using XSLT.
+Nodes (and comments) implement the `JsonSerializable` interface. As such, it is possible to JSON
+encode the AST directly using `json_encode()`:
 
 ```php
-<?php
 $code = <<<'CODE'
 <?php
 
@@ -106,97 +104,125 @@ printLine('Hello World!!!');
 CODE;
 
 $parser = (new PhpParser\ParserFactory)->create(PhpParser\ParserFactory::PREFER_PHP7);
-$serializer = new PhpParser\Serializer\XML;
 
 try {
     $stmts = $parser->parse($code);
 
-    echo $serializer->serialize($stmts);
+    echo json_encode($stmts, JSON_PRETTY_PRINT), "\n";
 } catch (PhpParser\Error $e) {
     echo 'Parse Error: ', $e->getMessage();
 }
 ```
 
-Produces:
+This will result in the following output (which includes attributes):
 
-```xml
-<?xml version="1.0" encoding="UTF-8"?>
-<AST xmlns:node="http://nikic.github.com/PHPParser/XML/node" xmlns:subNode="http://nikic.github.com/PHPParser/XML/subNode" xmlns:scalar="http://nikic.github.com/PHPParser/XML/scalar">
- <scalar:array>
-  <node:Stmt_Function line="2">
-   <subNode:byRef>
-    <scalar:false/>
-   </subNode:byRef>
-   <subNode:params>
-    <scalar:array>
-     <node:Param line="2">
-      <subNode:name>
-       <scalar:string>msg</scalar:string>
-      </subNode:name>
-      <subNode:default>
-       <scalar:null/>
-      </subNode:default>
-      <subNode:type>
-       <scalar:null/>
-      </subNode:type>
-      <subNode:byRef>
-       <scalar:false/>
-      </subNode:byRef>
-     </node:Param>
-    </scalar:array>
-   </subNode:params>
-   <subNode:stmts>
-    <scalar:array>
-     <node:Stmt_Echo line="3">
-      <subNode:exprs>
-       <scalar:array>
-        <node:Expr_Variable line="3">
-         <subNode:name>
-          <scalar:string>msg</scalar:string>
-         </subNode:name>
-        </node:Expr_Variable>
-        <node:Scalar_String line="3">
-         <subNode:value>
-          <scalar:string>
-</scalar:string>
-         </subNode:value>
-        </node:Scalar_String>
-       </scalar:array>
-      </subNode:exprs>
-     </node:Stmt_Echo>
-    </scalar:array>
-   </subNode:stmts>
-   <subNode:name>
-    <scalar:string>printLine</scalar:string>
-   </subNode:name>
-  </node:Stmt_Function>
-  <node:Expr_FuncCall line="6">
-   <subNode:name>
-    <node:Name line="6">
-     <subNode:parts>
-      <scalar:array>
-       <scalar:string>printLine</scalar:string>
-      </scalar:array>
-     </subNode:parts>
-    </node:Name>
-   </subNode:name>
-   <subNode:args>
-    <scalar:array>
-     <node:Arg line="6">
-      <subNode:value>
-       <node:Scalar_String line="6">
-        <subNode:value>
-         <scalar:string>Hello World!!!</scalar:string>
-        </subNode:value>
-       </node:Scalar_String>
-      </subNode:value>
-      <subNode:byRef>
-       <scalar:false/>
-      </subNode:byRef>
-     </node:Arg>
-    </scalar:array>
-   </subNode:args>
-  </node:Expr_FuncCall>
- </scalar:array>
-</AST>
-```
+```json
+[
+    {
+        "nodeType": "Stmt_Function",
+        "byRef": false,
+        "name": "printLine",
+        "params": [
+            {
+                "nodeType": "Param",
+                "type": null,
+                "byRef": false,
+                "variadic": false,
+                "name": "msg",
+                "default": null,
+                "attributes": {
+                    "startLine": 3,
+                    "endLine": 3
+                }
+            }
+        ],
+        "returnType": null,
+        "stmts": [
+            {
+                "nodeType": "Stmt_Echo",
+                "exprs": [
+                    {
+                        "nodeType": "Expr_Variable",
+                        "name": "msg",
+                        "attributes": {
+                            "startLine": 4,
+                            "endLine": 4
+                        }
+                    },
+                    {
+                        "nodeType": "Scalar_String",
+                        "value": "\n",
+                        "attributes": {
+                            "startLine": 4,
+                            "endLine": 4,
+                            "kind": 2
+                        }
+                    }
+                ],
+                "attributes": {
+                    "startLine": 4,
+                    "endLine": 4
+                }
+            }
+        ],
+        "attributes": {
+            "startLine": 3,
+            "endLine": 5
+        }
+    },
+    {
+        "nodeType": "Expr_FuncCall",
+        "name": {
+            "nodeType": "Name",
+            "parts": [
+                "printLine"
+            ],
+            "attributes": {
+                "startLine": 7,
+                "endLine": 7
+            }
+        },
+        "args": [
+            {
+                "nodeType": "Arg",
+                "value": {
+                    "nodeType": "Scalar_String",
+                    "value": "Hello World!!!",
+                    "attributes": {
+                        "startLine": 7,
+                        "endLine": 7,
+                        "kind": 1
+                    }
+                },
+                "byRef": false,
+                "unpack": false,
+                "attributes": {
+                    "startLine": 7,
+                    "endLine": 7
+                }
+            }
+        ],
+        "attributes": {
+            "startLine": 7,
+            "endLine": 7
+        }
+    }
+]
+```
+
+The JSON representation may be converted back into a node tree using the `JsonDecoder`:
+
+```php
+<?php
+
+$nodeDecoder = new PhpParser\NodeDecoder();
+$ast = $nodeDecoder->decode($json);
+```
+
+Note that not all ASTs can be represented using JSON. In particular:
+
+ * JSON only supports UTF-8 strings.
+ * JSON does not support non-finite floating-point numbers. This can occur if the original source
+   code contains non-representable floating-pointing literals such as `1e1000`.
+
+If the node tree is not representable in JSON, the initial `json_encode()` call will fail.

doc/4_Code_generation.markdown

@@ -21,14 +21,15 @@ use PhpParser\Node;
 $factory = new BuilderFactory;
 $node = $factory->namespace('Name\Space')
     ->addStmt($factory->use('Some\Other\Thingy')->as('SomeOtherClass'))
-    ->addStmt($factory->class('SomeClass')
-        ->extend('SomeOtherClass')
+    ->addStmt($factory->class('SomeOtherClass')
+        ->extend('SomeClass')
         ->implement('A\Few', '\Interfaces')
         ->makeAbstract() // ->makeFinal()
 
         ->addStmt($factory->method('someMethod')
             ->makePublic()
             ->makeAbstract() // ->makeFinal()
+            ->setReturnType('bool')
             ->addParam($factory->param('someParam')->setTypeHint('SomeClass'))
             ->setDocComment('/**
                               * This method does something.
@@ -65,7 +66,7 @@ This will produce the following output with the standard pretty printer:
 namespace Name\Space;
 
 use Some\Other\Thingy as SomeClass;
-abstract class SomeClass extends SomeOtherClass implements A\Few, \Interfaces
+abstract class SomeOtherClass extends SomeClass implements A\Few, \Interfaces
 {
     protected $someProperty;
     private $anotherProperty = array(1, 2, 3);
@@ -74,7 +75,7 @@ abstract class SomeClass extends SomeOtherClass implements A\Few, \Interfaces
      *
      * @param SomeClass And takes a parameter
      */
-    public abstract function someMethod(SomeClass $someParam);
+    public abstract function someMethod(SomeClass $someParam) : bool;
     protected function anotherMethod($someParam = 'test')
     {
         print $someParam;

doc/5_FAQ.markdown

@@ -0,0 +1,68 @@
+Frequently Asked Questions
+==========================
+
+ * [How can the parent of a node be obtained?](#how-can-the-parent-of-a-node-be-obtained)
+ * [How can the next/previous sibling of a node be obtained?](#how-can-the-nextprevious-sibling-of-a-node-be-obtained)
+
+How can the parent of a node be obtained?
+-----
+
+The AST does not store parent nodes by default. However, it is easy to add a custom parent node
+attribute using a custom node visitor:
+
+```php
+use PhpParser\Node;
+use PhpParser\NodeVisitorAbstract;
+
+class ParentConnector extends NodeVisitorAbstract {
+    private $stack;
+    public function beforeTraverse(array $nodes) {
+        $this->stack = [];
+    }
+    public function enterNode(Node $node) {
+        if (!empty($this->stack)) {
+            $node->setAttribute('parent', $this->stack[count($this->stack)-1]);
+        }
+        $this->stack[] = $node;
+    }
+    public function leaveNode(Node $node) {
+        array_pop($this->stack);
+    }
+}
+```
+
+After running this visitor, the parent node can be obtained through `$node->getAttribute('parent')`.
+
+How can the next/previous sibling of a node be obtained?
+-----
+
+Again, siblings are not stored by default, but the visitor from the previous entry can be easily
+extended to store the previous / next node with a common parent as well:
+
+```php
+use PhpParser\Node;
+use PhpParser\NodeVisitorAbstract;
+
+class NodeConnector extends NodeVisitorAbstract {
+    private $stack;
+    private $prev;
+    public function beforeTraverse(array $nodes) {
+        $this->stack = [];
+        $this->prev = null;
+    }
+    public function enterNode(Node $node) {
+        if (!empty($this->stack)) {
+            $node->setAttribute('parent', $this->stack[count($this->stack)-1]);
+        }
+        if ($this->prev && $this->prev->getAttribute('parent') == $node->getAttribute('parent')) {
+            $node->setAttribute('prev', $this->prev);
+            $this->prev->setAttribute('next', $node);
+        }
+        $this->stack[] = $node;
+    }
+    public function leaveNode(Node $node) {
+        $this->prev = $node;
+        array_pop($this->stack);
+    }
+}
+```

doc/README.md

@@ -0,0 +1,29 @@
+Table of Contents
+=================
+
+Guide
+-----
+
+  1. [Introduction](0_Introduction.markdown)
+  2. [Usage of basic components](2_Usage_of_basic_components.markdown)
+  3. [Other node tree representations](3_Other_node_tree_representations.markdown)
+  4. [Code generation](4_Code_generation.markdown)
+  5. [Frequently asked questions](5_FAQ.markdown)
+ 
+Component documentation
+-----------------------
+ 
+  * [Name resolution](component/Name_resolution.markdown)
+    * Name resolver options
+    * Name resolution context
+  * [Pretty printing](component/Pretty_printing.markdown)
+    * Converting AST back to PHP code
+    * Customizing formatting
+    * Formatting-preserving code transformations
+  * [Lexer](component/Lexer.markdown)
+    * Lexer options
+    * Token and file positions for nodes
+    * Custom attributes
+  * [Error handling](component/Error_handling.markdown)
+    * Column information for errors
+    * Error recovery (parsing of syntactically incorrect code)

doc/component/Error_handling.markdown

@@ -35,6 +35,8 @@ the source code of the parsed file. An example for printing an error:
 if ($e->hasColumnInfo()) {
     echo $e->getRawMessage() . ' from ' . $e->getStartLine() . ':' . $e->getStartColumn($code)
         . ' to ' . $e->getEndLine() . ':' . $e->getEndColumn($code);
+    // or:
+    echo $e->getMessageWithColumnInfo();
 } else {
     echo $e->getMessage();
 }
@@ -46,27 +48,23 @@ file.
 Error recovery
 --------------
 
-> **EXPERIMENTAL**
+The error behavior of the parser (and other components) is controlled by an `ErrorHandler`. Whenever an error is
+encountered, `ErrorHandler::handleError()` is invoked. The default error handling strategy is `ErrorHandler\Throwing`,
+which will immediately throw when an error is encountered.
 
-By default the parser will throw an exception upon encountering the first error during parsing. An alternative mode is
-also supported, in which the parser will remember the error, but try to continue parsing the rest of the source code.
-
-To enable this mode the `throwOnError` parser option needs to be disabled. Any errors that occurred during parsing can
-then be retrieved using `$parser->getErrors()`. The `$parser->parse()` method will either return a partial syntax tree
-or `null` if recovery fails.
-
-A usage example:
+To instead collect all encountered errors into an array, while trying to continue parsing the rest of the source code,
+an instance of `ErrorHandler\Collecting` can be passed to the `Parser::parse()` method. A usage example:
 
 ```php
-$parser = (new PhpParser\ParserFactory)->create(PhpParser\ParserFactory::PREFER_PHP7, null, array(
-    'throwOnError' => false,
-));
+$parser = (new PhpParser\ParserFactory)->create(PhpParser\ParserFactory::ONLY_PHP7);
+$errorHandler = new PhpParser\ErrorHandler\Collecting;
 
-$stmts = $parser->parse($code);
-$errors = $parser->getErrors();
+$stmts = $parser->parse($code, $errorHandler);
 
-foreach ($errors as $error) {
-    // $error is an ordinary PhpParser\Error
+if ($errorHandler->hasErrors()) {
+    foreach ($errorHandler->getErrors() as $error) {
+        // $error is an ordinary PhpParser\Error
+    }
 }
 
 if (null !== $stmts) {
@@ -74,4 +72,4 @@ if (null !== $stmts) {
 }
 ```
 
-The error recovery implementation is experimental -- it currently won't be able to recover from many types of errors.
+The `NameResolver` visitor also accepts an `ErrorHandler` as a constructor argument.

doc/component/Lexer.markdown

@@ -27,18 +27,23 @@ The attributes used in this example match the default behavior of the lexer. The
 
  * `comments`: Array of `PhpParser\Comment` or `PhpParser\Comment\Doc` instances, representing all comments that occurred
    between the previous non-discarded token and the current one. Use of this attribute is required for the
-   `$node->getDocComment()` method to work. The attribute is also needed if you wish the pretty printer to retain
-   comments present in the original code.
+   `$node->getComments()` and `$node->getDocComment()` methods to work. The attribute is also needed if you wish the pretty
+   printer to retain comments present in the original code.
  * `startLine`: Line in which the node starts. This attribute is required for the `$node->getLine()` to work. It is also
    required if syntax errors should contain line number information.
- * `endLine`: Line in which the node ends.
- * `startTokenPos`: Offset into the token array of the first token in the node.
- * `endTokenPos`: Offset into the token array of the last token in the node.
- * `startFilePos`: Offset into the code string of the first character that is part of the node.
- * `endFilePos`: Offset into the code string of the last character that is part of the node.
+ * `endLine`: Line in which the node ends. Required for `$node->getEndLine()`.
+ * `startTokenPos`: Offset into the token array of the first token in the node. Required for `$node->getStartTokenPos()`.
+ * `endTokenPos`: Offset into the token array of the last token in the node. Required for `$node->getEndTokenPos()`.
+ * `startFilePos`: Offset into the code string of the first character that is part of the node. Required for `$node->getStartFilePos()`.
+ * `endFilePos`: Offset into the code string of the last character that is part of the node. Required for `$node->getEndFilePos()`.
 
 ### Using token positions
 
+> **Note:** The example in this section is outdated in that this information is directly available in the AST: While
+> `$property->isPublic()` does not distinguish between `public` and `var`, directly checking `$property->flags` for
+> the `$property->flags & Class_::VISIBILITY_MODIFIER_MASK) === 0` allows making this distinction without resorting to
+> tokens. However the general idea behind the example still applies in other cases.
+
 The token offset information is useful if you wish to examine the exact formatting used for a node. For example the AST
 does not distinguish whether a property was declared using `public` or using `var`, but you can retrieve this
 information based on the token position:
@@ -72,7 +77,7 @@ $lexer = new PhpParser\Lexer(array(
         'comments', 'startLine', 'endLine', 'startTokenPos', 'endTokenPos'
     )
 ));
-$parser = (new PhpParser\ParserFactory)->create(PhpParser\ParserFactory::PREFER_PHP7, $lexer);
+$parser = (new PhpParser\ParserFactory)->create(PhpParser\ParserFactory::ONLY_PHP7, $lexer);
 
 $visitor = new MyNodeVisitor();
 $traverser = new PhpParser\NodeTraverser();
@@ -95,13 +100,16 @@ Lexer extension
 
 A lexer has to define the following public interface:
 
-    void startLexing(string $code);
-    array getTokens();
-    string handleHaltCompiler();
-    int getNextToken(string &$value = null, array &$startAttributes = null, array &$endAttributes = null);
+```php
+function startLexing(string $code, ErrorHandler $errorHandler = null): void;
+function getTokens(): array;
+function handleHaltCompiler(): string;
+function getNextToken(string &$value = null, array &$startAttributes = null, array &$endAttributes = null): int;
+```
 
 The `startLexing()` method is invoked with the source code that is to be lexed (including the opening tag) whenever the
-`parse()` method of the parser is called. It can be used to reset state or preprocess the source code or tokens.
+`parse()` method of the parser is called. It can be used to reset state or preprocess the source code or tokens. The
+passes `ErrorHandler` should be used to report lexing errors.
 
 The `getTokens()` method returns the current token array, in the usual `token_get_all()` format. This method is not
 used by the parser (which uses `getNextToken()`), but is useful in combination with the token position attributes.
@@ -122,9 +130,10 @@ node and the `$endAttributes` from the last token that is part of the node.
 E.g. if the tokens `T_FUNCTION T_STRING ... '{' ... '}'` constitute a node, then the `$startAttributes` from the
 `T_FUNCTION` token will be taken and the `$endAttributes` from the `'}'` token.
 
-An application of custom attributes is storing the original formatting of literals: The parser does not retain
-information about the formatting of integers (like decimal vs. hexadecimal) or strings (like used quote type or used
-escape sequences). This can be remedied by storing the original value in an attribute:
+An application of custom attributes is storing the exact original formatting of literals: While the parser does retain
+some information about the formatting of integers (like decimal vs. hexadecimal) or strings (like used quote type), it
+does not preserve the exact original formatting (e.g. leading zeros for integers or escape sequences in strings). This
+can be remedied by storing the original value in an attribute:
 
 ```php
 use PhpParser\Lexer;
@@ -135,9 +144,10 @@ class KeepOriginalValueLexer extends Lexer // or Lexer\Emulative
     public function getNextToken(&$value = null, &$startAttributes = null, &$endAttributes = null) {
         $tokenId = parent::getNextToken($value, $startAttributes, $endAttributes);
 
-        if ($tokenId == Tokens::T_CONSTANT_ENCAPSED_STRING // non-interpolated string
-            || $tokenId == Tokens::T_LNUMBER               // integer
-            || $tokenId == Tokens::T_DNUMBER               // floating point number
+        if ($tokenId == Tokens::T_CONSTANT_ENCAPSED_STRING   // non-interpolated string
+            || $tokenId == Tokens::T_ENCAPSED_AND_WHITESPACE // interpolated string
+            || $tokenId == Tokens::T_LNUMBER                 // integer
+            || $tokenId == Tokens::T_DNUMBER                 // floating point number
         ) {
             // could also use $startAttributes, doesn't really matter here
             $endAttributes['originalValue'] = $value;

doc/component/Name_resolution.markdown

@@ -0,0 +1,87 @@
+Name resolution
+===============
+
+Since the introduction of namespaces in PHP 5.3, literal names in PHP code are subject to a
+relatively complex name resolution process, which is based on the current namespace, the current
+import table state, as well the type of the referenced symbol. PHP-Parser implements name
+resolution and related functionality, both as reusable logic (NameContext), as well as a node
+visitor (NameResolver) based on it.
+
+The NameResolver visitor
+------------------------
+
+The `NameResolver` visitor can (and for nearly all uses of the AST, is) be applied to resolve names
+to their fully-qualified form, to the degree that this is possible.
+
+```php
+$nameResolver = new PhpParser\NodeVisitor\NameResolver;
+$nodeTraverser = new PhpParser\NodeTraverser;
+$nodeTraverser->addVisitor($nameResolver);
+
+// Resolve names
+$stmts = $nodeTraverser->traverse($stmts);
+```
+
+In the default configuration, the name resolver will perform three actions:
+
+ * Declarations of functions, classes, interfaces, traits and global constants will have a
+   `namespacedName` property added, which contains the function/class/etc name including the
+   namespace prefix. For historic reasons this is a **property** rather than an attribute.
+ * Names will be replaced by fully qualified resolved names, which are instances of
+   `Node\Name\FullyQualified`.
+ * Unqualified function and constant names inside a namespace cannot be statically resolved. Inside
+   a namespace `Foo`, a call to `strlen()` may either refer to the namespaced `\Foo\strlen()`, or
+   the global `\strlen()`. Because PHP-Parser does not have the necessary context to decide this,
+   such names are left unresolved. Additionally a `namespacedName` **attribute** is added to the
+   name node.
+
+The name resolver accepts an option array as the second argument, with the following default values:
+
+```php
+$nameResolver = new PhpParser\NodeVisitor\NameResolver(null, [
+    'preserveOriginalNames' => false,
+    'replaceNodes' => true,
+]);
+```
+
+If the `preserveOriginalNames` option is enabled, then the resolved (fully qualified) name will have
+an `originalName` attribute, which contains the unresolved name.
+
+If the `replaceNodes` option is disabled, then names will no longer be resolved in-place. Instead a
+`resolvedName` attribute will be added to each name, which contains the resolved (fully qualified)
+name. Once again, if an unqualified function or constant name cannot be resolved, then the
+`resolvedName` attribute will not be present, and instead a `namespacedName` attribute is added.
+
+The `replaceNodes` attribute is useful if you wish to perform modifications on the AST, as you
+probably do not wish the resoluting code to have fully resolved names as a side-effect.
+
+The NameContext
+---------------
+
+The actual name resolution logic is implemented in the `NameContext` class, which has the following
+public API:
+
+```php
+class NameContext {
+    public function __construct(ErrorHandler $errorHandler);
+    public function startNamespace(Name $namespace = null);
+    public function addAlias(Name $name, string $aliasName, int $type, array $errorAttrs = []);
+
+    public function getNamespace();
+    public function getResolvedName(Name $name, int $type);
+    public function getResolvedClassName(Name $name) : Name;
+    public function getPossibleNames(string $name, int $type) : array;
+    public function getShortName(string $name, int $type) : Name;
+}
+```
+
+The `$type` parameters accept on of the `Stmt\Use_::TYPE_*` constants, which represent the three
+basic symbol types in PHP (functions, constants and everything else).
+
+Next to name resolution, the `NameContext` also supports the reverse operation of finding a short
+representation of a name given the current name resolution environment.
+
+The name context is intended to be used for name resolution operations outside the AST itself, such
+as class names inside doc comments. A visitor running in parallel with the name resolver can access
+the name context using `$nameResolver->getNameContext()`. Alternatively a visitor can use an
+independent context and explicitly feed `Namespace` and `Use` nodes to it.

doc/component/Pretty_printing.markdown

@@ -0,0 +1,92 @@
+Pretty printing
+===============
+
+Pretty printing is the process of converting a syntax tree back to PHP code. In its basic mode of
+operation the pretty printer provided by this library will print the AST using a certain predefined
+code style and will discard (nearly) all formatting of the original code. Because programmers tend
+to be rather picky about their code formatting, this mode of operation is not very suitable for
+refactoring code, but can be used for automatically generated code, which is usually only read for
+debugging purposes.
+
+Basic usage
+-----------
+
+```php
+$stmts = $parser->parse($code);
+
+// MODIFY $stmts here
+
+$prettyPrinter = new PhpParser\PrettyPrinter\Standard;
+$newCode = $prettyPrinter->prettyPrintFile();
+```
+
+The pretty printer has three basic printing methods: `prettyPrint()`, `prettyPrintFile()` and
+`prettyPrintExpr()`. The one that is most commonly useful is `prettyPrintFile()`, which takes an
+array of statements and produces a full PHP file, including opening `<?php`.
+
+`prettyPrint()` also takes a statement array, but produces code which is valid inside an already
+open `<?php` context. Lastly, `prettyPrintExpr()` takes an `Expr` node and prints only a single
+expression.
+
+Customizing the formatting
+--------------------------
+
+Apart from an `shortArraySyntax` option, the default pretty printer does not provide any
+functionality to customize the formatting of the generated code. The pretty printer does respect a
+number of `kind` attributes used by some notes (e.g., whether an integer should be printed as
+decimal, hexadecimal, etc), but there are no options to control brace placement or similar.
+
+If you want to make minor changes to the formatting, the easiest way is to extend the pretty printer
+and override the methods responsible for the node types you are interested in.
+
+If you want to have more fine-grained formatting control, the recommended method is to combine the
+default pretty printer with an existing library for code reformatting, such as
+[PHP-CS-Fixer](https://github.com/FriendsOfPHP/PHP-CS-Fixer).
+
+Formatting-preserving pretty printing
+-------------------------------------
+
+> **Note:** This functionality is **experimental** and not yet complete.
+
+For automated code refactoring, migration and similar, you will usually only want to modify a small
+portion of the code and leave the remainder alone. The basic pretty printer is not suitable for
+this, because it will also reformat parts of the code, which have not been modified.
+
+Since PHP-Parser 4.0 an experimental formatting-preserving pretty-printing mode is available, which
+attempts to preserve the formatting of code, those AST nodes have not changed, and only reformat
+code which has been modified or newly inserted.
+
+Use of the formatting-preservation functionality currently requires some additional preparatory
+steps:
+
+```php
+use PhpParser\{Lexer, NodeTraverser, NodeVisitor, Parser, PrettyPrinter};
+
+$lexer = new Lexer\Emulative([
+    'usedAttributes' => [
+        'comments',
+        'startLine', 'endLine',
+        'startTokenPos', 'endTokenPos',
+    ],
+]);
+$parser = new Parser\Php7($lexer);
+
+$traverser = new NodeTraverser();
+$traverser->addVisitor(new NodeVisitor\CloningVisitor());
+
+$printer = new PrettyPrinter\Standard();
+
+$oldStmts = $parser->parse($code);
+$oldTokens = $lexer->getTokens();
+
+$newStmts = $traverser->traverse($oldStmts);
+
+// MODIFY $newStmts HERE
+
+$newCode = $printer->printFormatPreserving($newStmts, $oldStmts, $oldTokens);
+```
+
+This functionality is experimental and not yet fully implemented. It should not provide incorrect
+code, but it may sometimes reformat more code than necessary. Open issues are tracked in
+[issue #344](https://github.com/nikic/PHP-Parser/issues/344). If you encounter problems while using
+this functionality, please open an issue, so we know what to prioritize.

grammar/README.md

@@ -1,13 +1,12 @@
 What do all those files mean?
 =============================
 
- * `php5.y`:            PHP 5 grammar written in a pseudo language
- * `php7.y`:            PHP 7 grammar written in a pseudo language
- * `tokens.y`:          Tokens definition shared between PHP 5 and PHP 7 grammars
- * `parser.template`:   A `kmyacc` parser prototype file for PHP
- * `tokens.template`:   A `kmyacc` prototype file for the `Tokens` class
- * `analyze.php`:       Analyzes the grammer and outputs some info about it
- * `rebuildParser.php`: Preprocesses the grammar and builds the parser using `kmyacc`
+ * `php5.y`:             PHP 5 grammar written in a pseudo language
+ * `php7.y`:             PHP 7 grammar written in a pseudo language
+ * `tokens.y`:           Tokens definition shared between PHP 5 and PHP 7 grammars
+ * `parser.template`:    A `kmyacc` parser prototype file for PHP
+ * `tokens.template`:    A `kmyacc` prototype file for the `Tokens` class
+ * `rebuildParsers.php`: Preprocesses the grammar and builds the parser using `kmyacc`
 
 .phpy pseudo language
 =====================
@@ -17,13 +16,13 @@ applied to it:
 
  * Nodes are created using the syntax `Name[..., ...]`. This is transformed into
    `new Name(..., ..., attributes())`
- * Some function-like constructs are resolved (see `rebuildParser.php` for a list)
+ * Some function-like constructs are resolved (see `rebuildParsers.php` for a list)
 
 Building the parser
 ===================
 
 In order to rebuild the parser, you need [moriyoshi's fork of kmyacc](https://github.com/moriyoshi/kmyacc-forked).
-After you compiled/installed it, run the `rebuildParser.php` script.
+After you compiled/installed it, run the `rebuildParsers.php` script.
 
 By default only the `Parser.php` is built. If you want to additionally emit debug symbols and create `y.output`, run the
 script with `--debug`. If you want to retain the preprocessed grammar pass `--keep-tmp-grammar`.

grammar/analyze.php

@@ -1,96 +0,0 @@
-<?php
-
-const GRAMMAR_FILE = './php5.y';
-
-const LIB = '(?(DEFINE)
-    (?<singleQuotedString>\'[^\\\\\']*+(?:\\\\.[^\\\\\']*+)*+\')
-    (?<doubleQuotedString>"[^\\\\"]*+(?:\\\\.[^\\\\"]*+)*+")
-    (?<string>(?&singleQuotedString)|(?&doubleQuotedString))
-    (?<comment>/\*[^*]*+(?:\*(?!/)[^*]*+)*+\*/)
-    (?<code>\{[^\'"/{}]*+(?:(?:(?&string)|(?&comment)|(?&code)|/)[^\'"/{}]*+)*+})
-)';
-
-const RULE_BLOCK = '(?<name>[a-z_]++):(?<rules>[^\'"/{};]*+(?:(?:(?&string)|(?&comment)|(?&code)|/|})[^\'"/{};]*+)*+);';
-
-$usedTerminals = array_flip(array(
-    'T_VARIABLE', 'T_STRING', 'T_INLINE_HTML', 'T_ENCAPSED_AND_WHITESPACE',
-    'T_LNUMBER', 'T_DNUMBER', 'T_CONSTANT_ENCAPSED_STRING', 'T_STRING_VARNAME', 'T_NUM_STRING'
-));
-$unusedNonterminals = array_flip(array(
-    'case_separator', 'optional_comma'
-));
-
-function regex($regex) {
-    return '~' . LIB . '(?:' . str_replace('~', '\~', $regex) . ')~';
-}
-
-function magicSplit($regex, $string) {
-    $pieces = preg_split(regex('(?:(?&string)|(?&comment)|(?&code))(*SKIP)(*FAIL)|' . $regex), $string);
-
-    foreach ($pieces as &$piece) {
-        $piece = trim($piece);
-    }
-
-    return array_filter($pieces);
-}
-
-echo '<pre>';
-
-////////////////////
-////////////////////
-////////////////////
-
-list($defs, $ruleBlocks) = magicSplit('%%', file_get_contents(GRAMMAR_FILE));
-
-if ('' !== trim(preg_replace(regex(RULE_BLOCK), '', $ruleBlocks))) {
-    die('Not all rule blocks were properly recognized!');
-}
-
-preg_match_all(regex(RULE_BLOCK), $ruleBlocks, $ruleBlocksMatches, PREG_SET_ORDER);
-foreach ($ruleBlocksMatches as $match) {
-    $ruleBlockName = $match['name'];
-    $rules = magicSplit('\|', $match['rules']);
-
-    foreach ($rules as &$rule) {
-        $parts = magicSplit('\s+', $rule);
-        $usedParts = array();
-
-        foreach ($parts as $part) {
-            if ('{' === $part[0]) {
-                preg_match_all('~\$([0-9]+)~', $part, $backReferencesMatches, PREG_SET_ORDER);
-                foreach ($backReferencesMatches as $match) {
-                    $usedParts[$match[1]] = true;
-                }
-            }
-        }
-
-        $i = 1;
-        foreach ($parts as &$part) {
-            if ('/' === $part[0]) {
-                continue;
-            }
-
-            if (isset($usedParts[$i])) {
-                if ('\'' === $part[0] || '{' === $part[0]
-                    || (ctype_upper($part[0]) && !isset($usedTerminals[$part]))
-                    || (ctype_lower($part[0]) && isset($unusedNonterminals[$part]))
-                ) {
-                    $part = '<span style="background-color: red; color: white;">' . $part . '</span>';
-                } else {
-                    $part = '<strong><em>' . $part . '</em></strong>';
-                }
-            } elseif ((ctype_upper($part[0]) && isset($usedTerminals[$part]))
-                      || (ctype_lower($part[0]) && !isset($unusedNonterminals[$part]))
-
-            ) {
-                $part = '<span style="background-color: blue; color: white;">' . $part . '</span>';
-            }
-
-            ++$i;
-        }
-
-        $rule = implode(' ', $parts);
-    }
-
-    echo $ruleBlockName, ':', "\n", '      ', implode("\n" . '    | ', $rules), "\n", ';', "\n\n";
-}

grammar/parser.template

@@ -2,8 +2,8 @@
 $meta #
 #semval($) $this->semValue
 #semval($,%t) $this->semValue
-#semval(%n) $this->stackPos-(%l-%n)
-#semval(%n,%t) $this->stackPos-(%l-%n)
+#semval(%n) $stackPos-(%l-%n)
+#semval(%n,%t) $stackPos-(%l-%n)
 
 namespace PhpParser\Parser;
 
@@ -88,16 +88,19 @@ class #(-p) extends \PhpParser\ParserAbstract
         #production-strings;
     );
 #endif
+
+    protected function initReduceCallbacks() {
+        $this->reduceCallbacks = [
 #reduce
-
-    protected function reduceRule%n() {
-        %b
-    }
+            %n => function ($stackPos) {
+                %b
+            },
 #noact
-
-    protected function reduceRule%n() {
-        $this->semValue = $this->semStack[$this->stackPos];
-    }
+            %n => function ($stackPos) {
+                $this->semValue = $this->semStack[$stackPos];
+            },
 #endreduce
+        ];
+    }
 }
 #tailcode;

grammar/php5.y

@@ -9,11 +9,17 @@ start:
     top_statement_list                                      { $$ = $this->handleNamespaces($1); }
 ;
 
-top_statement_list:
-      top_statement_list top_statement                      { pushNormalizing($1, $2); }
+top_statement_list_ex:
+      top_statement_list_ex top_statement                   { pushNormalizing($1, $2); }
     | /* empty */                                           { init(); }
 ;
 
+top_statement_list:
+      top_statement_list_ex
+          { makeNop($nop, $this->lookaheadStartAttributes, $this->endAttributes);
+            if ($nop !== null) { $1[] = $nop; } $$ = $1; }
+;
+
 reserved_non_modifiers:
       T_INCLUDE | T_INCLUDE_ONCE | T_EVAL | T_REQUIRE | T_REQUIRE_ONCE | T_LOGICAL_OR | T_LOGICAL_XOR | T_LOGICAL_AND
     | T_INSTANCEOF | T_NEW | T_CLONE | T_EXIT | T_IF | T_ELSEIF | T_ELSE | T_ENDIF | T_ECHO | T_DO | T_WHILE
@@ -21,6 +27,7 @@ reserved_non_modifiers:
     | T_FINALLY | T_THROW | T_USE | T_INSTEADOF | T_GLOBAL | T_VAR | T_UNSET | T_ISSET | T_EMPTY | T_CONTINUE | T_GOTO
     | T_FUNCTION | T_CONST | T_RETURN | T_PRINT | T_YIELD | T_LIST | T_SWITCH | T_ENDSWITCH | T_CASE | T_DEFAULT
     | T_BREAK | T_ARRAY | T_CALLABLE | T_EXTENDS | T_IMPLEMENTS | T_NAMESPACE | T_TRAIT | T_INTERFACE | T_CLASS
+    | T_CLASS_C | T_TRAIT_C | T_FUNC_C | T_METHOD_C | T_LINE | T_FILE | T_DIR | T_NS_C | T_HALT_COMPILER
 ;
 
 semi_reserved:
@@ -28,9 +35,13 @@ semi_reserved:
     | T_STATIC | T_ABSTRACT | T_FINAL | T_PRIVATE | T_PROTECTED | T_PUBLIC
 ;
 
+identifier_ex:
+      T_STRING                                              { $$ = Node\Identifier[$1]; }
+    | semi_reserved                                         { $$ = Node\Identifier[$1]; }
+;
+
 identifier:
-      T_STRING                                              { $$ = $1; }
-    | semi_reserved                                         { $$ = $1; }
+      T_STRING                                              { $$ = Node\Identifier[$1]; }
 ;
 
 namespace_name_parts:
@@ -42,15 +53,28 @@ namespace_name:
       namespace_name_parts                                  { $$ = Name[$1]; }
 ;
 
+plain_variable:
+      T_VARIABLE                                            { $$ = Expr\Variable[parseVar($1)]; }
+;
+
 top_statement:
       statement                                             { $$ = $1; }
     | function_declaration_statement                        { $$ = $1; }
     | class_declaration_statement                           { $$ = $1; }
     | T_HALT_COMPILER
           { $$ = Stmt\HaltCompiler[$this->lexer->handleHaltCompiler()]; }
-    | T_NAMESPACE namespace_name ';'                        { $$ = Stmt\Namespace_[$2, null]; }
-    | T_NAMESPACE namespace_name '{' top_statement_list '}' { $$ = Stmt\Namespace_[$2, $4]; }
-    | T_NAMESPACE '{' top_statement_list '}'                { $$ = Stmt\Namespace_[null,     $3]; }
+    | T_NAMESPACE namespace_name ';'
+          { $$ = Stmt\Namespace_[$2, null];
+            $$->setAttribute('kind', Stmt\Namespace_::KIND_SEMICOLON);
+            $this->checkNamespace($$); }
+    | T_NAMESPACE namespace_name '{' top_statement_list '}'
+          { $$ = Stmt\Namespace_[$2, $4];
+            $$->setAttribute('kind', Stmt\Namespace_::KIND_BRACED);
+            $this->checkNamespace($$); }
+    | T_NAMESPACE '{' top_statement_list '}'
+          { $$ = Stmt\Namespace_[null, $3];
+            $$->setAttribute('kind', Stmt\Namespace_::KIND_BRACED);
+            $this->checkNamespace($$); }
     | T_USE use_declarations ';'                            { $$ = Stmt\Use_[$2, Stmt\Use_::TYPE_NORMAL]; }
     | T_USE use_type use_declarations ';'                   { $$ = Stmt\Use_[$3, $2]; }
     | group_use_declaration ';'                             { $$ = $1; }
@@ -65,13 +89,13 @@ use_type:
 /* Using namespace_name_parts here to avoid s/r conflict on T_NS_SEPARATOR */
 group_use_declaration:
       T_USE use_type namespace_name_parts T_NS_SEPARATOR '{' unprefixed_use_declarations '}'
-          { $$ = Stmt\GroupUse[Name[$3], $6, $2]; }
+          { $$ = Stmt\GroupUse[new Name($3, stackAttributes(#3)), $6, $2]; }
     | T_USE use_type T_NS_SEPARATOR namespace_name_parts T_NS_SEPARATOR '{' unprefixed_use_declarations '}'
-          { $$ = Stmt\GroupUse[Name[$4], $7, $2]; }
+          { $$ = Stmt\GroupUse[new Name($4, stackAttributes(#4)), $7, $2]; }
     | T_USE namespace_name_parts T_NS_SEPARATOR '{' inline_use_declarations '}'
-          { $$ = Stmt\GroupUse[Name[$2], $5, Stmt\Use_::TYPE_UNKNOWN]; }
+          { $$ = Stmt\GroupUse[new Name($2, stackAttributes(#2)), $5, Stmt\Use_::TYPE_UNKNOWN]; }
     | T_USE T_NS_SEPARATOR namespace_name_parts T_NS_SEPARATOR '{' inline_use_declarations '}'
-          { $$ = Stmt\GroupUse[Name[$3], $6, Stmt\Use_::TYPE_UNKNOWN]; }
+          { $$ = Stmt\GroupUse[new Name($3, stackAttributes(#3)), $6, Stmt\Use_::TYPE_UNKNOWN]; }
 ;
 
 unprefixed_use_declarations:
@@ -91,8 +115,10 @@ inline_use_declarations:
 ;
 
 unprefixed_use_declaration:
-      namespace_name                                        { $$ = Stmt\UseUse[$1, null, Stmt\Use_::TYPE_UNKNOWN]; }
-    | namespace_name T_AS T_STRING                          { $$ = Stmt\UseUse[$1, $3, Stmt\Use_::TYPE_UNKNOWN]; }
+      namespace_name
+          { $$ = Stmt\UseUse[$1, null, Stmt\Use_::TYPE_UNKNOWN]; $this->checkUseUse($$, #1); }
+    | namespace_name T_AS identifier
+          { $$ = Stmt\UseUse[$1, $3, Stmt\Use_::TYPE_UNKNOWN]; $this->checkUseUse($$, #3); }
 ;
 
 use_declaration:
@@ -111,7 +137,7 @@ constant_declaration_list:
 ;
 
 constant_declaration:
-    T_STRING '=' static_scalar                              { $$ = Node\Const_[$1, $3]; }
+    identifier '=' static_scalar                            { $$ = Node\Const_[$1, $3]; }
 ;
 
 class_const_list:
@@ -120,12 +146,18 @@ class_const_list:
 ;
 
 class_const:
-    identifier '=' static_scalar                            { $$ = Node\Const_[$1, $3]; }
+    identifier_ex '=' static_scalar                         { $$ = Node\Const_[$1, $3]; }
+;
+
+inner_statement_list_ex:
+      inner_statement_list_ex inner_statement               { pushNormalizing($1, $2); }
+    | /* empty */                                           { init(); }
 ;
 
 inner_statement_list:
-      inner_statement_list inner_statement                  { pushNormalizing($1, $2); }
-    | /* empty */                                           { init(); }
+      inner_statement_list_ex
+          { makeNop($nop, $this->lookaheadStartAttributes, $this->endAttributes);
+            if ($nop !== null) { $1[] = $nop; } $$ = $1; }
 ;
 
 inner_statement:
@@ -136,8 +168,16 @@ inner_statement:
           { throw new Error('__HALT_COMPILER() can only be used from the outermost scope', attributes()); }
 ;
 
-statement:
-      '{' inner_statement_list '}'                          { $$ = $2; }
+non_empty_statement:
+      '{' inner_statement_list '}'
+    {
+        if ($2) {
+            $$ = $2; prependLeadingComments($$);
+        } else {
+            makeNop($$, $this->startAttributeStack[#1], $this->endAttributes);
+            if (null === $$) { $$ = array(); }
+        }
+    }
     | T_IF parentheses_expr statement elseif_list else_single
           { $$ = Stmt\If_[$2, ['stmts' => toArray($3), 'elseifs' => $4, 'else' => $5]]; }
     | T_IF parentheses_expr ':' inner_statement_list new_elseif_list new_else_single T_ENDIF ';'
@@ -153,40 +193,47 @@ statement:
     | T_CONTINUE expr ';'                                   { $$ = Stmt\Continue_[$2]; }
     | T_RETURN ';'                                          { $$ = Stmt\Return_[null]; }
     | T_RETURN expr ';'                                     { $$ = Stmt\Return_[$2]; }
-    | yield_expr ';'                                        { $$ = $1; }
     | T_GLOBAL global_var_list ';'                          { $$ = Stmt\Global_[$2]; }
     | T_STATIC static_var_list ';'                          { $$ = Stmt\Static_[$2]; }
     | T_ECHO expr_list ';'                                  { $$ = Stmt\Echo_[$2]; }
     | T_INLINE_HTML                                         { $$ = Stmt\InlineHTML[$1]; }
-    | expr ';'                                              { $$ = $1; }
+    | yield_expr ';'                                        { $$ = Stmt\Expression[$1]; }
+    | expr ';'                                              { $$ = Stmt\Expression[$1]; }
     | T_UNSET '(' variables_list ')' ';'                    { $$ = Stmt\Unset_[$3]; }
     | T_FOREACH '(' expr T_AS foreach_variable ')' foreach_statement
           { $$ = Stmt\Foreach_[$3, $5[0], ['keyVar' => null, 'byRef' => $5[1], 'stmts' => $7]]; }
     | T_FOREACH '(' expr T_AS variable T_DOUBLE_ARROW foreach_variable ')' foreach_statement
           { $$ = Stmt\Foreach_[$3, $7[0], ['keyVar' => $5, 'byRef' => $7[1], 'stmts' => $9]]; }
     | T_DECLARE '(' declare_list ')' declare_statement      { $$ = Stmt\Declare_[$3, $5]; }
-    | ';'                                                   { $$ = array(); /* means: no statement */ }
     | T_TRY '{' inner_statement_list '}' catches optional_finally
-          { $$ = Stmt\TryCatch[$3, $5, $6]; }
+          { $$ = Stmt\TryCatch[$3, $5, $6]; $this->checkTryCatch($$); }
     | T_THROW expr ';'                                      { $$ = Stmt\Throw_[$2]; }
-    | T_GOTO T_STRING ';'                                   { $$ = Stmt\Goto_[$2]; }
-    | T_STRING ':'                                          { $$ = Stmt\Label[$1]; }
+    | T_GOTO identifier ';'                                 { $$ = Stmt\Goto_[$2]; }
+    | identifier ':'                                        { $$ = Stmt\Label[$1]; }
+    | expr error                                            { $$ = Stmt\Expression[$1]; }
     | error                                                 { $$ = array(); /* means: no statement */ }
 ;
 
+statement:
+      non_empty_statement                                   { $$ = $1; }
+    | ';'
+          { makeNop($$, $this->startAttributeStack[#1], $this->endAttributes);
+            if ($$ === null) $$ = array(); /* means: no statement */ }
+;
+
 catches:
       /* empty */                                           { init(); }
     | catches catch                                         { push($1, $2); }
 ;
 
 catch:
-    T_CATCH '(' name T_VARIABLE ')' '{' inner_statement_list '}'
-        { $$ = Stmt\Catch_[$3, parseVar($4), $7]; }
+    T_CATCH '(' name plain_variable ')' '{' inner_statement_list '}'
+        { $$ = Stmt\Catch_[array($3), $4, $7]; }
 ;
 
 optional_finally:
       /* empty */                                           { $$ = null; }
-    | T_FINALLY '{' inner_statement_list '}'                { $$ = $3; }
+    | T_FINALLY '{' inner_statement_list '}'                { $$ = Stmt\Finally_[$3]; }
 ;
 
 variables_list:
@@ -205,17 +252,19 @@ optional_ellipsis:
 ;
 
 function_declaration_statement:
-    T_FUNCTION optional_ref T_STRING '(' parameter_list ')' optional_return_type '{' inner_statement_list '}'
+    T_FUNCTION optional_ref identifier '(' parameter_list ')' optional_return_type '{' inner_statement_list '}'
         { $$ = Stmt\Function_[$3, ['byRef' => $2, 'params' => $5, 'returnType' => $7, 'stmts' => $9]]; }
 ;
 
 class_declaration_statement:
-      class_entry_type T_STRING extends_from implements_list '{' class_statement_list '}'
-          { $$ = Stmt\Class_[$2, ['type' => $1, 'extends' => $3, 'implements' => $4, 'stmts' => $6]]; }
-    | T_INTERFACE T_STRING interface_extends_list '{' class_statement_list '}'
-          { $$ = Stmt\Interface_[$2, ['extends' => $3, 'stmts' => $5]]; }
-    | T_TRAIT T_STRING '{' class_statement_list '}'
-          { $$ = Stmt\Trait_[$2, $4]; }
+      class_entry_type identifier extends_from implements_list '{' class_statement_list '}'
+          { $$ = Stmt\Class_[$2, ['type' => $1, 'extends' => $3, 'implements' => $4, 'stmts' => $6]];
+            $this->checkClass($$, #2); }
+    | T_INTERFACE identifier interface_extends_list '{' class_statement_list '}'
+          { $$ = Stmt\Interface_[$2, ['extends' => $3, 'stmts' => $5]];
+            $this->checkInterface($$, #2); }
+    | T_TRAIT identifier '{' class_statement_list '}'
+          { $$ = Stmt\Trait_[$2, ['stmts' => $4]]; }
 ;
 
 class_entry_type:
@@ -226,22 +275,22 @@ class_entry_type:
 
 extends_from:
       /* empty */                                           { $$ = null; }
-    | T_EXTENDS name                                        { $$ = $2; }
+    | T_EXTENDS class_name                                  { $$ = $2; }
 ;
 
 interface_extends_list:
       /* empty */                                           { $$ = array(); }
-    | T_EXTENDS name_list                                   { $$ = $2; }
+    | T_EXTENDS class_name_list                             { $$ = $2; }
 ;
 
 implements_list:
       /* empty */                                           { $$ = array(); }
-    | T_IMPLEMENTS name_list                                { $$ = $2; }
+    | T_IMPLEMENTS class_name_list                          { $$ = $2; }
 ;
 
-name_list:
-      name                                                  { init($1); }
-    | name_list ',' name                                    { push($1, $3); }
+class_name_list:
+      class_name                                            { init($1); }
+    | class_name_list ',' class_name                        { push($1, $3); }
 ;
 
 for_statement:
@@ -255,7 +304,8 @@ foreach_statement:
 ;
 
 declare_statement:
-      statement                                             { $$ = toArray($1); }
+      non_empty_statement                                   { $$ = toArray($1); }
+    | ';'                                                   { $$ = null; }
     | ':' inner_statement_list T_ENDDECLARE ';'             { $$ = $2; }
 ;
 
@@ -265,7 +315,7 @@ declare_list:
 ;
 
 declare_list_element:
-      T_STRING '=' static_scalar                            { $$ = Stmt\DeclareDeclare[$1, $3]; }
+      identifier '=' static_scalar                          { $$ = Stmt\DeclareDeclare[$1, $3]; }
 ;
 
 switch_case_list:
@@ -340,16 +390,16 @@ non_empty_parameter_list:
 ;
 
 parameter:
-      optional_param_type optional_ref optional_ellipsis T_VARIABLE
-          { $$ = Node\Param[parseVar($4), null, $1, $2, $3]; }
-    | optional_param_type optional_ref optional_ellipsis T_VARIABLE '=' static_scalar
-          { $$ = Node\Param[parseVar($4), $6, $1, $2, $3]; }
+      optional_param_type optional_ref optional_ellipsis plain_variable
+          { $$ = Node\Param[$4, null, $1, $2, $3]; $this->checkParam($$); }
+    | optional_param_type optional_ref optional_ellipsis plain_variable '=' static_scalar
+          { $$ = Node\Param[$4, $6, $1, $2, $3]; $this->checkParam($$); }
 ;
 
 type:
       name                                                  { $$ = $1; }
-    | T_ARRAY                                               { $$ = 'array'; }
-    | T_CALLABLE                                            { $$ = 'callable'; }
+    | T_ARRAY                                               { $$ = Node\Identifier['array']; }
+    | T_CALLABLE                                            { $$ = Node\Identifier['callable']; }
 ;
 
 optional_param_type:
@@ -385,7 +435,7 @@ global_var_list:
 ;
 
 global_var:
-      T_VARIABLE                                            { $$ = Expr\Variable[parseVar($1)]; }
+      plain_variable                                        { $$ = $1; }
     | '$' variable                                          { $$ = Expr\Variable[$2]; }
     | '$' '{' expr '}'                                      { $$ = Expr\Variable[$3]; }
 ;
@@ -396,8 +446,8 @@ static_var_list:
 ;
 
 static_var:
-      T_VARIABLE                                            { $$ = Stmt\StaticVar[parseVar($1), null]; }
-    | T_VARIABLE '=' static_scalar                          { $$ = Stmt\StaticVar[parseVar($1), $3]; }
+      plain_variable                                        { $$ = Stmt\StaticVar[$1, null]; }
+    | plain_variable '=' static_scalar                      { $$ = Stmt\StaticVar[$1, $3]; }
 ;
 
 class_statement_list:
@@ -406,11 +456,13 @@ class_statement_list:
 ;
 
 class_statement:
-      variable_modifiers property_declaration_list ';'      { $$ = Stmt\Property[$1, $2]; }
-    | T_CONST class_const_list ';'                          { $$ = Stmt\ClassConst[$2]; }
-    | method_modifiers T_FUNCTION optional_ref identifier '(' parameter_list ')' optional_return_type method_body
-          { $$ = Stmt\ClassMethod[$4, ['type' => $1, 'byRef' => $3, 'params' => $6, 'returnType' => $8, 'stmts' => $9]]; }
-    | T_USE name_list trait_adaptations                     { $$ = Stmt\TraitUse[$2, $3]; }
+      variable_modifiers property_declaration_list ';'
+          { $$ = Stmt\Property[$1, $2]; $this->checkProperty($$, #1); }
+    | T_CONST class_const_list ';'                          { $$ = Stmt\ClassConst[$2, 0]; }
+    | method_modifiers T_FUNCTION optional_ref identifier_ex '(' parameter_list ')' optional_return_type method_body
+          { $$ = Stmt\ClassMethod[$4, ['type' => $1, 'byRef' => $3, 'params' => $6, 'returnType' => $8, 'stmts' => $9]];
+            $this->checkClassMethod($$, #1); }
+    | T_USE class_name_list trait_adaptations               { $$ = Stmt\TraitUse[$2, $3]; }
 ;
 
 trait_adaptations:
@@ -424,24 +476,24 @@ trait_adaptation_list:
 ;
 
 trait_adaptation:
-      trait_method_reference_fully_qualified T_INSTEADOF name_list ';'
+      trait_method_reference_fully_qualified T_INSTEADOF class_name_list ';'
           { $$ = Stmt\TraitUseAdaptation\Precedence[$1[0], $1[1], $3]; }
-    | trait_method_reference T_AS member_modifier identifier ';'
+    | trait_method_reference T_AS member_modifier identifier_ex ';'
           { $$ = Stmt\TraitUseAdaptation\Alias[$1[0], $1[1], $3, $4]; }
     | trait_method_reference T_AS member_modifier ';'
           { $$ = Stmt\TraitUseAdaptation\Alias[$1[0], $1[1], $3, null]; }
-    | trait_method_reference T_AS T_STRING ';'
+    | trait_method_reference T_AS identifier ';'
           { $$ = Stmt\TraitUseAdaptation\Alias[$1[0], $1[1], null, $3]; }
     | trait_method_reference T_AS reserved_non_modifiers ';'
           { $$ = Stmt\TraitUseAdaptation\Alias[$1[0], $1[1], null, $3]; }
 ;
 
 trait_method_reference_fully_qualified:
-      name T_PAAMAYIM_NEKUDOTAYIM identifier                { $$ = array($1, $3); }
+      name T_PAAMAYIM_NEKUDOTAYIM identifier_ex             { $$ = array($1, $3); }
 ;
 trait_method_reference:
       trait_method_reference_fully_qualified                { $$ = $1; }
-    | identifier                                            { $$ = array(null, $1); }
+    | identifier_ex                                         { $$ = array(null, $1); }
 ;
 
 method_body:
@@ -461,7 +513,7 @@ method_modifiers:
 
 non_empty_member_modifiers:
       member_modifier                                       { $$ = $1; }
-    | non_empty_member_modifiers member_modifier            { Stmt\Class_::verifyModifier($1, $2); $$ = $1 | $2; }
+    | non_empty_member_modifiers member_modifier            { $this->checkModifier($1, $2, #2); $$ = $1 | $2; }
 ;
 
 member_modifier:
@@ -478,9 +530,13 @@ property_declaration_list:
     | property_declaration_list ',' property_declaration    { push($1, $3); }
 ;
 
+property_decl_name:
+      T_VARIABLE                                            { $$ = Node\VarLikeIdentifier[parseVar($1)]; }
+;
+
 property_declaration:
-      T_VARIABLE                                            { $$ = Stmt\PropertyProperty[parseVar($1), null]; }
-    | T_VARIABLE '=' static_scalar                          { $$ = Stmt\PropertyProperty[parseVar($1), $3]; }
+      property_decl_name                                    { $$ = Stmt\PropertyProperty[$1, null]; }
+    | property_decl_name '=' static_scalar                  { $$ = Stmt\PropertyProperty[$1, $3]; }
 ;
 
 expr_list:
@@ -568,7 +624,10 @@ expr:
     | T_OBJECT_CAST expr                                    { $$ = Expr\Cast\Object_ [$2]; }
     | T_BOOL_CAST expr                                      { $$ = Expr\Cast\Bool_   [$2]; }
     | T_UNSET_CAST expr                                     { $$ = Expr\Cast\Unset_  [$2]; }
-    | T_EXIT exit_expr                                      { $$ = Expr\Exit_        [$2]; }
+    | T_EXIT exit_expr
+          { $attrs = attributes();
+            $attrs['kind'] = strtolower($1) === 'exit' ? Expr\Exit_::KIND_EXIT : Expr\Exit_::KIND_DIE;
+            $$ = new Expr\Exit_($2, $attrs); }
     | '@' expr                                              { $$ = Expr\ErrorSuppress[$2]; }
     | scalar                                                { $$ = $1; }
     | array_expr                                            { $$ = $1; }
@@ -596,14 +655,19 @@ yield_expr:
 ;
 
 array_expr:
-      T_ARRAY '(' array_pair_list ')'                       { $$ = Expr\Array_[$3]; }
-    | '[' array_pair_list ']'                               { $$ = Expr\Array_[$2]; }
+      T_ARRAY '(' array_pair_list ')'
+          { $attrs = attributes(); $attrs['kind'] = Expr\Array_::KIND_LONG;
+            $$ = new Expr\Array_($3, $attrs); }
+    | '[' array_pair_list ']'
+          { $attrs = attributes(); $attrs['kind'] = Expr\Array_::KIND_SHORT;
+            $$ = new Expr\Array_($2, $attrs); }
 ;
 
 scalar_dereference:
       array_expr '[' dim_offset ']'                         { $$ = Expr\ArrayDimFetch[$1, $3]; }
     | T_CONSTANT_ENCAPSED_STRING '[' dim_offset ']'
-          { $$ = Expr\ArrayDimFetch[Scalar\String_[Scalar\String_::parse($1, false)], $3]; }
+          { $attrs = attributes(); $attrs['kind'] = strKind($1);
+            $$ = Expr\ArrayDimFetch[new Scalar\String_(Scalar\String_::parse($1), $attrs), $3]; }
     | constant '[' dim_offset ']'                           { $$ = Expr\ArrayDimFetch[$1, $3]; }
     | scalar_dereference '[' dim_offset ']'                 { $$ = Expr\ArrayDimFetch[$1, $3]; }
     /* alternative array syntax missing intentionally */
@@ -611,7 +675,9 @@ scalar_dereference:
 
 anonymous_class:
       T_CLASS ctor_arguments extends_from implements_list '{' class_statement_list '}'
-          { $$ = array(Stmt\Class_[null, ['type' => 0, 'extends' => $3, 'implements' => $4, 'stmts' => $6]], $2); }
+          { $$ = array(Stmt\Class_[null, ['type' => 0, 'extends' => $3, 'implements' => $4, 'stmts' => $6]], $2);
+            $this->checkClass($$[0], -1); }
+;
 
 new_expr:
       T_NEW class_name_reference ctor_arguments             { $$ = Expr\New_[$2, $3]; }
@@ -630,30 +696,17 @@ lexical_var_list:
 ;
 
 lexical_var:
-      optional_ref T_VARIABLE                               { $$ = Expr\ClosureUse[parseVar($2), $1]; }
+      optional_ref plain_variable                           { $$ = Expr\ClosureUse[$2, $1]; }
 ;
 
 function_call:
       name argument_list                                    { $$ = Expr\FuncCall[$1, $2]; }
-    | class_name_or_var T_PAAMAYIM_NEKUDOTAYIM identifier argument_list
+    | class_name_or_var T_PAAMAYIM_NEKUDOTAYIM identifier_ex argument_list
           { $$ = Expr\StaticCall[$1, $3, $4]; }
     | class_name_or_var T_PAAMAYIM_NEKUDOTAYIM '{' expr '}' argument_list
           { $$ = Expr\StaticCall[$1, $4, $6]; }
-    | static_property argument_list {
-            if ($1 instanceof Node\Expr\StaticPropertyFetch) {
-                $$ = Expr\StaticCall[$1->class, Expr\Variable[$1->name], $2];
-            } elseif ($1 instanceof Node\Expr\ArrayDimFetch) {
-                $tmp = $1;
-                while ($tmp->var instanceof Node\Expr\ArrayDimFetch) {
-                    $tmp = $tmp->var;
-                }
-
-                $$ = Expr\StaticCall[$tmp->var->class, $1, $2];
-                $tmp->var = Expr\Variable[$tmp->var->name];
-            } else {
-                throw new \Exception;
-            }
-          }
+    | static_property argument_list
+          { $$ = $this->fixupPhp5StaticPropCall($1, $2, attributes()); }
     | variable_without_objects argument_list
           { $$ = Expr\FuncCall[$1, $2]; }
     | function_call '[' dim_offset ']'                      { $$ = Expr\ArrayDimFetch[$1, $3]; }
@@ -714,9 +767,11 @@ ctor_arguments:
 ;
 
 common_scalar:
-      T_LNUMBER                                             { $$ = Scalar\LNumber[Scalar\LNumber::parse($1)]; }
+      T_LNUMBER                                             { $$ = $this->parseLNumber($1, attributes(), true); }
     | T_DNUMBER                                             { $$ = Scalar\DNumber[Scalar\DNumber::parse($1)]; }
-    | T_CONSTANT_ENCAPSED_STRING                            { $$ = Scalar\String_[Scalar\String_::parse($1, false)]; }
+    | T_CONSTANT_ENCAPSED_STRING
+          { $attrs = attributes(); $attrs['kind'] = strKind($1);
+            $$ = new Scalar\String_(Scalar\String_::parse($1, false), $attrs); }
     | T_LINE                                                { $$ = Scalar\MagicConst\Line[]; }
     | T_FILE                                                { $$ = Scalar\MagicConst\File[]; }
     | T_DIR                                                 { $$ = Scalar\MagicConst\Dir[]; }
@@ -726,14 +781,16 @@ common_scalar:
     | T_FUNC_C                                              { $$ = Scalar\MagicConst\Function_[]; }
     | T_NS_C                                                { $$ = Scalar\MagicConst\Namespace_[]; }
     | T_START_HEREDOC T_ENCAPSED_AND_WHITESPACE T_END_HEREDOC
-          { $$ = Scalar\String_[Scalar\String_::parseDocString($1, $2, false)]; }
+          { $attrs = attributes(); setDocStringAttrs($attrs, $1);
+            $$ = new Scalar\String_(Scalar\String_::parseDocString($1, $2, false), $attrs); }
     | T_START_HEREDOC T_END_HEREDOC
-          { $$ = Scalar\String_['']; }
+          { $attrs = attributes(); setDocStringAttrs($attrs, $1);
+            $$ = new Scalar\String_('', $attrs); }
 ;
 
 static_scalar:
       common_scalar                                         { $$ = $1; }
-    | class_name T_PAAMAYIM_NEKUDOTAYIM identifier          { $$ = Expr\ClassConstFetch[$1, $3]; }
+    | class_name T_PAAMAYIM_NEKUDOTAYIM identifier_ex       { $$ = Expr\ClassConstFetch[$1, $3]; }
     | name                                                  { $$ = Expr\ConstFetch[$1]; }
     | T_ARRAY '(' static_array_pair_list ')'                { $$ = Expr\Array_[$3]; }
     | '[' static_array_pair_list ']'                        { $$ = Expr\Array_[$2]; }
@@ -778,7 +835,7 @@ static_operation:
 
 constant:
       name                                                  { $$ = Expr\ConstFetch[$1]; }
-    | class_name_or_var T_PAAMAYIM_NEKUDOTAYIM identifier
+    | class_name_or_var T_PAAMAYIM_NEKUDOTAYIM identifier_ex
           { $$ = Expr\ClassConstFetch[$1, $3]; }
 ;
 
@@ -786,9 +843,11 @@ scalar:
       common_scalar                                         { $$ = $1; }
     | constant                                              { $$ = $1; }
     | '"' encaps_list '"'
-          { parseEncapsed($2, '"', false); $$ = Scalar\Encapsed[$2]; }
+          { $attrs = attributes(); $attrs['kind'] = Scalar\String_::KIND_DOUBLE_QUOTED;
+            parseEncapsed($2, '"', true); $$ = new Scalar\Encapsed($2, $attrs); }
     | T_START_HEREDOC encaps_list T_END_HEREDOC
-          { parseEncapsedDoc($2, false); $$ = Scalar\Encapsed[$2]; }
+          { $attrs = attributes(); setDocStringAttrs($attrs, $1);
+            parseEncapsedDoc($2, true); $$ = new Scalar\Encapsed($2, $attrs); }
 ;
 
 static_array_pair_list:
@@ -855,9 +914,14 @@ static_property:
     | static_property_with_arrays                           { $$ = $1; }
 ;
 
+static_property_simple_name:
+      T_VARIABLE
+          { $var = parseVar($1); $$ = \is_string($var) ? Node\VarLikeIdentifier[$var] : $var; }
+;
+
 static_property_with_arrays:
-      class_name_or_var T_PAAMAYIM_NEKUDOTAYIM T_VARIABLE
-          { $$ = Expr\StaticPropertyFetch[$1, parseVar($3)]; }
+      class_name_or_var T_PAAMAYIM_NEKUDOTAYIM static_property_simple_name
+          { $$ = Expr\StaticPropertyFetch[$1, $3]; }
     | class_name_or_var T_PAAMAYIM_NEKUDOTAYIM '$' '{' expr '}'
           { $$ = Expr\StaticPropertyFetch[$1, $5]; }
     | static_property_with_arrays '[' dim_offset ']'        { $$ = Expr\ArrayDimFetch[$1, $3]; }
@@ -867,7 +931,7 @@ static_property_with_arrays:
 reference_variable:
       reference_variable '[' dim_offset ']'                 { $$ = Expr\ArrayDimFetch[$1, $3]; }
     | reference_variable '{' expr '}'                       { $$ = Expr\ArrayDimFetch[$1, $3]; }
-    | T_VARIABLE                                            { $$ = Expr\Variable[parseVar($1)]; }
+    | plain_variable                                        { $$ = $1; }
     | '$' '{' expr '}'                                      { $$ = Expr\Variable[$3]; }
 ;
 
@@ -877,9 +941,10 @@ dim_offset:
 ;
 
 object_property:
-      T_STRING                                              { $$ = $1; }
+      identifier                                            { $$ = $1; }
     | '{' expr '}'                                          { $$ = $2; }
     | variable_without_objects                              { $$ = $1; }
+    | error                                                 { $$ = Expr\Error[]; $this->errorState = 2; }
 ;
 
 list_expr:
@@ -892,8 +957,8 @@ list_expr_elements:
 ;
 
 list_expr_element:
-      variable                                              { $$ = $1; }
-    | list_expr                                             { $$ = $1; }
+      variable                                              { $$ = Expr\ArrayItem[$1, null, false]; }
+    | list_expr                                             { $$ = Expr\ArrayItem[$1, null, false]; }
     | /* empty */                                           { $$ = null; }
 ;
 
@@ -925,21 +990,25 @@ encaps_string_part:
       T_ENCAPSED_AND_WHITESPACE                             { $$ = Scalar\EncapsedStringPart[$1]; }
 ;
 
+encaps_str_varname:
+      T_STRING_VARNAME                                      { $$ = Expr\Variable[$1]; }
+;
+
 encaps_var:
-      T_VARIABLE                                            { $$ = Expr\Variable[parseVar($1)]; }
-    | T_VARIABLE '[' encaps_var_offset ']'                  { $$ = Expr\ArrayDimFetch[Expr\Variable[parseVar($1)], $3]; }
-    | T_VARIABLE T_OBJECT_OPERATOR T_STRING                 { $$ = Expr\PropertyFetch[Expr\Variable[parseVar($1)], $3]; }
+      plain_variable                                        { $$ = $1; }
+    | plain_variable '[' encaps_var_offset ']'              { $$ = Expr\ArrayDimFetch[$1, $3]; }
+    | plain_variable T_OBJECT_OPERATOR identifier           { $$ = Expr\PropertyFetch[$1, $3]; }
     | T_DOLLAR_OPEN_CURLY_BRACES expr '}'                   { $$ = Expr\Variable[$2]; }
     | T_DOLLAR_OPEN_CURLY_BRACES T_STRING_VARNAME '}'       { $$ = Expr\Variable[$2]; }
-    | T_DOLLAR_OPEN_CURLY_BRACES T_STRING_VARNAME '[' expr ']' '}'
-          { $$ = Expr\ArrayDimFetch[Expr\Variable[$2], $4]; }
+    | T_DOLLAR_OPEN_CURLY_BRACES encaps_str_varname '[' expr ']' '}'
+          { $$ = Expr\ArrayDimFetch[$2, $4]; }
     | T_CURLY_OPEN variable '}'                             { $$ = $2; }
 ;
 
 encaps_var_offset:
       T_STRING                                              { $$ = Scalar\String_[$1]; }
-    | T_NUM_STRING                                          { $$ = Scalar\String_[$1]; }
-    | T_VARIABLE                                            { $$ = Expr\Variable[parseVar($1)]; }
+    | T_NUM_STRING                                          { $$ = $this->parseNumString($1, attributes()); }
+    | plain_variable                                        { $$ = $1; }
 ;
 
 %%

grammar/php7.y

@@ -9,11 +9,17 @@ start:
     top_statement_list                                      { $$ = $this->handleNamespaces($1); }
 ;
 
-top_statement_list:
-      top_statement_list top_statement                      { pushNormalizing($1, $2); }
+top_statement_list_ex:
+      top_statement_list_ex top_statement                   { pushNormalizing($1, $2); }
     | /* empty */                                           { init(); }
 ;
 
+top_statement_list:
+      top_statement_list_ex
+          { makeNop($nop, $this->lookaheadStartAttributes, $this->endAttributes);
+            if ($nop !== null) { $1[] = $nop; } $$ = $1; }
+;
+
 reserved_non_modifiers:
       T_INCLUDE | T_INCLUDE_ONCE | T_EVAL | T_REQUIRE | T_REQUIRE_ONCE | T_LOGICAL_OR | T_LOGICAL_XOR | T_LOGICAL_AND
     | T_INSTANCEOF | T_NEW | T_CLONE | T_EXIT | T_IF | T_ELSEIF | T_ELSE | T_ENDIF | T_ECHO | T_DO | T_WHILE
@@ -21,6 +27,7 @@ reserved_non_modifiers:
     | T_FINALLY | T_THROW | T_USE | T_INSTEADOF | T_GLOBAL | T_VAR | T_UNSET | T_ISSET | T_EMPTY | T_CONTINUE | T_GOTO
     | T_FUNCTION | T_CONST | T_RETURN | T_PRINT | T_YIELD | T_LIST | T_SWITCH | T_ENDSWITCH | T_CASE | T_DEFAULT
     | T_BREAK | T_ARRAY | T_CALLABLE | T_EXTENDS | T_IMPLEMENTS | T_NAMESPACE | T_TRAIT | T_INTERFACE | T_CLASS
+    | T_CLASS_C | T_TRAIT_C | T_FUNC_C | T_METHOD_C | T_LINE | T_FILE | T_DIR | T_NS_C | T_HALT_COMPILER
 ;
 
 semi_reserved:
@@ -28,9 +35,13 @@ semi_reserved:
     | T_STATIC | T_ABSTRACT | T_FINAL | T_PRIVATE | T_PROTECTED | T_PUBLIC
 ;
 
+identifier_ex:
+      T_STRING                                              { $$ = Node\Identifier[$1]; }
+    | semi_reserved                                         { $$ = Node\Identifier[$1]; }
+;
+
 identifier:
-      T_STRING                                              { $$ = $1; }
-    | semi_reserved                                         { $$ = $1; }
+      T_STRING                                              { $$ = Node\Identifier[$1]; }
 ;
 
 namespace_name_parts:
@@ -42,19 +53,46 @@ namespace_name:
       namespace_name_parts                                  { $$ = Name[$1]; }
 ;
 
+plain_variable:
+      T_VARIABLE                                            { $$ = Expr\Variable[parseVar($1)]; }
+;
+
+semi:
+      ';'                                                   { /* nothing */ }
+    | error                                                 { /* nothing */ }
+;
+
+no_comma:
+      /* empty */ { /* nothing */ }
+    | ',' { $this->emitError(new Error('A trailing comma is not allowed here', attributes())); }
+;
+
+optional_comma:
+      /* empty */
+    | ','
+
 top_statement:
       statement                                             { $$ = $1; }
     | function_declaration_statement                        { $$ = $1; }
     | class_declaration_statement                           { $$ = $1; }
     | T_HALT_COMPILER
           { $$ = Stmt\HaltCompiler[$this->lexer->handleHaltCompiler()]; }
-    | T_NAMESPACE namespace_name ';'                        { $$ = Stmt\Namespace_[$2, null]; }
-    | T_NAMESPACE namespace_name '{' top_statement_list '}' { $$ = Stmt\Namespace_[$2, $4]; }
-    | T_NAMESPACE '{' top_statement_list '}'                { $$ = Stmt\Namespace_[null,     $3]; }
-    | T_USE use_declarations ';'                            { $$ = Stmt\Use_[$2, Stmt\Use_::TYPE_NORMAL]; }
-    | T_USE use_type use_declarations ';'                   { $$ = Stmt\Use_[$3, $2]; }
-    | group_use_declaration ';'                             { $$ = $1; }
-    | T_CONST constant_declaration_list ';'                 { $$ = Stmt\Const_[$2]; }
+    | T_NAMESPACE namespace_name semi
+          { $$ = Stmt\Namespace_[$2, null];
+            $$->setAttribute('kind', Stmt\Namespace_::KIND_SEMICOLON);
+            $this->checkNamespace($$); }
+    | T_NAMESPACE namespace_name '{' top_statement_list '}'
+          { $$ = Stmt\Namespace_[$2, $4];
+            $$->setAttribute('kind', Stmt\Namespace_::KIND_BRACED);
+            $this->checkNamespace($$); }
+    | T_NAMESPACE '{' top_statement_list '}'
+          { $$ = Stmt\Namespace_[null, $3];
+            $$->setAttribute('kind', Stmt\Namespace_::KIND_BRACED);
+            $this->checkNamespace($$); }
+    | T_USE use_declarations semi                           { $$ = Stmt\Use_[$2, Stmt\Use_::TYPE_NORMAL]; }
+    | T_USE use_type use_declarations semi                  { $$ = Stmt\Use_[$3, $2]; }
+    | group_use_declaration semi                            { $$ = $1; }
+    | T_CONST constant_declaration_list semi                { $$ = Stmt\Const_[$2]; }
 ;
 
 use_type:
@@ -65,34 +103,49 @@ use_type:
 /* Using namespace_name_parts here to avoid s/r conflict on T_NS_SEPARATOR */
 group_use_declaration:
       T_USE use_type namespace_name_parts T_NS_SEPARATOR '{' unprefixed_use_declarations '}'
-          { $$ = Stmt\GroupUse[Name[$3], $6, $2]; }
+          { $$ = Stmt\GroupUse[new Name($3, stackAttributes(#3)), $6, $2]; }
     | T_USE use_type T_NS_SEPARATOR namespace_name_parts T_NS_SEPARATOR '{' unprefixed_use_declarations '}'
-          { $$ = Stmt\GroupUse[Name[$4], $7, $2]; }
+          { $$ = Stmt\GroupUse[new Name($4, stackAttributes(#4)), $7, $2]; }
     | T_USE namespace_name_parts T_NS_SEPARATOR '{' inline_use_declarations '}'
-          { $$ = Stmt\GroupUse[Name[$2], $5, Stmt\Use_::TYPE_UNKNOWN]; }
+          { $$ = Stmt\GroupUse[new Name($2, stackAttributes(#2)), $5, Stmt\Use_::TYPE_UNKNOWN]; }
     | T_USE T_NS_SEPARATOR namespace_name_parts T_NS_SEPARATOR '{' inline_use_declarations '}'
-          { $$ = Stmt\GroupUse[Name[$3], $6, Stmt\Use_::TYPE_UNKNOWN]; }
+          { $$ = Stmt\GroupUse[new Name($3, stackAttributes(#3)), $6, Stmt\Use_::TYPE_UNKNOWN]; }
 ;
 
 unprefixed_use_declarations:
-      unprefixed_use_declarations ',' unprefixed_use_declaration
+      non_empty_unprefixed_use_declarations optional_comma  { $$ = $1; }
+;
+
+non_empty_unprefixed_use_declarations:
+      non_empty_unprefixed_use_declarations ',' unprefixed_use_declaration
           { push($1, $3); }
     | unprefixed_use_declaration                            { init($1); }
 ;
 
 use_declarations:
-      use_declarations ',' use_declaration                  { push($1, $3); }
+      non_empty_use_declarations no_comma                   { $$ = $1; }
+;
+
+non_empty_use_declarations:
+      non_empty_use_declarations ',' use_declaration        { push($1, $3); }
     | use_declaration                                       { init($1); }
 ;
 
 inline_use_declarations:
-      inline_use_declarations ',' inline_use_declaration    { push($1, $3); }
+      non_empty_inline_use_declarations optional_comma      { $$ = $1; }
+;
+
+non_empty_inline_use_declarations:
+      non_empty_inline_use_declarations ',' inline_use_declaration
+          { push($1, $3); }
     | inline_use_declaration                                { init($1); }
 ;
 
 unprefixed_use_declaration:
-      namespace_name                                        { $$ = Stmt\UseUse[$1, null, Stmt\Use_::TYPE_UNKNOWN]; }
-    | namespace_name T_AS T_STRING                          { $$ = Stmt\UseUse[$1, $3, Stmt\Use_::TYPE_UNKNOWN]; }
+      namespace_name
+          { $$ = Stmt\UseUse[$1, null, Stmt\Use_::TYPE_UNKNOWN]; $this->checkUseUse($$, #1); }
+    | namespace_name T_AS identifier
+          { $$ = Stmt\UseUse[$1, $3, Stmt\Use_::TYPE_UNKNOWN]; $this->checkUseUse($$, #3); }
 ;
 
 use_declaration:
@@ -106,26 +159,41 @@ inline_use_declaration:
 ;
 
 constant_declaration_list:
-      constant_declaration_list ',' constant_declaration    { push($1, $3); }
+      non_empty_constant_declaration_list no_comma          { $$ = $1; }
+;
+
+non_empty_constant_declaration_list:
+      non_empty_constant_declaration_list ',' constant_declaration
+          { push($1, $3); }
     | constant_declaration                                  { init($1); }
 ;
 
 constant_declaration:
-    T_STRING '=' expr                                       { $$ = Node\Const_[$1, $3]; }
+    identifier '=' expr                                     { $$ = Node\Const_[$1, $3]; }
 ;
 
 class_const_list:
-      class_const_list ',' class_const                      { push($1, $3); }
+      non_empty_class_const_list no_comma                   { $$ = $1; }
+;
+
+non_empty_class_const_list:
+      non_empty_class_const_list ',' class_const            { push($1, $3); }
     | class_const                                           { init($1); }
 ;
 
 class_const:
-    identifier '=' expr                                     { $$ = Node\Const_[$1, $3]; }
+    identifier_ex '=' expr                                  { $$ = Node\Const_[$1, $3]; }
+;
+
+inner_statement_list_ex:
+      inner_statement_list_ex inner_statement               { pushNormalizing($1, $2); }
+    | /* empty */                                           { init(); }
 ;
 
 inner_statement_list:
-      inner_statement_list inner_statement                  { pushNormalizing($1, $2); }
-    | /* empty */                                           { init(); }
+      inner_statement_list_ex
+          { makeNop($nop, $this->lookaheadStartAttributes, $this->endAttributes);
+            if ($nop !== null) { $1[] = $nop; } $$ = $1; }
 ;
 
 inner_statement:
@@ -136,8 +204,16 @@ inner_statement:
           { throw new Error('__HALT_COMPILER() can only be used from the outermost scope', attributes()); }
 ;
 
-statement:
-      '{' inner_statement_list '}'                          { $$ = $2; }
+non_empty_statement:
+      '{' inner_statement_list '}'
+    {
+        if ($2) {
+            $$ = $2; prependLeadingComments($$);
+        } else {
+            makeNop($$, $this->startAttributeStack[#1], $this->endAttributes);
+            if (null === $$) { $$ = array(); }
+        }
+    }
     | T_IF '(' expr ')' statement elseif_list else_single
           { $$ = Stmt\If_[$3, ['stmts' => toArray($5), 'elseifs' => $6, 'else' => $7]]; }
     | T_IF '(' expr ')' ':' inner_statement_list new_elseif_list new_else_single T_ENDIF ';'
@@ -147,47 +223,62 @@ statement:
     | T_FOR '(' for_expr ';'  for_expr ';' for_expr ')' for_statement
           { $$ = Stmt\For_[['init' => $3, 'cond' => $5, 'loop' => $7, 'stmts' => $9]]; }
     | T_SWITCH '(' expr ')' switch_case_list                { $$ = Stmt\Switch_[$3, $5]; }
-    | T_BREAK optional_expr ';'                             { $$ = Stmt\Break_[$2]; }
-    | T_CONTINUE optional_expr ';'                          { $$ = Stmt\Continue_[$2]; }
-    | T_RETURN optional_expr ';'                            { $$ = Stmt\Return_[$2]; }
-    | T_GLOBAL global_var_list ';'                          { $$ = Stmt\Global_[$2]; }
-    | T_STATIC static_var_list ';'                          { $$ = Stmt\Static_[$2]; }
-    | T_ECHO expr_list ';'                                  { $$ = Stmt\Echo_[$2]; }
+    | T_BREAK optional_expr semi                            { $$ = Stmt\Break_[$2]; }
+    | T_CONTINUE optional_expr semi                         { $$ = Stmt\Continue_[$2]; }
+    | T_RETURN optional_expr semi                           { $$ = Stmt\Return_[$2]; }
+    | T_GLOBAL global_var_list semi                         { $$ = Stmt\Global_[$2]; }
+    | T_STATIC static_var_list semi                         { $$ = Stmt\Static_[$2]; }
+    | T_ECHO expr_list semi                                 { $$ = Stmt\Echo_[$2]; }
     | T_INLINE_HTML                                         { $$ = Stmt\InlineHTML[$1]; }
-    | expr ';'                                              { $$ = $1; }
-    | T_UNSET '(' variables_list ')' ';'                    { $$ = Stmt\Unset_[$3]; }
+    | expr semi                                             { $$ = Stmt\Expression[$1]; }
+    | T_UNSET '(' variables_list ')' semi                   { $$ = Stmt\Unset_[$3]; }
     | T_FOREACH '(' expr T_AS foreach_variable ')' foreach_statement
           { $$ = Stmt\Foreach_[$3, $5[0], ['keyVar' => null, 'byRef' => $5[1], 'stmts' => $7]]; }
     | T_FOREACH '(' expr T_AS variable T_DOUBLE_ARROW foreach_variable ')' foreach_statement
           { $$ = Stmt\Foreach_[$3, $7[0], ['keyVar' => $5, 'byRef' => $7[1], 'stmts' => $9]]; }
     | T_DECLARE '(' declare_list ')' declare_statement      { $$ = Stmt\Declare_[$3, $5]; }
-    | ';'                                                   { $$ = array(); /* means: no statement */ }
     | T_TRY '{' inner_statement_list '}' catches optional_finally
-          { $$ = Stmt\TryCatch[$3, $5, $6]; }
-    | T_THROW expr ';'                                      { $$ = Stmt\Throw_[$2]; }
-    | T_GOTO T_STRING ';'                                   { $$ = Stmt\Goto_[$2]; }
-    | T_STRING ':'                                          { $$ = Stmt\Label[$1]; }
+          { $$ = Stmt\TryCatch[$3, $5, $6]; $this->checkTryCatch($$); }
+    | T_THROW expr semi                                     { $$ = Stmt\Throw_[$2]; }
+    | T_GOTO identifier semi                                { $$ = Stmt\Goto_[$2]; }
+    | identifier ':'                                        { $$ = Stmt\Label[$1]; }
     | error                                                 { $$ = array(); /* means: no statement */ }
 ;
 
+statement:
+      non_empty_statement                                   { $$ = $1; }
+    | ';'
+          { makeNop($$, $this->startAttributeStack[#1], $this->endAttributes);
+            if ($$ === null) $$ = array(); /* means: no statement */ }
+;
+
 catches:
       /* empty */                                           { init(); }
     | catches catch                                         { push($1, $2); }
 ;
 
+name_union:
+      name                                                  { init($1); }
+    | name_union '|' name                                   { push($1, $3); }
+;
+
 catch:
-    T_CATCH '(' name T_VARIABLE ')' '{' inner_statement_list '}'
-        { $$ = Stmt\Catch_[$3, parseVar($4), $7]; }
+    T_CATCH '(' name_union plain_variable ')' '{' inner_statement_list '}'
+        { $$ = Stmt\Catch_[$3, $4, $7]; }
 ;
 
 optional_finally:
       /* empty */                                           { $$ = null; }
-    | T_FINALLY '{' inner_statement_list '}'                { $$ = $3; }
+    | T_FINALLY '{' inner_statement_list '}'                { $$ = Stmt\Finally_[$3]; }
 ;
 
 variables_list:
+      non_empty_variables_list no_comma                     { $$ = $1; }
+;
+
+non_empty_variables_list:
       variable                                              { init($1); }
-    | variables_list ',' variable                           { push($1, $3); }
+    | non_empty_variables_list ',' variable                 { push($1, $3); }
 ;
 
 optional_ref:
@@ -201,17 +292,19 @@ optional_ellipsis:
 ;
 
 function_declaration_statement:
-    T_FUNCTION optional_ref T_STRING '(' parameter_list ')' optional_return_type '{' inner_statement_list '}'
+    T_FUNCTION optional_ref identifier '(' parameter_list ')' optional_return_type '{' inner_statement_list '}'
         { $$ = Stmt\Function_[$3, ['byRef' => $2, 'params' => $5, 'returnType' => $7, 'stmts' => $9]]; }
 ;
 
 class_declaration_statement:
-      class_entry_type T_STRING extends_from implements_list '{' class_statement_list '}'
-          { $$ = Stmt\Class_[$2, ['type' => $1, 'extends' => $3, 'implements' => $4, 'stmts' => $6]]; }
-    | T_INTERFACE T_STRING interface_extends_list '{' class_statement_list '}'
-          { $$ = Stmt\Interface_[$2, ['extends' => $3, 'stmts' => $5]]; }
-    | T_TRAIT T_STRING '{' class_statement_list '}'
-          { $$ = Stmt\Trait_[$2, $4]; }
+      class_entry_type identifier extends_from implements_list '{' class_statement_list '}'
+          { $$ = Stmt\Class_[$2, ['type' => $1, 'extends' => $3, 'implements' => $4, 'stmts' => $6]];
+            $this->checkClass($$, #2); }
+    | T_INTERFACE identifier interface_extends_list '{' class_statement_list '}'
+          { $$ = Stmt\Interface_[$2, ['extends' => $3, 'stmts' => $5]];
+            $this->checkInterface($$, #2); }
+    | T_TRAIT identifier '{' class_statement_list '}'
+          { $$ = Stmt\Trait_[$2, ['stmts' => $4]]; }
 ;
 
 class_entry_type:
@@ -222,22 +315,26 @@ class_entry_type:
 
 extends_from:
       /* empty */                                           { $$ = null; }
-    | T_EXTENDS name                                        { $$ = $2; }
+    | T_EXTENDS class_name                                  { $$ = $2; }
 ;
 
 interface_extends_list:
       /* empty */                                           { $$ = array(); }
-    | T_EXTENDS name_list                                   { $$ = $2; }
+    | T_EXTENDS class_name_list                             { $$ = $2; }
 ;
 
 implements_list:
       /* empty */                                           { $$ = array(); }
-    | T_IMPLEMENTS name_list                                { $$ = $2; }
+    | T_IMPLEMENTS class_name_list                          { $$ = $2; }
 ;
 
-name_list:
-      name                                                  { init($1); }
-    | name_list ',' name                                    { push($1, $3); }
+class_name_list:
+      non_empty_class_name_list no_comma                    { $$ = $1; }
+;
+
+non_empty_class_name_list:
+      class_name                                            { init($1); }
+    | non_empty_class_name_list ',' class_name              { push($1, $3); }
 ;
 
 for_statement:
@@ -251,17 +348,22 @@ foreach_statement:
 ;
 
 declare_statement:
-      statement                                             { $$ = toArray($1); }
+      non_empty_statement                                   { $$ = toArray($1); }
+    | ';'                                                   { $$ = null; }
     | ':' inner_statement_list T_ENDDECLARE ';'             { $$ = $2; }
 ;
 
 declare_list:
+      non_empty_declare_list no_comma                       { $$ = $1; }
+;
+
+non_empty_declare_list:
       declare_list_element                                  { init($1); }
-    | declare_list ',' declare_list_element                 { push($1, $3); }
+    | non_empty_declare_list ',' declare_list_element       { push($1, $3); }
 ;
 
 declare_list_element:
-      T_STRING '=' expr                                     { $$ = Stmt\DeclareDeclare[$1, $3]; }
+      identifier '=' expr                                   { $$ = Stmt\DeclareDeclare[$1, $3]; }
 ;
 
 switch_case_list:
@@ -323,10 +425,11 @@ foreach_variable:
       variable                                              { $$ = array($1, false); }
     | '&' variable                                          { $$ = array($2, true); }
     | list_expr                                             { $$ = array($1, false); }
+    | array_short_syntax                                    { $$ = array($1, false); }
 ;
 
 parameter_list:
-      non_empty_parameter_list                              { $$ = $1; }
+      non_empty_parameter_list no_comma                     { $$ = $1; }
     | /* empty */                                           { $$ = array(); }
 ;
 
@@ -336,31 +439,36 @@ non_empty_parameter_list:
 ;
 
 parameter:
-      optional_param_type optional_ref optional_ellipsis T_VARIABLE
-          { $$ = Node\Param[parseVar($4), null, $1, $2, $3]; }
-    | optional_param_type optional_ref optional_ellipsis T_VARIABLE '=' expr
-          { $$ = Node\Param[parseVar($4), $6, $1, $2, $3]; }
+      optional_param_type optional_ref optional_ellipsis plain_variable
+          { $$ = Node\Param[$4, null, $1, $2, $3]; $this->checkParam($$); }
+    | optional_param_type optional_ref optional_ellipsis plain_variable '=' expr
+          { $$ = Node\Param[$4, $6, $1, $2, $3]; $this->checkParam($$); }
+;
+
+type_expr:
+      type                                                  { $$ = $1; }
+    | '?' type                                              { $$ = Node\NullableType[$2]; }
 ;
 
 type:
-      name                                                  { $$ = $this->handleScalarTypes($1); }
-    | T_ARRAY                                               { $$ = 'array'; }
-    | T_CALLABLE                                            { $$ = 'callable'; }
+      name                                                  { $$ = $this->handleBuiltinTypes($1); }
+    | T_ARRAY                                               { $$ = Node\Identifier['array']; }
+    | T_CALLABLE                                            { $$ = Node\Identifier['callable']; }
 ;
 
 optional_param_type:
       /* empty */                                           { $$ = null; }
-    | type                                                  { $$ = $1; }
+    | type_expr                                             { $$ = $1; }
 ;
 
 optional_return_type:
       /* empty */                                           { $$ = null; }
-    | ':' type                                              { $$ = $2; }
+    | ':' type_expr                                         { $$ = $2; }
 ;
 
 argument_list:
       '(' ')'                                               { $$ = array(); }
-    | '(' non_empty_argument_list ')'                       { $$ = $2; }
+    | '(' non_empty_argument_list no_comma ')'              { $$ = $2; }
 ;
 
 non_empty_argument_list:
@@ -375,7 +483,11 @@ argument:
 ;
 
 global_var_list:
-      global_var_list ',' global_var                        { push($1, $3); }
+      non_empty_global_var_list no_comma                    { $$ = $1; }
+;
+
+non_empty_global_var_list:
+      non_empty_global_var_list ',' global_var              { push($1, $3); }
     | global_var                                            { init($1); }
 ;
 
@@ -384,13 +496,17 @@ global_var:
 ;
 
 static_var_list:
-      static_var_list ',' static_var                        { push($1, $3); }
+      non_empty_static_var_list no_comma                    { $$ = $1; }
+;
+
+non_empty_static_var_list:
+      non_empty_static_var_list ',' static_var              { push($1, $3); }
     | static_var                                            { init($1); }
 ;
 
 static_var:
-      T_VARIABLE                                            { $$ = Stmt\StaticVar[parseVar($1), null]; }
-    | T_VARIABLE '=' expr                                   { $$ = Stmt\StaticVar[parseVar($1), $3]; }
+      plain_variable                                        { $$ = Stmt\StaticVar[$1, null]; }
+    | plain_variable '=' expr                               { $$ = Stmt\StaticVar[$1, $3]; }
 ;
 
 class_statement_list:
@@ -399,11 +515,14 @@ class_statement_list:
 ;
 
 class_statement:
-      variable_modifiers property_declaration_list ';'      { $$ = Stmt\Property[$1, $2]; }
-    | T_CONST class_const_list ';'                          { $$ = Stmt\ClassConst[$2]; }
-    | method_modifiers T_FUNCTION optional_ref identifier '(' parameter_list ')' optional_return_type method_body
-          { $$ = Stmt\ClassMethod[$4, ['type' => $1, 'byRef' => $3, 'params' => $6, 'returnType' => $8, 'stmts' => $9]]; }
-    | T_USE name_list trait_adaptations                     { $$ = Stmt\TraitUse[$2, $3]; }
+      variable_modifiers property_declaration_list ';'
+          { $$ = Stmt\Property[$1, $2]; $this->checkProperty($$, #1); }
+    | method_modifiers T_CONST class_const_list ';'
+          { $$ = Stmt\ClassConst[$3, $1]; $this->checkClassConst($$, #1); }
+    | method_modifiers T_FUNCTION optional_ref identifier_ex '(' parameter_list ')' optional_return_type method_body
+          { $$ = Stmt\ClassMethod[$4, ['type' => $1, 'byRef' => $3, 'params' => $6, 'returnType' => $8, 'stmts' => $9]];
+            $this->checkClassMethod($$, #1); }
+    | T_USE class_name_list trait_adaptations               { $$ = Stmt\TraitUse[$2, $3]; }
 ;
 
 trait_adaptations:
@@ -417,24 +536,24 @@ trait_adaptation_list:
 ;
 
 trait_adaptation:
-      trait_method_reference_fully_qualified T_INSTEADOF name_list ';'
+      trait_method_reference_fully_qualified T_INSTEADOF class_name_list ';'
           { $$ = Stmt\TraitUseAdaptation\Precedence[$1[0], $1[1], $3]; }
-    | trait_method_reference T_AS member_modifier identifier ';'
+    | trait_method_reference T_AS member_modifier identifier_ex ';'
           { $$ = Stmt\TraitUseAdaptation\Alias[$1[0], $1[1], $3, $4]; }
     | trait_method_reference T_AS member_modifier ';'
           { $$ = Stmt\TraitUseAdaptation\Alias[$1[0], $1[1], $3, null]; }
-    | trait_method_reference T_AS T_STRING ';'
+    | trait_method_reference T_AS identifier ';'
           { $$ = Stmt\TraitUseAdaptation\Alias[$1[0], $1[1], null, $3]; }
     | trait_method_reference T_AS reserved_non_modifiers ';'
           { $$ = Stmt\TraitUseAdaptation\Alias[$1[0], $1[1], null, $3]; }
 ;
 
 trait_method_reference_fully_qualified:
-      name T_PAAMAYIM_NEKUDOTAYIM identifier                { $$ = array($1, $3); }
+      name T_PAAMAYIM_NEKUDOTAYIM identifier_ex             { $$ = array($1, $3); }
 ;
 trait_method_reference:
       trait_method_reference_fully_qualified                { $$ = $1; }
-    | identifier                                            { $$ = array(null, $1); }
+    | identifier_ex                                         { $$ = array(null, $1); }
 ;
 
 method_body:
@@ -454,7 +573,7 @@ method_modifiers:
 
 non_empty_member_modifiers:
       member_modifier                                       { $$ = $1; }
-    | non_empty_member_modifiers member_modifier            { Stmt\Class_::verifyModifier($1, $2); $$ = $1 | $2; }
+    | non_empty_member_modifiers member_modifier            { $this->checkModifier($1, $2, #2); $$ = $1 | $2; }
 ;
 
 member_modifier:
@@ -467,17 +586,30 @@ member_modifier:
 ;
 
 property_declaration_list:
+      non_empty_property_declaration_list no_comma          { $$ = $1; }
+;
+
+non_empty_property_declaration_list:
       property_declaration                                  { init($1); }
-    | property_declaration_list ',' property_declaration    { push($1, $3); }
+    | non_empty_property_declaration_list ',' property_declaration
+          { push($1, $3); }
+;
+
+property_decl_name:
+      T_VARIABLE                                            { $$ = Node\VarLikeIdentifier[parseVar($1)]; }
 ;
 
 property_declaration:
-      T_VARIABLE                                            { $$ = Stmt\PropertyProperty[parseVar($1), null]; }
-    | T_VARIABLE '=' expr                                   { $$ = Stmt\PropertyProperty[parseVar($1), $3]; }
+      property_decl_name                                    { $$ = Stmt\PropertyProperty[$1, null]; }
+    | property_decl_name '=' expr                           { $$ = Stmt\PropertyProperty[$1, $3]; }
 ;
 
 expr_list:
-      expr_list ',' expr                                    { push($1, $3); }
+      non_empty_expr_list no_comma                          { $$ = $1; }
+;
+
+non_empty_expr_list:
+      non_empty_expr_list ',' expr                          { push($1, $3); }
     | expr                                                  { init($1); }
 ;
 
@@ -489,9 +621,9 @@ for_expr:
 expr:
       variable                                              { $$ = $1; }
     | list_expr '=' expr                                    { $$ = Expr\Assign[$1, $3]; }
+    | array_short_syntax '=' expr                           { $$ = Expr\Assign[$1, $3]; }
     | variable '=' expr                                     { $$ = Expr\Assign[$1, $3]; }
     | variable '=' '&' variable                             { $$ = Expr\AssignRef[$1, $4]; }
-    | variable '=' '&' new_expr                             { $$ = Expr\AssignRef[$1, $4]; }
     | new_expr                                              { $$ = $1; }
     | T_CLONE expr                                          { $$ = Expr\Clone_[$2]; }
     | variable T_PLUS_EQUAL expr                            { $$ = Expr\AssignOp\Plus      [$1, $3]; }
@@ -559,7 +691,10 @@ expr:
     | T_OBJECT_CAST expr                                    { $$ = Expr\Cast\Object_ [$2]; }
     | T_BOOL_CAST expr                                      { $$ = Expr\Cast\Bool_   [$2]; }
     | T_UNSET_CAST expr                                     { $$ = Expr\Cast\Unset_  [$2]; }
-    | T_EXIT exit_expr                                      { $$ = Expr\Exit_        [$2]; }
+    | T_EXIT exit_expr
+          { $attrs = attributes();
+            $attrs['kind'] = strtolower($1) === 'exit' ? Expr\Exit_::KIND_EXIT : Expr\Exit_::KIND_DIE;
+            $$ = new Expr\Exit_($2, $attrs); }
     | '@' expr                                              { $$ = Expr\ErrorSuppress[$2]; }
     | scalar                                                { $$ = $1; }
     | '`' backticks_expr '`'                                { $$ = Expr\ShellExec[$2]; }
@@ -578,7 +713,9 @@ expr:
 
 anonymous_class:
       T_CLASS ctor_arguments extends_from implements_list '{' class_statement_list '}'
-          { $$ = array(Stmt\Class_[null, ['type' => 0, 'extends' => $3, 'implements' => $4, 'stmts' => $6]], $2); }
+          { $$ = array(Stmt\Class_[null, ['type' => 0, 'extends' => $3, 'implements' => $4, 'stmts' => $6]], $2);
+            $this->checkClass($$[0], -1); }
+;
 
 new_expr:
       T_NEW class_name_reference ctor_arguments             { $$ = Expr\New_[$2, $3]; }
@@ -592,12 +729,16 @@ lexical_vars:
 ;
 
 lexical_var_list:
+      non_empty_lexical_var_list no_comma                   { $$ = $1; }
+;
+
+non_empty_lexical_var_list:
       lexical_var                                           { init($1); }
-    | lexical_var_list ',' lexical_var                      { push($1, $3); }
+    | non_empty_lexical_var_list ',' lexical_var            { push($1, $3); }
 ;
 
 lexical_var:
-      optional_ref T_VARIABLE                               { $$ = Expr\ClosureUse[parseVar($2), $1]; }
+      optional_ref plain_variable                           { $$ = Expr\ClosureUse[$2, $1]; }
 ;
 
 function_call:
@@ -621,6 +762,7 @@ name:
 class_name_reference:
       class_name                                            { $$ = $1; }
     | new_variable                                          { $$ = $1; }
+    | error                                                 { $$ = Expr\Error[]; $this->errorState = 2; }
 ;
 
 class_name_or_var:
@@ -647,18 +789,32 @@ ctor_arguments:
 
 constant:
       name                                                  { $$ = Expr\ConstFetch[$1]; }
-    | class_name_or_var T_PAAMAYIM_NEKUDOTAYIM identifier
+    | class_name_or_var T_PAAMAYIM_NEKUDOTAYIM identifier_ex
           { $$ = Expr\ClassConstFetch[$1, $3]; }
+    /* We interpret and isolated FOO:: as an unfinished class constant fetch. It could also be
+       an unfinished static property fetch or unfinished scoped call. */
+    | class_name_or_var T_PAAMAYIM_NEKUDOTAYIM error
+          { $$ = Expr\ClassConstFetch[$1, new Expr\Error(stackAttributes(#3))]; $this->errorState = 2; }
+;
+
+array_short_syntax:
+      '[' array_pair_list ']'
+          { $attrs = attributes(); $attrs['kind'] = Expr\Array_::KIND_SHORT;
+            $$ = new Expr\Array_($2, $attrs); }
 ;
 
 dereferencable_scalar:
-      T_ARRAY '(' array_pair_list ')'                       { $$ = Expr\Array_[$3]; }
-    | '[' array_pair_list ']'                               { $$ = Expr\Array_[$2]; }
-    | T_CONSTANT_ENCAPSED_STRING                            { $$ = Scalar\String_[Scalar\String_::parse($1)]; }
+      T_ARRAY '(' array_pair_list ')'
+          { $attrs = attributes(); $attrs['kind'] = Expr\Array_::KIND_LONG;
+            $$ = new Expr\Array_($3, $attrs); }
+    | array_short_syntax                                    { $$ = $1; }
+    | T_CONSTANT_ENCAPSED_STRING
+          { $attrs = attributes(); $attrs['kind'] = strKind($1);
+            $$ = new Scalar\String_(Scalar\String_::parse($1), $attrs); }
 ;
 
 scalar:
-      T_LNUMBER                                             { $$ = Scalar\LNumber[Scalar\LNumber::parse($1)]; }
+      T_LNUMBER                                             { $$ = $this->parseLNumber($1, attributes()); }
     | T_DNUMBER                                             { $$ = Scalar\DNumber[Scalar\DNumber::parse($1)]; }
     | T_LINE                                                { $$ = Scalar\MagicConst\Line[]; }
     | T_FILE                                                { $$ = Scalar\MagicConst\File[]; }
@@ -671,18 +827,17 @@ scalar:
     | dereferencable_scalar                                 { $$ = $1; }
     | constant                                              { $$ = $1; }
     | T_START_HEREDOC T_ENCAPSED_AND_WHITESPACE T_END_HEREDOC
-          { $$ = Scalar\String_[Scalar\String_::parseDocString($1, $2)]; }
+          { $attrs = attributes(); setDocStringAttrs($attrs, $1);
+            $$ = new Scalar\String_(Scalar\String_::parseDocString($1, $2), $attrs); }
     | T_START_HEREDOC T_END_HEREDOC
-          { $$ = Scalar\String_['']; }
+          { $attrs = attributes(); setDocStringAttrs($attrs, $1);
+            $$ = new Scalar\String_('', $attrs); }
     | '"' encaps_list '"'
-          { parseEncapsed($2, '"', true); $$ = Scalar\Encapsed[$2]; }
+          { $attrs = attributes(); $attrs['kind'] = Scalar\String_::KIND_DOUBLE_QUOTED;
+            parseEncapsed($2, '"', true); $$ = new Scalar\Encapsed($2, $attrs); }
     | T_START_HEREDOC encaps_list T_END_HEREDOC
-          { parseEncapsedDoc($2, true); $$ = Scalar\Encapsed[$2]; }
-;
-
-optional_comma:
-      /* empty */
-    | ','
+          { $attrs = attributes(); setDocStringAttrs($attrs, $1);
+            parseEncapsedDoc($2, true); $$ = new Scalar\Encapsed($2, $attrs); }
 ;
 
 optional_expr:
@@ -722,10 +877,16 @@ simple_variable:
       T_VARIABLE                                            { $$ = parseVar($1); }
     | '$' '{' expr '}'                                      { $$ = $3; }
     | '$' simple_variable                                   { $$ = Expr\Variable[$2]; }
+    | '$' error                                             { $$ = Expr\Error[]; $this->errorState = 2; }
+;
+
+static_member_prop_name:
+      simple_variable
+          { $var = $1; $$ = \is_string($var) ? Node\VarLikeIdentifier[$var] : $var; }
 ;
 
 static_member:
-      class_name_or_var T_PAAMAYIM_NEKUDOTAYIM simple_variable
+      class_name_or_var T_PAAMAYIM_NEKUDOTAYIM static_member_prop_name
           { $$ = Expr\StaticPropertyFetch[$1, $3]; }
 ;
 
@@ -734,20 +895,23 @@ new_variable:
     | new_variable '[' optional_expr ']'                    { $$ = Expr\ArrayDimFetch[$1, $3]; }
     | new_variable '{' expr '}'                             { $$ = Expr\ArrayDimFetch[$1, $3]; }
     | new_variable T_OBJECT_OPERATOR property_name          { $$ = Expr\PropertyFetch[$1, $3]; }
-    | class_name T_PAAMAYIM_NEKUDOTAYIM simple_variable     { $$ = Expr\StaticPropertyFetch[$1, $3]; }
-    | new_variable T_PAAMAYIM_NEKUDOTAYIM simple_variable   { $$ = Expr\StaticPropertyFetch[$1, $3]; }
+    | class_name T_PAAMAYIM_NEKUDOTAYIM static_member_prop_name
+          { $$ = Expr\StaticPropertyFetch[$1, $3]; }
+    | new_variable T_PAAMAYIM_NEKUDOTAYIM static_member_prop_name
+          { $$ = Expr\StaticPropertyFetch[$1, $3]; }
 ;
 
 member_name:
-      identifier                                            { $$ = $1; }
+      identifier_ex                                         { $$ = $1; }
     | '{' expr '}'	                                        { $$ = $2; }
     | simple_variable	                                    { $$ = Expr\Variable[$1]; }
 ;
 
 property_name:
-      T_STRING                                              { $$ = $1; }
+      identifier                                            { $$ = $1; }
     | '{' expr '}'	                                        { $$ = $2; }
     | simple_variable	                                    { $$ = Expr\Variable[$1]; }
+    | error                                                 { $$ = Expr\Error[]; $this->errorState = 2; }
 ;
 
 list_expr:
@@ -760,18 +924,20 @@ list_expr_elements:
 ;
 
 list_expr_element:
-      variable                                              { $$ = $1; }
-    | list_expr                                             { $$ = $1; }
+      variable                                              { $$ = Expr\ArrayItem[$1, null, false]; }
+    | list_expr                                             { $$ = Expr\ArrayItem[$1, null, false]; }
+    | expr T_DOUBLE_ARROW variable                          { $$ = Expr\ArrayItem[$3, $1, false]; }
+    | expr T_DOUBLE_ARROW list_expr                         { $$ = Expr\ArrayItem[$3, $1, false]; }
     | /* empty */                                           { $$ = null; }
 ;
 
 array_pair_list:
-      /* empty */                                           { $$ = array(); }
-    | non_empty_array_pair_list optional_comma              { $$ = $1; }
+      inner_array_pair_list
+          { $$ = $1; $end = count($$)-1; if ($$[$end] === null) unset($$[$end]); }
 ;
 
-non_empty_array_pair_list:
-      non_empty_array_pair_list ',' array_pair              { push($1, $3); }
+inner_array_pair_list:
+      inner_array_pair_list ',' array_pair                  { push($1, $3); }
     | array_pair                                            { init($1); }
 ;
 
@@ -780,6 +946,7 @@ array_pair:
     | expr                                                  { $$ = Expr\ArrayItem[$1, null, false]; }
     | expr T_DOUBLE_ARROW '&' variable                      { $$ = Expr\ArrayItem[$4, $1,   true]; }
     | '&' variable                                          { $$ = Expr\ArrayItem[$2, null, true]; }
+    | /* empty */                                           { $$ = null; }
 ;
 
 encaps_list:
@@ -793,21 +960,26 @@ encaps_string_part:
       T_ENCAPSED_AND_WHITESPACE                             { $$ = Scalar\EncapsedStringPart[$1]; }
 ;
 
+encaps_str_varname:
+      T_STRING_VARNAME                                      { $$ = Expr\Variable[$1]; }
+;
+
 encaps_var:
-      T_VARIABLE                                            { $$ = Expr\Variable[parseVar($1)]; }
-    | T_VARIABLE '[' encaps_var_offset ']'                  { $$ = Expr\ArrayDimFetch[Expr\Variable[parseVar($1)], $3]; }
-    | T_VARIABLE T_OBJECT_OPERATOR T_STRING                 { $$ = Expr\PropertyFetch[Expr\Variable[parseVar($1)], $3]; }
+      plain_variable                                        { $$ = $1; }
+    | plain_variable '[' encaps_var_offset ']'              { $$ = Expr\ArrayDimFetch[$1, $3]; }
+    | plain_variable T_OBJECT_OPERATOR identifier           { $$ = Expr\PropertyFetch[$1, $3]; }
     | T_DOLLAR_OPEN_CURLY_BRACES expr '}'                   { $$ = Expr\Variable[$2]; }
     | T_DOLLAR_OPEN_CURLY_BRACES T_STRING_VARNAME '}'       { $$ = Expr\Variable[$2]; }
-    | T_DOLLAR_OPEN_CURLY_BRACES T_STRING_VARNAME '[' expr ']' '}'
-          { $$ = Expr\ArrayDimFetch[Expr\Variable[$2], $4]; }
+    | T_DOLLAR_OPEN_CURLY_BRACES encaps_str_varname '[' expr ']' '}'
+          { $$ = Expr\ArrayDimFetch[$2, $4]; }
     | T_CURLY_OPEN variable '}'                             { $$ = $2; }
 ;
 
 encaps_var_offset:
       T_STRING                                              { $$ = Scalar\String_[$1]; }
-    | T_NUM_STRING                                          { $$ = Scalar\String_[$1]; }
-    | T_VARIABLE                                            { $$ = Expr\Variable[parseVar($1)]; }
+    | T_NUM_STRING                                          { $$ = $this->parseNumString($1, attributes()); }
+    | '-' T_NUM_STRING                                      { $$ = $this->parseNumString('-' . $2, attributes()); }
+    | plain_variable                                        { $$ = $1; }
 ;
 
 %%

grammar/rebuildParsers.php

@@ -85,7 +85,7 @@ foreach ($grammarFileToName as $grammarFile => $name) {
 
 function resolveNodes($code) {
     return preg_replace_callback(
-        '~(?<name>[A-Z][a-zA-Z_\\\\]++)\s*' . PARAMS . '~',
+        '~\b(?<name>[A-Z][a-zA-Z_\\\\]++)\s*' . PARAMS . '~',
         function($matches) {
             // recurse
             $matches['params'] = resolveNodes($matches['params']);
@@ -124,6 +124,12 @@ function resolveMacros($code) {
                 return '$this->startAttributeStack[#1] + $this->endAttributes';
             }
 
+            if ('stackAttributes' == $name) {
+                assertArgs(1, $args, $name);
+                return '$this->startAttributeStack[' . $args[0] . ']'
+                     . ' + $this->endAttributeStack[' . $args[0] . ']';
+            }
+
             if ('init' == $name) {
                 return '$$ = array(' . implode(', ', $args) . ')';
             }
@@ -169,6 +175,41 @@ function resolveMacros($code) {
                      . ' if (\'\' === $s->value) array_pop(' . $args[0] . ');';
             }
 
+            if ('makeNop' == $name) {
+                assertArgs(3, $args, $name);
+
+                return '$startAttributes = ' . $args[1] . ';'
+                . ' if (isset($startAttributes[\'comments\']))'
+                . ' { ' . $args[0] . ' = new Stmt\Nop($startAttributes + ' . $args[2] . '); }'
+                . ' else { ' . $args[0] . ' = null; }';
+            }
+
+            if ('strKind' == $name) {
+                assertArgs(1, $args, $name);
+
+                return '(' . $args[0] . '[0] === "\'" || (' . $args[0] . '[1] === "\'" && '
+                     . '(' . $args[0] . '[0] === \'b\' || ' . $args[0] . '[0] === \'B\')) '
+                     . '? Scalar\String_::KIND_SINGLE_QUOTED : Scalar\String_::KIND_DOUBLE_QUOTED)';
+            }
+
+            if ('setDocStringAttrs' == $name) {
+                assertArgs(2, $args, $name);
+
+                return $args[0] . '[\'kind\'] = strpos(' . $args[1] . ', "\'") === false '
+                     . '? Scalar\String_::KIND_HEREDOC : Scalar\String_::KIND_NOWDOC; '
+                     . 'preg_match(\'/\A[bB]?<<<[ \t]*[\\\'"]?([a-zA-Z_\x7f-\xff][a-zA-Z0-9_\x7f-\xff]*)[\\\'"]?(?:\r\n|\n|\r)\z/\', ' . $args[1] . ', $matches); '
+                     . $args[0] . '[\'docLabel\'] = $matches[1];';
+            }
+
+            if ('prependLeadingComments' == $name) {
+                assertArgs(1, $args, $name);
+
+                return '$attrs = $this->startAttributeStack[#1]; $stmts = ' . $args[0] . '; '
+                . 'if (!empty($attrs[\'comments\'])) {'
+                . '$stmts[0]->setAttribute(\'comments\', '
+                . 'array_merge($attrs[\'comments\'], $stmts[0]->getAttribute(\'comments\', []))); }';
+            }
+
             return $matches[0];
         },
         $code

lib/PhpParser/Autoloader.php

@@ -1,4 +1,4 @@
-<?php
+<?php declare(strict_types=1);
 
 namespace PhpParser;
 
@@ -15,12 +15,12 @@ class Autoloader
      *
      * @param bool $prepend Whether to prepend the autoloader instead of appending
      */
-    static public function register($prepend = false) {
+    static public function register(bool $prepend = false) {
         if (self::$registered === true) {
             return;
         }
 
-        spl_autoload_register(array(__CLASS__, 'autoload'), true, $prepend);
+        spl_autoload_register([__CLASS__, 'autoload'], true, $prepend);
         self::$registered = true;
     }
 
@@ -29,9 +29,9 @@ class Autoloader
      *
      * @param string $class A class name.
      */
-    static public function autoload($class) {
+    static public function autoload(string $class) {
         if (0 === strpos($class, 'PhpParser\\')) {
-            $fileName = dirname(__DIR__) . '/' . strtr($class, '\\', '/') . '.php';
+            $fileName = __DIR__ . strtr(substr($class, 9), '\\', '/') . '.php';
             if (file_exists($fileName)) {
                 require $fileName;
             }

lib/PhpParser/Builder.php

@@ -1,4 +1,4 @@
-<?php
+<?php declare(strict_types=1);
 
 namespace PhpParser;
 
@@ -9,5 +9,5 @@ interface Builder
      *
      * @return Node The built node
      */
-    public function getNode();
+    public function getNode() : Node;
 }

lib/PhpParser/Builder/Class_.php

@@ -1,8 +1,9 @@
-<?php
+<?php declare(strict_types=1);
 
 namespace PhpParser\Builder;
 
 use PhpParser;
+use PhpParser\BuilderHelpers;
 use PhpParser\Node\Name;
 use PhpParser\Node\Stmt;
 
@@ -11,20 +12,20 @@ class Class_ extends Declaration
     protected $name;
 
     protected $extends = null;
-    protected $implements = array();
-    protected $type = 0;
+    protected $implements = [];
+    protected $flags = 0;
 
-    protected $uses = array();
-    protected $constants = array();
-    protected $properties = array();
-    protected $methods = array();
+    protected $uses = [];
+    protected $constants = [];
+    protected $properties = [];
+    protected $methods = [];
 
     /**
      * Creates a class builder.
      *
      * @param string $name Name of the class
      */
-    public function __construct($name) {
+    public function __construct(string $name) {
         $this->name = $name;
     }
 
@@ -36,7 +37,7 @@ class Class_ extends Declaration
      * @return $this The builder instance (for fluid interface)
      */
     public function extend($class) {
-        $this->extends = $this->normalizeName($class);
+        $this->extends = BuilderHelpers::normalizeName($class);
 
         return $this;
     }
@@ -44,14 +45,13 @@ class Class_ extends Declaration
     /**
      * Implements one or more interfaces.
      *
-     * @param Name|string $interface Name of interface to implement
-     * @param Name|string $...       More interfaces to implement
+     * @param Name|string ...$interfaces Names of interfaces to implement
      *
      * @return $this The builder instance (for fluid interface)
      */
-    public function implement() {
-        foreach (func_get_args() as $interface) {
-            $this->implements[] = $this->normalizeName($interface);
+    public function implement(...$interfaces) {
+        foreach ($interfaces as $interface) {
+            $this->implements[] = BuilderHelpers::normalizeName($interface);
         }
 
         return $this;
@@ -63,7 +63,7 @@ class Class_ extends Declaration
      * @return $this The builder instance (for fluid interface)
      */
     public function makeAbstract() {
-        $this->setModifier(Stmt\Class_::MODIFIER_ABSTRACT);
+        $this->flags = BuilderHelpers::addModifier($this->flags, Stmt\Class_::MODIFIER_ABSTRACT);
 
         return $this;
     }
@@ -74,7 +74,7 @@ class Class_ extends Declaration
      * @return $this The builder instance (for fluid interface)
      */
     public function makeFinal() {
-        $this->setModifier(Stmt\Class_::MODIFIER_FINAL);
+        $this->flags = BuilderHelpers::addModifier($this->flags, Stmt\Class_::MODIFIER_FINAL);
 
         return $this;
     }
@@ -87,14 +87,14 @@ class Class_ extends Declaration
      * @return $this The builder instance (for fluid interface)
      */
     public function addStmt($stmt) {
-        $stmt = $this->normalizeNode($stmt);
+        $stmt = BuilderHelpers::normalizeNode($stmt);
 
-        $targets = array(
+        $targets = [
             'Stmt_TraitUse'    => &$this->uses,
             'Stmt_ClassConst'  => &$this->constants,
             'Stmt_Property'    => &$this->properties,
             'Stmt_ClassMethod' => &$this->methods,
-        );
+        ];
 
         $type = $stmt->getType();
         if (!isset($targets[$type])) {
@@ -111,12 +111,12 @@ class Class_ extends Declaration
      *
      * @return Stmt\Class_ The built class node
      */
-    public function getNode() {
-        return new Stmt\Class_($this->name, array(
-            'type' => $this->type,
+    public function getNode() : PhpParser\Node {
+        return new Stmt\Class_($this->name, [
+            'flags' => $this->flags,
             'extends' => $this->extends,
             'implements' => $this->implements,
             'stmts' => array_merge($this->uses, $this->constants, $this->properties, $this->methods),
-        ), $this->attributes);
+        ], $this->attributes);
     }
 }

lib/PhpParser/Builder/Declaration.php

@@ -1,14 +1,13 @@
-<?php
+<?php declare(strict_types=1);
 
 namespace PhpParser\Builder;
 
 use PhpParser;
-use PhpParser\Node;
-use PhpParser\Node\Stmt;
+use PhpParser\BuilderHelpers;
 
-abstract class Declaration extends PhpParser\BuilderAbstract
+abstract class Declaration implements PhpParser\Builder
 {
-    protected $attributes = array();
+    protected $attributes = [];
 
     abstract public function addStmt($stmt);
 
@@ -35,9 +34,9 @@ abstract class Declaration extends PhpParser\BuilderAbstract
      * @return $this The builder instance (for fluid interface)
      */
     public function setDocComment($docComment) {
-        $this->attributes['comments'] = array(
-            $this->normalizeDocComment($docComment)
-        );
+        $this->attributes['comments'] = [
+            BuilderHelpers::normalizeDocComment($docComment)
+        ];
 
         return $this;
     }

lib/PhpParser/Builder/FunctionLike.php

@@ -1,15 +1,17 @@
-<?php
+<?php declare(strict_types=1);
 
 namespace PhpParser\Builder;
 
-use PhpParser;
+use PhpParser\BuilderHelpers;
 use PhpParser\Node;
-use PhpParser\Node\Stmt;
 
 abstract class FunctionLike extends Declaration
 {
     protected $returnByRef = false;
-    protected $params = array();
+    protected $params = [];
+
+    /** @var string|Node\Name|Node\NullableType|null */
+    protected $returnType = null;
 
     /**
      * Make the function return by reference.
@@ -30,7 +32,7 @@ abstract class FunctionLike extends Declaration
      * @return $this The builder instance (for fluid interface)
      */
     public function addParam($param) {
-        $param = $this->normalizeNode($param);
+        $param = BuilderHelpers::normalizeNode($param);
 
         if (!$param instanceof Node\Param) {
             throw new \LogicException(sprintf('Expected parameter node, got "%s"', $param->getType()));
@@ -55,4 +57,18 @@ abstract class FunctionLike extends Declaration
 
         return $this;
     }
-}
+
+    /**
+     * Sets the return type for PHP 7.
+     *
+     * @param string|Node\Name|Node\NullableType $type One of array, callable, string, int, float,
+     *                                                 bool, iterable, or a class/interface name.
+     *
+     * @return $this The builder instance (for fluid interface)
+     */
+    public function setReturnType($type) {
+        $this->returnType = BuilderHelpers::normalizeType($type);
+
+        return $this;
+    }
+}

lib/PhpParser/Builder/Function_.php

@@ -1,22 +1,23 @@
-<?php
+<?php declare(strict_types=1);
 
 namespace PhpParser\Builder;
 
 use PhpParser;
+use PhpParser\BuilderHelpers;
 use PhpParser\Node;
 use PhpParser\Node\Stmt;
 
 class Function_ extends FunctionLike
 {
     protected $name;
-    protected $stmts = array();
+    protected $stmts = [];
 
     /**
      * Creates a function builder.
      *
      * @param string $name Name of the function
      */
-    public function __construct($name) {
+    public function __construct(string $name) {
         $this->name = $name;
     }
 
@@ -28,7 +29,7 @@ class Function_ extends FunctionLike
      * @return $this The builder instance (for fluid interface)
      */
     public function addStmt($stmt) {
-        $this->stmts[] = $this->normalizeNode($stmt);
+        $this->stmts[] = BuilderHelpers::normalizeStmt($stmt);
 
         return $this;
     }
@@ -38,11 +39,12 @@ class Function_ extends FunctionLike
      *
      * @return Stmt\Function_ The built function node
      */
-    public function getNode() {
-        return new Stmt\Function_($this->name, array(
-            'byRef'  => $this->returnByRef,
-            'params' => $this->params,
-            'stmts'  => $this->stmts,
-        ), $this->attributes);
+    public function getNode() : Node {
+        return new Stmt\Function_($this->name, [
+            'byRef'      => $this->returnByRef,
+            'params'     => $this->params,
+            'returnType' => $this->returnType,
+            'stmts'      => $this->stmts,
+        ], $this->attributes);
     }
-}
+}

lib/PhpParser/Builder/Interface_.php

@@ -1,38 +1,38 @@
-<?php
+<?php declare(strict_types=1);
 
 namespace PhpParser\Builder;
 
 use PhpParser;
+use PhpParser\BuilderHelpers;
 use PhpParser\Node\Name;
 use PhpParser\Node\Stmt;
 
 class Interface_ extends Declaration
 {
     protected $name;
-    protected $extends = array();
-    protected $constants = array();
-    protected $methods = array();
+    protected $extends = [];
+    protected $constants = [];
+    protected $methods = [];
 
     /**
      * Creates an interface builder.
      *
      * @param string $name Name of the interface
      */
-    public function __construct($name) {
+    public function __construct(string $name) {
         $this->name = $name;
     }
 
     /**
      * Extends one or more interfaces.
      *
-     * @param Name|string $interface Name of interface to extend
-     * @param Name|string $...       More interfaces to extend
+     * @param Name|string ...$interfaces Names of interfaces to extend
      *
      * @return $this The builder instance (for fluid interface)
      */
-    public function extend() {
-        foreach (func_get_args() as $interface) {
-            $this->extends[] = $this->normalizeName($interface);
+    public function extend(...$interfaces) {
+        foreach ($interfaces as $interface) {
+            $this->extends[] = BuilderHelpers::normalizeName($interface);
         }
 
         return $this;
@@ -46,7 +46,7 @@ class Interface_ extends Declaration
      * @return $this The builder instance (for fluid interface)
      */
     public function addStmt($stmt) {
-        $stmt = $this->normalizeNode($stmt);
+        $stmt = BuilderHelpers::normalizeNode($stmt);
 
         $type = $stmt->getType();
         switch ($type) {
@@ -72,10 +72,10 @@ class Interface_ extends Declaration
      *
      * @return Stmt\Interface_ The built interface node
      */
-    public function getNode() {
-        return new Stmt\Interface_($this->name, array(
+    public function getNode() : PhpParser\Node {
+        return new Stmt\Interface_($this->name, [
             'extends' => $this->extends,
             'stmts' => array_merge($this->constants, $this->methods),
-        ), $this->attributes);
+        ], $this->attributes);
     }
 }

lib/PhpParser/Builder/Method.php

@@ -1,23 +1,26 @@
-<?php
+<?php declare(strict_types=1);
 
 namespace PhpParser\Builder;
 
 use PhpParser;
+use PhpParser\BuilderHelpers;
 use PhpParser\Node;
 use PhpParser\Node\Stmt;
 
 class Method extends FunctionLike
 {
     protected $name;
-    protected $type = 0;
-    protected $stmts = array();
+    protected $flags = 0;
+
+    /** @var array|null */
+    protected $stmts = [];
 
     /**
      * Creates a method builder.
      *
      * @param string $name Name of the method
      */
-    public function __construct($name) {
+    public function __construct(string $name) {
         $this->name = $name;
     }
 
@@ -27,7 +30,7 @@ class Method extends FunctionLike
      * @return $this The builder instance (for fluid interface)
      */
     public function makePublic() {
-        $this->setModifier(Stmt\Class_::MODIFIER_PUBLIC);
+        $this->flags = BuilderHelpers::addModifier($this->flags, Stmt\Class_::MODIFIER_PUBLIC);
 
         return $this;
     }
@@ -38,7 +41,7 @@ class Method extends FunctionLike
      * @return $this The builder instance (for fluid interface)
      */
     public function makeProtected() {
-        $this->setModifier(Stmt\Class_::MODIFIER_PROTECTED);
+        $this->flags = BuilderHelpers::addModifier($this->flags, Stmt\Class_::MODIFIER_PROTECTED);
 
         return $this;
     }
@@ -49,7 +52,7 @@ class Method extends FunctionLike
      * @return $this The builder instance (for fluid interface)
      */
     public function makePrivate() {
-        $this->setModifier(Stmt\Class_::MODIFIER_PRIVATE);
+        $this->flags = BuilderHelpers::addModifier($this->flags, Stmt\Class_::MODIFIER_PRIVATE);
 
         return $this;
     }
@@ -60,7 +63,7 @@ class Method extends FunctionLike
      * @return $this The builder instance (for fluid interface)
      */
     public function makeStatic() {
-        $this->setModifier(Stmt\Class_::MODIFIER_STATIC);
+        $this->flags = BuilderHelpers::addModifier($this->flags, Stmt\Class_::MODIFIER_STATIC);
 
         return $this;
     }
@@ -75,7 +78,7 @@ class Method extends FunctionLike
             throw new \LogicException('Cannot make method with statements abstract');
         }
 
-        $this->setModifier(Stmt\Class_::MODIFIER_ABSTRACT);
+        $this->flags = BuilderHelpers::addModifier($this->flags, Stmt\Class_::MODIFIER_ABSTRACT);
         $this->stmts = null; // abstract methods don't have statements
 
         return $this;
@@ -87,7 +90,7 @@ class Method extends FunctionLike
      * @return $this The builder instance (for fluid interface)
      */
     public function makeFinal() {
-        $this->setModifier(Stmt\Class_::MODIFIER_FINAL);
+        $this->flags = BuilderHelpers::addModifier($this->flags, Stmt\Class_::MODIFIER_FINAL);
 
         return $this;
     }
@@ -104,7 +107,7 @@ class Method extends FunctionLike
             throw new \LogicException('Cannot add statements to an abstract method');
         }
 
-        $this->stmts[] = $this->normalizeNode($stmt);
+        $this->stmts[] = BuilderHelpers::normalizeStmt($stmt);
 
         return $this;
     }
@@ -114,12 +117,13 @@ class Method extends FunctionLike
      *
      * @return Stmt\ClassMethod The built method node
      */
-    public function getNode() {
-        return new Stmt\ClassMethod($this->name, array(
-            'type'   => $this->type,
-            'byRef'  => $this->returnByRef,
-            'params' => $this->params,
-            'stmts'  => $this->stmts,
-        ), $this->attributes);
+    public function getNode() : Node {
+        return new Stmt\ClassMethod($this->name, [
+            'flags'      => $this->flags,
+            'byRef'      => $this->returnByRef,
+            'params'     => $this->params,
+            'returnType' => $this->returnType,
+            'stmts'      => $this->stmts,
+        ], $this->attributes);
     }
 }

lib/PhpParser/Builder/Namespace_.php

@@ -1,15 +1,16 @@
-<?php
+<?php declare(strict_types=1);
 
 namespace PhpParser\Builder;
 
 use PhpParser;
+use PhpParser\BuilderHelpers;
 use PhpParser\Node;
 use PhpParser\Node\Stmt;
 
-class Namespace_ extends PhpParser\BuilderAbstract
+class Namespace_ implements PhpParser\Builder
 {
     private $name;
-    private $stmts = array();
+    private $stmts = [];
 
     /**
      * Creates a namespace builder.
@@ -17,7 +18,7 @@ class Namespace_ extends PhpParser\BuilderAbstract
      * @param Node\Name|string|null $name Name of the namespace
      */
     public function __construct($name) {
-        $this->name = null !== $name ? $this->normalizeName($name) : null;
+        $this->name = null !== $name ? BuilderHelpers::normalizeName($name) : null;
     }
 
     /**
@@ -28,7 +29,7 @@ class Namespace_ extends PhpParser\BuilderAbstract
      * @return $this The builder instance (for fluid interface)
      */
     public function addStmt($stmt) {
-        $this->stmts[] = $this->normalizeNode($stmt);
+        $this->stmts[] = BuilderHelpers::normalizeStmt($stmt);
 
         return $this;
     }
@@ -53,7 +54,7 @@ class Namespace_ extends PhpParser\BuilderAbstract
      *
      * @return Node The built node
      */
-    public function getNode() {
+    public function getNode() : Node {
         return new Stmt\Namespace_($this->name, $this->stmts);
     }
 }

lib/PhpParser/Builder/Param.php

@@ -1,24 +1,30 @@
-<?php
+<?php declare(strict_types=1);
 
 namespace PhpParser\Builder;
 
 use PhpParser;
+use PhpParser\BuilderHelpers;
 use PhpParser\Node;
 
-class Param extends PhpParser\BuilderAbstract
+class Param implements PhpParser\Builder
 {
     protected $name;
 
     protected $default = null;
+
+    /** @var string|Node\Name|Node\NullableType|null */
     protected $type = null;
+
     protected $byRef = false;
 
+    protected $variadic = false;
+
     /**
      * Creates a parameter builder.
      *
      * @param string $name Name of the parameter
      */
-    public function __construct($name) {
+    public function __construct(string $name) {
         $this->name = $name;
     }
 
@@ -30,7 +36,7 @@ class Param extends PhpParser\BuilderAbstract
      * @return $this The builder instance (for fluid interface)
      */
     public function setDefault($value) {
-        $this->default = $this->normalizeValue($value);
+        $this->default = BuilderHelpers::normalizeValue($value);
 
         return $this;
     }
@@ -38,15 +44,14 @@ class Param extends PhpParser\BuilderAbstract
     /**
      * Sets type hint for the parameter.
      *
-     * @param string|Node\Name $type Type hint to use
+     * @param string|Node\Name|Node\NullableType $type Type hint to use
      *
      * @return $this The builder instance (for fluid interface)
      */
     public function setTypeHint($type) {
-        if ($type === 'array' || $type === 'callable') {
-            $this->type = $type;
-        } else {
-            $this->type = $this->normalizeName($type);
+        $this->type = BuilderHelpers::normalizeType($type);
+        if ($this->type == 'void') {
+            throw new \LogicException('Parameter type cannot be void');
         }
 
         return $this;
@@ -63,14 +68,26 @@ class Param extends PhpParser\BuilderAbstract
         return $this;
     }
 
+    /**
+     * Make the parameter variadic
+     *
+     * @return $this The builder instance (for fluid interface)
+     */
+    public function makeVariadic() {
+        $this->variadic = true;
+
+        return $this;
+    }
+
     /**
      * Returns the built parameter node.
      *
      * @return Node\Param The built parameter node
      */
-    public function getNode() {
+    public function getNode() : Node {
         return new Node\Param(
-            $this->name, $this->default, $this->type, $this->byRef
+            new Node\Expr\Variable($this->name),
+            $this->default, $this->type, $this->byRef, $this->variadic
         );
     }
-}
+}

lib/PhpParser/Builder/Property.php

@@ -1,24 +1,25 @@
-<?php
+<?php declare(strict_types=1);
 
 namespace PhpParser\Builder;
 
 use PhpParser;
+use PhpParser\BuilderHelpers;
 use PhpParser\Node\Stmt;
 
-class Property extends PhpParser\BuilderAbstract
+class Property implements PhpParser\Builder
 {
     protected $name;
 
-    protected $type = 0;
+    protected $flags = 0;
     protected $default = null;
-    protected $attributes = array();
+    protected $attributes = [];
 
     /**
      * Creates a property builder.
      *
      * @param string $name Name of the property
      */
-    public function __construct($name) {
+    public function __construct(string $name) {
         $this->name = $name;
     }
 
@@ -28,7 +29,7 @@ class Property extends PhpParser\BuilderAbstract
      * @return $this The builder instance (for fluid interface)
      */
     public function makePublic() {
-        $this->setModifier(Stmt\Class_::MODIFIER_PUBLIC);
+        $this->flags = BuilderHelpers::addModifier($this->flags, Stmt\Class_::MODIFIER_PUBLIC);
 
         return $this;
     }
@@ -39,7 +40,7 @@ class Property extends PhpParser\BuilderAbstract
      * @return $this The builder instance (for fluid interface)
      */
     public function makeProtected() {
-        $this->setModifier(Stmt\Class_::MODIFIER_PROTECTED);
+        $this->flags = BuilderHelpers::addModifier($this->flags, Stmt\Class_::MODIFIER_PROTECTED);
 
         return $this;
     }
@@ -50,7 +51,7 @@ class Property extends PhpParser\BuilderAbstract
      * @return $this The builder instance (for fluid interface)
      */
     public function makePrivate() {
-        $this->setModifier(Stmt\Class_::MODIFIER_PRIVATE);
+        $this->flags = BuilderHelpers::addModifier($this->flags, Stmt\Class_::MODIFIER_PRIVATE);
 
         return $this;
     }
@@ -61,7 +62,7 @@ class Property extends PhpParser\BuilderAbstract
      * @return $this The builder instance (for fluid interface)
      */
     public function makeStatic() {
-        $this->setModifier(Stmt\Class_::MODIFIER_STATIC);
+        $this->flags = BuilderHelpers::addModifier($this->flags, Stmt\Class_::MODIFIER_STATIC);
 
         return $this;
     }
@@ -74,7 +75,7 @@ class Property extends PhpParser\BuilderAbstract
      * @return $this The builder instance (for fluid interface)
      */
     public function setDefault($value) {
-        $this->default = $this->normalizeValue($value);
+        $this->default = BuilderHelpers::normalizeValue($value);
 
         return $this;
     }
@@ -87,9 +88,9 @@ class Property extends PhpParser\BuilderAbstract
      * @return $this The builder instance (for fluid interface)
      */
     public function setDocComment($docComment) {
-        $this->attributes = array(
-            'comments' => array($this->normalizeDocComment($docComment))
-        );
+        $this->attributes = [
+            'comments' => [BuilderHelpers::normalizeDocComment($docComment)]
+        ];
 
         return $this;
     }
@@ -99,12 +100,12 @@ class Property extends PhpParser\BuilderAbstract
      *
      * @return Stmt\Property The built property node
      */
-    public function getNode() {
+    public function getNode() : PhpParser\Node {
         return new Stmt\Property(
-            $this->type !== 0 ? $this->type : Stmt\Class_::MODIFIER_PUBLIC,
-            array(
+            $this->flags !== 0 ? $this->flags : Stmt\Class_::MODIFIER_PUBLIC,
+            [
                 new Stmt\PropertyProperty($this->name, $this->default)
-            ),
+            ],
             $this->attributes
         );
     }

lib/PhpParser/Builder/Trait_.php

@@ -1,23 +1,24 @@
-<?php
+<?php declare(strict_types=1);
 
 namespace PhpParser\Builder;
 
 use PhpParser;
-use PhpParser\Node\Name;
+use PhpParser\BuilderHelpers;
 use PhpParser\Node\Stmt;
 
 class Trait_ extends Declaration
 {
     protected $name;
-    protected $properties = array();
-    protected $methods = array();
+    protected $uses = [];
+    protected $properties = [];
+    protected $methods = [];
 
     /**
      * Creates an interface builder.
      *
      * @param string $name Name of the interface
      */
-    public function __construct($name) {
+    public function __construct(string $name) {
         $this->name = $name;
     }
 
@@ -29,12 +30,14 @@ class Trait_ extends Declaration
      * @return $this The builder instance (for fluid interface)
      */
     public function addStmt($stmt) {
-        $stmt = $this->normalizeNode($stmt);
+        $stmt = BuilderHelpers::normalizeNode($stmt);
 
         if ($stmt instanceof Stmt\Property) {
             $this->properties[] = $stmt;
         } else if ($stmt instanceof Stmt\ClassMethod) {
             $this->methods[] = $stmt;
+        } else if ($stmt instanceof Stmt\TraitUse) {
+            $this->uses[] = $stmt;
         } else {
             throw new \LogicException(sprintf('Unexpected node of type "%s"', $stmt->getType()));
         }
@@ -47,9 +50,11 @@ class Trait_ extends Declaration
      *
      * @return Stmt\Trait_ The built interface node
      */
-    public function getNode() {
+    public function getNode() : PhpParser\Node {
         return new Stmt\Trait_(
-            $this->name, array_merge($this->properties, $this->methods), $this->attributes
+            $this->name, [
+                'stmts' => array_merge($this->uses, $this->properties, $this->methods)
+            ], $this->attributes
         );
     }
 }

lib/PhpParser/Builder/Use_.php

@@ -1,15 +1,13 @@
-<?php
+<?php declare(strict_types=1);
 
 namespace PhpParser\Builder;
 
-use PhpParser\BuilderAbstract;
+use PhpParser\Builder;
+use PhpParser\BuilderHelpers;
 use PhpParser\Node;
 use PhpParser\Node\Stmt;
 
-/**
- * @method $this as(string $alias) Sets alias for used name.
- */
-class Use_ extends BuilderAbstract {
+class Use_ implements Builder {
     protected $name;
     protected $type;
     protected $alias = null;
@@ -20,8 +18,8 @@ class Use_ extends BuilderAbstract {
      * @param Node\Name|string $name Name of the entity (namespace, class, function, constant) to alias
      * @param int              $type One of the Stmt\Use_::TYPE_* constants
      */
-    public function __construct($name, $type) {
-        $this->name = $this->normalizeName($name);
+    public function __construct($name, int $type) {
+        $this->name = BuilderHelpers::normalizeName($name);
         $this->type = $type;
     }
 
@@ -32,27 +30,19 @@ class Use_ extends BuilderAbstract {
      *
      * @return $this The builder instance (for fluid interface)
      */
-    protected function as_($alias) {
+    public function as(string $alias) {
         $this->alias = $alias;
         return $this;
     }
-    public function __call($name, $args) {
-        if (method_exists($this, $name . '_')) {
-            return call_user_func_array(array($this, $name . '_'), $args);
-        }
-
-        throw new \LogicException(sprintf('Method "%s" does not exist', $name));
-    }
 
     /**
      * Returns the built node.
      *
      * @return Node The built node
      */
-    public function getNode() {
-        $alias = null !== $this->alias ? $this->alias : $this->name->getLast();
-        return new Stmt\Use_(array(
-            new Stmt\UseUse($this->name, $alias)
-        ), $this->type);
+    public function getNode() : Node {
+        return new Stmt\Use_([
+            new Stmt\UseUse($this->name, $this->alias)
+        ], $this->type);
     }
 }

lib/PhpParser/BuilderAbstract.php

@@ -1,131 +0,0 @@
-<?php
-
-namespace PhpParser;
-
-use PhpParser\Node\Name;
-use PhpParser\Node\Expr;
-use PhpParser\Node\Stmt;
-use PhpParser\Node\Scalar;
-use PhpParser\Comment;
-
-abstract class BuilderAbstract implements Builder {
-    /**
-     * Normalizes a node: Converts builder objects to nodes.
-     *
-     * @param Node|Builder $node The node to normalize
-     *
-     * @return Node The normalized node
-     */
-    protected function normalizeNode($node) {
-        if ($node instanceof Builder) {
-            return $node->getNode();
-        } elseif ($node instanceof Node) {
-            return $node;
-        }
-
-        throw new \LogicException('Expected node or builder object');
-    }
-
-    /**
-     * Normalizes a name: Converts plain string names to PhpParser\Node\Name.
-     *
-     * @param Name|string $name The name to normalize
-     *
-     * @return Name The normalized name
-     */
-    protected function normalizeName($name) {
-        if ($name instanceof Name) {
-            return $name;
-        } elseif (is_string($name)) {
-            if (!$name) {
-                throw new \LogicException('Name cannot be empty');
-            }
-
-            if ($name[0] == '\\') {
-                return new Name\FullyQualified(substr($name, 1));
-            } elseif (0 === strpos($name, 'namespace\\')) {
-                return new Name\Relative(substr($name, strlen('namespace\\')));
-            } else {
-                return new Name($name);
-            }
-        }
-
-        throw new \LogicException('Name must be a string or an instance of PhpParser\Node\Name');
-    }
-
-    /**
-     * Normalizes a value: Converts nulls, booleans, integers,
-     * floats, strings and arrays into their respective nodes
-     *
-     * @param mixed $value The value to normalize
-     *
-     * @return Expr The normalized value
-     */
-    protected function normalizeValue($value) {
-        if ($value instanceof Node) {
-            return $value;
-        } elseif (is_null($value)) {
-            return new Expr\ConstFetch(
-                new Name('null')
-            );
-        } elseif (is_bool($value)) {
-            return new Expr\ConstFetch(
-                new Name($value ? 'true' : 'false')
-            );
-        } elseif (is_int($value)) {
-            return new Scalar\LNumber($value);
-        } elseif (is_float($value)) {
-            return new Scalar\DNumber($value);
-        } elseif (is_string($value)) {
-            return new Scalar\String_($value);
-        } elseif (is_array($value)) {
-            $items = array();
-            $lastKey = -1;
-            foreach ($value as $itemKey => $itemValue) {
-                // for consecutive, numeric keys don't generate keys
-                if (null !== $lastKey && ++$lastKey === $itemKey) {
-                    $items[] = new Expr\ArrayItem(
-                        $this->normalizeValue($itemValue)
-                    );
-                } else {
-                    $lastKey = null;
-                    $items[] = new Expr\ArrayItem(
-                        $this->normalizeValue($itemValue),
-                        $this->normalizeValue($itemKey)
-                    );
-                }
-            }
-
-            return new Expr\Array_($items);
-        } else {
-            throw new \LogicException('Invalid value');
-        }
-    }
-
-    /**
-     * Normalizes a doc comment: Converts plain strings to PhpParser\Comment\Doc.
-     *
-     * @param Comment\Doc|string $docComment The doc comment to normalize
-     *
-     * @return Comment\Doc The normalized doc comment
-     */
-    protected function normalizeDocComment($docComment) {
-        if ($docComment instanceof Comment\Doc) {
-            return $docComment;
-        } else if (is_string($docComment)) {
-            return new Comment\Doc($docComment);
-        } else {
-            throw new \LogicException('Doc comment must be a string or an instance of PhpParser\Comment\Doc');
-        }
-    }
-
-    /**
-     * Sets a modifier in the $this->type property.
-     *
-     * @param int $modifier Modifier to set
-     */
-    protected function setModifier($modifier) {
-        Stmt\Class_::verifyModifier($this->type, $modifier);
-        $this->type |= $modifier;
-    }
-}

lib/PhpParser/BuilderFactory.php

@@ -1,31 +1,23 @@
-<?php
+<?php declare(strict_types=1);
 
 namespace PhpParser;
 
-use PhpParser\Builder;
+use PhpParser\Node\Arg;
+use PhpParser\Node\Expr;
+use PhpParser\Node\Expr\BinaryOp\Concat;
+use PhpParser\Node\Scalar\String_;
 use PhpParser\Node\Stmt\Use_;
 
-/**
- * The following methods use reserved keywords, so their implementation is defined with an underscore and made available
- * with the reserved name through __call() magic.
- *
- * @method Builder\Namespace_ namespace(string $name) Creates a namespace builder.
- * @method Builder\Class_     class(string $name)     Creates a class builder.
- * @method Builder\Interface_ interface(string $name) Creates an interface builder.
- * @method Builder\Trait_     trait(string $name)     Creates a trait builder.
- * @method Builder\Function_  function(string $name)  Creates a function builder.
- * @method Builder\Use_       use(string $name)       Creates a namespace/class use builder.
- */
 class BuilderFactory
 {
     /**
      * Creates a namespace builder.
-     * 
+     *
      * @param null|string|Node\Name $name Name of the namespace
      *
      * @return Builder\Namespace_ The created namespace builder
      */
-    protected function _namespace($name) {
+    public function namespace($name) : Builder\Namespace_ {
         return new Builder\Namespace_($name);
     }
 
@@ -36,7 +28,7 @@ class BuilderFactory
      *
      * @return Builder\Class_ The created class builder
      */
-    protected function _class($name) {
+    public function class(string $name) : Builder\Class_ {
         return new Builder\Class_($name);
     }
 
@@ -47,7 +39,7 @@ class BuilderFactory
      *
      * @return Builder\Interface_ The created interface builder
      */
-    protected function _interface($name) {
+    public function interface(string $name) : Builder\Interface_ {
         return new Builder\Interface_($name);
     }
 
@@ -58,7 +50,7 @@ class BuilderFactory
      *
      * @return Builder\Trait_ The created trait builder
      */
-    protected function _trait($name) {
+    public function trait(string $name) : Builder\Trait_ {
         return new Builder\Trait_($name);
     }
 
@@ -69,7 +61,7 @@ class BuilderFactory
      *
      * @return Builder\Method The created method builder
      */
-    public function method($name) {
+    public function method(string $name) : Builder\Method {
         return new Builder\Method($name);
     }
 
@@ -80,7 +72,7 @@ class BuilderFactory
      *
      * @return Builder\Param The created parameter builder
      */
-    public function param($name) {
+    public function param(string $name) : Builder\Param {
         return new Builder\Param($name);
     }
 
@@ -91,7 +83,7 @@ class BuilderFactory
      *
      * @return Builder\Property The created property builder
      */
-    public function property($name) {
+    public function property(string $name) : Builder\Property {
         return new Builder\Property($name);
     }
 
@@ -102,7 +94,7 @@ class BuilderFactory
      *
      * @return Builder\Function_ The created function builder
      */
-    protected function _function($name) {
+    public function function(string $name) : Builder\Function_ {
         return new Builder\Function_($name);
     }
 
@@ -113,15 +105,71 @@ class BuilderFactory
      *
      * @return Builder\Use_ The create use builder
      */
-    protected function _use($name) {
+    public function use($name) : Builder\Use_ {
         return new Builder\Use_($name, Use_::TYPE_NORMAL);
     }
 
-    public function __call($name, array $args) {
-        if (method_exists($this, '_' . $name)) {
-            return call_user_func_array(array($this, '_' . $name), $args);
+    /**
+     * Creates node a for a literal value.
+     *
+     * @param Expr|bool|null|int|float|string|array $value $value
+     *
+     * @return Expr
+     */
+    public function val($value) : Expr {
+        return BuilderHelpers::normalizeValue($value);
+    }
+
+    /**
+     * Normalizes an argument list.
+     *
+     * Creates Arg nodes for all arguments and converts literal values to expressions.
+     *
+     * @param array $args List of arguments to normalize
+     *
+     * @return Arg[]
+     */
+    public function args(array $args) : array {
+        $normalizedArgs = [];
+        foreach ($args as $arg) {
+            if ($arg instanceof Arg) {
+                $normalizedArgs[] = $arg;
+            } else {
+                $normalizedArgs[] = new Arg(BuilderHelpers::normalizeValue($arg));
+            }
+        }
+        return $normalizedArgs;
+    }
+
+    /**
+     * Creates nested Concat nodes from a list of expressions.
+     *
+     * @param Expr|string ...$exprs Expressions or literal strings
+     *
+     * @return Concat
+     */
+    public function concat(...$exprs) : Concat {
+        $numExprs = count($exprs);
+        if ($numExprs < 2) {
+            throw new \LogicException('Expected at least two expressions');
         }
 
-        throw new \LogicException(sprintf('Method "%s" does not exist', $name));
+        $lastConcat = $this->normalizeStringExpr($exprs[0]);
+        for ($i = 1; $i < $numExprs; $i++) {
+            $lastConcat = new Concat($lastConcat, $this->normalizeStringExpr($exprs[$i]));
+        }
+        return $lastConcat;
+    }
+
+    private function normalizeStringExpr($expr) {
+        if ($expr instanceof Expr) {
+            return $expr;
+        }
+
+        if (is_string($expr)) {
+            return new String_($expr);
+        }
+
+        throw new \LogicException('Expected string or Expr');
     }
 }

lib/PhpParser/BuilderHelpers.php

@@ -0,0 +1,206 @@
+<?php declare(strict_types=1);
+
+namespace PhpParser;
+
+use PhpParser\Node\Expr;
+use PhpParser\Node\Identifier;
+use PhpParser\Node\Name;
+use PhpParser\Node\NullableType;
+use PhpParser\Node\Scalar;
+use PhpParser\Node\Stmt;
+
+/**
+ * This class defines helpers used in the implementation of builders. Don't use it directly.
+ *
+ * @internal
+ */
+final class BuilderHelpers {
+    /**
+     * Normalizes a node: Converts builder objects to nodes.
+     *
+     * @param Node|Builder $node The node to normalize
+     *
+     * @return Node The normalized node
+     */
+    public static function normalizeNode($node) : Node {
+        if ($node instanceof Builder) {
+            return $node->getNode();
+        } elseif ($node instanceof Node) {
+            return $node;
+        }
+
+        throw new \LogicException('Expected node or builder object');
+    }
+
+    /**
+     * Normalizes a node to a statement.
+     *
+     * Expressions are wrapped in a Stmt\Expression node.
+     *
+     * @param Node|Builder $node The node to normalize
+     *
+     * @return Stmt The normalized statement node
+     */
+    public static function normalizeStmt($node) : Stmt {
+        $node = self::normalizeNode($node);
+        if ($node instanceof Stmt) {
+            return $node;
+        }
+
+        if ($node instanceof Expr) {
+            return new Stmt\Expression($node);
+        }
+
+        throw new \LogicException('Expected statement or expression node');
+    }
+
+    /**
+     * Normalizes a name: Converts plain string names to PhpParser\Node\Name.
+     *
+     * @param Name|string $name The name to normalize
+     *
+     * @return Name The normalized name
+     */
+    public static function normalizeName($name) : Name {
+        if ($name instanceof Name) {
+            return $name;
+        } elseif (is_string($name)) {
+            if (!$name) {
+                throw new \LogicException('Name cannot be empty');
+            }
+
+            if ($name[0] === '\\') {
+                return new Name\FullyQualified(substr($name, 1));
+            } elseif (0 === strpos($name, 'namespace\\')) {
+                return new Name\Relative(substr($name, strlen('namespace\\')));
+            } else {
+                return new Name($name);
+            }
+        }
+
+        throw new \LogicException('Name must be a string or an instance of PhpParser\Node\Name');
+    }
+
+    /**
+     * Normalizes a type: Converts plain-text type names into proper AST representation.
+     *
+     * In particular, builtin types become Identifiers, custom types become Names and nullables
+     * are wrapped in NullableType nodes.
+     *
+     * @param string|Name|Identifier|NullableType $type The type to normalize
+     *
+     * @return Name|Identifier|NullableType The normalized type
+     */
+    public static function normalizeType($type) {
+        if (!is_string($type)) {
+            if (!$type instanceof Name && !$type instanceof Identifier
+                    && !$type instanceof NullableType) {
+                throw new \LogicException(
+                    'Type must be a string, or an instance of Name, Identifier or NullableType');
+            }
+            return $type;
+        }
+
+        $nullable = false;
+        if (strlen($type) > 0 && $type[0] === '?') {
+            $nullable = true;
+            $type = substr($type, 1);
+        }
+
+        $builtinTypes = [
+            'array', 'callable', 'string', 'int', 'float', 'bool', 'iterable', 'void', 'object'
+        ];
+
+        $lowerType = strtolower($type);
+        if (in_array($lowerType, $builtinTypes)) {
+            $type = new Identifier($lowerType);
+        } else {
+            $type = self::normalizeName($type);
+        }
+
+        if ($nullable && (string) $type === 'void') {
+            throw new \LogicException('void type cannot be nullable');
+        }
+
+        return $nullable ? new Node\NullableType($type) : $type;
+    }
+
+    /**
+     * Normalizes a value: Converts nulls, booleans, integers,
+     * floats, strings and arrays into their respective nodes
+     *
+     * @param Node\Expr|bool|null|int|float|string|array $value The value to normalize
+     *
+     * @return Expr The normalized value
+     */
+    public static function normalizeValue($value) : Expr {
+        if ($value instanceof Node\Expr) {
+            return $value;
+        } elseif (is_null($value)) {
+            return new Expr\ConstFetch(
+                new Name('null')
+            );
+        } elseif (is_bool($value)) {
+            return new Expr\ConstFetch(
+                new Name($value ? 'true' : 'false')
+            );
+        } elseif (is_int($value)) {
+            return new Scalar\LNumber($value);
+        } elseif (is_float($value)) {
+            return new Scalar\DNumber($value);
+        } elseif (is_string($value)) {
+            return new Scalar\String_($value);
+        } elseif (is_array($value)) {
+            $items = [];
+            $lastKey = -1;
+            foreach ($value as $itemKey => $itemValue) {
+                // for consecutive, numeric keys don't generate keys
+                if (null !== $lastKey && ++$lastKey === $itemKey) {
+                    $items[] = new Expr\ArrayItem(
+                        self::normalizeValue($itemValue)
+                    );
+                } else {
+                    $lastKey = null;
+                    $items[] = new Expr\ArrayItem(
+                        self::normalizeValue($itemValue),
+                        self::normalizeValue($itemKey)
+                    );
+                }
+            }
+
+            return new Expr\Array_($items);
+        } else {
+            throw new \LogicException('Invalid value');
+        }
+    }
+
+    /**
+     * Normalizes a doc comment: Converts plain strings to PhpParser\Comment\Doc.
+     *
+     * @param Comment\Doc|string $docComment The doc comment to normalize
+     *
+     * @return Comment\Doc The normalized doc comment
+     */
+    public static function normalizeDocComment($docComment) : Comment\Doc {
+        if ($docComment instanceof Comment\Doc) {
+            return $docComment;
+        } else if (is_string($docComment)) {
+            return new Comment\Doc($docComment);
+        } else {
+            throw new \LogicException('Doc comment must be a string or an instance of PhpParser\Comment\Doc');
+        }
+    }
+
+    /**
+     * Adds a modifier and returns new modifier bitmask.
+     *
+     * @param int $modifiers Existing modifiers
+     * @param int $modifier  Modifier to set
+     *
+     * @return int New modifiers
+     */
+    public static function addModifier(int $modifiers, int $modifier) : int {
+        Stmt\Class_::verifyModifier($modifiers, $modifier);
+        return $modifiers | $modifier;
+    }
+}

lib/PhpParser/Comment.php

@@ -1,21 +1,24 @@
-<?php
+<?php declare(strict_types=1);
 
 namespace PhpParser;
 
-class Comment
+class Comment implements \JsonSerializable
 {
     protected $text;
     protected $line;
+    protected $filePos;
 
     /**
      * Constructs a comment node.
      *
-     * @param string $text Comment text (including comment delimiters like /*)
-     * @param int    $line Line number the comment started on
+     * @param string $text         Comment text (including comment delimiters like /*)
+     * @param int    $startLine    Line number the comment started on
+     * @param int    $startFilePos File offset the comment started on
      */
-    public function __construct($text, $line = -1) {
+    public function __construct(string $text, int $startLine = -1, int $startFilePos = -1) {
         $this->text = $text;
-        $this->line = $line;
+        $this->line = $startLine;
+        $this->filePos = $startFilePos;
     }
 
     /**
@@ -23,35 +26,26 @@ class Comment
      *
      * @return string The comment text (including comment delimiters like /*)
      */
-    public function getText() {
+    public function getText() : string {
         return $this->text;
     }
 
-    /**
-     * Sets the comment text.
-     *
-     * @param string $text The comment text (including comment delimiters like /*)
-     */
-    public function setText($text) {
-        $this->text = $text;
-    }
-
     /**
      * Gets the line number the comment started on.
      *
      * @return int Line number
      */
-    public function getLine() {
+    public function getLine() : int {
         return $this->line;
     }
 
     /**
-     * Sets the line number the comment started on.
+     * Gets the file offset the comment started on.
      *
-     * @param int $line Line number
+     * @return int File offset
      */
-    public function setLine($line) {
-        $this->line = $line;
+    public function getFilePos() : int {
+        return $this->filePos;
     }
 
     /**
@@ -59,7 +53,7 @@ class Comment
      *
      * @return string The comment text (including comment delimiters like /*)
      */
-    public function __toString() {
+    public function __toString() : string {
         return $this->text;
     }
 
@@ -75,7 +69,8 @@ class Comment
      */
     public function getReformattedText() {
         $text = trim($this->text);
-        if (false === strpos($text, "\n")) {
+        $newlinePos = strpos($text, "\n");
+        if (false === $newlinePos) {
             // Single line comments don't need further processing
             return $text;
         } elseif (preg_match('((*BSR_ANYCRLF)(*ANYCRLF)^.*(?:\R\s+\*.*)+$)', $text)) {
@@ -105,15 +100,53 @@ class Comment
             //
             //     /* Some text.
             //        Some more text.
+            //          Indented text.
             //        Even more text. */
             //
-            // is handled by taking the length of the "/* " segment and leaving only that
-            // many space characters before the lines. Thus in the above example only three
-            // space characters are left at the start of every line.
-            return preg_replace('(^\s*(?= {' . strlen($matches[0]) . '}(?!\s)))m', '', $text);
+            // is handled by removing the difference between the shortest whitespace prefix on all
+            // lines and the length of the "/* " opening sequence.
+            $prefixLen = $this->getShortestWhitespacePrefixLen(substr($text, $newlinePos + 1));
+            $removeLen = $prefixLen - strlen($matches[0]);
+            return preg_replace('(^\s{' . $removeLen . '})m', '', $text);
         }
 
         // No idea how to format this comment, so simply return as is
         return $text;
     }
+
+    /**
+     * Get length of shortest whitespace prefix (at the start of a line).
+     *
+     * If there is a line with no prefix whitespace, 0 is a valid return value.
+     *
+     * @param string $str String to check
+     * @return int Length in characters. Tabs count as single characters.
+     */
+    private function getShortestWhitespacePrefixLen(string $str) : int {
+        $lines = explode("\n", $str);
+        $shortestPrefixLen = INF;
+        foreach ($lines as $line) {
+            preg_match('(^\s*)', $line, $matches);
+            $prefixLen = strlen($matches[0]);
+            if ($prefixLen < $shortestPrefixLen) {
+                $shortestPrefixLen = $prefixLen;
+            }
+        }
+        return $shortestPrefixLen;
+    }
+
+    /**
+     * @return       array
+     * @psalm-return array{nodeType:string, text:mixed, line:mixed, filePos:mixed}
+     */
+    public function jsonSerialize() : array {
+        // Technically not a node, but we make it look like one anyway
+        $type = $this instanceof Comment\Doc ? 'Comment_Doc' : 'Comment';
+        return [
+            'nodeType' => $type,
+            'text' => $this->text,
+            'line' => $this->line,
+            'filePos' => $this->filePos,
+        ];
+    }
 }

lib/PhpParser/Comment/Doc.php

@@ -1,4 +1,4 @@
-<?php
+<?php declare(strict_types=1);
 
 namespace PhpParser\Comment;
 

lib/PhpParser/ConstExprEvaluationException.php

@@ -0,0 +1,5 @@
+<?php
+
+namespace PhpParser;
+
+class ConstExprEvaluationException extends \Exception {}

lib/PhpParser/ConstExprEvaluator.php

@@ -0,0 +1,186 @@
+<?php
+
+namespace PhpParser;
+
+use PhpParser\Node\Expr;
+use PhpParser\Node\Scalar;
+
+/**
+ * Evaluates constant expressions.
+ *
+ * This evaluator is able to evaluate all constant expressions (as defined by PHP), which can be
+ * evaluated without further context. If a subexpression is not of this type, a user-provided
+ * fallback evaluator is invoked. To support all constant expressions that are also supported by
+ * PHP (and not already handled by this class), the fallback evaluator must be able to handle the
+ * following node types:
+ *
+ *  * All Scalar\MagicConst\* nodes.
+ *  * Expr\ConstFetch nodes. Only null/false/true are already handled by this class.
+ *  * Expr\ClassConstFetch nodes.
+ *
+ * The fallback evaluator should throw ConstExprEvaluationException for nodes it cannot evaluate.
+ *
+ * The evaluation is performed as PHP would perform it, and as such may generate notices, warnings
+ * or Errors. For example, if the expression `1%0` is evaluated, an ArithmeticError is thrown. It is
+ * left to the consumer to handle these as appropriate.
+ *
+ * The evaluation is also dependent on runtime configuration in two respects: Firstly, floating
+ * point to string conversions are affected by the precision ini setting. Secondly, they are also
+ * affected by the LC_NUMERIC locale.
+ */
+class ConstExprEvaluator {
+    private $fallbackEvaluator;
+
+    /**
+     * Create a constant expression evaluator.
+     *
+     * The provided fallback evaluator is invoked whenever a subexpression cannot be evaluated. See
+     * class doc comment for more information.
+     *
+     * @param callable|null $fallbackEvaluator To call if subexpression cannot be evaluated
+     */
+    public function __construct(callable $fallbackEvaluator = null) {
+        $this->fallbackEvaluator = $fallbackEvaluator ?? function(Expr $expr) {
+            throw new ConstExprEvaluationException(
+                "Expression of type {$expr->getType()} cannot be evaluated"
+            );
+        };
+    }
+
+    /**
+     * Evaluates a constant expression into a PHP value.
+     *
+     * If some part of the expression cannot be evaluated, the fallback evaluator passed to the
+     * constructor will be invoked. By default, if no fallback is provided, an exception of type
+     * ConstExprEvaluationException is thrown.
+     *
+     * See class doc comment for caveats and limitations.
+     *
+     * @param Expr $expr Constant expression to evaluate
+     * @return mixed Result of evaluation
+     * @throws ConstExprEvaluationException if the expression cannot be evaluated
+     */
+    public function evaluate(Expr $expr) {
+        if ($expr instanceof Scalar\LNumber
+            || $expr instanceof Scalar\DNumber
+            || $expr instanceof Scalar\String_
+        ) {
+            return $expr->value;
+        }
+
+        if ($expr instanceof Expr\Array_) {
+            return $this->evaluateArray($expr);
+        }
+
+        // Unary operators
+        if ($expr instanceof Expr\UnaryPlus) {
+            return +$this->evaluate($expr->expr);
+        }
+        if ($expr instanceof Expr\UnaryMinus) {
+            return -$this->evaluate($expr->expr);
+        }
+        if ($expr instanceof Expr\BooleanNot) {
+            return !$this->evaluate($expr->expr);
+        }
+        if ($expr instanceof Expr\BitwiseNot) {
+            return ~$this->evaluate($expr->expr);
+        }
+
+        if ($expr instanceof Expr\BinaryOp) {
+            return $this->evaluateBinaryOp($expr);
+        }
+
+        if ($expr instanceof Expr\Ternary) {
+            return $this->evaluateTernary($expr);
+        }
+
+        if ($expr instanceof Expr\ArrayDimFetch && null !== $expr->dim) {
+            return $this->evaluate($expr->var)[$this->evaluate($expr->dim)];
+        }
+
+        if ($expr instanceof Expr\ConstFetch) {
+            return $this->evaluateConstFetch($expr);
+        }
+
+        return ($this->fallbackEvaluator)($expr);
+    }
+
+    private function evaluateArray(Expr\Array_ $expr) {
+        $array = [];
+        foreach ($expr->items as $item) {
+            if (null !== $item->key) {
+                $array[$this->evaluate($item->key)] = $this->evaluate($item->value);
+            } else {
+                $array[] = $this->evaluate($item->value);
+            }
+        }
+        return $array;
+    }
+
+    private function evaluateTernary(Expr\Ternary $expr) {
+        if (null === $expr->if) {
+            return $this->evaluate($expr->cond) ?: $this->evaluate($expr->else);
+        }
+
+        return $this->evaluate($expr->cond)
+            ? $this->evaluate($expr->if)
+            : $this->evaluate($expr->else);
+    }
+
+    private function evaluateBinaryOp(Expr\BinaryOp $expr) {
+        if ($expr instanceof Expr\BinaryOp\Coalesce
+            && $expr->left instanceof Expr\ArrayDimFetch
+        ) {
+            // This needs to be special cased to respect BP_VAR_IS fetch semantics
+            return $this->evaluate($expr->left->var)[$this->evaluate($expr->left->dim)]
+                ?? $this->evaluate($expr->right);
+        }
+
+        // The evaluate() calls are repeated in each branch, because some of the operators are
+        // short-circuiting and evaluating the RHS in advance may be illegal in that case
+        $l = $expr->left;
+        $r = $expr->right;
+        switch ($expr->getOperatorSigil()) {
+            case '&':   return $this->evaluate($l) &   $this->evaluate($r);
+            case '|':   return $this->evaluate($l) |   $this->evaluate($r);
+            case '^':   return $this->evaluate($l) ^   $this->evaluate($r);
+            case '&&':  return $this->evaluate($l) &&  $this->evaluate($r);
+            case '||':  return $this->evaluate($l) ||  $this->evaluate($r);
+            case '??':  return $this->evaluate($l) ??  $this->evaluate($r);
+            case '.':   return $this->evaluate($l) .   $this->evaluate($r);
+            case '/':   return $this->evaluate($l) /   $this->evaluate($r);
+            case '==':  return $this->evaluate($l) ==  $this->evaluate($r);
+            case '>':   return $this->evaluate($l) >   $this->evaluate($r);
+            case '>=':  return $this->evaluate($l) >=  $this->evaluate($r);
+            case '===': return $this->evaluate($l) === $this->evaluate($r);
+            case 'and': return $this->evaluate($l) and $this->evaluate($r);
+            case 'or':  return $this->evaluate($l) or  $this->evaluate($r);
+            case 'xor': return $this->evaluate($l) xor $this->evaluate($r);
+            case '-':   return $this->evaluate($l) -   $this->evaluate($r);
+            case '%':   return $this->evaluate($l) %   $this->evaluate($r);
+            case '*':   return $this->evaluate($l) *   $this->evaluate($r);
+            case '!=':  return $this->evaluate($l) !=  $this->evaluate($r);
+            case '!==': return $this->evaluate($l) !== $this->evaluate($r);
+            case '+':   return $this->evaluate($l) +   $this->evaluate($r);
+            case '**':  return $this->evaluate($l) **  $this->evaluate($r);
+            case '<<':  return $this->evaluate($l) <<  $this->evaluate($r);
+            case '>>':  return $this->evaluate($l) >>  $this->evaluate($r);
+            case '<':   return $this->evaluate($l) <   $this->evaluate($r);
+            case '<=':  return $this->evaluate($l) <=  $this->evaluate($r);
+            case '<=>': return $this->evaluate($l) <=> $this->evaluate($r);
+        }
+
+        throw new \Exception('Should not happen');
+    }
+
+    private function evaluateConstFetch(Expr\ConstFetch $expr) {
+        $name = $expr->name->toLowerString();
+        switch ($name) {
+            case 'null': return null;
+            case 'false': return false;
+            case 'true': return true;
+        }
+
+        return ($this->fallbackEvaluator)($expr);
+    }
+}

lib/PhpParser/Error.php

@@ -1,4 +1,4 @@
-<?php
+<?php declare(strict_types=1);
 
 namespace PhpParser;
 
@@ -14,12 +14,12 @@ class Error extends \RuntimeException
      * @param array|int $attributes Attributes of node/token where error occurred
      *                              (or start line of error -- deprecated)
      */
-    public function __construct($message, $attributes = array()) {
+    public function __construct(string $message, $attributes = []) {
         $this->rawMessage = (string) $message;
         if (is_array($attributes)) {
             $this->attributes = $attributes;
         } else {
-            $this->attributes = array('startLine' => $attributes);
+            $this->attributes = ['startLine' => $attributes];
         }
         $this->updateMessage();
     }
@@ -29,7 +29,7 @@ class Error extends \RuntimeException
      *
      * @return string Error message
      */
-    public function getRawMessage() {
+    public function getRawMessage() : string {
         return $this->rawMessage;
     }
 
@@ -38,8 +38,8 @@ class Error extends \RuntimeException
      *
      * @return int Error start line
      */
-    public function getStartLine() {
-        return isset($this->attributes['startLine']) ? $this->attributes['startLine'] : -1;
+    public function getStartLine() : int {
+        return $this->attributes['startLine'] ?? -1;
     }
 
     /**
@@ -47,8 +47,8 @@ class Error extends \RuntimeException
      *
      * @return int Error end line
      */
-    public function getEndLine() {
-        return isset($this->attributes['endLine']) ? $this->attributes['endLine'] : -1;
+    public function getEndLine() : int {
+        return $this->attributes['endLine'] ?? -1;
     }
 
 
@@ -57,16 +57,26 @@ class Error extends \RuntimeException
      *
      * @return array
      */
-    public function getAttributes() {
+    public function getAttributes() : array {
         return $this->attributes;
     }
 
+    /**
+     * Sets the attributes of the node/token the error occured at.
+     *
+     * @param array $attributes
+     */
+    public function setAttributes(array $attributes) {
+        $this->attributes = $attributes;
+        $this->updateMessage();
+    }
+
     /**
      * Sets the line of the PHP file the error occurred in.
      *
      * @param string $message Error message
      */
-    public function setRawMessage($message) {
+    public function setRawMessage(string $message) {
         $this->rawMessage = (string) $message;
         $this->updateMessage();
     }
@@ -76,7 +86,7 @@ class Error extends \RuntimeException
      *
      * @param int $line Error start line
      */
-    public function setStartLine($line) {
+    public function setStartLine(int $line) {
         $this->attributes['startLine'] = (int) $line;
         $this->updateMessage();
     }
@@ -88,7 +98,7 @@ class Error extends \RuntimeException
      *
      * @return bool
      */
-    public function hasColumnInfo() {
+    public function hasColumnInfo() : bool {
         return isset($this->attributes['startFilePos']) && isset($this->attributes['endFilePos']);
     }
 
@@ -98,7 +108,7 @@ class Error extends \RuntimeException
      * @param string $code Source code of the file
      * @return int
      */
-    public function getStartColumn($code) {
+    public function getStartColumn(string $code) : int {
         if (!$this->hasColumnInfo()) {
             throw new \RuntimeException('Error does not have column information');
         }
@@ -112,7 +122,7 @@ class Error extends \RuntimeException
      * @param string $code Source code of the file
      * @return int
      */
-    public function getEndColumn($code) {
+    public function getEndColumn(string $code) : int {
         if (!$this->hasColumnInfo()) {
             throw new \RuntimeException('Error does not have column information');
         }
@@ -120,7 +130,30 @@ class Error extends \RuntimeException
         return $this->toColumn($code, $this->attributes['endFilePos']);
     }
 
-    private function toColumn($code, $pos) {
+    /**
+     * Formats message including line and column information.
+     *
+     * @param string $code Source code associated with the error, for calculation of the columns
+     *
+     * @return string Formatted message
+     */
+    public function getMessageWithColumnInfo(string $code) : string {
+        return sprintf(
+            '%s from %d:%d to %d:%d', $this->getRawMessage(),
+            $this->getStartLine(), $this->getStartColumn($code),
+            $this->getEndLine(), $this->getEndColumn($code)
+        );
+    }
+
+    /**
+     * Converts a file offset into a column.
+     *
+     * @param string $code Source code that $pos indexes into
+     * @param int    $pos  0-based position in $code
+     *
+     * @return int 1-based column (relative to start of line)
+     */
+    private function toColumn(string $code, int $pos) : int {
         if ($pos > strlen($code)) {
             throw new \RuntimeException('Invalid position information');
         }
@@ -145,14 +178,4 @@ class Error extends \RuntimeException
             $this->message .= ' on line ' . $this->getStartLine();
         }
     }
-
-    /** @deprecated Use getStartLine() instead */
-    public function getRawLine() {
-        return $this->getStartLine();
-    }
-
-    /** @deprecated Use setStartLine() instead */
-    public function setRawLine($line) {
-        $this->setStartLine($line);
-    }
 }

lib/PhpParser/ErrorHandler.php

@@ -0,0 +1,13 @@
+<?php declare(strict_types=1);
+
+namespace PhpParser;
+
+interface ErrorHandler
+{
+    /**
+     * Handle an error generated during lexing, parsing or some other operation.
+     *
+     * @param Error $error The error that needs to be handled
+     */
+    public function handleError(Error $error);
+}

lib/PhpParser/ErrorHandler/Collecting.php

@@ -0,0 +1,46 @@
+<?php declare(strict_types=1);
+
+namespace PhpParser\ErrorHandler;
+
+use PhpParser\Error;
+use PhpParser\ErrorHandler;
+
+/**
+ * Error handler that collects all errors into an array.
+ *
+ * This allows graceful handling of errors.
+ */
+class Collecting implements ErrorHandler
+{
+    /** @var Error[] Collected errors */
+    private $errors = [];
+
+    public function handleError(Error $error) {
+        $this->errors[] = $error;
+    }
+
+    /**
+     * Get collected errors.
+     *
+     * @return Error[]
+     */
+    public function getErrors() : array {
+        return $this->errors;
+    }
+
+    /**
+     * Check whether there are any errors.
+     *
+     * @return bool
+     */
+    public function hasErrors() : bool {
+        return !empty($this->errors);
+    }
+
+    /**
+     * Reset/clear collected errors.
+     */
+    public function clearErrors() {
+        $this->errors = [];
+    }
+}

lib/PhpParser/ErrorHandler/Throwing.php

@@ -0,0 +1,18 @@
+<?php declare(strict_types=1);
+
+namespace PhpParser\ErrorHandler;
+
+use PhpParser\Error;
+use PhpParser\ErrorHandler;
+
+/**
+ * Error handler that handles all errors by throwing them.
+ *
+ * This is the default strategy used by all components.
+ */
+class Throwing implements ErrorHandler
+{
+    public function handleError(Error $error) {
+        throw $error;
+    }
+}

lib/PhpParser/Internal/DiffElem.php

@@ -0,0 +1,26 @@
+<?php
+
+namespace PhpParser\Internal;
+
+/**
+ * @internal
+ */
+class DiffElem {
+    const TYPE_KEEP = 0;
+    const TYPE_REMOVE = 1;
+    const TYPE_ADD = 2;
+    const TYPE_REPLACE = 3;
+
+    /** @var int One of the TYPE_* constants */
+    public $type;
+    /** @var mixed Is null for add operations */
+    public $old;
+    /** @var mixed Is null for remove operations */
+    public $new;
+
+    public function __construct(int $type, $old, $new) {
+        $this->type = $type;
+        $this->old = $old;
+        $this->new = $new;
+    }
+}

lib/PhpParser/Internal/Differ.php

@@ -0,0 +1,163 @@
+<?php
+
+namespace PhpParser\Internal;
+
+/**
+ * Implements the Myers diff algorithm.
+ *
+ * Myers, Eugene W. "An O (ND) difference algorithm and its variations."
+ * Algorithmica 1.1 (1986): 251-266.
+ *
+ * @internal
+ */
+class Differ {
+    private $isEqual;
+
+    /**
+     * Create differ over the given equality relation.
+     *
+     * @param callable $isEqual Equality relation with signature function($a, $b): bool
+     */
+    public function __construct(callable $isEqual) {
+        $this->isEqual = $isEqual;
+    }
+
+    /**
+     * Calculate diff (edit script) from $old to $new.
+     *
+     * @param array $old Original array
+     * @param array $new New array
+     *
+     * @return DiffElem[] Diff (edit script)
+     */
+    public function diff(array $old, array $new) {
+        list($trace, $x, $y) = $this->calculateTrace($old, $new);
+        return $this->extractDiff($trace, $x, $y, $old, $new);
+    }
+
+    /**
+     * Calculate diff, including "replace" operations.
+     *
+     * If a sequence of remove operations is followed by the same number of add operations, these
+     * will be coalesced into replace operations.
+     *
+     * @param array $old Original array
+     * @param array $new New array
+     *
+     * @return DiffElem[] Diff (edit script), including replace operations
+     */
+    public function diffWithReplacements(array $old, array $new) {
+        return $this->coalesceReplacements($this->diff($old, $new));
+    }
+
+    private function calculateTrace(array $a, array $b) {
+        $n = \count($a);
+        $m = \count($b);
+        $max = $n + $m;
+        $v = [1 => 0];
+        $trace = [];
+        for ($d = 0; $d <= $max; $d++) {
+            $trace[] = $v;
+            for ($k = -$d; $k <= $d; $k += 2) {
+                if ($k === -$d || ($k !== $d && $v[$k-1] < $v[$k+1])) {
+                    $x = $v[$k+1];
+                } else {
+                    $x = $v[$k-1] + 1;
+                }
+
+                $y = $x - $k;
+                while ($x < $n && $y < $m && ($this->isEqual)($a[$x], $b[$y])) {
+                    $x++;
+                    $y++;
+                }
+
+                $v[$k] = $x;
+                if ($x >= $n && $y >= $m) {
+                    return [$trace, $x, $y];
+                }
+            }
+        }
+        throw new \Exception('Should not happen');
+    }
+
+    private function extractDiff(array $trace, int $x, int $y, array $a, array $b) {
+        $result = [];
+        for ($d = \count($trace) - 1; $d >= 0; $d--) {
+            $v = $trace[$d];
+            $k = $x - $y;
+
+            if ($k === -$d || ($k !== $d && $v[$k-1] < $v[$k+1])) {
+                $prevK = $k + 1;
+            } else {
+                $prevK = $k - 1;
+            }
+
+            $prevX = $v[$prevK];
+            $prevY = $prevX - $prevK;
+
+            while ($x > $prevX && $y > $prevY) {
+                $result[] = new DiffElem(DiffElem::TYPE_KEEP, $a[$x-1], $b[$y-1]);
+                $x--;
+                $y--;
+            }
+
+            if ($d === 0) {
+                break;
+            }
+
+            while ($x > $prevX) {
+                $result[] = new DiffElem(DiffElem::TYPE_REMOVE, $a[$x-1], null);
+                $x--;
+            }
+
+            while ($y > $prevY) {
+                $result[] = new DiffElem(DiffElem::TYPE_ADD, null, $b[$y-1]);
+                $y--;
+            }
+        }
+        return array_reverse($result);
+    }
+
+    /**
+     * Coalesce equal-length sequences of remove+add into a replace operation.
+     *
+     * @param DiffElem[] $diff
+     * @return DiffElem[]
+     */
+    private function coalesceReplacements(array $diff) {
+        $newDiff = [];
+        $c = \count($diff);
+        for ($i = 0; $i < $c; $i++) {
+            $diffType = $diff[$i]->type;
+            if ($diffType !== DiffElem::TYPE_REMOVE) {
+                $newDiff[] = $diff[$i];
+                continue;
+            }
+
+            $j = $i;
+            while ($j < $c && $diff[$j]->type === DiffElem::TYPE_REMOVE) {
+                $j++;
+            }
+
+            $k = $j;
+            while ($k < $c && $diff[$k]->type === DiffElem::TYPE_ADD) {
+                $k++;
+            }
+
+            if ($j - $i === $k - $j) {
+                $len = $j - $i;
+                for ($n = 0; $n < $len; $n++) {
+                    $newDiff[] = new DiffElem(
+                        DiffElem::TYPE_REPLACE, $diff[$i + $n]->old, $diff[$j + $n]->new
+                    );
+                }
+            } else {
+                for (; $i < $k; $i++) {
+                    $newDiff[] = $diff[$i];
+                }
+            }
+            $i = $k - 1;
+        }
+        return $newDiff;
+    }
+}

lib/PhpParser/JsonDecoder.php

@@ -0,0 +1,98 @@
+<?php declare(strict_types=1);
+
+namespace PhpParser;
+
+class JsonDecoder {
+    /** @var \ReflectionClass[] Node type to reflection class map */
+    private $reflectionClassCache;
+
+    public function decode(string $json) {
+        $value = json_decode($json, true);
+        if (json_last_error()) {
+            throw new \RuntimeException('JSON decoding error: ' . json_last_error_msg());
+        }
+
+        return $this->decodeRecursive($value);
+    }
+
+    private function decodeRecursive($value) {
+        if (\is_array($value)) {
+            if (isset($value['nodeType'])) {
+                if ($value['nodeType'] === 'Comment' || $value['nodeType'] === 'Comment_Doc') {
+                    return $this->decodeComment($value);
+                }
+                return $this->decodeNode($value);
+            }
+            return $this->decodeArray($value);
+        }
+        return $value;
+    }
+
+    private function decodeArray(array $array) : array {
+        $decodedArray = [];
+        foreach ($array as $key => $value) {
+            $decodedArray[$key] = $this->decodeRecursive($value);
+        }
+        return $decodedArray;
+    }
+
+    private function decodeNode(array $value) : Node {
+        $nodeType = $value['nodeType'];
+        if (!\is_string($nodeType)) {
+            throw new \RuntimeException('Node type must be a string');
+        }
+
+        $reflectionClass = $this->reflectionClassFromNodeType($nodeType);
+        /** @var Node $node */
+        $node = $reflectionClass->newInstanceWithoutConstructor();
+
+        if (isset($value['attributes'])) {
+            if (!\is_array($value['attributes'])) {
+                throw new \RuntimeException('Attributes must be an array');
+            }
+
+            $node->setAttributes($this->decodeArray($value['attributes']));
+        }
+
+        foreach ($value as $name => $subNode) {
+            if ($name === 'nodeType' || $name === 'attributes') {
+                continue;
+            }
+
+            $node->$name = $this->decodeRecursive($subNode);
+        }
+
+        return $node;
+    }
+
+    private function decodeComment(array $value) : Comment {
+        $className = $value['nodeType'] === 'Comment' ? Comment::class : Comment\Doc::class;
+        if (!isset($value['text'])) {
+            throw new \RuntimeException('Comment must have text');
+        }
+
+        return new $className($value['text'], $value['line'] ?? -1, $value['filePos'] ?? -1);
+    }
+
+    private function reflectionClassFromNodeType(string $nodeType) : \ReflectionClass {
+        if (!isset($this->reflectionClassCache[$nodeType])) {
+            $className = $this->classNameFromNodeType($nodeType);
+            $this->reflectionClassCache[$nodeType] = new \ReflectionClass($className);
+        }
+        return $this->reflectionClassCache[$nodeType];
+    }
+
+    private function classNameFromNodeType(string $nodeType) : string {
+        $className = 'PhpParser\\Node\\' . strtr($nodeType, '_', '\\');
+        if (class_exists($className)) {
+            return $className;
+        }
+
+        $className .= '_';
+        if (class_exists($className)) {
+            return $className;
+        }
+
+        throw new \RuntimeException("Unknown node type \"$nodeType\"");
+    }
+}

lib/PhpParser/Lexer.php

@@ -1,4 +1,4 @@
-<?php
+<?php declare(strict_types=1);
 
 namespace PhpParser;
 
@@ -11,6 +11,7 @@ class Lexer
     protected $pos;
     protected $line;
     protected $filePos;
+    protected $prevCloseTagHasNewline;
 
     protected $tokenMap;
     protected $dropTokens;
@@ -21,88 +22,177 @@ class Lexer
      * Creates a Lexer.
      *
      * @param array $options Options array. Currently only the 'usedAttributes' option is supported,
-     *                       which is an array of attributes to add to the AST nodes. Possible attributes
-     *                       are: 'comments', 'startLine', 'endLine', 'startTokenPos', 'endTokenPos',
-     *                       'startFilePos', 'endFilePos'. The option defaults to the first three.
-     *                       For more info see getNextToken() docs.
+     *                       which is an array of attributes to add to the AST nodes. Possible
+     *                       attributes are: 'comments', 'startLine', 'endLine', 'startTokenPos',
+     *                       'endTokenPos', 'startFilePos', 'endFilePos'. The option defaults to the
+     *                       first three. For more info see getNextToken() docs.
      */
-    public function __construct(array $options = array()) {
+    public function __construct(array $options = []) {
         // map from internal tokens to PhpParser tokens
         $this->tokenMap = $this->createTokenMap();
 
         // map of tokens to drop while lexing (the map is only used for isset lookup,
         // that's why the value is simply set to 1; the value is never actually used.)
-        $this->dropTokens = array_fill_keys(array(T_WHITESPACE, T_OPEN_TAG), 1);
+        $this->dropTokens = array_fill_keys(
+            [T_WHITESPACE, T_OPEN_TAG, T_COMMENT, T_DOC_COMMENT], 1
+        );
 
         // the usedAttributes member is a map of the used attribute names to a dummy
         // value (here "true")
-        $options += array(
-            'usedAttributes' => array('comments', 'startLine', 'endLine'),
-        );
+        $options += [
+            'usedAttributes' => ['comments', 'startLine', 'endLine'],
+        ];
         $this->usedAttributes = array_fill_keys($options['usedAttributes'], true);
     }
 
     /**
      * Initializes the lexer for lexing the provided source code.
      *
-     * @param string $code The source code to lex
+     * This function does not throw if lexing errors occur. Instead, errors may be retrieved using
+     * the getErrors() method.
      *
-     * @throws Error on lexing errors (unterminated comment or unexpected character)
+     * @param string $code The source code to lex
+     * @param ErrorHandler|null $errorHandler Error handler to use for lexing errors. Defaults to
+     *                                        ErrorHandler\Throwing
      */
-    public function startLexing($code) {
-        $scream = ini_set('xdebug.scream', '0');
-
-        $this->resetErrors();
-        $this->tokens = @token_get_all($code);
-        $this->handleErrors();
-
-        if (false !== $scream) {
-            ini_set('xdebug.scream', $scream);
+    public function startLexing(string $code, ErrorHandler $errorHandler = null) {
+        if (null === $errorHandler) {
+            $errorHandler = new ErrorHandler\Throwing();
         }
 
         $this->code = $code; // keep the code around for __halt_compiler() handling
         $this->pos  = -1;
         $this->line =  1;
         $this->filePos = 0;
-    }
 
-    protected function resetErrors() {
-        if (function_exists('error_clear_last')) {
-            error_clear_last();
-        } else {
-            // set error_get_last() to defined state by forcing an undefined variable error
-            set_error_handler(function() { return false; }, 0);
-            @$undefinedVariable;
-            restore_error_handler();
+        // If inline HTML occurs without preceding code, treat it as if it had a leading newline.
+        // This ensures proper composability, because having a newline is the "safe" assumption.
+        $this->prevCloseTagHasNewline = true;
+
+        $scream = ini_set('xdebug.scream', '0');
+
+        error_clear_last();
+        $this->tokens = @token_get_all($code);
+        $this->handleErrors($errorHandler);
+
+        if (false !== $scream) {
+            ini_set('xdebug.scream', $scream);
         }
     }
 
-    protected function handleErrors() {
-        $error = error_get_last();
-        if (null === $error) {
+    private function handleInvalidCharacterRange($start, $end, $line, ErrorHandler $errorHandler) {
+        for ($i = $start; $i < $end; $i++) {
+            $chr = $this->code[$i];
+            if ($chr === 'b' || $chr === 'B') {
+                // HHVM does not treat b" tokens correctly, so ignore these
+                continue;
+            }
+
+            if ($chr === "\0") {
+                // PHP cuts error message after null byte, so need special case
+                $errorMsg = 'Unexpected null byte';
+            } else {
+                $errorMsg = sprintf(
+                    'Unexpected character "%s" (ASCII %d)', $chr, ord($chr)
+                );
+            }
+
+            $errorHandler->handleError(new Error($errorMsg, [
+                'startLine' => $line,
+                'endLine' => $line,
+                'startFilePos' => $i,
+                'endFilePos' => $i,
+            ]));
+        }
+    }
+
+    /**
+     * Check whether comment token is unterminated.
+     *
+     * @return bool
+     */
+    private function isUnterminatedComment($token) : bool {
+        return ($token[0] === T_COMMENT || $token[0] === T_DOC_COMMENT)
+            && substr($token[1], 0, 2) === '/*'
+            && substr($token[1], -2) !== '*/';
+    }
+
+    /**
+     * Check whether an error *may* have occurred during tokenization.
+     *
+     * @return bool
+     */
+    private function errorMayHaveOccurred() : bool {
+        if (defined('HHVM_VERSION')) {
+            // In HHVM token_get_all() does not throw warnings, so we need to conservatively
+            // assume that an error occurred
+            return true;
+        }
+
+        return null !== error_get_last();
+    }
+
+    protected function handleErrors(ErrorHandler $errorHandler) {
+        if (!$this->errorMayHaveOccurred()) {
             return;
         }
 
-        if (preg_match(
-            '~^Unterminated comment starting line ([0-9]+)$~',
-            $error['message'], $matches
-        )) {
-            throw new Error('Unterminated comment', (int) $matches[1]);
+        // PHP's error handling for token_get_all() is rather bad, so if we want detailed
+        // error information we need to compute it ourselves. Invalid character errors are
+        // detected by finding "gaps" in the token array. Unterminated comments are detected
+        // by checking if a trailing comment has a "*/" at the end.
+
+        $filePos = 0;
+        $line = 1;
+        foreach ($this->tokens as $token) {
+            $tokenValue = \is_string($token) ? $token : $token[1];
+            $tokenLen = \strlen($tokenValue);
+
+            if (substr($this->code, $filePos, $tokenLen) !== $tokenValue) {
+                // Something is missing, must be an invalid character
+                $nextFilePos = strpos($this->code, $tokenValue, $filePos);
+                $this->handleInvalidCharacterRange(
+                    $filePos, $nextFilePos, $line, $errorHandler);
+                $filePos = (int)$nextFilePos;
+            }
+
+            $filePos += $tokenLen;
+            $line += substr_count($tokenValue, "\n");
         }
 
-        if (preg_match(
-            '~^Unexpected character in input:  \'(.)\' \(ASCII=([0-9]+)\)~s',
-            $error['message'], $matches
-        )) {
-            throw new Error(sprintf(
-                'Unexpected character "%s" (ASCII %d)',
-                $matches[1], $matches[2]
-            ));
+        if ($filePos !== \strlen($this->code)) {
+            if (substr($this->code, $filePos, 2) === '/*') {
+                // Unlike PHP, HHVM will drop unterminated comments entirely
+                $comment = substr($this->code, $filePos);
+                $errorHandler->handleError(new Error('Unterminated comment', [
+                    'startLine' => $line,
+                    'endLine' => $line + substr_count($comment, "\n"),
+                    'startFilePos' => $filePos,
+                    'endFilePos' => $filePos + \strlen($comment),
+                ]));
+
+                // Emulate the PHP behavior
+                $isDocComment = isset($comment[3]) && $comment[3] === '*';
+                $this->tokens[] = [$isDocComment ? T_DOC_COMMENT : T_COMMENT, $comment, $line];
+            } else {
+                // Invalid characters at the end of the input
+                $this->handleInvalidCharacterRange(
+                    $filePos, \strlen($this->code), $line, $errorHandler);
+            }
+            return;
         }
 
-        // PHP cuts error message after null byte, so need special case
-        if (preg_match('~^Unexpected character in input:  \'$~', $error['message'])) {
-            throw new Error('Unexpected null byte');
+        if (count($this->tokens) > 0) {
+            // Check for unterminated comment
+            $lastToken = $this->tokens[count($this->tokens) - 1];
+            if ($this->isUnterminatedComment($lastToken)) {
+                $errorHandler->handleError(new Error('Unterminated comment', [
+                    'startLine' => $line - substr_count($lastToken[1], "\n"),
+                    'endLine' => $line,
+                    'startFilePos' => $filePos - \strlen($lastToken[1]),
+                    'endFilePos' => $filePos,
+                ]));
+            }
         }
     }
 
@@ -120,7 +210,7 @@ class Lexer
      *  * 'startTokenPos' => Offset into the token array of the first token in the node.
      *  * 'endTokenPos'   => Offset into the token array of the last token in the node.
      *  * 'startFilePos'  => Offset into the code string of the first character that is part of the node.
-     *  * 'endFilePos'    => Offset into the code string of the last character that is part of the node
+     *  * 'endFilePos'    => Offset into the code string of the last character that is part of the node.
      *
      * @param mixed $value           Variable to store token content in
      * @param mixed $startAttributes Variable to store start attributes in
@@ -128,9 +218,9 @@ class Lexer
      *
      * @return int Token id
      */
-    public function getNextToken(&$value = null, &$startAttributes = null, &$endAttributes = null) {
-        $startAttributes = array();
-        $endAttributes   = array();
+    public function getNextToken(&$value = null, &$startAttributes = null, &$endAttributes = null) : int {
+        $startAttributes = [];
+        $endAttributes   = [];
 
         while (1) {
             if (isset($this->tokens[++$this->pos])) {
@@ -140,6 +230,9 @@ class Lexer
                 $token = "\0";
             }
 
+            if (isset($this->usedAttributes['startLine'])) {
+                $startAttributes['startLine'] = $this->line;
+            }
             if (isset($this->usedAttributes['startTokenPos'])) {
                 $startAttributes['startTokenPos'] = $this->pos;
             }
@@ -147,63 +240,53 @@ class Lexer
                 $startAttributes['startFilePos'] = $this->filePos;
             }
 
-            if (is_string($token)) {
-                // bug in token_get_all
-                if ('b"' === $token) {
-                    $value = 'b"';
+            if (\is_string($token)) {
+                $value = $token;
+                if (isset($token[1])) {
+                    // bug in token_get_all
                     $this->filePos += 2;
                     $id = ord('"');
                 } else {
-                    $value = $token;
                     $this->filePos += 1;
                     $id = ord($token);
                 }
-
-                if (isset($this->usedAttributes['startLine'])) {
-                    $startAttributes['startLine'] = $this->line;
-                }
-                if (isset($this->usedAttributes['endLine'])) {
-                    $endAttributes['endLine'] = $this->line;
-                }
-                if (isset($this->usedAttributes['endTokenPos'])) {
-                    $endAttributes['endTokenPos'] = $this->pos;
-                }
-                if (isset($this->usedAttributes['endFilePos'])) {
-                    $endAttributes['endFilePos'] = $this->filePos - 1;
+            } elseif (!isset($this->dropTokens[$token[0]])) {
+                $value = $token[1];
+                $id = $this->tokenMap[$token[0]];
+                if (T_CLOSE_TAG === $token[0]) {
+                    $this->prevCloseTagHasNewline = false !== strpos($token[1], "\n");
+                } else if (T_INLINE_HTML === $token[0]) {
+                    $startAttributes['hasLeadingNewline'] = $this->prevCloseTagHasNewline;
                 }
 
-                return $id;
+                $this->line += substr_count($value, "\n");
+                $this->filePos += \strlen($value);
             } else {
-                $this->line += substr_count($token[1], "\n");
-                $this->filePos += strlen($token[1]);
-
-                if (T_COMMENT === $token[0]) {
+                if (T_COMMENT === $token[0] || T_DOC_COMMENT === $token[0]) {
                     if (isset($this->usedAttributes['comments'])) {
-                        $startAttributes['comments'][] = new Comment($token[1], $token[2]);
+                        $comment = T_DOC_COMMENT === $token[0]
+                            ? new Comment\Doc($token[1], $this->line, $this->filePos)
+                            : new Comment($token[1], $this->line, $this->filePos);
+                        $startAttributes['comments'][] = $comment;
                     }
-                } elseif (T_DOC_COMMENT === $token[0]) {
-                    if (isset($this->usedAttributes['comments'])) {
-                        $startAttributes['comments'][] = new Comment\Doc($token[1], $token[2]);
-                    }
-                } elseif (!isset($this->dropTokens[$token[0]])) {
-                    $value = $token[1];
-
-                    if (isset($this->usedAttributes['startLine'])) {
-                        $startAttributes['startLine'] = $token[2];
-                    }
-                    if (isset($this->usedAttributes['endLine'])) {
-                        $endAttributes['endLine'] = $this->line;
-                    }
-                    if (isset($this->usedAttributes['endTokenPos'])) {
-                        $endAttributes['endTokenPos'] = $this->pos;
-                    }
-                    if (isset($this->usedAttributes['endFilePos'])) {
-                        $endAttributes['endFilePos'] = $this->filePos - 1;
-                    }
-
-                    return $this->tokenMap[$token[0]];
                 }
+
+                $this->line += substr_count($token[1], "\n");
+                $this->filePos += \strlen($token[1]);
+                continue;
             }
+
+            if (isset($this->usedAttributes['endLine'])) {
+                $endAttributes['endLine'] = $this->line;
+            }
+            if (isset($this->usedAttributes['endTokenPos'])) {
+                $endAttributes['endTokenPos'] = $this->pos;
+            }
+            if (isset($this->usedAttributes['endFilePos'])) {
+                $endAttributes['endFilePos'] = $this->filePos - 1;
+            }
+
+            return $id;
         }
 
         throw new \RuntimeException('Reached end of lexer loop');
@@ -219,7 +302,7 @@ class Lexer
      *
      * @return array Array of tokens in token_get_all() format
      */
-    public function getTokens() {
+    public function getTokens() : array {
         return $this->tokens;
     }
 
@@ -228,7 +311,7 @@ class Lexer
      *
      * @return string Remaining text
      */
-    public function handleHaltCompiler() {
+    public function handleHaltCompiler() : string {
         // text after T_HALT_COMPILER, still including ();
         $textAfter = substr($this->code, $this->filePos);
 
@@ -243,7 +326,7 @@ class Lexer
         $this->pos = count($this->tokens);
 
         // return with (); removed
-        return (string) substr($textAfter, strlen($matches[0])); // (string) converts false to ''
+        return substr($textAfter, strlen($matches[0]));
     }
 
     /**
@@ -255,8 +338,8 @@ class Lexer
      *
      * @return array The token map
      */
-    protected function createTokenMap() {
-        $tokenMap = array();
+    protected function createTokenMap() : array {
+        $tokenMap = [];
 
         // 256 is the minimum possible token number, as everything below
         // it is an ASCII value

lib/PhpParser/Lexer/Emulative.php

@@ -1,205 +1,9 @@
-<?php
+<?php declare(strict_types=1);
 
 namespace PhpParser\Lexer;
 
-use PhpParser\Parser\Tokens;
 
-/**
- * ATTENTION: This code is WRITE-ONLY. Do not try to read it.
- */
 class Emulative extends \PhpParser\Lexer
 {
-    protected $newKeywords;
-    protected $inObjectAccess;
-
-    const T_ELLIPSIS   = 1001;
-    const T_POW        = 1002;
-    const T_POW_EQUAL  = 1003;
-    const T_COALESCE   = 1004;
-    const T_SPACESHIP  = 1005;
-    const T_YIELD_FROM = 1006;
-
-    const PHP_7_0 = '7.0.0dev';
-    const PHP_5_6 = '5.6.0rc1';
-    const PHP_5_5 = '5.5.0beta1';
-
-    public function __construct(array $options = array()) {
-        parent::__construct($options);
-
-        $newKeywordsPerVersion = array(
-            self::PHP_5_5 => array(
-                'finally'       => Tokens::T_FINALLY,
-                'yield'         => Tokens::T_YIELD,
-            ),
-        );
-
-        $this->newKeywords = array();
-        foreach ($newKeywordsPerVersion as $version => $newKeywords) {
-            if (version_compare(PHP_VERSION, $version, '>=')) {
-                break;
-            }
-
-            $this->newKeywords += $newKeywords;
-        }
-
-        if (version_compare(PHP_VERSION, self::PHP_7_0, '>=')) {
-            return;
-        }
-        $this->tokenMap[self::T_COALESCE]   = Tokens::T_COALESCE;
-        $this->tokenMap[self::T_SPACESHIP]  = Tokens::T_SPACESHIP;
-        $this->tokenMap[self::T_YIELD_FROM] = Tokens::T_YIELD_FROM;
-
-        if (version_compare(PHP_VERSION, self::PHP_5_6, '>=')) {
-            return;
-        }
-        $this->tokenMap[self::T_ELLIPSIS]  = Tokens::T_ELLIPSIS;
-        $this->tokenMap[self::T_POW]       = Tokens::T_POW;
-        $this->tokenMap[self::T_POW_EQUAL] = Tokens::T_POW_EQUAL;
-    }
-
-    public function startLexing($code) {
-        $this->inObjectAccess = false;
-
-        $preprocessedCode = $this->preprocessCode($code);
-        parent::startLexing($preprocessedCode);
-        if ($preprocessedCode !== $code) {
-            $this->postprocessTokens();
-        }
-
-        // Set code property back to the original code, so __halt_compiler()
-        // handling and (start|end)FilePos attributes use the correct offsets
-        $this->code = $code;
-    }
-
-    /*
-     * Replaces new features in the code by ~__EMU__{NAME}__{DATA}__~ sequences.
-     * ~LABEL~ is never valid PHP code, that's why we can (to some degree) safely
-     * use it here.
-     * Later when preprocessing the tokens these sequences will either be replaced
-     * by real tokens or replaced with their original content (e.g. if they occurred
-     * inside a string, i.e. a place where they don't have a special meaning).
-     */
-    protected function preprocessCode($code) {
-        if (version_compare(PHP_VERSION, self::PHP_7_0, '>=')) {
-            return $code;
-        }
-
-        $code = str_replace('??', '~__EMU__COALESCE__~', $code);
-        $code = str_replace('<=>', '~__EMU__SPACESHIP__~', $code);
-        $code = preg_replace_callback('(yield[ \n\r\t]+from)', function($matches) {
-            // Encoding $0 in order to preserve exact whitespace
-            return '~__EMU__YIELDFROM__' . bin2hex($matches[0]) . '__~';
-        }, $code);
-
-        if (version_compare(PHP_VERSION, self::PHP_5_6, '>=')) {
-            return $code;
-        }
-
-        $code = str_replace('...', '~__EMU__ELLIPSIS__~', $code);
-        $code = preg_replace('((?<!/)\*\*=)', '~__EMU__POWEQUAL__~', $code);
-        $code = preg_replace('((?<!/)\*\*(?!/))', '~__EMU__POW__~', $code);
-
-        return $code;
-    }
-
-    /*
-     * Replaces the ~__EMU__...~ sequences with real tokens or their original
-     * value.
-     */
-    protected function postprocessTokens() {
-        // we need to manually iterate and manage a count because we'll change
-        // the tokens array on the way
-        for ($i = 0, $c = count($this->tokens); $i < $c; ++$i) {
-            // first check that the following tokens are of form ~LABEL~,
-            // then match the __EMU__... sequence.
-            if ('~' === $this->tokens[$i]
-                && isset($this->tokens[$i + 2])
-                && '~' === $this->tokens[$i + 2]
-                && T_STRING === $this->tokens[$i + 1][0]
-                && preg_match('(^__EMU__([A-Z]++)__(?:([A-Za-z0-9]++)__)?$)', $this->tokens[$i + 1][1], $matches)
-            ) {
-                if ('ELLIPSIS' === $matches[1]) {
-                    $replace = array(
-                        array(self::T_ELLIPSIS, '...', $this->tokens[$i + 1][2])
-                    );
-                } else if ('POW' === $matches[1]) {
-                    $replace = array(
-                        array(self::T_POW, '**', $this->tokens[$i + 1][2])
-                    );
-                } else if ('POWEQUAL' === $matches[1]) {
-                    $replace = array(
-                        array(self::T_POW_EQUAL, '**=', $this->tokens[$i + 1][2])
-                    );
-                } else if ('COALESCE' === $matches[1]) {
-                    $replace = array(
-                        array(self::T_COALESCE, '??', $this->tokens[$i + 1][2])
-                    );
-                } else if ('SPACESHIP' === $matches[1]) {
-                    $replace = array(
-                        array(self::T_SPACESHIP, '<=>', $this->tokens[$i + 1][2]),
-                    );
-                } else if ('YIELDFROM' === $matches[1]) {
-                    $content = hex2bin($matches[2]);
-                    $replace = array(
-                        array(self::T_YIELD_FROM, $content, $this->tokens[$i + 1][2] - substr_count($content, "\n"))
-                    );
-                } else {
-                    throw new \RuntimeException('Invalid __EMU__ sequence');
-                }
-
-                array_splice($this->tokens, $i, 3, $replace);
-                $c -= 3 - count($replace);
-            // for multichar tokens (e.g. strings) replace any ~__EMU__...~ sequences
-            // in their content with the original character sequence
-            } elseif (is_array($this->tokens[$i])
-                      && 0 !== strpos($this->tokens[$i][1], '__EMU__')
-            ) {
-                $this->tokens[$i][1] = preg_replace_callback(
-                    '(~__EMU__([A-Z]++)__(?:([A-Za-z0-9]++)__)?~)',
-                    array($this, 'restoreContentCallback'),
-                    $this->tokens[$i][1]
-                );
-            }
-        }
-    }
-
-    /*
-     * This method is a callback for restoring EMU sequences in
-     * multichar tokens (like strings) to their original value.
-     */
-    public function restoreContentCallback(array $matches) {
-        if ('ELLIPSIS' === $matches[1]) {
-            return '...';
-        } else if ('POW' === $matches[1]) {
-            return '**';
-        } else if ('POWEQUAL' === $matches[1]) {
-            return '**=';
-        } else if ('COALESCE' === $matches[1]) {
-            return '??';
-        } else if ('SPACESHIP' === $matches[1]) {
-            return '<=>';
-        } else if ('YIELDFROM' === $matches[1]) {
-            return hex2bin($matches[2]);
-        } else {
-            return $matches[0];
-        }
-    }
-
-    public function getNextToken(&$value = null, &$startAttributes = null, &$endAttributes = null) {
-        $token = parent::getNextToken($value, $startAttributes, $endAttributes);
-
-        // replace new keywords by their respective tokens. This is not done
-        // if we currently are in an object access (e.g. in $obj->namespace
-        // "namespace" stays a T_STRING tokens and isn't converted to T_NAMESPACE)
-        if (Tokens::T_STRING === $token && !$this->inObjectAccess) {
-            if (isset($this->newKeywords[strtolower($value)])) {
-                return $this->newKeywords[strtolower($value)];
-            }
-        } else {
-            // keep track of whether we currently are in an object access (after ->)
-            $this->inObjectAccess = Tokens::T_OBJECT_OPERATOR === $token;
-        }
-
-        return $token;
-    }
-}
+    /* No features requiring emulation have been added in PHP > 7.0 */
+}

lib/PhpParser/NameContext.php

@@ -0,0 +1,284 @@
+<?php declare(strict_types=1);
+
+namespace PhpParser;
+
+use PhpParser\Node\Name;
+use PhpParser\Node\Name\FullyQualified;
+use PhpParser\Node\Stmt;
+
+class NameContext {
+    /** @var null|Name Current namespace */
+    protected $namespace;
+
+    /** @var Name[][] Map of format [aliasType => [aliasName => originalName]] */
+    protected $aliases = [];
+
+    /** @var Name[][] Same as $aliases but preserving original case */
+    protected $origAliases = [];
+
+    /** @var ErrorHandler Error handler */
+    protected $errorHandler;
+
+    /**
+     * Create a name context.
+     *
+     * @param ErrorHandler $errorHandler Error handling used to report errors
+     */
+    public function __construct(ErrorHandler $errorHandler) {
+        $this->errorHandler = $errorHandler;
+    }
+
+    /**
+     * Start a new namespace.
+     *
+     * This also resets the alias table.
+     *
+     * @param Name|null $namespace Null is the global namespace
+     */
+    public function startNamespace(Name $namespace = null) {
+        $this->namespace = $namespace;
+        $this->origAliases = $this->aliases = [
+            Stmt\Use_::TYPE_NORMAL   => [],
+            Stmt\Use_::TYPE_FUNCTION => [],
+            Stmt\Use_::TYPE_CONSTANT => [],
+        ];
+    }
+
+    /**
+     * Add an alias / import.
+     *
+     * @param Name   $name        Original name
+     * @param string $aliasName   Aliased name
+     * @param int    $type        One of Stmt\Use_::TYPE_*
+     * @param array  $errorAttrs Attributes to use to report an error
+     */
+    public function addAlias(Name $name, string $aliasName, int $type, array $errorAttrs = []) {
+        // Constant names are case sensitive, everything else case insensitive
+        if ($type === Stmt\Use_::TYPE_CONSTANT) {
+            $aliasLookupName = $aliasName;
+        } else {
+            $aliasLookupName = strtolower($aliasName);
+        }
+
+        if (isset($this->aliases[$type][$aliasLookupName])) {
+            $typeStringMap = [
+                Stmt\Use_::TYPE_NORMAL   => '',
+                Stmt\Use_::TYPE_FUNCTION => 'function ',
+                Stmt\Use_::TYPE_CONSTANT => 'const ',
+            ];
+
+            $this->errorHandler->handleError(new Error(
+                sprintf(
+                    'Cannot use %s%s as %s because the name is already in use',
+                    $typeStringMap[$type], $name, $aliasName
+                ),
+                $errorAttrs
+            ));
+            return;
+        }
+
+        $this->aliases[$type][$aliasLookupName] = $name;
+        $this->origAliases[$type][$aliasName] = $name;
+    }
+
+    /**
+     * Get current namespace.
+     *
+     * @return null|Name Namespace (or null if global namespace)
+     */
+    public function getNamespace() {
+        return $this->namespace;
+    }
+
+    /**
+     * Get resolved name.
+     *
+     * @param Name $name Name to resolve
+     * @param int  $type One of Stmt\Use_::TYPE_{FUNCTION|CONSTANT}
+     *
+     * @return null|Name Resolved name, or null if static resolution is not possible
+     */
+    public function getResolvedName(Name $name, int $type) {
+        // don't resolve special class names
+        if ($type === Stmt\Use_::TYPE_NORMAL && $name->isSpecialClassName()) {
+            if (!$name->isUnqualified()) {
+                $this->errorHandler->handleError(new Error(
+                    sprintf("'\\%s' is an invalid class name", $name->toString()),
+                    $name->getAttributes()
+                ));
+            }
+            return $name;
+        }
+
+        // fully qualified names are already resolved
+        if ($name->isFullyQualified()) {
+            return $name;
+        }
+
+        // Try to resolve aliases
+        if (null !== $resolvedName = $this->resolveAlias($name, $type)) {
+            return $resolvedName;
+        }
+
+        if ($type !== Stmt\Use_::TYPE_NORMAL && $name->isUnqualified()) {
+            if (null === $this->namespace) {
+                // outside of a namespace unaliased unqualified is same as fully qualified
+                return new FullyQualified($name, $name->getAttributes());
+            }
+
+            // Cannot resolve statically
+            return null;
+        }
+
+        // if no alias exists prepend current namespace
+        return FullyQualified::concat($this->namespace, $name, $name->getAttributes());
+    }
+
+    /**
+     * Get resolved class name.
+     *
+     * @param Name $name Class ame to resolve
+     *
+     * @return Name Resolved name
+     */
+    public function getResolvedClassName(Name $name) : Name {
+        return $this->getResolvedName($name, Stmt\Use_::TYPE_NORMAL);
+    }
+
+    /**
+     * Get possible ways of writing a fully qualified name (e.g., by making use of aliases).
+     *
+     * @param string $name Fully-qualified name (without leading namespace separator)
+     * @param int    $type One of Stmt\Use_::TYPE_*
+     *
+     * @return Name[] Possible representations of the name
+     */
+    public function getPossibleNames(string $name, int $type) : array {
+        $lcName = strtolower($name);
+
+        if ($type === Stmt\Use_::TYPE_NORMAL) {
+            // self, parent and static must always be unqualified
+            if ($lcName === "self" || $lcName === "parent" || $lcName === "static") {
+                return [new Name($name)];
+            }
+        }
+
+        // Collect possible ways to write this name, starting with the fully-qualified name
+        $possibleNames = [new FullyQualified($name)];
+
+        if (null !== $nsRelativeName = $this->getNamespaceRelativeName($name, $lcName, $type)) {
+            // Make sure there is no alias that makes the normally namespace-relative name
+            // into something else
+            if (null === $this->resolveAlias($nsRelativeName, $type)) {
+                $possibleNames[] = $nsRelativeName;
+            }
+        }
+
+        // Check for relevant namespace use statements
+        foreach ($this->origAliases[Stmt\Use_::TYPE_NORMAL] as $alias => $orig) {
+            $lcOrig = $orig->toLowerString();
+            if (0 === strpos($lcName, $lcOrig . '\\')) {
+                $possibleNames[] = new Name($alias . substr($name, strlen($lcOrig)));
+            }
+        }
+
+        // Check for relevant type-specific use statements
+        foreach ($this->origAliases[$type] as $alias => $orig) {
+            if ($type === Stmt\Use_::TYPE_CONSTANT) {
+                // Constants are are complicated-sensitive
+                $normalizedOrig = $this->normalizeConstName($orig->toString());
+                if ($normalizedOrig === $this->normalizeConstName($name)) {
+                    $possibleNames[] = new Name($alias);
+                }
+            } else {
+                // Everything else is case-insensitive
+                if ($orig->toLowerString() === $lcName) {
+                    $possibleNames[] = new Name($alias);
+                }
+            }
+        }
+
+        return $possibleNames;
+    }
+
+    /**
+     * Get shortest representation of this fully-qualified name.
+     *
+     * @param string $name Fully-qualified name (without leading namespace separator)
+     * @param int    $type One of Stmt\Use_::TYPE_*
+     *
+     * @return Name Shortest representation
+     */
+    public function getShortName(string $name, int $type) : Name {
+        $possibleNames = $this->getPossibleNames($name, $type);
+
+        // Find shortest name
+        $shortestName = null;
+        $shortestLength = INF;
+        foreach ($possibleNames as $possibleName) {
+            $length = strlen($possibleName->toCodeString());
+            if ($length < $shortestLength) {
+                $shortestName = $possibleName;
+                $shortestLength = $length;
+            }
+        }
+
+       return $shortestName;
+    }
+
+    private function resolveAlias(Name $name, $type) {
+        $firstPart = $name->getFirst();
+
+        if ($name->isQualified()) {
+            // resolve aliases for qualified names, always against class alias table
+            $checkName = strtolower($firstPart);
+            if (isset($this->aliases[Stmt\Use_::TYPE_NORMAL][$checkName])) {
+                $alias = $this->aliases[Stmt\Use_::TYPE_NORMAL][$checkName];
+                return FullyQualified::concat($alias, $name->slice(1), $name->getAttributes());
+            }
+        } elseif ($name->isUnqualified()) {
+            // constant aliases are case-sensitive, function aliases case-insensitive
+            $checkName = $type === Stmt\Use_::TYPE_CONSTANT ? $firstPart : strtolower($firstPart);
+            if (isset($this->aliases[$type][$checkName])) {
+                // resolve unqualified aliases
+                return new FullyQualified($this->aliases[$type][$checkName], $name->getAttributes());
+            }
+        }
+
+        // No applicable aliases
+        return null;
+    }
+
+    private function getNamespaceRelativeName(string $name, string $lcName, int $type) {
+        if (null === $this->namespace) {
+            return new Name($name);
+        }
+
+        if ($type === Stmt\Use_::TYPE_CONSTANT) {
+            // The constants true/false/null always resolve to the global symbols, even inside a
+            // namespace, so they may be used without qualification
+            if ($lcName === "true" || $lcName === "false" || $lcName === "null") {
+                return new Name($name);
+            }
+        }
+
+        $namespacePrefix = strtolower($this->namespace . '\\');
+        if (0 === strpos($lcName, $namespacePrefix)) {
+            return new Name(substr($name, strlen($namespacePrefix)));
+        }
+
+        return null;
+    }
+
+    private function normalizeConstName(string $name) {
+        $nsSep = strrpos($name, '\\');
+        if (false === $nsSep) {
+            return $name;
+        }
+
+        // Constants have case-insensitive namespace and case-sensitive short-name
+        $ns = substr($name, 0, $nsSep);
+        $shortName = substr($name, $nsSep + 1);
+        return strtolower($ns) . '\\' . $shortName;
+    }
+}

lib/PhpParser/Node.php

@@ -1,4 +1,4 @@
-<?php
+<?php declare(strict_types=1);
 
 namespace PhpParser;
 
@@ -9,28 +9,88 @@ interface Node
      *
      * @return string Type of the node
      */
-    public function getType();
+    public function getType() : string;
 
     /**
      * Gets the names of the sub nodes.
      *
      * @return array Names of sub nodes
      */
-    public function getSubNodeNames();
+    public function getSubNodeNames() : array;
+
+    /**
+     * Gets line the node started in (alias of getStartLine).
+     *
+     * @return int Start line (or -1 if not available)
+     */
+    public function getLine() : int;
 
     /**
      * Gets line the node started in.
      *
-     * @return int Line
+     * Requires the 'startLine' attribute to be enabled in the lexer (enabled by default).
+     *
+     * @return int Start line (or -1 if not available)
      */
-    public function getLine();
+    public function getStartLine() : int;
 
     /**
-     * Sets line the node started in.
+     * Gets the line the node ended in.
      *
-     * @param int $line Line
+     * Requires the 'endLine' attribute to be enabled in the lexer (enabled by default).
+     *
+     * @return int End line (or -1 if not available)
      */
-    public function setLine($line);
+    public function getEndLine() : int;
+
+    /**
+     * Gets the token offset of the first token that is part of this node.
+     *
+     * The offset is an index into the array returned by Lexer::getTokens().
+     *
+     * Requires the 'startTokenPos' attribute to be enabled in the lexer (DISABLED by default).
+     *
+     * @return int Token start position (or -1 if not available)
+     */
+    public function getStartTokenPos() : int;
+
+    /**
+     * Gets the token offset of the last token that is part of this node.
+     *
+     * The offset is an index into the array returned by Lexer::getTokens().
+     *
+     * Requires the 'endTokenPos' attribute to be enabled in the lexer (DISABLED by default).
+     *
+     * @return int Token end position (or -1 if not available)
+     */
+    public function getEndTokenPos() : int;
+
+    /**
+     * Gets the file offset of the first character that is part of this node.
+     *
+     * Requires the 'startFilePos' attribute to be enabled in the lexer (DISABLED by default).
+     *
+     * @return int File start position (or -1 if not available)
+     */
+    public function getStartFilePos() : int;
+
+    /**
+     * Gets the file offset of the last character that is part of this node.
+     *
+     * Requires the 'endFilePos' attribute to be enabled in the lexer (DISABLED by default).
+     *
+     * @return int File end position (or -1 if not available)
+     */
+    public function getEndFilePos() : int;
+
+    /**
+     * Gets all comments directly preceding this node.
+     *
+     * The comments are also available through the "comments" attribute.
+     *
+     * @return Comment[]
+     */
+    public function getComments(): array;
 
     /**
      * Gets the doc comment of the node.
@@ -41,13 +101,22 @@ interface Node
      */
     public function getDocComment();
 
+    /**
+     * Sets the doc comment of the node.
+     *
+     * This will either replace an existing doc comment or add it to the comments array.
+     *
+     * @param Comment\Doc $docComment Doc comment to set
+     */
+    public function setDocComment(Comment\Doc $docComment);
+
     /**
      * Sets an attribute on a node.
      *
      * @param string $key
      * @param mixed  $value
      */
-    public function setAttribute($key, $value);
+    public function setAttribute(string $key, $value);
 
     /**
      * Returns whether an attribute exists.
@@ -56,7 +125,7 @@ interface Node
      *
      * @return bool
      */
-    public function hasAttribute($key);
+    public function hasAttribute(string $key) : bool;
 
     /**
      * Returns the value of an attribute.
@@ -66,12 +135,19 @@ interface Node
      *
      * @return mixed
      */
-    public function &getAttribute($key, $default = null);
+    public function getAttribute(string $key, $default = null);
 
     /**
-     * Returns all attributes for the given node.
+     * Returns all the attributes of this node.
      *
      * @return array
      */
-    public function getAttributes();
-}
+    public function getAttributes() : array;
+
+    /**
+     * Replaces all the attributes of this node.
+     *
+     * @param array $attributes
+     */
+    public function setAttributes(array $attributes);
+}

lib/PhpParser/Node/Arg.php

@@ -1,4 +1,4 @@
-<?php
+<?php declare(strict_types=1);
 
 namespace PhpParser\Node;
 
@@ -21,14 +21,14 @@ class Arg extends NodeAbstract
      * @param bool  $unpack     Whether to unpack the argument
      * @param array $attributes Additional attributes
      */
-    public function __construct(Expr $value, $byRef = false, $unpack = false, array $attributes = array()) {
+    public function __construct(Expr $value, bool $byRef = false, bool $unpack = false, array $attributes = []) {
         parent::__construct($attributes);
         $this->value = $value;
         $this->byRef = $byRef;
         $this->unpack = $unpack;
     }
 
-    public function getSubNodeNames() {
-        return array('value', 'byRef', 'unpack');
+    public function getSubNodeNames() : array {
+        return ['value', 'byRef', 'unpack'];
     }
 }

lib/PhpParser/Node/Const_.php

@@ -1,12 +1,15 @@
-<?php
+<?php declare(strict_types=1);
 
 namespace PhpParser\Node;
 
 use PhpParser\NodeAbstract;
 
+/**
+ * @property Name $namespacedName Namespaced name (for class constants, if using NameResolver)
+ */
 class Const_ extends NodeAbstract
 {
-    /** @var string Name */
+    /** @var Identifier Name */
     public $name;
     /** @var Expr Value */
     public $value;
@@ -14,17 +17,17 @@ class Const_ extends NodeAbstract
     /**
      * Constructs a const node for use in class const and const statements.
      *
-     * @param string  $name       Name
-     * @param Expr    $value      Value
-     * @param array   $attributes Additional attributes
+     * @param string|Identifier $name       Name
+     * @param Expr              $value      Value
+     * @param array             $attributes Additional attributes
      */
-    public function __construct($name, Expr $value, array $attributes = array()) {
+    public function __construct($name, Expr $value, array $attributes = []) {
         parent::__construct($attributes);
-        $this->name = $name;
+        $this->name = \is_string($name) ? new Identifier($name) : $name;
         $this->value = $value;
     }
 
-    public function getSubNodeNames() {
-        return array('name', 'value');
+    public function getSubNodeNames() : array {
+        return ['name', 'value'];
     }
 }

lib/PhpParser/Node/Expr.php

@@ -1,4 +1,4 @@
-<?php
+<?php declare(strict_types=1);
 
 namespace PhpParser\Node;
 

lib/PhpParser/Node/Expr/ArrayDimFetch.php

@@ -1,4 +1,4 @@
-<?php
+<?php declare(strict_types=1);
 
 namespace PhpParser\Node\Expr;
 
@@ -18,13 +18,13 @@ class ArrayDimFetch extends Expr
      * @param null|Expr $dim        Array index / dim
      * @param array     $attributes Additional attributes
      */
-    public function __construct(Expr $var, Expr $dim = null, array $attributes = array()) {
+    public function __construct(Expr $var, Expr $dim = null, array $attributes = []) {
         parent::__construct($attributes);
         $this->var = $var;
         $this->dim = $dim;
     }
 
-    public function getSubnodeNames() {
-        return array('var', 'dim');
+    public function getSubNodeNames() : array {
+        return ['var', 'dim'];
     }
 }

lib/PhpParser/Node/Expr/ArrayItem.php

@@ -1,4 +1,4 @@
-<?php
+<?php declare(strict_types=1);
 
 namespace PhpParser\Node\Expr;
 
@@ -21,14 +21,14 @@ class ArrayItem extends Expr
      * @param bool      $byRef      Whether to assign by reference
      * @param array     $attributes Additional attributes
      */
-    public function __construct(Expr $value, Expr $key = null, $byRef = false, array $attributes = array()) {
+    public function __construct(Expr $value, Expr $key = null, bool $byRef = false, array $attributes = []) {
         parent::__construct($attributes);
         $this->key = $key;
         $this->value = $value;
         $this->byRef = $byRef;
     }
 
-    public function getSubNodeNames() {
-        return array('key', 'value', 'byRef');
+    public function getSubNodeNames() : array {
+        return ['key', 'value', 'byRef'];
     }
 }

lib/PhpParser/Node/Expr/Array_.php

@@ -1,4 +1,4 @@
-<?php
+<?php declare(strict_types=1);
 
 namespace PhpParser\Node\Expr;
 
@@ -6,6 +6,10 @@ use PhpParser\Node\Expr;
 
 class Array_ extends Expr
 {
+    // For use in "kind" attribute
+    const KIND_LONG = 1;  // array() syntax
+    const KIND_SHORT = 2; // [] syntax
+
     /** @var ArrayItem[] Items */
     public $items;
 
@@ -15,12 +19,12 @@ class Array_ extends Expr
      * @param ArrayItem[] $items      Items of the array
      * @param array       $attributes Additional attributes
      */
-    public function __construct(array $items = array(), array $attributes = array()) {
+    public function __construct(array $items = [], array $attributes = []) {
         parent::__construct($attributes);
         $this->items = $items;
     }
 
-    public function getSubNodeNames() {
-        return array('items');
+    public function getSubNodeNames() : array {
+        return ['items'];
     }
 }

lib/PhpParser/Node/Expr/Assign.php

@@ -1,4 +1,4 @@
-<?php
+<?php declare(strict_types=1);
 
 namespace PhpParser\Node\Expr;
 
@@ -18,13 +18,13 @@ class Assign extends Expr
      * @param Expr  $expr       Expression
      * @param array $attributes Additional attributes
      */
-    public function __construct(Expr $var, Expr $expr, array $attributes = array()) {
+    public function __construct(Expr $var, Expr $expr, array $attributes = []) {
         parent::__construct($attributes);
         $this->var = $var;
         $this->expr = $expr;
     }
 
-    public function getSubNodeNames() {
-        return array('var', 'expr');
+    public function getSubNodeNames() : array {
+        return ['var', 'expr'];
     }
 }

lib/PhpParser/Node/Expr/AssignOp.php

@@ -1,13 +1,9 @@
-<?php
+<?php declare(strict_types=1);
 
 namespace PhpParser\Node\Expr;
 
 use PhpParser\Node\Expr;
 
-/**
- * @property Expr $var  Variable
- * @property Expr $expr Expression
- */
 abstract class AssignOp extends Expr
 {
     /** @var Expr Variable */
@@ -22,13 +18,13 @@ abstract class AssignOp extends Expr
      * @param Expr  $expr       Expression
      * @param array $attributes Additional attributes
      */
-    public function __construct(Expr $var, Expr $expr, array $attributes = array()) {
+    public function __construct(Expr $var, Expr $expr, array $attributes = []) {
         parent::__construct($attributes);
         $this->var = $var;
         $this->expr = $expr;
     }
 
-    public function getSubNodeNames() {
-        return array('var', 'expr');
+    public function getSubNodeNames() : array {
+        return ['var', 'expr'];
     }
 }

lib/PhpParser/Node/Expr/AssignOp/BitwiseAnd.php

@@ -1,4 +1,4 @@
-<?php
+<?php declare(strict_types=1);
 
 namespace PhpParser\Node\Expr\AssignOp;
 

lib/PhpParser/Node/Expr/AssignOp/BitwiseOr.php

@@ -1,4 +1,4 @@
-<?php
+<?php declare(strict_types=1);
 
 namespace PhpParser\Node\Expr\AssignOp;
 

lib/PhpParser/Node/Expr/AssignOp/BitwiseXor.php

@@ -1,4 +1,4 @@
-<?php
+<?php declare(strict_types=1);
 
 namespace PhpParser\Node\Expr\AssignOp;
 

lib/PhpParser/Node/Expr/AssignOp/Concat.php

@@ -1,4 +1,4 @@
-<?php
+<?php declare(strict_types=1);
 
 namespace PhpParser\Node\Expr\AssignOp;
 

lib/PhpParser/Node/Expr/AssignOp/Div.php

@@ -1,4 +1,4 @@
-<?php
+<?php declare(strict_types=1);
 
 namespace PhpParser\Node\Expr\AssignOp;
 

lib/PhpParser/Node/Expr/AssignOp/Minus.php

@@ -1,4 +1,4 @@
-<?php
+<?php declare(strict_types=1);
 
 namespace PhpParser\Node\Expr\AssignOp;
 

lib/PhpParser/Node/Expr/AssignOp/Mod.php

@@ -1,4 +1,4 @@
-<?php
+<?php declare(strict_types=1);
 
 namespace PhpParser\Node\Expr\AssignOp;
 

lib/PhpParser/Node/Expr/AssignOp/Mul.php

@@ -1,4 +1,4 @@
-<?php
+<?php declare(strict_types=1);
 
 namespace PhpParser\Node\Expr\AssignOp;
 

lib/PhpParser/Node/Expr/AssignOp/Plus.php

@@ -1,4 +1,4 @@
-<?php
+<?php declare(strict_types=1);
 
 namespace PhpParser\Node\Expr\AssignOp;
 

lib/PhpParser/Node/Expr/AssignOp/Pow.php

@@ -1,4 +1,4 @@
-<?php
+<?php declare(strict_types=1);
 
 namespace PhpParser\Node\Expr\AssignOp;
 

lib/PhpParser/Node/Expr/AssignOp/ShiftLeft.php

@@ -1,4 +1,4 @@
-<?php
+<?php declare(strict_types=1);
 
 namespace PhpParser\Node\Expr\AssignOp;
 

lib/PhpParser/Node/Expr/AssignOp/ShiftRight.php

@@ -1,4 +1,4 @@
-<?php
+<?php declare(strict_types=1);
 
 namespace PhpParser\Node\Expr\AssignOp;
 

lib/PhpParser/Node/Expr/AssignRef.php

@@ -1,13 +1,9 @@
-<?php
+<?php declare(strict_types=1);
 
 namespace PhpParser\Node\Expr;
 
 use PhpParser\Node\Expr;
 
-/**
- * @property Expr $var  Variable reference is assigned to
- * @property Expr $expr Variable which is referenced
- */
 class AssignRef extends Expr
 {
     /** @var Expr Variable reference is assigned to */
@@ -22,13 +18,13 @@ class AssignRef extends Expr
      * @param Expr  $expr       Expression
      * @param array $attributes Additional attributes
      */
-    public function __construct(Expr $var, Expr $expr, array $attributes = array()) {
+    public function __construct(Expr $var, Expr $expr, array $attributes = []) {
         parent::__construct($attributes);
         $this->var = $var;
         $this->expr = $expr;
     }
 
-    public function getSubNodeNames() {
-        return array('var', 'expr');
+    public function getSubNodeNames() : array {
+        return ['var', 'expr'];
     }
 }

lib/PhpParser/Node/Expr/BinaryOp.php

@@ -1,4 +1,4 @@
-<?php
+<?php declare(strict_types=1);
 
 namespace PhpParser\Node\Expr;
 
@@ -18,13 +18,23 @@ abstract class BinaryOp extends Expr
      * @param Expr  $right      The right hand side expression
      * @param array $attributes Additional attributes
      */
-    public function __construct(Expr $left, Expr $right, array $attributes = array()) {
+    public function __construct(Expr $left, Expr $right, array $attributes = []) {
         parent::__construct($attributes);
         $this->left = $left;
         $this->right = $right;
     }
 
-    public function getSubNodeNames() {
-        return array('left', 'right');
+    public function getSubNodeNames() : array {
+        return ['left', 'right'];
     }
+
+    /**
+     * Get the operator sigil for this binary operation.
+     *
+     * In the case there are multiple possible sigils for an operator, this method does not
+     * necessarily return the one used in the parsed code.
+     *
+     * @return string
+     */
+    abstract public function getOperatorSigil() : string;
 }

lib/PhpParser/Node/Expr/BinaryOp/BitwiseAnd.php

@@ -1,4 +1,4 @@
-<?php
+<?php declare(strict_types=1);
 
 namespace PhpParser\Node\Expr\BinaryOp;
 
@@ -6,4 +6,7 @@ use PhpParser\Node\Expr\BinaryOp;
 
 class BitwiseAnd extends BinaryOp
 {
+    public function getOperatorSigil() : string {
+        return '&';
+    }
 }

lib/PhpParser/Node/Expr/BinaryOp/BitwiseOr.php

@@ -1,4 +1,4 @@
-<?php
+<?php declare(strict_types=1);
 
 namespace PhpParser\Node\Expr\BinaryOp;
 
@@ -6,4 +6,7 @@ use PhpParser\Node\Expr\BinaryOp;
 
 class BitwiseOr extends BinaryOp
 {
+    public function getOperatorSigil() : string {
+        return '|';
+    }
 }

lib/PhpParser/Node/Expr/BinaryOp/BitwiseXor.php

@@ -1,4 +1,4 @@
-<?php
+<?php declare(strict_types=1);
 
 namespace PhpParser\Node\Expr\BinaryOp;
 
@@ -6,4 +6,7 @@ use PhpParser\Node\Expr\BinaryOp;
 
 class BitwiseXor extends BinaryOp
 {
+    public function getOperatorSigil() : string {
+        return '^';
+    }
 }

lib/PhpParser/Node/Expr/BinaryOp/BooleanAnd.php

@@ -1,4 +1,4 @@
-<?php
+<?php declare(strict_types=1);
 
 namespace PhpParser\Node\Expr\BinaryOp;
 
@@ -6,4 +6,7 @@ use PhpParser\Node\Expr\BinaryOp;
 
 class BooleanAnd extends BinaryOp
 {
+    public function getOperatorSigil() : string {
+        return '&&';
+    }
 }

lib/PhpParser/Node/Expr/BinaryOp/BooleanOr.php

@@ -1,4 +1,4 @@
-<?php
+<?php declare(strict_types=1);
 
 namespace PhpParser\Node\Expr\BinaryOp;
 
@@ -6,4 +6,7 @@ use PhpParser\Node\Expr\BinaryOp;
 
 class BooleanOr extends BinaryOp
 {
+    public function getOperatorSigil() : string {
+        return '||';
+    }
 }

lib/PhpParser/Node/Expr/BinaryOp/Coalesce.php

@@ -1,4 +1,4 @@
-<?php
+<?php declare(strict_types=1);
 
 namespace PhpParser\Node\Expr\BinaryOp;
 
@@ -6,4 +6,7 @@ use PhpParser\Node\Expr\BinaryOp;
 
 class Coalesce extends BinaryOp
 {
+    public function getOperatorSigil() : string {
+        return '??';
+    }
 }

lib/PhpParser/Node/Expr/BinaryOp/Concat.php

@@ -1,4 +1,4 @@
-<?php
+<?php declare(strict_types=1);
 
 namespace PhpParser\Node\Expr\BinaryOp;
 
@@ -6,4 +6,7 @@ use PhpParser\Node\Expr\BinaryOp;
 
 class Concat extends BinaryOp
 {
+    public function getOperatorSigil() : string {
+        return '.';
+    }
 }

lib/PhpParser/Node/Expr/BinaryOp/Div.php

@@ -1,4 +1,4 @@
-<?php
+<?php declare(strict_types=1);
 
 namespace PhpParser\Node\Expr\BinaryOp;
 
@@ -6,4 +6,7 @@ use PhpParser\Node\Expr\BinaryOp;
 
 class Div extends BinaryOp
 {
+    public function getOperatorSigil() : string {
+        return '/';
+    }
 }

lib/PhpParser/Node/Expr/BinaryOp/Equal.php

@@ -1,4 +1,4 @@
-<?php
+<?php declare(strict_types=1);
 
 namespace PhpParser\Node\Expr\BinaryOp;
 
@@ -6,4 +6,7 @@ use PhpParser\Node\Expr\BinaryOp;
 
 class Equal extends BinaryOp
 {
+    public function getOperatorSigil() : string {
+        return '==';
+    }
 }

lib/PhpParser/Node/Expr/BinaryOp/Greater.php

@@ -1,4 +1,4 @@
-<?php
+<?php declare(strict_types=1);
 
 namespace PhpParser\Node\Expr\BinaryOp;
 
@@ -6,4 +6,7 @@ use PhpParser\Node\Expr\BinaryOp;
 
 class Greater extends BinaryOp
 {
+    public function getOperatorSigil() : string {
+        return '>';
+    }
 }

lib/PhpParser/Node/Expr/BinaryOp/GreaterOrEqual.php

@@ -1,4 +1,4 @@
-<?php
+<?php declare(strict_types=1);
 
 namespace PhpParser\Node\Expr\BinaryOp;
 
@@ -6,4 +6,7 @@ use PhpParser\Node\Expr\BinaryOp;
 
 class GreaterOrEqual extends BinaryOp
 {
+    public function getOperatorSigil() : string {
+        return '>=';
+    }
 }

lib/PhpParser/Node/Expr/BinaryOp/Identical.php

@@ -1,4 +1,4 @@
-<?php
+<?php declare(strict_types=1);
 
 namespace PhpParser\Node\Expr\BinaryOp;
 
@@ -6,4 +6,7 @@ use PhpParser\Node\Expr\BinaryOp;
 
 class Identical extends BinaryOp
 {
+    public function getOperatorSigil() : string {
+        return '===';
+    }
 }

lib/PhpParser/Node/Expr/BinaryOp/LogicalAnd.php

@@ -1,4 +1,4 @@
-<?php
+<?php declare(strict_types=1);
 
 namespace PhpParser\Node\Expr\BinaryOp;
 
@@ -6,4 +6,7 @@ use PhpParser\Node\Expr\BinaryOp;
 
 class LogicalAnd extends BinaryOp
 {
+    public function getOperatorSigil() : string {
+        return 'and';
+    }
 }

lib/PhpParser/Node/Expr/BinaryOp/LogicalOr.php

@@ -1,4 +1,4 @@
-<?php
+<?php declare(strict_types=1);
 
 namespace PhpParser\Node\Expr\BinaryOp;
 
@@ -6,4 +6,7 @@ use PhpParser\Node\Expr\BinaryOp;
 
 class LogicalOr extends BinaryOp
 {
+    public function getOperatorSigil() : string {
+        return 'or';
+    }
 }

lib/PhpParser/Node/Expr/BinaryOp/LogicalXor.php

@@ -1,4 +1,4 @@
-<?php
+<?php declare(strict_types=1);
 
 namespace PhpParser\Node\Expr\BinaryOp;
 
@@ -6,4 +6,7 @@ use PhpParser\Node\Expr\BinaryOp;
 
 class LogicalXor extends BinaryOp
 {
+    public function getOperatorSigil() : string {
+        return 'xor';
+    }
 }

lib/PhpParser/Node/Expr/BinaryOp/Minus.php

@@ -1,4 +1,4 @@
-<?php
+<?php declare(strict_types=1);
 
 namespace PhpParser\Node\Expr\BinaryOp;
 
@@ -6,4 +6,7 @@ use PhpParser\Node\Expr\BinaryOp;
 
 class Minus extends BinaryOp
 {
+    public function getOperatorSigil() : string {
+        return '-';
+    }
 }

lib/PhpParser/Node/Expr/BinaryOp/Mod.php

@@ -1,4 +1,4 @@
-<?php
+<?php declare(strict_types=1);
 
 namespace PhpParser\Node\Expr\BinaryOp;
 
@@ -6,4 +6,7 @@ use PhpParser\Node\Expr\BinaryOp;
 
 class Mod extends BinaryOp
 {
+    public function getOperatorSigil() : string {
+        return '%';
+    }
 }

lib/PhpParser/Node/Expr/BinaryOp/Mul.php

@@ -1,4 +1,4 @@
-<?php
+<?php declare(strict_types=1);
 
 namespace PhpParser\Node\Expr\BinaryOp;
 
@@ -6,4 +6,7 @@ use PhpParser\Node\Expr\BinaryOp;
 
 class Mul extends BinaryOp
 {
+    public function getOperatorSigil() : string {
+        return '*';
+    }
 }

lib/PhpParser/Node/Expr/BinaryOp/NotEqual.php

@@ -1,4 +1,4 @@
-<?php
+<?php declare(strict_types=1);
 
 namespace PhpParser\Node\Expr\BinaryOp;
 
@@ -6,4 +6,7 @@ use PhpParser\Node\Expr\BinaryOp;
 
 class NotEqual extends BinaryOp
 {
+    public function getOperatorSigil() : string {
+        return '!=';
+    }
 }

lib/PhpParser/Node/Expr/BinaryOp/NotIdentical.php

@@ -1,4 +1,4 @@
-<?php
+<?php declare(strict_types=1);
 
 namespace PhpParser\Node\Expr\BinaryOp;
 
@@ -6,4 +6,7 @@ use PhpParser\Node\Expr\BinaryOp;
 
 class NotIdentical extends BinaryOp
 {
+    public function getOperatorSigil() : string {
+        return '!==';
+    }
 }

lib/PhpParser/Node/Expr/BinaryOp/Plus.php

@@ -1,4 +1,4 @@
-<?php
+<?php declare(strict_types=1);
 
 namespace PhpParser\Node\Expr\BinaryOp;
 
@@ -6,4 +6,7 @@ use PhpParser\Node\Expr\BinaryOp;
 
 class Plus extends BinaryOp
 {
+    public function getOperatorSigil() : string {
+        return '+';
+    }
 }

lib/PhpParser/Node/Expr/BinaryOp/Pow.php

@@ -1,4 +1,4 @@
-<?php
+<?php declare(strict_types=1);
 
 namespace PhpParser\Node\Expr\BinaryOp;
 
@@ -6,4 +6,7 @@ use PhpParser\Node\Expr\BinaryOp;
 
 class Pow extends BinaryOp
 {
+    public function getOperatorSigil() : string {
+        return '**';
+    }
 }

lib/PhpParser/Node/Expr/BinaryOp/ShiftLeft.php

@@ -1,4 +1,4 @@
-<?php
+<?php declare(strict_types=1);
 
 namespace PhpParser\Node\Expr\BinaryOp;
 
@@ -6,4 +6,7 @@ use PhpParser\Node\Expr\BinaryOp;
 
 class ShiftLeft extends BinaryOp
 {
+    public function getOperatorSigil() : string {
+        return '<<';
+    }
 }

lib/PhpParser/Node/Expr/BinaryOp/ShiftRight.php

@@ -1,4 +1,4 @@
-<?php
+<?php declare(strict_types=1);
 
 namespace PhpParser\Node\Expr\BinaryOp;
 
@@ -6,4 +6,7 @@ use PhpParser\Node\Expr\BinaryOp;
 
 class ShiftRight extends BinaryOp
 {
+    public function getOperatorSigil() : string {
+        return '>>';
+    }
 }