Release PHP-Parser 2.1.0

Document that XDebug is super slow
Update CHANGELOG
2025-07-09 08:32:35 +02:00 · 2016-04-19 15:41:41 +02:00 · 2016-04-19 15:40:08 +02:00 · 2016-04-19 15:32:57 +02:00 · 2016-04-19 15:10:51 +02:00 · 2016-04-19 14:51:05 +02:00
93 changed files with 5185 additions and 3971 deletions
--- a/.travis.yml
+++ b/.travis.yml
@ -2,6 +2,10 @@ language: php

 sudo: false

+cache:
+  directories:
+    - $HOME/.composer/cache
+
 php:
  - 5.4
  - 5.5
@ -10,11 +14,19 @@ php:
  - nightly
  - hhvm

-install: composer install --prefer-source
+install:
+  - if [ $TRAVIS_PHP_VERSION = '5.6' ]; then composer require satooshi/php-coveralls '~1.0'; fi
+  - composer install --prefer-dist

 matrix:
  allow_failures:
    - php: nightly
  fast_finish: true

-script: vendor/bin/phpunit
+script:
+  - if [ $TRAVIS_PHP_VERSION = '5.6' ]; then vendor/bin/phpunit --coverage-clover build/logs/clover.xml; else vendor/bin/phpunit; fi
+  - if [ $TRAVIS_PHP_VERSION = '7.0' ]; then test_old/run-php-src.sh; fi
+
+after_success:
+  if [ $TRAVIS_PHP_VERSION = '5.6' ]; then php vendor/bin/coveralls; fi
+
--- a/CHANGELOG.md
+++ b/CHANGELOG.md
@ -1,8 +1,67 @@
-Version 2.0.1-dev
+Version 2.1.1-dev
 -----------------

 Nothing yet.

+Version 2.1.0 (2016-04-19)
+--------------------------
+
+### Fixed
+
+* Properly support `B""` strings (with uppercase `B`) in a number of places.
+* Fixed reformatting of indented parts in a certain non-standard comment style.
+
+### Added
+
+* Added `dumpComments` option to node dumper, to enable dumping of comments associated with nodes.
+* Added `Stmt\Nop` node, that is used to collect comments located at the end of a block or at the
+  end of a file (without a following node with which they could otherwise be associated).
+* Added `kind` attribute to `Expr\Exit` to distinguish between `exit` and `die`.
+* Added `kind` attribute to `Scalar\LNumber` to distinguish between decimal, binary, octal and
+  hexadecimal numbers.
+* Added `kind` attribtue to `Expr\Array` to distinguish between `array()` and `[]`.
+* Added `kind` attribute to `Scalar\String` and `Scalar\Encapsed` to distinguish between
+  single-quoted, double-quoted, heredoc and nowdoc string.
+* Added `docLabel` attribute to `Scalar\String` and `Scalar\Encapsed`, if it is a heredoc or
+  nowdoc string.
+* Added start file offset information to `Comment` nodes.
+* Added `setReturnType()` method to function and method builders.
+* Added `-h` and `--help` options to `php-parse` script.
+
+### Changed
+
+* Invalid octal literals now throw a parse error in PHP 7 mode.
+* The pretty printer takes all the new attributes mentioned in the previous section into account.
+* The protected `AbstractPrettyPrinter::pComments()` method no longer returns a trailing newline.
+* The bundled autoloader supports library files being stored in a different directory than
+  `PhpParser` for easier downstream distribution.
+
+### Deprecated
+
+* The `Comment::setLine()` and `Comment::setText()` methods have been deprecated. Construct new
+  objects instead.
+
+### Removed
+
+* The internal (but public) method `Scalar\LNumber::parse()` has been removed. A non-internal
+  `LNumber::fromString()` method has been added instead.
+
+Version 2.0.1 (2016-02-28)
+--------------------------
+
+### Fixed
+
+* `declare() {}` and `declare();` are not semantically equivalent and will now result in different
+  ASTs. The format case will have an empty `stmts` array, while the latter will set `stmts` to
+  `null`.
+* Magic constants are now supported as semi-reserved keywords.
+* A shebang line like `#!/usr/bin/env php` is now allowed at the start of a namespaced file.
+  Previously this generated an exception.
+* The `prettyPrintFile()` method will not strip a trailing `?>` from the raw data that follows a
+  `__halt_compiler()` statement.
+* The `prettyPrintFile()` method will not strip an opening `<?php` if the file starts with a
+  comment followed by InlineHTML.
+
 Version 2.0.0 (2015-12-04)
 --------------------------

@ -72,242 +131,8 @@ A more detailed description of backwards incompatible changes can be found in th
 * The constructor for `Scalar` nodes no longer has a default value. E.g. `new LNumber()` should now
  be written as `new LNumber(0)`.

-Version 1.4.0 (2015-07-14)
--------------------------
-
-### Added
-
-* Added interface `PhpParser\Node\FunctionLike`, which is implemented by `Stmt\ClassMethod`,
-  `Stmt\Function_` and `Expr\Closure` nodes. This interface provides getters for their common
-  subnodes.
-* Added `Node\Stmt\ClassLike::getMethod()` to look up a specific method on a class/interface/trait.
-
-### Fixed
-
-* Fixed `isPublic()` return value for implicitly public properties and methods that define and
-  additional modifier like `static` or `abstract`.
-* Properties are now accepted by the trait builder.
-* Fixed `__HALT_COMPILER_OFFSET__` support on HHVM.
-
-Version 1.3.0 (2015-05-02)
--------------------------
-
-### Added
-
-* Errors can now store the attributes of the node/token where the error occurred. Previously only the start line was
-  stored.
-* If file positions are enabled in the lexer, errors can now provide column information if it is available. See
-  [documentation](https://github.com/nikic/PHP-Parser/blob/master/doc/component/Error.markdown#column-information).
-* The parser now provides an experimental error recovery mode, which can be enabled by disabling the `throwOnError`
-  parser option. In this mode the parser will try to construct a partial AST even if the code is not valid PHP. See
-  [documentation](https://github.com/nikic/PHP-Parser/blob/master/doc/component/Error.markdown#error-recovery).
-* Added support for PHP 7 `yield from` expression. It is represented by `Expr\YieldFrom`.
-* Added support for PHP 7 anonymous classes. These are represented by ordinary `Stmt\Class_` nodes with the name set to
-  `null`. Furthermore this implies that `Expr\New_` can now contain a `Stmt\Class_` in its `class` subnode.
-
-### Fixed
-
-* Fixed registration of PHP 7 aliases, for the case where the old name was used before the new name.
-* Fixed handling of precedence when pretty-printing `print` expressions.
-* Floating point numbers are now pretty-printed with a higher precision.
-* Checks for special class names like `self` are now case-insensitive.
-
-Version 1.2.2 (2015-04-03)
--------------------------
-
-* The `NameResolver` now resolves parameter type hints when entering the function/method/closure node. As such other
-  visitors running after it will be able to make use of the resolved names at that point already.
-* The autoloader no longer sets the `unserialize_callback_func` ini option on registration - this is not necessary and
-  may cause issues when running PhpUnit tests with process isolation.
-
-Version 1.2.1 (2015-03-24)
--------------------------
-
-* Fixed registration of the aliases introduced in 1.2.0. Previously the old class names could not be used in
-  `instanceof` checks under some circumstances.
-
-Version 1.2.0 (2015-03-22)
--------------------------
-
-### Changed
-
-* To ensure compatibility with PHP 7, the following node classes have been renamed:
-
-        OLD                             => NEW
-        PhpParser\Node\Expr\Cast\Bool   => PhpParser\Node\Expr\Cast\Bool_
-        PhpParser\Node\Expr\Cast\Int    => PhpParser\Node\Expr\Cast\Int_
-        PhpParser\Node\Expr\Cast\Object => PhpParser\Node\Expr\Cast\Object_
-        PhpParser\Node\Expr\Cast\String => PhpParser\Node\Expr\Cast\String_
-        PhpParser\Node\Scalar\String    => PhpParser\Node\Scalar\String_
-
-  **The previous class names are still supported as aliases.** However it is strongly encouraged to use the new names
-  in order to make your code compatible with PHP 7.
-
-* Subnodes are now stored using real properties instead of an array. This improves performance and memory usage of the
-  initial parse and subsequent node tree operations. The `NodeAbstract` class still supports the old way of specifying
-  subnodes, however this is *deprecated*. In any case properties that are assigned to a node after creation will no
-  longer be considered as subnodes.
-
-* Methods and property declarations will no longer set the `Stmt\Class_::MODIFIER_PUBLIC` flag if no visibility is
-  explicitly given. However the `isPublic()` method will continue to return true. This allows you to distinguish whether
-  a method/property is explicitly or implicitly public and control the pretty printer output more precisely.
-
-* The `Stmt\Class_`, `Stmt\Interface_` and `Stmt\Trait_` nodes now inherit from `Stmt\ClassLike`, which provides a
-  `getMethods()` method. Previously this method was only available on `Stmt\Class_`.
-
-* Support including the `bootstrap.php` file multiple times.
-
-* Make documentation and tests part of the release tarball again.
-
-* Improve support for HHVM and PHP 7.
-
-### Added
-
-* Added support for PHP 7 return type declarations. This adds an additional `returnType` subnode to `Stmt\Function_`,
-  `Stmt\ClassMethod` and `Expr\Closure`.
-
-* Added support for the PHP 7 null coalesce operator `??`. The operator is represented by `Expr\BinaryOp\Coalesce`.
-
-* Added support for the PHP 7 spaceship operator `<=>`. The operator is represented by `Expr\BinaryOp\Spaceship`.
-
-* Added use builder.
-
-* Added global namespace support to the namespace builder.
-
-* Added a constructor flag to `NodeTraverser`, which disables cloning of nodes.
-
-Version 1.1.0 (2015-01-18)
--------------------------
-
-* Methods that do not specify an explicit visibility (e.g. `function method()`) will now have the `MODIFIER_PUBLIC`
-  flag set. This also means that their `isPublic()` method will return true.
-
-* Declaring a property as abstract or final is now an error.
-
-* The `Lexer` and `Lexer\Emulative` classes now accept an `$options` array in their constructors. Currently only the
-  `usedAttributes` option is supported, which determines which attributes will be added to AST nodes. In particular
-  it is now possible to add information on the token and file positions corresponding to a node. For more details see
-  the [Lexer component](https://github.com/nikic/PHP-Parser/blob/master/doc/component/Lexer.markdown) documentation.
-
-* Node visitors can now return `NodeTraverser::DONT_TRAVERSE_CHILDREN` from `enterNode()` in order to skip all children
-  of the current node, for all visitors.
-
-* Added builders for traits and namespaces.
-
-* The class, interface, trait, function, method and property builders now support adding doc comments using the
-  `setDocComment()` method.
-
-* Added support for fully-qualified and namespace-relative names in builders. No longer allow use of name component
-  arrays.
-
-* Do not add documentation and tests to distribution archive files.
-
-Version 1.0.2 (2014-11-04)
--------------------------
-
-* The `NameResolver` visitor now also resolves names in trait adaptations (aliases and precedence declarations).
-
-* Remove stray whitespace when pretty-printing trait adaptations that only change visibility.
-
-Version 1.0.1 (2014-10-14)
--------------------------
-
-* Disallow `new` expressions without a class name. Previously `new;` was accidentally considered to be valid code.
-
-* Support T_ONUMBER token used by HHVM.
-
-* Add ability to directly pass code to the `php-parse.php` script.
-
-* Prevent truncation of `var_dump()` output in the `php-parse.php` script if XDebug is used.
-
-Version 1.0.0 (2014-09-12)
--------------------------
-
-* [BC] Removed deprecated `Template` and `TemplateLoader` classes.
-
-* Fixed XML unserializer to properly work with new namespaced node names.
-
-Version 1.0.0-beta2 (2014-08-31)
--------------------------------
-
-* [PHP 5.6] Updated support for constant scalar expressions to comply with latest changes. This means that arrays
-  and array dimension fetches are now supported as well.
-
-* [PHP 5.6] Direct array dereferencing of constants is supported now, i.e. both `FOO[0]` and `Foo::BAR[0]` are valid
-  now.
-
-* Fixed handling of special class names (`self`, `parent` and `static`) in the name resolver to be case insensitive.
-  Additionally the name resolver now enforces that special class names are only used as unqualified names, e.g. `\self`
-  is considered invalid.
-
-* The case of references to the `static` class name is now preserved. Previously `static` was always lowercased,
-  regardless of the case used in the source code.
-
-* The autoloader now only requires a file if it exists. This allows usages like
-  `class_exists('PhpParser\NotExistingClass')`.
-
-* Added experimental `bin/php-parse.php` script, which is intended to help exploring and debugging the node tree.
-
-* Separated the parser implemention (in `lib/PhpParser/ParserAbstract.php`) and the generated data (in
-  `lib/PhpParser/Parser.php`). Furthermore the parser now uses meaningful variable names and contains comments
-  explaining their usage.
-
-Version 1.0.0-beta1 (2014-03-27)
--------------------------------
-
-* [BC] PHP-Parser now requires PHP 5.3 or newer to run. It is however still possible to *parse* PHP 5.2 source code,
-  while running on a newer version.
-
-* [BC] The library has been moved to use namespaces with the `PhpParser` vendor prefix. However, the old names using
-  underscores are still available as aliases, as such most code should continue running on the new version without
-  further changes.
-
-  However, code performing dispatch operations on `Node::getType()` may be affected by some of the name changes. For
-  example a `+` node will now return type `Expr_BinaryOp_Plus` instead of `Expr_Plus`. In particular this may affect
-  custom pretty printers.
-
-  Due to conflicts with reserved keywords, some class names now end with an underscore, e.g. `PHPParser_Node_Stmt_Class`
-  is now `PhpParser\Node\Stmt\Class_`. (But as usual, the old name is still available)
-
-* [PHP 5.6] Added support for the power operator `**` (node `Expr\BinaryOp\Pow`) and the compound power assignment
-  operator `**=` (node `Expr\AssignOp\Pow`).
-
-* [PHP 5.6] Added support for variadic functions: `Param` nodes now have `variadic` as a boolean subnode.
-
-* [PHP 5.6] Added support for argument unpacking: `Arg` nodes now have `unpack` as a boolean subnode.
-
-* [PHP 5.6] Added support for aliasing of functions and constants. `Stmt\Use_` nodes now have an integral `type`
-  subnode, which is one of `Stmt\Use_::TYPE_NORMAL` (`use`), `Stmt\Use_::TYPE_FUNCTION` (`use function`) or
-  `Stmt\Use_::TYPE_CONSTANT` (`use const`).
-
-  The `NameResolver` now also supports resolution of such aliases.
-
-* [PHP 5.6] Added support for constant scalar expressions. This means that certain expressions are now allowed as the
-  initializer for constants, properties, parameters, static variables, etc.
-
-* [BC] Improved pretty printing of empty statements lists, which are now printed as `{\n}` instead of `{\n    \n}`.
-  This changes the behavior of the protected `PrettyPrinterAbstract::pStmts()` method, so custom pretty printing code
-  making use it of may need to be adjusted.
-
-* Changed the order of some subnodes to be consistent with their order in the sour code. For example `Stmt\If->cond`
-  will now appear before `Stmt\If->stmts` etc.
-
-* Added `Scalar\MagicConstant->getName()`, which returns the name of the magic constant (e.g. `__CLASS__`).
-
-**The following changes are also included in 0.9.5**:
-
-* [BC] Deprecated `PHPParser_Template` and `PHPParser_TemplateLoader`. This functionality does not belong in the main project
-  and - as far as I know - nobody is using it.
-
-* Add `NodeTraverser::removeVisitor()` method, which removes a visitor from the node traverser. This also modifies the
-  corresponding `NodeTraverserInterface`.
-
-* Fix alias resolution in `NameResolver`: Class names are now correctly handled as case-insensitive.
-
-* The undefined variable error, which is used to the lexer to reset the error state, will no longer interfere with
-  custom error handlers.
-
 ---

-**This changelog only includes changes from the 1.0 and 2.0 series. For older changes see the
-[0.9 series changelog][https://github.com/nikic/PHP-Parser/blob/0.9/CHANGELOG.md].**
+**This changelog only includes changes from the 2.0 series. For older changes see the
+[1.x series changelog](https://github.com/nikic/PHP-Parser/blob/1.x/CHANGELOG.md) and the
+[0.9 series changelog](https://github.com/nikic/PHP-Parser/blob/0.9/CHANGELOG.md).**
--- a/README.md
+++ b/README.md
@ -1,13 +1,14 @@
 PHP Parser
 ==========

+[![Build Status](https://travis-ci.org/nikic/PHP-Parser.svg?branch=master)](https://travis-ci.org/nikic/PHP-Parser) [![Coverage Status](https://coveralls.io/repos/github/nikic/PHP-Parser/badge.svg?branch=master)](https://coveralls.io/github/nikic/PHP-Parser?branch=master)
+
 This is a PHP 5.2 to PHP 7.0 parser written in PHP. Its purpose is to simplify static code analysis and
 manipulation.

-
 [**Documentation for version 2.x**][doc_master] (stable; for running on PHP >= 5.4; for parsing PHP 5.2 to PHP 7.0).

-[Documentation for version 1.x][doc_1_x] (stable; for running on PHP >= 5.3; for parsing PHP 5.2 to PHP 5.6).
+[Documentation for version 1.x][doc_1_x] (unsupported; for running on PHP >= 5.3; for parsing PHP 5.2 to PHP 5.6).

 In a Nutshell
 -------------
--- a/bin/php-parse
+++ b/bin/php-parse
@ -27,10 +27,10 @@ if (empty($files)) {
 }

 $lexer = new PhpParser\Lexer\Emulative(array('usedAttributes' => array(
-    'startLine', 'endLine', 'startFilePos', 'endFilePos'
+    'startLine', 'endLine', 'startFilePos', 'endFilePos', 'comments'
 )));
 $parser = (new PhpParser\ParserFactory)->create(PhpParser\ParserFactory::PREFER_PHP7, $lexer);
-$dumper = new PhpParser\NodeDumper;
+$dumper = new PhpParser\NodeDumper(['dumpComments' => true]);
 $prettyPrinter = new PhpParser\PrettyPrinter\Standard;
 $serializer = new PhpParser\Serializer\XML;

@ -86,9 +86,11 @@ foreach ($files as $file) {
    }
 }

-function showHelp($error) {
-    die($error . "\n\n" .
-        <<<OUTPUT
+function showHelp($error = '') {
+    if ($error) {
+        echo $error . "\n\n";
+    }
+    die(<<<OUTPUT
 Usage: php-parse [operations] file1.php [file2.php ...]
   or: php-parse [operations] "<?php code"
 Turn PHP source code into an abstract syntax tree.
@ -101,6 +103,7 @@ Operations is a list of the following options (--dump by default):
        --var-dump          var_dump() nodes (for exact structure)
    -N, --resolve-names     Resolve names using NodeVisitor\NameResolver
    -c, --with-column-info  Show column-numbers for errors (if available)
+    -h, --help              Display this page

 Example:
    php-parse -d -p -N -d file.php
@ -150,6 +153,10 @@ function parseArgs($args) {
            case '-c';
                $attributes['with-column-info'] = true;
                break;
+            case '--help':
+            case '-h';
+                showHelp();
+                break;
            case '--':
                $parseOptions = false;
                break;
--- a/composer.json
+++ b/composer.json
@ -24,7 +24,7 @@
    "bin": ["bin/php-parse"],
    "extra": {
        "branch-alias": {
-            "dev-master": "2.0-dev"
+            "dev-master": "2.1-dev"
        }
    }
 }
--- a/doc/0_Introduction.markdown
+++ b/doc/0_Introduction.markdown
@ -18,7 +18,7 @@ For example an AST abstracts away the fact that in PHP variables can be written
 as `$$bar`, `${'foobar'}` or even `${!${''}=barfoo()}`. You don't have to worry about recognizing
 all the different syntaxes from a stream of tokens.

-Another questions is: Why would I want to have a PHP parser *written in PHP*? Well, PHP might not be
+Another question is: Why would I want to have a PHP parser *written in PHP*? Well, PHP might not be
 a language especially suited for fast parsing, but processing the AST is much easier in PHP than it
 would be in other, faster languages like C. Furthermore the people most probably wanting to do
 programmatic PHP code analysis are incidentally PHP developers, not C developers.
--- a/doc/2_Usage_of_basic_components.markdown
+++ b/doc/2_Usage_of_basic_components.markdown
@ -18,7 +18,9 @@ Additionally you may want to set the `xdebug.max_nesting_level` ini option to a
 ini_set('xdebug.max_nesting_level', 3000);
 ```

-This ensures that there will be no errors when traversing highly nested node trees.
+This ensures that there will be no errors when traversing highly nested node trees. However, it is
+preferable to disable XDebug completely, as it can easily make this library more than five times
+slower.

 Parsing
 -------
@ -268,7 +270,7 @@ The `enterNode()` method can additionally return the value `NodeTraverser::DONT_
 which instructs the traverser to skip all children of the current node.

 The `leaveNode()` method can additionally return the value `NodeTraverser::REMOVE_NODE`, in which
-case the current node will be removed from the parent array. Furthermove it is possible to return
+case the current node will be removed from the parent array. Furthermore it is possible to return
 an array of nodes, which will be merged into the parent array at the offset of the current node.
 I.e. if in `array(A, B, C)` the node `B` should be replaced with `array(X, Y, Z)` the result will
 be `array(A, X, Y, Z, C)`.
@ -376,7 +378,7 @@ create a name with backslashes either write `$node->toString()` or `(string) $no
 a new name from the string and return it. Returning a new node replaces the old node.

 Another thing we need to do is change the class/function/const declarations. Currently they contain
-only the shortname (i.e. the last part of the name), but they need to contain the complete name inclduing
+only the shortname (i.e. the last part of the name), but they need to contain the complete name including
 the namespace prefix:

 ```php
--- a/doc/4_Code_generation.markdown
+++ b/doc/4_Code_generation.markdown
@ -29,6 +29,7 @@ $node = $factory->namespace('Name\Space')
        ->addStmt($factory->method('someMethod')
            ->makePublic()
            ->makeAbstract() // ->makeFinal()
+            ->setReturnType('bool')
            ->addParam($factory->param('someParam')->setTypeHint('SomeClass'))
            ->setDocComment('/**
                              * This method does something.
@ -74,7 +75,7 @@ abstract class SomeClass extends SomeOtherClass implements A\Few, \Interfaces
     *
     * @param SomeClass And takes a parameter
     */
-    public abstract function someMethod(SomeClass $someParam);
+    public abstract function someMethod(SomeClass $someParam) : bool;
    protected function anotherMethod($someParam = 'test')
    {
        print $someParam;
--- a/grammar/php5.y
+++ b/grammar/php5.y
@ -9,11 +9,17 @@ start:
    top_statement_list                                      { $$ = $this->handleNamespaces($1); }
 ;

-top_statement_list:
-      top_statement_list top_statement                      { pushNormalizing($1, $2); }
+top_statement_list_ex:
+      top_statement_list_ex top_statement                   { pushNormalizing($1, $2); }
    | /* empty */                                           { init(); }
 ;

+top_statement_list:
+      top_statement_list_ex
+          { makeNop($nop, $this->lookaheadStartAttributes);
+            if ($nop !== null) { $1[] = $nop; } $$ = $1; }
+;
+
 reserved_non_modifiers:
      T_INCLUDE | T_INCLUDE_ONCE | T_EVAL | T_REQUIRE | T_REQUIRE_ONCE | T_LOGICAL_OR | T_LOGICAL_XOR | T_LOGICAL_AND
    | T_INSTANCEOF | T_NEW | T_CLONE | T_EXIT | T_IF | T_ELSEIF | T_ELSE | T_ENDIF | T_ECHO | T_DO | T_WHILE
@ -21,6 +27,7 @@ reserved_non_modifiers:
    | T_FINALLY | T_THROW | T_USE | T_INSTEADOF | T_GLOBAL | T_VAR | T_UNSET | T_ISSET | T_EMPTY | T_CONTINUE | T_GOTO
    | T_FUNCTION | T_CONST | T_RETURN | T_PRINT | T_YIELD | T_LIST | T_SWITCH | T_ENDSWITCH | T_CASE | T_DEFAULT
    | T_BREAK | T_ARRAY | T_CALLABLE | T_EXTENDS | T_IMPLEMENTS | T_NAMESPACE | T_TRAIT | T_INTERFACE | T_CLASS
+    | T_CLASS_C | T_TRAIT_C | T_FUNC_C | T_METHOD_C | T_LINE | T_FILE | T_DIR | T_NS_C | T_HALT_COMPILER
 ;

 semi_reserved:
@ -123,11 +130,17 @@ class_const:
    identifier '=' static_scalar                            { $$ = Node\Const_[$1, $3]; }
 ;

-inner_statement_list:
-      inner_statement_list inner_statement                  { pushNormalizing($1, $2); }
+inner_statement_list_ex:
+      inner_statement_list_ex inner_statement               { pushNormalizing($1, $2); }
    | /* empty */                                           { init(); }
 ;

+inner_statement_list:
+      inner_statement_list_ex
+          { makeNop($nop, $this->lookaheadStartAttributes);
+            if ($nop !== null) { $1[] = $nop; } $$ = $1; }
+;
+
 inner_statement:
      statement                                             { $$ = $1; }
    | function_declaration_statement                        { $$ = $1; }
@ -136,7 +149,7 @@ inner_statement:
          { throw new Error('__HALT_COMPILER() can only be used from the outermost scope', attributes()); }
 ;

-statement:
+non_empty_statement:
      '{' inner_statement_list '}'                          { $$ = $2; }
    | T_IF parentheses_expr statement elseif_list else_single
          { $$ = Stmt\If_[$2, ['stmts' => toArray($3), 'elseifs' => $4, 'else' => $5]]; }
@ -165,7 +178,6 @@ statement:
    | T_FOREACH '(' expr T_AS variable T_DOUBLE_ARROW foreach_variable ')' foreach_statement
          { $$ = Stmt\Foreach_[$3, $7[0], ['keyVar' => $5, 'byRef' => $7[1], 'stmts' => $9]]; }
    | T_DECLARE '(' declare_list ')' declare_statement      { $$ = Stmt\Declare_[$3, $5]; }
-    | ';'                                                   { $$ = array(); /* means: no statement */ }
    | T_TRY '{' inner_statement_list '}' catches optional_finally
          { $$ = Stmt\TryCatch[$3, $5, $6]; }
    | T_THROW expr ';'                                      { $$ = Stmt\Throw_[$2]; }
@ -174,6 +186,13 @@ statement:
    | error                                                 { $$ = array(); /* means: no statement */ }
 ;

+statement:
+      non_empty_statement                                   { $$ = $1; }
+    | ';'
+          { makeNop($$, $this->startAttributeStack[#1]);
+            if ($$ === null) $$ = array(); /* means: no statement */ }
+;
+
 catches:
      /* empty */                                           { init(); }
    | catches catch                                         { push($1, $2); }
@ -255,7 +274,8 @@ foreach_statement:
 ;

 declare_statement:
-      statement                                             { $$ = toArray($1); }
+      non_empty_statement                                   { $$ = toArray($1); }
+    | ';'                                                   { $$ = null; }
    | ':' inner_statement_list T_ENDDECLARE ';'             { $$ = $2; }
 ;

@ -568,7 +588,10 @@ expr:
    | T_OBJECT_CAST expr                                    { $$ = Expr\Cast\Object_ [$2]; }
    | T_BOOL_CAST expr                                      { $$ = Expr\Cast\Bool_   [$2]; }
    | T_UNSET_CAST expr                                     { $$ = Expr\Cast\Unset_  [$2]; }
-    | T_EXIT exit_expr                                      { $$ = Expr\Exit_        [$2]; }
+    | T_EXIT exit_expr
+          { $attrs = attributes();
+            $attrs['kind'] = strtolower($1) === 'exit' ? Expr\Exit_::KIND_EXIT : Expr\Exit_::KIND_DIE;
+            $$ = new Expr\Exit_($2, $attrs); }
    | '@' expr                                              { $$ = Expr\ErrorSuppress[$2]; }
    | scalar                                                { $$ = $1; }
    | array_expr                                            { $$ = $1; }
@ -596,14 +619,19 @@ yield_expr:
 ;

 array_expr:
-      T_ARRAY '(' array_pair_list ')'                       { $$ = Expr\Array_[$3]; }
-    | '[' array_pair_list ']'                               { $$ = Expr\Array_[$2]; }
+      T_ARRAY '(' array_pair_list ')'
+          { $attrs = attributes(); $attrs['kind'] = Expr\Array_::KIND_LONG;
+            $$ = new Expr\Array_($3, $attrs); }
+    | '[' array_pair_list ']'
+          { $attrs = attributes(); $attrs['kind'] = Expr\Array_::KIND_SHORT;
+            $$ = new Expr\Array_($2, $attrs); }
 ;

 scalar_dereference:
      array_expr '[' dim_offset ']'                         { $$ = Expr\ArrayDimFetch[$1, $3]; }
    | T_CONSTANT_ENCAPSED_STRING '[' dim_offset ']'
-          { $$ = Expr\ArrayDimFetch[Scalar\String_[Scalar\String_::parse($1, false)], $3]; }
+          { $attrs = attributes(); $attrs['kind'] = strKind($1);
+            $$ = Expr\ArrayDimFetch[new Scalar\String_(Scalar\String_::parse($1), $attrs), $3]; }
    | constant '[' dim_offset ']'                           { $$ = Expr\ArrayDimFetch[$1, $3]; }
    | scalar_dereference '[' dim_offset ']'                 { $$ = Expr\ArrayDimFetch[$1, $3]; }
    /* alternative array syntax missing intentionally */
@ -714,9 +742,11 @@ ctor_arguments:
 ;

 common_scalar:
-      T_LNUMBER                                             { $$ = Scalar\LNumber[Scalar\LNumber::parse($1)]; }
+      T_LNUMBER                                             { $$ = Scalar\LNumber::fromString($1, attributes(), true); }
    | T_DNUMBER                                             { $$ = Scalar\DNumber[Scalar\DNumber::parse($1)]; }
-    | T_CONSTANT_ENCAPSED_STRING                            { $$ = Scalar\String_[Scalar\String_::parse($1, false)]; }
+    | T_CONSTANT_ENCAPSED_STRING
+          { $attrs = attributes(); $attrs['kind'] = strKind($1);
+            $$ = new Scalar\String_(Scalar\String_::parse($1, false), $attrs); }
    | T_LINE                                                { $$ = Scalar\MagicConst\Line[]; }
    | T_FILE                                                { $$ = Scalar\MagicConst\File[]; }
    | T_DIR                                                 { $$ = Scalar\MagicConst\Dir[]; }
@ -726,9 +756,11 @@ common_scalar:
    | T_FUNC_C                                              { $$ = Scalar\MagicConst\Function_[]; }
    | T_NS_C                                                { $$ = Scalar\MagicConst\Namespace_[]; }
    | T_START_HEREDOC T_ENCAPSED_AND_WHITESPACE T_END_HEREDOC
-          { $$ = Scalar\String_[Scalar\String_::parseDocString($1, $2, false)]; }
+          { $attrs = attributes(); setDocStringAttrs($attrs, $1);
+            $$ = new Scalar\String_(Scalar\String_::parseDocString($1, $2, false), $attrs); }
    | T_START_HEREDOC T_END_HEREDOC
-          { $$ = Scalar\String_['']; }
+          { $attrs = attributes(); setDocStringAttrs($attrs, $1);
+            $$ = new Scalar\String_('', $attrs); }
 ;

 static_scalar:
@ -786,9 +818,11 @@ scalar:
      common_scalar                                         { $$ = $1; }
    | constant                                              { $$ = $1; }
    | '"' encaps_list '"'
-          { parseEncapsed($2, '"', false); $$ = Scalar\Encapsed[$2]; }
+          { $attrs = attributes(); $attrs['kind'] = Scalar\String_::KIND_DOUBLE_QUOTED;
+            parseEncapsed($2, '"', true); $$ = new Scalar\Encapsed($2, $attrs); }
    | T_START_HEREDOC encaps_list T_END_HEREDOC
-          { parseEncapsedDoc($2, false); $$ = Scalar\Encapsed[$2]; }
+          { $attrs = attributes(); setDocStringAttrs($attrs, $1);
+            parseEncapsedDoc($2, true); $$ = new Scalar\Encapsed($2, $attrs); }
 ;

 static_array_pair_list:
--- a/grammar/php7.y
+++ b/grammar/php7.y
@ -9,11 +9,17 @@ start:
    top_statement_list                                      { $$ = $this->handleNamespaces($1); }
 ;

-top_statement_list:
-      top_statement_list top_statement                      { pushNormalizing($1, $2); }
+top_statement_list_ex:
+      top_statement_list_ex top_statement                   { pushNormalizing($1, $2); }
    | /* empty */                                           { init(); }
 ;

+top_statement_list:
+      top_statement_list_ex
+          { makeNop($nop, $this->lookaheadStartAttributes);
+            if ($nop !== null) { $1[] = $nop; } $$ = $1; }
+;
+
 reserved_non_modifiers:
      T_INCLUDE | T_INCLUDE_ONCE | T_EVAL | T_REQUIRE | T_REQUIRE_ONCE | T_LOGICAL_OR | T_LOGICAL_XOR | T_LOGICAL_AND
    | T_INSTANCEOF | T_NEW | T_CLONE | T_EXIT | T_IF | T_ELSEIF | T_ELSE | T_ENDIF | T_ECHO | T_DO | T_WHILE
@ -21,6 +27,7 @@ reserved_non_modifiers:
    | T_FINALLY | T_THROW | T_USE | T_INSTEADOF | T_GLOBAL | T_VAR | T_UNSET | T_ISSET | T_EMPTY | T_CONTINUE | T_GOTO
    | T_FUNCTION | T_CONST | T_RETURN | T_PRINT | T_YIELD | T_LIST | T_SWITCH | T_ENDSWITCH | T_CASE | T_DEFAULT
    | T_BREAK | T_ARRAY | T_CALLABLE | T_EXTENDS | T_IMPLEMENTS | T_NAMESPACE | T_TRAIT | T_INTERFACE | T_CLASS
+    | T_CLASS_C | T_TRAIT_C | T_FUNC_C | T_METHOD_C | T_LINE | T_FILE | T_DIR | T_NS_C | T_HALT_COMPILER
 ;

 semi_reserved:
@ -123,11 +130,17 @@ class_const:
    identifier '=' expr                                     { $$ = Node\Const_[$1, $3]; }
 ;

-inner_statement_list:
-      inner_statement_list inner_statement                  { pushNormalizing($1, $2); }
+inner_statement_list_ex:
+      inner_statement_list_ex inner_statement               { pushNormalizing($1, $2); }
    | /* empty */                                           { init(); }
 ;

+inner_statement_list:
+      inner_statement_list_ex
+          { makeNop($nop, $this->lookaheadStartAttributes);
+            if ($nop !== null) { $1[] = $nop; } $$ = $1; }
+;
+
 inner_statement:
      statement                                             { $$ = $1; }
    | function_declaration_statement                        { $$ = $1; }
@ -136,7 +149,7 @@ inner_statement:
          { throw new Error('__HALT_COMPILER() can only be used from the outermost scope', attributes()); }
 ;

-statement:
+non_empty_statement:
      '{' inner_statement_list '}'                          { $$ = $2; }
    | T_IF '(' expr ')' statement elseif_list else_single
          { $$ = Stmt\If_[$3, ['stmts' => toArray($5), 'elseifs' => $6, 'else' => $7]]; }
@ -161,7 +174,6 @@ statement:
    | T_FOREACH '(' expr T_AS variable T_DOUBLE_ARROW foreach_variable ')' foreach_statement
          { $$ = Stmt\Foreach_[$3, $7[0], ['keyVar' => $5, 'byRef' => $7[1], 'stmts' => $9]]; }
    | T_DECLARE '(' declare_list ')' declare_statement      { $$ = Stmt\Declare_[$3, $5]; }
-    | ';'                                                   { $$ = array(); /* means: no statement */ }
    | T_TRY '{' inner_statement_list '}' catches optional_finally
          { $$ = Stmt\TryCatch[$3, $5, $6]; }
    | T_THROW expr ';'                                      { $$ = Stmt\Throw_[$2]; }
@ -170,6 +182,13 @@ statement:
    | error                                                 { $$ = array(); /* means: no statement */ }
 ;

+statement:
+      non_empty_statement                                   { $$ = $1; }
+    | ';'
+          { makeNop($$, $this->startAttributeStack[#1]);
+            if ($$ === null) $$ = array(); /* means: no statement */ }
+;
+
 catches:
      /* empty */                                           { init(); }
    | catches catch                                         { push($1, $2); }
@ -251,7 +270,8 @@ foreach_statement:
 ;

 declare_statement:
-      statement                                             { $$ = toArray($1); }
+      non_empty_statement                                   { $$ = toArray($1); }
+    | ';'                                                   { $$ = null; }
    | ':' inner_statement_list T_ENDDECLARE ';'             { $$ = $2; }
 ;

@ -559,7 +579,10 @@ expr:
    | T_OBJECT_CAST expr                                    { $$ = Expr\Cast\Object_ [$2]; }
    | T_BOOL_CAST expr                                      { $$ = Expr\Cast\Bool_   [$2]; }
    | T_UNSET_CAST expr                                     { $$ = Expr\Cast\Unset_  [$2]; }
-    | T_EXIT exit_expr                                      { $$ = Expr\Exit_        [$2]; }
+    | T_EXIT exit_expr
+          { $attrs = attributes();
+            $attrs['kind'] = strtolower($1) === 'exit' ? Expr\Exit_::KIND_EXIT : Expr\Exit_::KIND_DIE;
+            $$ = new Expr\Exit_($2, $attrs); }
    | '@' expr                                              { $$ = Expr\ErrorSuppress[$2]; }
    | scalar                                                { $$ = $1; }
    | '`' backticks_expr '`'                                { $$ = Expr\ShellExec[$2]; }
@ -652,13 +675,19 @@ constant:
 ;

 dereferencable_scalar:
-      T_ARRAY '(' array_pair_list ')'                       { $$ = Expr\Array_[$3]; }
-    | '[' array_pair_list ']'                               { $$ = Expr\Array_[$2]; }
-    | T_CONSTANT_ENCAPSED_STRING                            { $$ = Scalar\String_[Scalar\String_::parse($1)]; }
+      T_ARRAY '(' array_pair_list ')'
+          { $attrs = attributes(); $attrs['kind'] = Expr\Array_::KIND_LONG;
+            $$ = new Expr\Array_($3, $attrs); }
+    | '[' array_pair_list ']'
+          { $attrs = attributes(); $attrs['kind'] = Expr\Array_::KIND_SHORT;
+            $$ = new Expr\Array_($2, $attrs); }
+    | T_CONSTANT_ENCAPSED_STRING
+          { $attrs = attributes(); $attrs['kind'] = strKind($1);
+            $$ = new Scalar\String_(Scalar\String_::parse($1), $attrs); }
 ;

 scalar:
-      T_LNUMBER                                             { $$ = Scalar\LNumber[Scalar\LNumber::parse($1)]; }
+      T_LNUMBER                                             { $$ = Scalar\LNumber::fromString($1, attributes()); }
    | T_DNUMBER                                             { $$ = Scalar\DNumber[Scalar\DNumber::parse($1)]; }
    | T_LINE                                                { $$ = Scalar\MagicConst\Line[]; }
    | T_FILE                                                { $$ = Scalar\MagicConst\File[]; }
@ -671,13 +700,17 @@ scalar:
    | dereferencable_scalar                                 { $$ = $1; }
    | constant                                              { $$ = $1; }
    | T_START_HEREDOC T_ENCAPSED_AND_WHITESPACE T_END_HEREDOC
-          { $$ = Scalar\String_[Scalar\String_::parseDocString($1, $2)]; }
+          { $attrs = attributes(); setDocStringAttrs($attrs, $1);
+            $$ = new Scalar\String_(Scalar\String_::parseDocString($1, $2), $attrs); }
    | T_START_HEREDOC T_END_HEREDOC
-          { $$ = Scalar\String_['']; }
+          { $attrs = attributes(); setDocStringAttrs($attrs, $1);
+            $$ = new Scalar\String_('', $attrs); }
    | '"' encaps_list '"'
-          { parseEncapsed($2, '"', true); $$ = Scalar\Encapsed[$2]; }
+          { $attrs = attributes(); $attrs['kind'] = Scalar\String_::KIND_DOUBLE_QUOTED;
+            parseEncapsed($2, '"', true); $$ = new Scalar\Encapsed($2, $attrs); }
    | T_START_HEREDOC encaps_list T_END_HEREDOC
-          { parseEncapsedDoc($2, true); $$ = Scalar\Encapsed[$2]; }
+          { $attrs = attributes(); setDocStringAttrs($attrs, $1);
+            parseEncapsedDoc($2, true); $$ = new Scalar\Encapsed($2, $attrs); }
 ;

 optional_comma:
--- a/grammar/rebuildParsers.php
+++ b/grammar/rebuildParsers.php
@ -85,7 +85,7 @@ foreach ($grammarFileToName as $grammarFile => $name) {

 function resolveNodes($code) {
    return preg_replace_callback(
-        '~(?<name>[A-Z][a-zA-Z_\\\\]++)\s*' . PARAMS . '~',
+        '~\b(?<name>[A-Z][a-zA-Z_\\\\]++)\s*' . PARAMS . '~',
        function($matches) {
            // recurse
            $matches['params'] = resolveNodes($matches['params']);
@ -169,6 +169,32 @@ function resolveMacros($code) {
                     . ' if (\'\' === $s->value) array_pop(' . $args[0] . ');';
            }

+            if ('makeNop' == $name) {
+                assertArgs(2, $args, $name);
+
+                return '$startAttributes = ' . $args[1] . ';'
+                . ' if (isset($startAttributes[\'comments\']))'
+                . ' { ' . $args[0] . ' = new Stmt\Nop([\'comments\' => $startAttributes[\'comments\']]); }'
+                . ' else { ' . $args[0] . ' = null; }';
+            }
+
+            if ('strKind' == $name) {
+                assertArgs(1, $args, $name);
+
+                return '(' . $args[0] . '[0] === "\'" || (' . $args[0] . '[1] === "\'" && '
+                     . '(' . $args[0] . '[0] === \'b\' || ' . $args[0] . '[0] === \'B\')) '
+                     . '? Scalar\String_::KIND_SINGLE_QUOTED : Scalar\String_::KIND_DOUBLE_QUOTED)';
+            }
+
+            if ('setDocStringAttrs' == $name) {
+                assertArgs(2, $args, $name);
+
+                return $args[0] . '[\'kind\'] = strpos(' . $args[1] . ', "\'") === false '
+                     . '? Scalar\String_::KIND_HEREDOC : Scalar\String_::KIND_NOWDOC; '
+                     . 'preg_match(\'/\A[bB]?<<<[ \t]*[\\\'"]?([a-zA-Z_\x7f-\xff][a-zA-Z0-9_\x7f-\xff]*)[\\\'"]?(?:\r\n|\n|\r)\z/\', ' . $args[1] . ', $matches); '
+                     . $args[0] . '[\'docLabel\'] = $matches[1];';
+            }
+
            return $matches[0];
        },
        $code
--- a/lib/PhpParser/Autoloader.php
+++ b/lib/PhpParser/Autoloader.php
@ -31,7 +31,7 @@ class Autoloader
     */
    static public function autoload($class) {
        if (0 === strpos($class, 'PhpParser\\')) {
-            $fileName = dirname(__DIR__) . '/' . strtr($class, '\\', '/') . '.php';
+            $fileName = __DIR__ . strtr(substr($class, 9), '\\', '/') . '.php';
            if (file_exists($fileName)) {
                require $fileName;
            }
--- a/lib/PhpParser/Builder/Class_.php
+++ b/lib/PhpParser/Builder/Class_.php
@ -44,8 +44,7 @@ class Class_ extends Declaration
    /**
     * Implements one or more interfaces.
     *
-     * @param Name|string $interface Name of interface to implement
-     * @param Name|string $...       More interfaces to implement
+     * @param Name|string ...$interfaces Names of interfaces to implement
     *
     * @return $this The builder instance (for fluid interface)
     */
--- a/lib/PhpParser/Builder/FunctionLike.php
+++ b/lib/PhpParser/Builder/FunctionLike.php
@ -10,6 +10,7 @@ abstract class FunctionLike extends Declaration
 {
    protected $returnByRef = false;
    protected $params = array();
+    protected $returnType = null;

    /**
     * Make the function return by reference.
@ -55,4 +56,23 @@ abstract class FunctionLike extends Declaration

        return $this;
    }
-}
+
+    /**
+     * Sets the return type for PHP 7.
+     *
+     * @param string|Node\Name $type One of array, callable, string, int, float, bool,
+     *                               or a class/interface name.
+     *
+     * @return $this The builder instance (for fluid interface)
+     */
+    public function setReturnType($type)
+    {
+        if (in_array($type, array('array', 'callable', 'string', 'int', 'float', 'bool'))) {
+            $this->returnType = $type;
+        } else {
+            $this->returnType = $this->normalizeName($type);
+        }
+
+        return $this;
+    }
+}
--- a/lib/PhpParser/Builder/Function_.php
+++ b/lib/PhpParser/Builder/Function_.php
@ -40,9 +40,10 @@ class Function_ extends FunctionLike
     */
    public function getNode() {
        return new Stmt\Function_($this->name, array(
-            'byRef'  => $this->returnByRef,
-            'params' => $this->params,
-            'stmts'  => $this->stmts,
+            'byRef'      => $this->returnByRef,
+            'params'     => $this->params,
+            'returnType' => $this->returnType,
+            'stmts'      => $this->stmts,
        ), $this->attributes);
    }
-}
+}
--- a/lib/PhpParser/Builder/Interface_.php
+++ b/lib/PhpParser/Builder/Interface_.php
@ -25,8 +25,7 @@ class Interface_ extends Declaration
    /**
     * Extends one or more interfaces.
     *
-     * @param Name|string $interface Name of interface to extend
-     * @param Name|string $...       More interfaces to extend
+     * @param Name|string ...$interfaces Names of interfaces to extend
     *
     * @return $this The builder instance (for fluid interface)
     */
--- a/lib/PhpParser/Builder/Method.php
+++ b/lib/PhpParser/Builder/Method.php
@ -116,10 +116,11 @@ class Method extends FunctionLike
     */
    public function getNode() {
        return new Stmt\ClassMethod($this->name, array(
-            'type'   => $this->type,
-            'byRef'  => $this->returnByRef,
-            'params' => $this->params,
-            'stmts'  => $this->stmts,
+            'type'       => $this->type,
+            'byRef'      => $this->returnByRef,
+            'params'     => $this->params,
+            'returnType' => $this->returnType,
+            'stmts'      => $this->stmts,
        ), $this->attributes);
    }
 }
--- a/lib/PhpParser/Builder/Param.php
+++ b/lib/PhpParser/Builder/Param.php
@ -43,7 +43,7 @@ class Param extends PhpParser\BuilderAbstract
     * @return $this The builder instance (for fluid interface)
     */
    public function setTypeHint($type) {
-        if ($type === 'array' || $type === 'callable') {
+        if (in_array($type, array('array', 'callable', 'string', 'int', 'float', 'bool'))) {
            $this->type = $type;
        } else {
            $this->type = $this->normalizeName($type);
@ -73,4 +73,4 @@ class Param extends PhpParser\BuilderAbstract
            $this->name, $this->default, $this->type, $this->byRef
        );
    }
-}
+}
--- a/lib/PhpParser/BuilderFactory.php
+++ b/lib/PhpParser/BuilderFactory.php
@ -20,7 +20,7 @@ class BuilderFactory
 {
    /**
     * Creates a namespace builder.
-     * 
+     *
     * @param null|string|Node\Name $name Name of the namespace
     *
     * @return Builder\Namespace_ The created namespace builder
--- a/lib/PhpParser/Comment.php
+++ b/lib/PhpParser/Comment.php
@ -6,16 +6,19 @@ class Comment
 {
    protected $text;
    protected $line;
+    protected $filePos;

    /**
     * Constructs a comment node.
     *
-     * @param string $text Comment text (including comment delimiters like /*)
-     * @param int    $line Line number the comment started on
+     * @param string $text         Comment text (including comment delimiters like /*)
+     * @param int    $startLine    Line number the comment started on
+     * @param int    $startFilePos File offset the comment started on
     */
-    public function __construct($text, $line = -1) {
+    public function __construct($text, $startLine = -1, $startFilePos = -1) {
        $this->text = $text;
-        $this->line = $line;
+        $this->line = $startLine;
+        $this->filePos = $startFilePos;
    }

    /**
@ -31,6 +34,8 @@ class Comment
     * Sets the comment text.
     *
     * @param string $text The comment text (including comment delimiters like /*)
+     *
+     * @deprecated Construct a new comment instead
     */
    public function setText($text) {
        $this->text = $text;
@ -49,11 +54,22 @@ class Comment
     * Sets the line number the comment started on.
     *
     * @param int $line Line number
+     *
+     * @deprecated Construct a new comment instead
     */
    public function setLine($line) {
        $this->line = $line;
    }

+    /**
+     * Gets the file offset the comment started on.
+     *
+     * @return int File offset
+     */
+    public function getFilePos() {
+        return $this->filePos;
+    }
+
    /**
     * Gets the comment text.
     *
@ -75,7 +91,8 @@ class Comment
     */
    public function getReformattedText() {
        $text = trim($this->text);
-        if (false === strpos($text, "\n")) {
+        $newlinePos = strpos($text, "\n");
+        if (false === $newlinePos) {
            // Single line comments don't need further processing
            return $text;
        } elseif (preg_match('((*BSR_ANYCRLF)(*ANYCRLF)^.*(?:\R\s+\*.*)+$)', $text)) {
@ -105,15 +122,30 @@ class Comment
            //
            //     /* Some text.
            //        Some more text.
+            //          Indented text.
            //        Even more text. */
            //
-            // is handled by taking the length of the "/* " segment and leaving only that
-            // many space characters before the lines. Thus in the above example only three
-            // space characters are left at the start of every line.
-            return preg_replace('(^\s*(?= {' . strlen($matches[0]) . '}(?!\s)))m', '', $text);
+            // is handled by removing the difference between the shortest whitespace prefix on all
+            // lines and the length of the "/* " opening sequence.
+            $prefixLen = $this->getShortestWhitespacePrefixLen(substr($text, $newlinePos + 1));
+            $removeLen = $prefixLen - strlen($matches[0]);
+            return preg_replace('(^\s{' . $removeLen . '})m', '', $text);
        }

        // No idea how to format this comment, so simply return as is
        return $text;
    }
+
+    private function getShortestWhitespacePrefixLen($str) {
+        $lines = explode("\n", $str);
+        $shortestPrefixLen = INF;
+        foreach ($lines as $line) {
+            preg_match('(^\s*)', $line, $matches);
+            $prefixLen = strlen($matches[0]);
+            if ($prefixLen < $shortestPrefixLen) {
+                $shortestPrefixLen = $prefixLen;
+            }
+        }
+        return $shortestPrefixLen;
+    }
 }
--- a/lib/PhpParser/Lexer.php
+++ b/lib/PhpParser/Lexer.php
@ -2,6 +2,7 @@

 namespace PhpParser;

+use PhpParser\Node\Scalar\LNumber;
 use PhpParser\Parser\Tokens;

 class Lexer
@ -21,10 +22,10 @@ class Lexer
     * Creates a Lexer.
     *
     * @param array $options Options array. Currently only the 'usedAttributes' option is supported,
-     *                       which is an array of attributes to add to the AST nodes. Possible attributes
-     *                       are: 'comments', 'startLine', 'endLine', 'startTokenPos', 'endTokenPos',
-     *                       'startFilePos', 'endFilePos'. The option defaults to the first three.
-     *                       For more info see getNextToken() docs.
+     *                       which is an array of attributes to add to the AST nodes. Possible
+     *                       attributes are: 'comments', 'startLine', 'endLine', 'startTokenPos',
+     *                       'endTokenPos', 'startFilePos', 'endFilePos'. The option defaults to the
+     *                       first three. For more info see getNextToken() docs.
     */
    public function __construct(array $options = array()) {
        // map from internal tokens to PhpParser tokens
@ -32,7 +33,9 @@ class Lexer

        // map of tokens to drop while lexing (the map is only used for isset lookup,
        // that's why the value is simply set to 1; the value is never actually used.)
-        $this->dropTokens = array_fill_keys(array(T_WHITESPACE, T_OPEN_TAG), 1);
+        $this->dropTokens = array_fill_keys(
+            array(T_WHITESPACE, T_OPEN_TAG, T_COMMENT, T_DOC_COMMENT), 1
+        );

        // the usedAttributes member is a map of the used attribute names to a dummy
        // value (here "true")
@ -120,7 +123,7 @@ class Lexer
     *  * 'startTokenPos' => Offset into the token array of the first token in the node.
     *  * 'endTokenPos'   => Offset into the token array of the last token in the node.
     *  * 'startFilePos'  => Offset into the code string of the first character that is part of the node.
-     *  * 'endFilePos'    => Offset into the code string of the last character that is part of the node
+     *  * 'endFilePos'    => Offset into the code string of the last character that is part of the node.
     *
     * @param mixed $value           Variable to store token content in
     * @param mixed $startAttributes Variable to store start attributes in
@ -140,6 +143,9 @@ class Lexer
                $token = "\0";
            }

+            if (isset($this->usedAttributes['startLine'])) {
+                $startAttributes['startLine'] = $this->line;
+            }
            if (isset($this->usedAttributes['startTokenPos'])) {
                $startAttributes['startTokenPos'] = $this->pos;
            }
@ -147,63 +153,48 @@ class Lexer
                $startAttributes['startFilePos'] = $this->filePos;
            }

-            if (is_string($token)) {
-                // bug in token_get_all
-                if ('b"' === $token) {
-                    $value = 'b"';
+            if (\is_string($token)) {
+                $value = $token;
+                if (isset($token[1])) {
+                    // bug in token_get_all
                    $this->filePos += 2;
                    $id = ord('"');
                } else {
-                    $value = $token;
                    $this->filePos += 1;
                    $id = ord($token);
                }
+            } elseif (!isset($this->dropTokens[$token[0]])) {
+                $value = $token[1];
+                $id = $this->tokenMap[$token[0]];

-                if (isset($this->usedAttributes['startLine'])) {
-                    $startAttributes['startLine'] = $this->line;
-                }
-                if (isset($this->usedAttributes['endLine'])) {
-                    $endAttributes['endLine'] = $this->line;
-                }
-                if (isset($this->usedAttributes['endTokenPos'])) {
-                    $endAttributes['endTokenPos'] = $this->pos;
-                }
-                if (isset($this->usedAttributes['endFilePos'])) {
-                    $endAttributes['endFilePos'] = $this->filePos - 1;
-                }
-
-                return $id;
+                $this->line += substr_count($value, "\n");
+                $this->filePos += \strlen($value);
            } else {
-                $this->line += substr_count($token[1], "\n");
-                $this->filePos += strlen($token[1]);
-
-                if (T_COMMENT === $token[0]) {
+                if (T_COMMENT === $token[0] || T_DOC_COMMENT === $token[0]) {
                    if (isset($this->usedAttributes['comments'])) {
-                        $startAttributes['comments'][] = new Comment($token[1], $token[2]);
+                        $comment = T_DOC_COMMENT === $token[0]
+                            ? new Comment\Doc($token[1], $this->line, $this->filePos)
+                            : new Comment($token[1], $this->line, $this->filePos);
+                        $startAttributes['comments'][] = $comment;
                    }
-                } elseif (T_DOC_COMMENT === $token[0]) {
-                    if (isset($this->usedAttributes['comments'])) {
-                        $startAttributes['comments'][] = new Comment\Doc($token[1], $token[2]);
-                    }
-                } elseif (!isset($this->dropTokens[$token[0]])) {
-                    $value = $token[1];
-
-                    if (isset($this->usedAttributes['startLine'])) {
-                        $startAttributes['startLine'] = $token[2];
-                    }
-                    if (isset($this->usedAttributes['endLine'])) {
-                        $endAttributes['endLine'] = $this->line;
-                    }
-                    if (isset($this->usedAttributes['endTokenPos'])) {
-                        $endAttributes['endTokenPos'] = $this->pos;
-                    }
-                    if (isset($this->usedAttributes['endFilePos'])) {
-                        $endAttributes['endFilePos'] = $this->filePos - 1;
-                    }
-
-                    return $this->tokenMap[$token[0]];
                }
+
+                $this->line += substr_count($token[1], "\n");
+                $this->filePos += \strlen($token[1]);
+                continue;
            }
+
+            if (isset($this->usedAttributes['endLine'])) {
+                $endAttributes['endLine'] = $this->line;
+            }
+            if (isset($this->usedAttributes['endTokenPos'])) {
+                $endAttributes['endTokenPos'] = $this->pos;
+            }
+            if (isset($this->usedAttributes['endFilePos'])) {
+                $endAttributes['endFilePos'] = $this->filePos - 1;
+            }
+
+            return $id;
        }

        throw new \RuntimeException('Reached end of lexer loop');
--- a/lib/PhpParser/Node/Expr/Array_.php
+++ b/lib/PhpParser/Node/Expr/Array_.php
@ -6,6 +6,10 @@ use PhpParser\Node\Expr;

 class Array_ extends Expr
 {
+    // For use in "kind" attribute
+    const KIND_LONG = 1;  // array() syntax
+    const KIND_SHORT = 2; // [] syntax
+
    /** @var ArrayItem[] Items */
    public $items;

--- a/lib/PhpParser/Node/Expr/AssignOp.php
+++ b/lib/PhpParser/Node/Expr/AssignOp.php
@ -4,10 +4,6 @@ namespace PhpParser\Node\Expr;

 use PhpParser\Node\Expr;

-/**
- * @property Expr $var  Variable
- * @property Expr $expr Expression
- */
 abstract class AssignOp extends Expr
 {
    /** @var Expr Variable */
--- a/lib/PhpParser/Node/Expr/AssignRef.php
+++ b/lib/PhpParser/Node/Expr/AssignRef.php
@ -4,10 +4,6 @@ namespace PhpParser\Node\Expr;

 use PhpParser\Node\Expr;

-/**
- * @property Expr $var  Variable reference is assigned to
- * @property Expr $expr Variable which is referenced
- */
 class AssignRef extends Expr
 {
    /** @var Expr Variable reference is assigned to */
--- a/lib/PhpParser/Node/Expr/Exit_.php
+++ b/lib/PhpParser/Node/Expr/Exit_.php
@ -6,6 +6,10 @@ use PhpParser\Node\Expr;

 class Exit_ extends Expr
 {
+    /* For use in "kind" attribute */
+    const KIND_EXIT = 1;
+    const KIND_DIE = 2;
+
    /** @var null|Expr Expression */
    public $expr;

--- a/lib/PhpParser/Node/Scalar/LNumber.php
+++ b/lib/PhpParser/Node/Scalar/LNumber.php
@ -2,10 +2,17 @@

 namespace PhpParser\Node\Scalar;

+use PhpParser\Error;
 use PhpParser\Node\Scalar;

 class LNumber extends Scalar
 {
+    /* For use in "kind" attribute */
+    const KIND_BIN = 2;
+    const KIND_OCT = 8;
+    const KIND_DEC = 10;
+    const KIND_HEX = 16;
+
    /** @var int Number value */
    public $value;

@ -25,37 +32,36 @@ class LNumber extends Scalar
    }

    /**
-     * @internal
+     * Constructs an LNumber node from a string number literal.
     *
-     * Parses an LNUMBER token (dec, hex, oct and bin notations) like PHP would.
+     * @param string $str               String number literal (decimal, octal, hex or binary)
+     * @param array  $attributes        Additional attributes
+     * @param bool   $allowInvalidOctal Whether to allow invalid octal numbers (PHP 5)
     *
-     * @param string $str A string number
-     *
-     * @return int The parsed number
+     * @return LNumber The constructed LNumber, including kind attribute
     */
-    public static function parse($str) {
-        // handle plain 0 specially
-        if ('0' === $str) {
-            return 0;
+    public static function fromString($str, array $attributes = array(), $allowInvalidOctal = false) {
+        if ('0' !== $str[0] || '0' === $str) {
+            $attributes['kind'] = LNumber::KIND_DEC;
+            return new LNumber((int) $str, $attributes);
        }

-        // if first char is 0 (and number isn't 0) it's a special syntax
-        if ('0' === $str[0]) {
-            // hex
-            if ('x' === $str[1] || 'X' === $str[1]) {
-                return hexdec($str);
-            }
-
-            // bin
-            if ('b' === $str[1] || 'B' === $str[1]) {
-                return bindec($str);
-            }
-
-            // oct (intval instead of octdec to get proper cutting behavior with malformed numbers)
-            return intval($str, 8);
+        if ('x' === $str[1] || 'X' === $str[1]) {
+            $attributes['kind'] = LNumber::KIND_HEX;
+            return new LNumber(hexdec($str), $attributes);
        }

-        // dec
-        return (int) $str;
+        if ('b' === $str[1] || 'B' === $str[1]) {
+            $attributes['kind'] = LNumber::KIND_BIN;
+            return new LNumber(bindec($str), $attributes);
+        }
+
+        if (!$allowInvalidOctal && strpbrk($str, '89')) {
+            throw new Error('Invalid numeric literal', $attributes);
+        }
+
+        // use intval instead of octdec to get proper cutting behavior with malformed numbers
+        $attributes['kind'] = LNumber::KIND_OCT;
+        return new LNumber(intval($str, 8), $attributes);
    }
 }
--- a/lib/PhpParser/Node/Scalar/String_.php
+++ b/lib/PhpParser/Node/Scalar/String_.php
@ -7,6 +7,12 @@ use PhpParser\Node\Scalar;

 class String_ extends Scalar
 {
+    /* For use in "kind" attribute */
+    const KIND_SINGLE_QUOTED = 1;
+    const KIND_DOUBLE_QUOTED = 2;
+    const KIND_HEREDOC = 3;
+    const KIND_NOWDOC = 4;
+
    /** @var string String value */
    public $value;

@ -48,7 +54,7 @@ class String_ extends Scalar
     */
    public static function parse($str, $parseUnicodeEscape = true) {
        $bLength = 0;
-        if ('b' === $str[0]) {
+        if ('b' === $str[0] || 'B' === $str[0]) {
            $bLength = 1;
        }

--- a/lib/PhpParser/Node/Stmt/Declare_.php
+++ b/lib/PhpParser/Node/Stmt/Declare_.php
@ -14,10 +14,10 @@ class Declare_ extends Node\Stmt
     * Constructs a declare node.
     *
     * @param DeclareDeclare[] $declares   List of declares
-     * @param Node[]           $stmts      Statements
+     * @param Node[]|null      $stmts      Statements
     * @param array            $attributes Additional attributes
     */
-    public function __construct(array $declares, array $stmts, array $attributes = array()) {
+    public function __construct(array $declares, array $stmts = null, array $attributes = array()) {
        parent::__construct($attributes);
        $this->declares = $declares;
        $this->stmts = $stmts;
--- a/lib/PhpParser/Node/Stmt/Nop.php
+++ b/lib/PhpParser/Node/Stmt/Nop.php
@ -0,0 +1,13 @@
+<?php
+
+namespace PhpParser\Node\Stmt;
+
+use PhpParser\Node;
+
+/** Nop/empty statement (;). */
+class Nop extends Node\Stmt
+{
+    public function getSubNodeNames() {
+        return array();
+    }
+}
--- a/lib/PhpParser/NodeDumper.php
+++ b/lib/PhpParser/NodeDumper.php
@ -4,6 +4,18 @@ namespace PhpParser;

 class NodeDumper
 {
+    private $dumpComments;
+
+    /**
+     * Constructs a NodeDumper.
+     *
+     * @param array $options Boolean option 'dumpComments' controls whether comments should be
+     *                       dumped
+     */
+    public function __construct(array $options = []) {
+        $this->dumpComments = !empty($options['dumpComments']);
+    }
+
    /**
     * Dumps a node or array.
     *
@ -31,6 +43,10 @@ class NodeDumper
                    $r .= str_replace("\n", "\n    ", $this->dump($value));
                }
            }
+
+            if ($this->dumpComments && $comments = $node->getAttribute('comments')) {
+                $r .= "\n    comments: " . str_replace("\n", "\n    ", $this->dump($comments));
+            }
        } elseif (is_array($node)) {
            $r = 'array(';

@ -49,6 +65,8 @@ class NodeDumper
                    $r .= str_replace("\n", "\n    ", $this->dump($value));
                }
            }
+        } elseif ($node instanceof Comment) {
+            return $node->getReformattedText();
        } else {
            throw new \InvalidArgumentException('Can only dump nodes and arrays.');
        }
--- a/lib/PhpParser/NodeTraverser.php
+++ b/lib/PhpParser/NodeTraverser.php
@ -99,6 +99,12 @@ class NodeTraverser implements NodeTraverserInterface

                foreach ($this->visitors as $visitor) {
                    if (null !== $return = $visitor->leaveNode($subNode)) {
+                        if (is_array($return)) {
+                            throw new \LogicException(
+                                'leaveNode() may only return an array ' .
+                                'if the parent structure is an array'
+                            );
+                        }
                        $subNode = $return;
                    }
                }
--- a/lib/PhpParser/Parser/Php5.php
+++ b/lib/PhpParser/Parser/Php5.php
--- a/lib/PhpParser/Parser/Php7.php
+++ b/lib/PhpParser/Parser/Php7.php
--- a/lib/PhpParser/ParserAbstract.php
+++ b/lib/PhpParser/ParserAbstract.php
@ -88,6 +88,8 @@ abstract class ParserAbstract implements Parser
    protected $startAttributeStack;
    /** @var array End attributes of last *shifted* token */
    protected $endAttributes;
+    /** @var array Start attributes of last *read* token */
+    protected $lookaheadStartAttributes;

    /** @var bool Whether to throw on first error */
    protected $throwOnError;
@ -185,6 +187,7 @@ abstract class ParserAbstract implements Parser
                    // This is necessary to assign some meaningful attributes to /* empty */ productions. They'll get
                    // the attributes of the next token, even though they don't contain it themselves.
                    $this->startAttributeStack[$this->stackPos+1] = $startAttributes;
+                    $this->lookaheadStartAttributes = $startAttributes;

                    //$this->traceRead($symbol);
                }
@ -446,7 +449,7 @@ abstract class ParserAbstract implements Parser
    private function getNamespacingStyle(array $stmts) {
        $style = null;
        $hasNotAllowedStmts = false;
-        foreach ($stmts as $stmt) {
+        foreach ($stmts as $i => $stmt) {
            if ($stmt instanceof Node\Stmt\Namespace_) {
                $currentStyle = null === $stmt->stmts ? 'semicolon' : 'brace';
                if (null === $style) {
@ -457,9 +460,23 @@ abstract class ParserAbstract implements Parser
                } elseif ($style !== $currentStyle) {
                    throw new Error('Cannot mix bracketed namespace declarations with unbracketed namespace declarations', $stmt->getLine());
                }
-            } elseif (!$stmt instanceof Node\Stmt\Declare_ && !$stmt instanceof Node\Stmt\HaltCompiler) {
-                $hasNotAllowedStmts = true;
+                continue;
            }
+
+            /* declare(), __halt_compiler() and nops can be used before a namespace declaration */
+            if ($stmt instanceof Node\Stmt\Declare_
+                || $stmt instanceof Node\Stmt\HaltCompiler
+                || $stmt instanceof Node\Stmt\Nop) {
+                continue;
+            }
+
+            /* There may be a hashbang line at the very start of the file */
+            if ($i == 0 && $stmt instanceof Node\Stmt\InlineHTML && preg_match('/\A#!.*\r?\n\z/', $stmt->value)) {
+                continue;
+            }
+
+            /* Everything else if forbidden before namespace declarations */
+            $hasNotAllowedStmts = true;
        }
        return $style;
    }
--- a/lib/PhpParser/PrettyPrinter/Standard.php
+++ b/lib/PhpParser/PrettyPrinter/Standard.php
@ -84,15 +84,71 @@ class Standard extends PrettyPrinterAbstract
    // Scalars

    public function pScalar_String(Scalar\String_ $node) {
-        return '\'' . $this->pNoIndent(addcslashes($node->value, '\'\\')) . '\'';
+        $kind = $node->getAttribute('kind', Scalar\String_::KIND_SINGLE_QUOTED);
+        switch ($kind) {
+            case Scalar\String_::KIND_NOWDOC:
+                $label = $node->getAttribute('docLabel');
+                if ($label && !$this->containsEndLabel($node->value, $label)) {
+                    if ($node->value === '') {
+                        return $this->pNoIndent("<<<'$label'\n$label") . $this->docStringEndToken;
+                    }
+
+                    return $this->pNoIndent("<<<'$label'\n$node->value\n$label")
+                         . $this->docStringEndToken;
+                }
+                /* break missing intentionally */
+            case Scalar\String_::KIND_SINGLE_QUOTED:
+                return '\'' . $this->pNoIndent(addcslashes($node->value, '\'\\')) . '\'';
+            case Scalar\String_::KIND_HEREDOC:
+                $label = $node->getAttribute('docLabel');
+                if ($label && !$this->containsEndLabel($node->value, $label)) {
+                    if ($node->value === '') {
+                        return $this->pNoIndent("<<<$label\n$label") . $this->docStringEndToken;
+                    }
+
+                    $escaped = $this->escapeString($node->value, null);
+                    return $this->pNoIndent("<<<$label\n" . $escaped ."\n$label")
+                         . $this->docStringEndToken;
+                }
+            /* break missing intentionally */
+            case Scalar\String_::KIND_DOUBLE_QUOTED:
+                return '"' . $this->escapeString($node->value, '"') . '"';
+        }
+        throw new \Exception('Invalid string kind');
    }

    public function pScalar_Encapsed(Scalar\Encapsed $node) {
+        if ($node->getAttribute('kind') === Scalar\String_::KIND_HEREDOC) {
+            $label = $node->getAttribute('docLabel');
+            if ($label && !$this->encapsedContainsEndLabel($node->parts, $label)) {
+                if (count($node->parts) === 1
+                    && $node->parts[0] instanceof Scalar\EncapsedStringPart
+                    && $node->parts[0]->value === ''
+                ) {
+                    return $this->pNoIndent("<<<$label\n$label") . $this->docStringEndToken;
+                }
+
+                return $this->pNoIndent(
+                    "<<<$label\n" . $this->pEncapsList($node->parts, null) . "\n$label"
+                ) . $this->docStringEndToken;
+            }
+        }
        return '"' . $this->pEncapsList($node->parts, '"') . '"';
    }

    public function pScalar_LNumber(Scalar\LNumber $node) {
-        return (string) $node->value;
+        $str = (string) $node->value;
+        switch ($node->getAttribute('kind', Scalar\LNumber::KIND_DEC)) {
+            case Scalar\LNumber::KIND_BIN:
+                return '0b' . base_convert($str, 10, 2);
+            case Scalar\LNumber::KIND_OCT:
+                return '0' . base_convert($str, 10, 8);
+            case Scalar\LNumber::KIND_DEC:
+                return $str;
+            case Scalar\LNumber::KIND_HEX:
+                return '0x' . base_convert($str, 10, 16);
+        }
+        throw new \Exception('Invalid number kind');
    }

    public function pScalar_DNumber(Scalar\DNumber $node) {
@ -422,7 +478,9 @@ class Standard extends PrettyPrinterAbstract
    }

    public function pExpr_Array(Expr\Array_ $node) {
-        if ($this->options['shortArraySyntax']) {
+        $syntax = $node->getAttribute('kind',
+            $this->options['shortArraySyntax'] ? Expr\Array_::KIND_SHORT : Expr\Array_::KIND_LONG);
+        if ($syntax === Expr\Array_::KIND_SHORT) {
            return '[' . $this->pCommaSeparated($node->items) . ']';
        } else {
            return 'array(' . $this->pCommaSeparated($node->items) . ')';
@ -493,7 +551,9 @@ class Standard extends PrettyPrinterAbstract
    }

    public function pExpr_Exit(Expr\Exit_ $node) {
-        return 'die' . (null !== $node->expr ? '(' . $this->p($node->expr) . ')' : '');
+        $kind = $node->getAttribute('kind', Expr\Exit_::KIND_DIE);
+        return ($kind === Expr\Exit_::KIND_EXIT ? 'exit' : 'die')
+             . (null !== $node->expr ? '(' . $this->p($node->expr) . ')' : '');
    }

    public function pExpr_Yield(Expr\Yield_ $node) {
@ -609,12 +669,12 @@ class Standard extends PrettyPrinterAbstract
    }

    public function pStmt_Declare(Stmt\Declare_ $node) {
-        return 'declare (' . $this->pCommaSeparated($node->declares) . ') {'
-             . $this->pStmts($node->stmts) . "\n" . '}';
+        return 'declare (' . $this->pCommaSeparated($node->declares) . ')'
+             . (null !== $node->stmts ? ' {' . $this->pStmts($node->stmts) . "\n" . '}' : ';');
    }

    public function pStmt_DeclareDeclare(Stmt\DeclareDeclare $node) {
-        return $node->key . ' = ' . $this->p($node->value);
+        return $node->key . '=' . $this->p($node->value);
    }

    // Control flow
@ -738,6 +798,10 @@ class Standard extends PrettyPrinterAbstract
        return '__halt_compiler();' . $node->remaining;
    }

+    public function pStmt_Nop(Stmt\Nop $node) {
+        return '';
+    }
+
    // Helpers

    protected function pType($node) {
@ -773,7 +837,7 @@ class Standard extends PrettyPrinterAbstract
        $return = '';
        foreach ($encapsList as $element) {
            if ($element instanceof Scalar\EncapsedStringPart) {
-                $return .= addcslashes($element->value, "\n\r\t\f\v$" . $quote . "\\");
+                $return .= $this->escapeString($element->value, $quote);
            } else {
                $return .= '{' . $this->p($element) . '}';
            }
@ -782,6 +846,34 @@ class Standard extends PrettyPrinterAbstract
        return $return;
    }

+    protected function escapeString($string, $quote) {
+        if (null === $quote) {
+            // For doc strings, don't escape newlines
+            return addcslashes($string, "\t\f\v$\\");
+        }
+        return addcslashes($string, "\n\r\t\f\v$" . $quote . "\\");
+    }
+
+    protected function containsEndLabel($string, $label, $atStart = true, $atEnd = true) {
+        $start = $atStart ? '(?:^|[\r\n])' : '[\r\n]';
+        $end = $atEnd ? '(?:$|[;\r\n])' : '[;\r\n]';
+        return false !== strpos($string, $label)
+            && preg_match('/' . $start . $label . $end . '/', $string);
+    }
+
+    protected function encapsedContainsEndLabel(array $parts, $label) {
+        foreach ($parts as $i => $part) {
+            $atStart = $i === 0;
+            $atEnd = $i === count($parts) - 1;
+            if ($part instanceof Scalar\EncapsedStringPart
+                && $this->containsEndLabel($part->value, $label, $atStart, $atEnd)
+            ) {
+                return true;
+            }
+        }
+        return false;
+    }
+
    protected function pDereferenceLhs(Node $node) {
        if ($node instanceof Expr\Variable
            || $node instanceof Name
--- a/lib/PhpParser/PrettyPrinterAbstract.php
+++ b/lib/PhpParser/PrettyPrinterAbstract.php
@ -75,6 +75,7 @@ abstract class PrettyPrinterAbstract
    );

    protected $noIndentToken;
+    protected $docStringEndToken;
    protected $canUseSemicolonNamespaces;
    protected $options;

@ -82,12 +83,14 @@ abstract class PrettyPrinterAbstract
     * Creates a pretty printer instance using the given options.
     *
     * Supported options:
-     *  * bool $shortArraySyntax = false: Whether to use [] instead of array()
+     *  * bool $shortArraySyntax = false: Whether to use [] instead of array() as the default array
+     *                                    syntax, if the node does not specify a format.
     *
     * @param array $options Dictionary of formatting options
     */
    public function __construct(array $options = []) {
        $this->noIndentToken = '_NO_INDENT_' . mt_rand();
+        $this->docStringEndToken = '_DOC_STRING_END_' . mt_rand();

        $defaultOptions = ['shortArraySyntax' => false];
        $this->options = $options + $defaultOptions;
@ -103,7 +106,7 @@ abstract class PrettyPrinterAbstract
    public function prettyPrint(array $stmts) {
        $this->preprocessNodes($stmts);

-        return ltrim(str_replace("\n" . $this->noIndentToken, "\n", $this->pStmts($stmts, false)));
+        return ltrim($this->handleMagicTokens($this->pStmts($stmts, false)));
    }

    /**
@ -114,7 +117,7 @@ abstract class PrettyPrinterAbstract
     * @return string Pretty printed node
     */
    public function prettyPrintExpr(Expr $node) {
-        return str_replace("\n" . $this->noIndentToken, "\n", $this->p($node));
+        return $this->handleMagicTokens($this->p($node));
    }

    /**
@ -125,13 +128,17 @@ abstract class PrettyPrinterAbstract
     * @return string Pretty printed statements
     */
    public function prettyPrintFile(array $stmts) {
-        $p = rtrim($this->prettyPrint($stmts));
+        if (!$stmts) {
+            return "<?php\n\n";
+        }

-        $p = preg_replace('/^\?>\n?/', '', $p, -1, $count);
-        $p = preg_replace('/<\?php$/', '', $p);
+        $p = "<?php\n\n" . $this->prettyPrint($stmts);

-        if (!$count) {
-            $p = "<?php\n\n" . $p;
+        if ($stmts[0] instanceof Stmt\InlineHTML) {
+            $p = preg_replace('/^<\?php\s+\?>\n?/', '', $p);
+        }
+        if ($stmts[count($stmts) - 1] instanceof Stmt\InlineHTML) {
+            $p = preg_replace('/<\?php$/', '', rtrim($p));
        }

        return $p;
@ -152,6 +159,17 @@ abstract class PrettyPrinterAbstract
        }
    }

+    protected function handleMagicTokens($str) {
+        // Drop no-indent tokens
+        $str = str_replace($this->noIndentToken, '', $str);
+
+        // Replace doc-string-end tokens with nothing or a newline
+        $str = str_replace($this->docStringEndToken . ";\n", ";\n", $str);
+        $str = str_replace($this->docStringEndToken, "\n", $str);
+
+        return $str;
+    }
+
    /**
     * Pretty prints an array of nodes (statements) and indents them optionally.
     *
@ -163,10 +181,15 @@ abstract class PrettyPrinterAbstract
    protected function pStmts(array $nodes, $indent = true) {
        $result = '';
        foreach ($nodes as $node) {
-            $result .= "\n"
-                    . $this->pComments($node->getAttribute('comments', array()))
-                    . $this->p($node)
-                    . ($node instanceof Expr ? ';' : '');
+            $comments = $node->getAttribute('comments', array());
+            if ($comments) {
+                $result .= "\n" . $this->pComments($comments);
+                if ($node instanceof Stmt\Nop) {
+                    continue;
+                }
+            }
+
+            $result .= "\n" . $this->p($node) . ($node instanceof Expr ? ';' : '');
        }

        if ($indent) {
@ -264,19 +287,26 @@ abstract class PrettyPrinterAbstract
     *
     * @param string $string Not to be indented string
     *
-     * @return mixed String marked with $this->noIndentToken's.
+     * @return string String marked with $this->noIndentToken's.
     */
    protected function pNoIndent($string) {
        return str_replace("\n", "\n" . $this->noIndentToken, $string);
    }

+    /**
+     * Prints reformatted text of the passed comments.
+     *
+     * @param Comment[] $comments List of comments
+     *
+     * @return string Reformatted text of comments
+     */
    protected function pComments(array $comments) {
-        $result = '';
+        $formattedComments = [];

        foreach ($comments as $comment) {
-            $result .= $comment->getReformattedText() . "\n";
+            $formattedComments[] = $comment->getReformattedText();
        }

-        return $result;
+        return implode("\n", $formattedComments);
    }
 }
--- a/test/PhpParser/Builder/FunctionTest.php
+++ b/test/PhpParser/Builder/FunctionTest.php
@ -76,6 +76,16 @@ class FunctionTest extends \PHPUnit_Framework_TestCase
        )), $node);
    }

+    public function testReturnType() {
+        $node = $this->createFunctionBuilder('test')
+            ->setReturnType('bool')
+            ->getNode();
+
+        $this->assertEquals(new Stmt\Function_('test', array(
+            'returnType' => 'bool'
+        ), array()), $node);
+    }
+
    /**
     * @expectedException \LogicException
     * @expectedExceptionMessage Expected parameter node, got "Name"
--- a/test/PhpParser/Builder/MethodTest.php
+++ b/test/PhpParser/Builder/MethodTest.php
@ -120,6 +120,15 @@ class MethodTest extends \PHPUnit_Framework_TestCase
        )), $node);
    }

+    public function testReturnType() {
+        $node = $this->createMethodBuilder('test')
+            ->setReturnType('bool')
+            ->getNode();
+        $this->assertEquals(new Stmt\ClassMethod('test', array(
+            'returnType' => 'bool'
+        ), array()), $node);
+    }
+
    /**
     * @expectedException \LogicException
     * @expectedExceptionMessage Cannot add statements to an abstract method
--- a/test/PhpParser/CodeParsingTest.php
+++ b/test/PhpParser/CodeParsingTest.php
@ -13,7 +13,7 @@ class CodeParsingTest extends CodeTestAbstract
     */
    public function testParse($name, $code, $expected, $mode) {
        $lexer = new Lexer\Emulative(array('usedAttributes' => array(
-            'startLine', 'endLine', 'startFilePos', 'endFilePos'
+            'startLine', 'endLine', 'startFilePos', 'endFilePos', 'comments'
        )));
        $parser5 = new Parser\Php5($lexer, array(
            'throwOnError' => false,
@ -47,7 +47,7 @@ class CodeParsingTest extends CodeTestAbstract
        }

        if (null !== $stmts) {
-            $dumper = new NodeDumper;
+            $dumper = new NodeDumper(['dumpComments' => true]);
            $output .= $dumper->dump($stmts);
        }

--- a/test/PhpParser/CodeTestAbstract.php
+++ b/test/PhpParser/CodeTestAbstract.php
@ -13,6 +13,7 @@ abstract class CodeTestAbstract extends \PHPUnit_Framework_TestCase
        foreach ($it as $file) {
            $fileName = realpath($file->getPathname());
            $fileContents = file_get_contents($fileName);
+            $fileContents = canonicalize($fileContents);

            // evaluate @@{expr}@@ expressions
            $fileContents = preg_replace_callback(
@ -24,7 +25,7 @@ abstract class CodeTestAbstract extends \PHPUnit_Framework_TestCase
            );

            // parse sections
-            $parts = array_map('trim', explode('-----', $fileContents));
+            $parts = preg_split("/\n-----(?:\n|$)/", $fileContents);

            // first part is the name
            $name = array_shift($parts) . ' (' . $fileName . ')';
@ -34,7 +35,7 @@ abstract class CodeTestAbstract extends \PHPUnit_Framework_TestCase
            $chunks = array_chunk($parts, 2);
            foreach ($chunks as $i => $chunk) {
                $dataSetName = $shortName . (count($chunks) > 1 ? '#' . $i : '');
-                list($expected, $mode) = $this->extractMode(canonicalize($chunk[1]));
+                list($expected, $mode) = $this->extractMode($chunk[1]);
                $tests[$dataSetName] = array($name, $chunk[0], $expected, $mode);
            }
        }
--- a/test/PhpParser/CommentTest.php
+++ b/test/PhpParser/CommentTest.php
@ -5,11 +5,12 @@ namespace PhpParser;
 class CommentTest extends \PHPUnit_Framework_TestCase
 {
    public function testGetSet() {
-        $comment = new Comment('/* Some comment */', 1);
+        $comment = new Comment('/* Some comment */', 1, 10);

        $this->assertSame('/* Some comment */', $comment->getText());
        $this->assertSame('/* Some comment */', (string) $comment);
        $this->assertSame(1, $comment->getLine());
+        $this->assertSame(10, $comment->getFilePos());

        $comment->setText('/* Some other comment */');
        $comment->setLine(10);
@ -58,6 +59,14 @@ class CommentTest extends \PHPUnit_Framework_TestCase
                '/* Some text.
   More text.
   Even more text. */'
+            ),
+            array(
+                '/* Some text.
+       More text.
+         Indented text. */',
+                '/* Some text.
+   More text.
+     Indented text. */',
            ),
            // invalid comment -> no reformatting
            array(
--- a/test/PhpParser/LexerTest.php
+++ b/test/PhpParser/LexerTest.php
@ -93,7 +93,9 @@ class LexerTest extends \PHPUnit_Framework_TestCase
                        ord('$'), '$',
                        array(
                            'startLine' => 3,
-                            'comments' => array(new Comment\Doc('/** doc' . "\n" . 'comment */', 2))
+                            'comments' => array(
+                                new Comment\Doc('/** doc' . "\n" . 'comment */', 2, 14),
+                            )
                        ),
                        array('endLine' => 3)
                    ),
@ -109,10 +111,10 @@ class LexerTest extends \PHPUnit_Framework_TestCase
                        array(
                            'startLine' => 2,
                            'comments' => array(
-                                new Comment('/* comment */', 1),
-                                new Comment('// comment' . "\n", 1),
-                                new Comment\Doc('/** docComment 1 */', 2),
-                                new Comment\Doc('/** docComment 2 */', 2),
+                                new Comment('/* comment */', 1, 6),
+                                new Comment('// comment' . "\n", 1, 20),
+                                new Comment\Doc('/** docComment 1 */', 2, 31),
+                                new Comment\Doc('/** docComment 2 */', 2, 50),
                            ),
                        ),
                        array('endLine' => 2)
--- a/test/PhpParser/NodeTraverserTest.php
+++ b/test/PhpParser/NodeTraverserTest.php
@ -200,4 +200,19 @@ class NodeTraverserTest extends \PHPUnit_Framework_TestCase

        $this->assertSame($stmts, $traverser->traverse($stmts));
    }
+
+    /**
+     * @expectedException \LogicException
+     * @expectedExceptionMessage leaveNode() may only return an array if the parent structure is an array
+     */
+    public function testReplaceByArrayOnlyAllowedIfParentIsArray() {
+        $stmts = array(new Node\Expr\UnaryMinus(new Node\Scalar\LNumber(42)));
+
+        $visitor = $this->getMock('PhpParser\NodeVisitor');
+        $visitor->method('leaveNode')->willReturn(array(new Node\Scalar\DNumber(42.0)));
+
+        $traverser = new NodeTraverser();
+        $traverser->addVisitor($visitor);
+        $traverser->traverse($stmts);
+    }
 }
--- a/test/PhpParser/Parser/MultipleTest.php
+++ b/test/PhpParser/Parser/MultipleTest.php
@ -61,7 +61,7 @@ class MultipleTest extends ParserTest {
                $this->getPrefer5(),
                [
                    new Expr\Variable(
-                        new Expr\ArrayDimFetch(new Expr\Variable('a'), new LNumber(0))
+                        new Expr\ArrayDimFetch(new Expr\Variable('a'), LNumber::fromString('0'))
                    )
                ]
            ],
@ -71,7 +71,7 @@ class MultipleTest extends ParserTest {
                $this->getPrefer7(),
                [
                    new Expr\ArrayDimFetch(
-                        new Expr\Variable(new Expr\Variable('a')), new LNumber(0)
+                        new Expr\Variable(new Expr\Variable('a')), LNumber::fromString('0')
                    )
                ]
            ],
--- a/test/PhpParser/ParserTest.php
+++ b/test/PhpParser/ParserTest.php
@ -3,6 +3,9 @@
 namespace PhpParser;

 use PhpParser\Comment;
+use PhpParser\Node\Expr;
+use PhpParser\Node\Scalar;
+use PhpParser\Node\Scalar\String_;

 abstract class ParserTest extends \PHPUnit_Framework_TestCase
 {
@ -54,7 +57,7 @@ EOC;
        $this->assertInstanceOf('PhpParser\Node\Stmt\Function_', $fn);
        $this->assertEquals(array(
            'comments' => array(
-                new Comment\Doc('/** Doc comment */', 2),
+                new Comment\Doc('/** Doc comment */', 2, 6),
            ),
            'startLine' => 3,
            'endLine' => 7,
@ -76,8 +79,8 @@ EOC;
        $this->assertInstanceOf('PhpParser\Node\Stmt\Echo_', $echo);
        $this->assertEquals(array(
            'comments' => array(
-                new Comment("// Line\n", 4),
-                new Comment("// Comments\n", 5),
+                new Comment("// Line\n", 4, 49),
+                new Comment("// Comments\n", 5, 61),
            ),
            'startLine' => 6,
            'endLine' => 6,
@ -105,6 +108,58 @@ EOC;
        $parser = $this->getParser($lexer);
        $parser->parse('dummy');
    }
+
+    /**
+     * @dataProvider provideTestKindAttributes
+     */
+    public function testKindAttributes($code, $expectedAttributes) {
+        $parser = $this->getParser(new Lexer);
+        $stmts = $parser->parse("<?php $code;");
+        $attributes = $stmts[0]->getAttributes();
+        foreach ($expectedAttributes as $name => $value) {
+            $this->assertSame($value, $attributes[$name]);
+        }
+    }
+
+    public function provideTestKindAttributes() {
+        return array(
+            array('0', ['kind' => Scalar\LNumber::KIND_DEC]),
+            array('9', ['kind' => Scalar\LNumber::KIND_DEC]),
+            array('07', ['kind' => Scalar\LNumber::KIND_OCT]),
+            array('0xf', ['kind' => Scalar\LNumber::KIND_HEX]),
+            array('0XF', ['kind' => Scalar\LNumber::KIND_HEX]),
+            array('0b1', ['kind' => Scalar\LNumber::KIND_BIN]),
+            array('0B1', ['kind' => Scalar\LNumber::KIND_BIN]),
+            array('[]', ['kind' => Expr\Array_::KIND_SHORT]),
+            array('array()', ['kind' => Expr\Array_::KIND_LONG]),
+            array("'foo'", ['kind' => String_::KIND_SINGLE_QUOTED]),
+            array("b'foo'", ['kind' => String_::KIND_SINGLE_QUOTED]),
+            array("B'foo'", ['kind' => String_::KIND_SINGLE_QUOTED]),
+            array('"foo"', ['kind' => String_::KIND_DOUBLE_QUOTED]),
+            array('b"foo"', ['kind' => String_::KIND_DOUBLE_QUOTED]),
+            array('B"foo"', ['kind' => String_::KIND_DOUBLE_QUOTED]),
+            array('"foo$bar"', ['kind' => String_::KIND_DOUBLE_QUOTED]),
+            array('b"foo$bar"', ['kind' => String_::KIND_DOUBLE_QUOTED]),
+            array('B"foo$bar"', ['kind' => String_::KIND_DOUBLE_QUOTED]),
+            array("<<<'STR'\nSTR\n", ['kind' => String_::KIND_NOWDOC, 'docLabel' => 'STR']),
+            array("<<<STR\nSTR\n", ['kind' => String_::KIND_HEREDOC, 'docLabel' => 'STR']),
+            array("<<<\"STR\"\nSTR\n", ['kind' => String_::KIND_HEREDOC, 'docLabel' => 'STR']),
+            array("b<<<'STR'\nSTR\n", ['kind' => String_::KIND_NOWDOC, 'docLabel' => 'STR']),
+            array("B<<<'STR'\nSTR\n", ['kind' => String_::KIND_NOWDOC, 'docLabel' => 'STR']),
+            array("<<< \t 'STR'\nSTR\n", ['kind' => String_::KIND_NOWDOC, 'docLabel' => 'STR']),
+            // HHVM doesn't support this due to a lexer bug
+            // (https://github.com/facebook/hhvm/issues/6970)
+            // array("<<<'\xff'\n\xff\n", ['kind' => String_::KIND_NOWDOC, 'docLabel' => "\xff"]),
+            array("<<<\"STR\"\n\$a\nSTR\n", ['kind' => String_::KIND_HEREDOC, 'docLabel' => 'STR']),
+            array("b<<<\"STR\"\n\$a\nSTR\n", ['kind' => String_::KIND_HEREDOC, 'docLabel' => 'STR']),
+            array("B<<<\"STR\"\n\$a\nSTR\n", ['kind' => String_::KIND_HEREDOC, 'docLabel' => 'STR']),
+            array("<<< \t \"STR\"\n\$a\nSTR\n", ['kind' => String_::KIND_HEREDOC, 'docLabel' => 'STR']),
+            array("die", ['kind' => Expr\Exit_::KIND_DIE]),
+            array("die('done')", ['kind' => Expr\Exit_::KIND_DIE]),
+            array("exit", ['kind' => Expr\Exit_::KIND_EXIT]),
+            array("exit(1)", ['kind' => Expr\Exit_::KIND_EXIT]),
+        );
+    }
 }

 class InvalidTokenLexer extends Lexer {
--- a/test/PhpParser/PrettyPrinterTest.php
+++ b/test/PhpParser/PrettyPrinterTest.php
@ -2,7 +2,10 @@

 namespace PhpParser;

+use PhpParser\Comment;
 use PhpParser\Node\Expr;
+use PhpParser\Node\Scalar\Encapsed;
+use PhpParser\Node\Scalar\EncapsedStringPart;
 use PhpParser\Node\Scalar\String_;
 use PhpParser\Node\Stmt;
 use PhpParser\PrettyPrinter\Standard;
@ -87,10 +90,75 @@ class PrettyPrinterTest extends CodeTestAbstract
        $this->assertEquals("function () {\n    return 'a\nb';\n}", $prettyPrinter->prettyPrintExpr($expr));
    }

+    public function testCommentBeforeInlineHTML() {
+        $prettyPrinter = new PrettyPrinter\Standard;
+        $comment = new Comment\Doc("/**\n * This is a comment\n */");
+        $stmts = [new Stmt\InlineHTML('Hello World!', ['comments' => [$comment]])];
+        $expected = "<?php\n\n/**\n * This is a comment\n */\n?>\nHello World!";
+        $this->assertSame($expected, $prettyPrinter->prettyPrintFile($stmts));
+    }
+
    private function parseModeLine($modeLine) {
        $parts = explode(' ', $modeLine, 2);
        $version = isset($parts[0]) ? $parts[0] : 'both';
        $options = isset($parts[1]) ? json_decode($parts[1], true) : [];
        return [$version, $options];
    }
+
+    public function testArraySyntaxDefault() {
+        $prettyPrinter = new Standard(['shortArraySyntax' => true]);
+        $expr = new Expr\Array_([
+            new Expr\ArrayItem(new String_('val'), new String_('key'))
+        ]);
+        $expected = "['key' => 'val']";
+        $this->assertSame($expected, $prettyPrinter->prettyPrintExpr($expr));
+    }
+
+    /**
+     * @dataProvider provideTestKindAttributes
+     */
+    public function testKindAttributes($node, $expected) {
+        $prttyPrinter = new PrettyPrinter\Standard;
+        $result = $prttyPrinter->prettyPrintExpr($node);
+        $this->assertSame($expected, $result);
+    }
+
+    public function provideTestKindAttributes() {
+        $nowdoc = ['kind' => String_::KIND_NOWDOC, 'docLabel' => 'STR'];
+        $heredoc = ['kind' => String_::KIND_HEREDOC, 'docLabel' => 'STR'];
+        return [
+            // Defaults to single quoted
+            [new String_('foo'), "'foo'"],
+            // Explicit single/double quoted
+            [new String_('foo', ['kind' => String_::KIND_SINGLE_QUOTED]), "'foo'"],
+            [new String_('foo', ['kind' => String_::KIND_DOUBLE_QUOTED]), '"foo"'],
+            // Fallback from doc string if no label
+            [new String_('foo', ['kind' => String_::KIND_NOWDOC]), "'foo'"],
+            [new String_('foo', ['kind' => String_::KIND_HEREDOC]), '"foo"'],
+            // Fallback if string contains label
+            [new String_("A\nB\nC", ['kind' => String_::KIND_NOWDOC, 'docLabel' => 'A']), "'A\nB\nC'"],
+            [new String_("A\nB\nC", ['kind' => String_::KIND_NOWDOC, 'docLabel' => 'B']), "'A\nB\nC'"],
+            [new String_("A\nB\nC", ['kind' => String_::KIND_NOWDOC, 'docLabel' => 'C']), "'A\nB\nC'"],
+            [new String_("STR;", ['kind' => String_::KIND_NOWDOC, 'docLabel' => 'STR']), "'STR;'"],
+            // Doc string if label not contained (or not in ending position)
+            [new String_("foo", $nowdoc), "<<<'STR'\nfoo\nSTR\n"],
+            [new String_("foo", $heredoc), "<<<STR\nfoo\nSTR\n"],
+            [new String_("STRx", $nowdoc), "<<<'STR'\nSTRx\nSTR\n"],
+            [new String_("xSTR", $nowdoc), "<<<'STR'\nxSTR\nSTR\n"],
+            // Empty doc string variations (encapsed variant does not occur naturally)
+            [new String_("", $nowdoc), "<<<'STR'\nSTR\n"],
+            [new String_("", $heredoc), "<<<STR\nSTR\n"],
+            [new Encapsed([new EncapsedStringPart('')], $heredoc), "<<<STR\nSTR\n"],
+            // Encapsed doc string variations
+            [new Encapsed([new EncapsedStringPart('foo')], $heredoc), "<<<STR\nfoo\nSTR\n"],
+            [new Encapsed([new EncapsedStringPart('foo'), new Expr\Variable('y')], $heredoc), "<<<STR\nfoo{\$y}\nSTR\n"],
+            [new Encapsed([new EncapsedStringPart("\nSTR"), new Expr\Variable('y')], $heredoc), "<<<STR\n\nSTR{\$y}\nSTR\n"],
+            [new Encapsed([new EncapsedStringPart("\nSTR"), new Expr\Variable('y')], $heredoc), "<<<STR\n\nSTR{\$y}\nSTR\n"],
+            [new Encapsed([new Expr\Variable('y'), new EncapsedStringPart("STR\n")], $heredoc), "<<<STR\n{\$y}STR\n\nSTR\n"],
+            // Encapsed doc string fallback
+            [new Encapsed([new Expr\Variable('y'), new EncapsedStringPart("\nSTR")], $heredoc), '"{$y}\\nSTR"'],
+            [new Encapsed([new EncapsedStringPart("STR\n"), new Expr\Variable('y')], $heredoc), '"STR\\n{$y}"'],
+            [new Encapsed([new EncapsedStringPart("STR")], $heredoc), '"STR"'],
+        ];
+    }
 }
--- a/test/PhpParser/Serializer/XMLTest.php
+++ b/test/PhpParser/Serializer/XMLTest.php
@ -23,6 +23,9 @@ CODE;
 <AST xmlns:node="http://nikic.github.com/PHPParser/XML/node" xmlns:subNode="http://nikic.github.com/PHPParser/XML/subNode" xmlns:attribute="http://nikic.github.com/PHPParser/XML/attribute" xmlns:scalar="http://nikic.github.com/PHPParser/XML/scalar">
 <scalar:array>
  <node:Stmt_Function>
+   <attribute:startLine>
+    <scalar:int>4</scalar:int>
+   </attribute:startLine>
   <attribute:comments>
    <scalar:array>
     <comment isDocComment="false" line="2">// comment
@ -30,9 +33,6 @@ CODE;
     <comment isDocComment="true" line="3">/** doc comment */</comment>
    </scalar:array>
   </attribute:comments>
-   <attribute:startLine>
-    <scalar:int>4</scalar:int>
-   </attribute:startLine>
   <attribute:endLine>
    <scalar:int>6</scalar:int>
   </attribute:endLine>
@ -71,6 +71,9 @@ CODE;
        <attribute:endLine>
         <scalar:int>4</scalar:int>
        </attribute:endLine>
+        <attribute:kind>
+         <scalar:int>10</scalar:int>
+        </attribute:kind>
        <subNode:value>
         <scalar:int>0</scalar:int>
        </subNode:value>
@ -133,6 +136,9 @@ CODE;
         <attribute:endLine>
          <scalar:int>5</scalar:int>
         </attribute:endLine>
+         <attribute:kind>
+          <scalar:int>1</scalar:int>
+         </attribute:kind>
         <subNode:value>
          <scalar:string>Foo</scalar:string>
         </subNode:value>
--- a/test/bootstrap.php
+++ b/test/bootstrap.php
@ -5,12 +5,16 @@ namespace PhpParser;
 require __DIR__ . '/../vendor/autoload.php';

 function canonicalize($str) {
-    // trim from both sides
-    $str = trim($str);
+    // normalize EOL style
+    $str = str_replace("\r\n", "\n", $str);

-    // normalize EOL to \n
-    $str = str_replace(array("\r\n", "\r"), "\n", $str);
+    // trim newlines at end
+    $str = rtrim($str, "\n");

-    // trim right side of all lines
-    return implode("\n", array_map('rtrim', explode("\n", $str)));
+    // remove trailing whitespace on all lines
+    $lines = explode("\n", $str);
+    $lines = array_map(function($line) {
+        return rtrim($line, " \t");
+    }, $lines);
+    return implode("\n", $lines);
 }
--- a/test/code/parser/comments.test
+++ b/test/code/parser/comments.test
@ -0,0 +1,100 @@
+Comments
+-----
+<?php
+
+/** doc 1 */
+/* foobar 1 */
+// foo 1
+// bar 1
+$var;
+
+if ($cond) {
+    /** doc 2 */
+    /* foobar 2 */
+    // foo 2
+    // bar 2
+}
+
+/** doc 3 */
+/* foobar 3 */
+// foo 3
+// bar 3
+-----
+array(
+    0: Expr_Variable(
+        name: var
+        comments: array(
+            0: /** doc 1 */
+            1: /* foobar 1 */
+            2: // foo 1
+            3: // bar 1
+        )
+    )
+    1: Stmt_If(
+        cond: Expr_Variable(
+            name: cond
+        )
+        stmts: array(
+            0: Stmt_Nop(
+                comments: array(
+                    0: /** doc 2 */
+                    1: /* foobar 2 */
+                    2: // foo 2
+                    3: // bar 2
+                )
+            )
+        )
+        elseifs: array(
+        )
+        else: null
+    )
+    2: Stmt_Nop(
+        comments: array(
+            0: /** doc 3 */
+            1: /* foobar 3 */
+            2: // foo 3
+            3: // bar 3
+        )
+    )
+)
+-----
+<?php
+
+/** doc */
+/* foobar */
+// foo
+// bar
+
+?>
+-----
+array(
+    0: Stmt_Nop(
+        comments: array(
+            0: /** doc */
+            1: /* foobar */
+            2: // foo
+            3: // bar
+        )
+    )
+)
+-----
+<?php
+
+// comment
+if (42) {}
+-----
+array(
+    0: Stmt_If(
+        cond: Scalar_LNumber(
+            value: 42
+        )
+        stmts: array(
+        )
+        elseifs: array(
+        )
+        else: null
+        comments: array(
+            0: // comment
+        )
+    )
+)
--- a/test/code/parser/errorHandling/eofError.test
+++ b/test/code/parser/errorHandling/eofError.test
@ -10,4 +10,9 @@ array(
 -----
 Syntax error, unexpected EOF from 1:20 to 1:20
 array(
+    0: Stmt_Nop(
+        comments: array(
+            0: /* bar */
+        )
+    )
 )
--- a/test/code/parser/errorHandling/recovery.test
+++ b/test/code/parser/errorHandling/recovery.test
@ -172,6 +172,11 @@ array(
            value: 2
        )
    )
+    3: Stmt_Nop(
+        comments: array(
+            0: // The output here drops the loop - would require Error node to handle this
+        )
+    )
 )
 -----
 <?php
--- a/test/code/parser/expr/arrayDef.test
+++ b/test/code/parser/expr/arrayDef.test
@ -97,6 +97,9 @@ array(
    5: Expr_Array(
        items: array(
        )
+        comments: array(
+            0: // short array syntax
+        )
    )
    6: Expr_Array(
        items: array(
--- a/test/code/parser/expr/assign.test
+++ b/test/code/parser/expr/assign.test
@ -40,18 +40,30 @@ array(
    0: Expr_Assign(
        var: Expr_Variable(
            name: a
+            comments: array(
+                0: // simple assign
+            )
        )
        expr: Expr_Variable(
            name: b
        )
+        comments: array(
+            0: // simple assign
+        )
    )
    1: Expr_AssignOp_BitwiseAnd(
        var: Expr_Variable(
            name: a
+            comments: array(
+                0: // combined assign
+            )
        )
        expr: Expr_Variable(
            name: b
        )
+        comments: array(
+            0: // combined assign
+        )
    )
    2: Expr_AssignOp_BitwiseOr(
        var: Expr_Variable(
@ -144,6 +156,9 @@ array(
    13: Expr_Assign(
        var: Expr_Variable(
            name: a
+            comments: array(
+                0: // chained assign
+            )
        )
        expr: Expr_AssignOp_Mul(
            var: Expr_Variable(
@ -158,14 +173,23 @@ array(
                )
            )
        )
+        comments: array(
+            0: // chained assign
+        )
    )
    14: Expr_AssignRef(
        var: Expr_Variable(
            name: a
+            comments: array(
+                0: // by ref assign
+            )
        )
        expr: Expr_Variable(
            name: b
        )
+        comments: array(
+            0: // by ref assign
+        )
    )
    15: Expr_AssignRef(
        var: Expr_Variable(
@ -188,10 +212,16 @@ array(
                    name: a
                )
            )
+            comments: array(
+                0: // list() assign
+            )
        )
        expr: Expr_Variable(
            name: b
        )
+        comments: array(
+            0: // list() assign
+        )
    )
    17: Expr_Assign(
        var: Expr_List(
@ -236,6 +266,9 @@ array(
        var: Expr_Variable(
            name: a
        )
+        comments: array(
+            0: // inc/dec
+        )
    )
    20: Expr_PostInc(
        var: Expr_Variable(
--- a/test/code/parser/expr/fetchAndCall/funcCall.test
+++ b/test/code/parser/expr/fetchAndCall/funcCall.test
@ -21,9 +21,15 @@ array(
            parts: array(
                0: a
            )
+            comments: array(
+                0: // function name variations
+            )
        )
        args: array(
        )
+        comments: array(
+            0: // function name variations
+        )
    )
    1: Expr_FuncCall(
        name: Expr_Variable(
@ -106,12 +112,21 @@ array(
                parts: array(
                    0: a
                )
+                comments: array(
+                    0: // array dereferencing
+                )
            )
            args: array(
            )
+            comments: array(
+                0: // array dereferencing
+            )
        )
        dim: Scalar_String(
            value: b
        )
+        comments: array(
+            0: // array dereferencing
+        )
    )
 )
--- a/test/code/parser/expr/fetchAndCall/objectAccess.test
+++ b/test/code/parser/expr/fetchAndCall/objectAccess.test
@ -22,8 +22,14 @@ array(
    0: Expr_PropertyFetch(
        var: Expr_Variable(
            name: a
+            comments: array(
+                0: // property fetch variations
+            )
        )
        name: b
+        comments: array(
+            0: // property fetch variations
+        )
    )
    1: Expr_ArrayDimFetch(
        var: Expr_PropertyFetch(
@ -50,10 +56,16 @@ array(
    3: Expr_MethodCall(
        var: Expr_Variable(
            name: a
+            comments: array(
+                0: // method call variations
+            )
        )
        name: b
        args: array(
        )
+        comments: array(
+            0: // method call variations
+        )
    )
    4: Expr_MethodCall(
        var: Expr_Variable(
@ -94,14 +106,23 @@ array(
        var: Expr_MethodCall(
            var: Expr_Variable(
                name: a
+                comments: array(
+                    0: // array dereferencing
+                )
            )
            name: b
            args: array(
            )
+            comments: array(
+                0: // array dereferencing
+            )
        )
        dim: Scalar_String(
            value: c
        )
+        comments: array(
+            0: // array dereferencing
+        )
    )
    8: Expr_ArrayDimFetch(
        var: Expr_MethodCall(
@ -116,4 +137,9 @@ array(
            value: c
        )
    )
-)
+    9: Stmt_Nop(
+        comments: array(
+            0: // invalid PHP: drop Support?
+        )
+    )
+)
--- a/test/code/parser/expr/fetchAndCall/staticCall.test
+++ b/test/code/parser/expr/fetchAndCall/staticCall.test
@ -25,10 +25,16 @@ array(
            parts: array(
                0: A
            )
+            comments: array(
+                0: // method name variations
+            )
        )
        name: b
        args: array(
        )
+        comments: array(
+            0: // method name variations
+        )
    )
    1: Expr_StaticCall(
        class: Name(
@ -99,24 +105,39 @@ array(
                parts: array(
                    0: A
                )
+                comments: array(
+                    0: // array dereferencing
+                )
            )
            name: b
            args: array(
            )
+            comments: array(
+                0: // array dereferencing
+            )
        )
        dim: Scalar_String(
            value: c
        )
+        comments: array(
+            0: // array dereferencing
+        )
    )
    6: Expr_StaticCall(
        class: Name(
            parts: array(
                0: static
            )
+            comments: array(
+                0: // class name variations
+            )
        )
        name: b
        args: array(
        )
+        comments: array(
+            0: // class name variations
+        )
    )
    7: Expr_StaticCall(
        class: Expr_Variable(
@ -149,4 +170,4 @@ array(
        args: array(
        )
    )
-)
+)
--- a/test/code/parser/expr/fetchAndCall/staticPropertyFetch.test
+++ b/test/code/parser/expr/fetchAndCall/staticPropertyFetch.test
@ -19,8 +19,14 @@ array(
            parts: array(
                0: A
            )
+            comments: array(
+                0: // property name variations
+            )
        )
        name: b
+        comments: array(
+            0: // property name variations
+        )
    )
    1: Expr_StaticPropertyFetch(
        class: Name(
@ -48,12 +54,21 @@ array(
                parts: array(
                    0: A
                )
+                comments: array(
+                    0: // array access
+                )
            )
            name: b
+            comments: array(
+                0: // array access
+            )
        )
        dim: Scalar_String(
            value: c
        )
+        comments: array(
+            0: // array access
+        )
    )
    4: Expr_ArrayDimFetch(
        var: Expr_StaticPropertyFetch(
@ -68,4 +83,9 @@ array(
            value: c
        )
    )
+    5: Stmt_Nop(
+        comments: array(
+            0: // class name variations can be found in staticCall.test
+        )
+    )
 )
--- a/test/code/parser/expr/logic.test
+++ b/test/code/parser/expr/logic.test
@ -24,10 +24,16 @@ array(
    0: Expr_BinaryOp_BooleanAnd(
        left: Expr_Variable(
            name: a
+            comments: array(
+                0: // boolean ops
+            )
        )
        right: Expr_Variable(
            name: b
        )
+        comments: array(
+            0: // boolean ops
+        )
    )
    1: Expr_BinaryOp_BooleanOr(
        left: Expr_Variable(
@ -52,10 +58,16 @@ array(
    4: Expr_BinaryOp_LogicalAnd(
        left: Expr_Variable(
            name: a
+            comments: array(
+                0: // logical ops
+            )
        )
        right: Expr_Variable(
            name: b
        )
+        comments: array(
+            0: // logical ops
+        )
    )
    5: Expr_BinaryOp_LogicalOr(
        left: Expr_Variable(
@ -77,10 +89,16 @@ array(
        left: Expr_BinaryOp_BooleanAnd(
            left: Expr_Variable(
                name: a
+                comments: array(
+                    0: // precedence
+                )
            )
            right: Expr_Variable(
                name: b
            )
+            comments: array(
+                0: // precedence
+            )
        )
        right: Expr_BinaryOp_BooleanAnd(
            left: Expr_Variable(
@ -90,6 +108,9 @@ array(
                name: d
            )
        )
+        comments: array(
+            0: // precedence
+        )
    )
    8: Expr_BinaryOp_BooleanAnd(
        left: Expr_BinaryOp_BooleanAnd(
--- a/test/code/parser/expr/math.test
+++ b/test/code/parser/expr/math.test
@ -38,6 +38,9 @@ array(
        expr: Expr_Variable(
            name: a
        )
+        comments: array(
+            0: // unary ops
+        )
    )
    1: Expr_UnaryPlus(
        expr: Expr_Variable(
@ -52,10 +55,16 @@ array(
    3: Expr_BinaryOp_BitwiseAnd(
        left: Expr_Variable(
            name: a
+            comments: array(
+                0: // binary ops
+            )
        )
        right: Expr_Variable(
            name: b
        )
+        comments: array(
+            0: // binary ops
+        )
    )
    4: Expr_BinaryOp_BitwiseOr(
        left: Expr_Variable(
@ -149,14 +158,23 @@ array(
        left: Expr_BinaryOp_Mul(
            left: Expr_Variable(
                name: a
+                comments: array(
+                    0: // associativity
+                )
            )
            right: Expr_Variable(
                name: b
            )
+            comments: array(
+                0: // associativity
+            )
        )
        right: Expr_Variable(
            name: c
        )
+        comments: array(
+            0: // associativity
+        )
    )
    16: Expr_BinaryOp_Mul(
        left: Expr_Variable(
@ -174,6 +192,9 @@ array(
    17: Expr_BinaryOp_Plus(
        left: Expr_Variable(
            name: a
+            comments: array(
+                0: // precedence
+            )
        )
        right: Expr_BinaryOp_Mul(
            left: Expr_Variable(
@ -183,6 +204,9 @@ array(
                name: c
            )
        )
+        comments: array(
+            0: // precedence
+        )
    )
    18: Expr_BinaryOp_Mul(
        left: Expr_BinaryOp_Plus(
@ -200,6 +224,9 @@ array(
    19: Expr_BinaryOp_Pow(
        left: Expr_Variable(
            name: a
+            comments: array(
+                0: // pow is special
+            )
        )
        right: Expr_BinaryOp_Pow(
            left: Expr_Variable(
@ -209,6 +236,9 @@ array(
                name: c
            )
        )
+        comments: array(
+            0: // pow is special
+        )
    )
    20: Expr_BinaryOp_Pow(
        left: Expr_BinaryOp_Pow(
--- a/test/code/parser/expr/ternaryAndCoalesce.test
+++ b/test/code/parser/expr/ternaryAndCoalesce.test
@ -20,6 +20,9 @@ array(
    0: Expr_Ternary(
        cond: Expr_Variable(
            name: a
+            comments: array(
+                0: // ternary
+            )
        )
        if: Expr_Variable(
            name: b
@ -27,6 +30,9 @@ array(
        else: Expr_Variable(
            name: c
        )
+        comments: array(
+            0: // ternary
+        )
    )
    1: Expr_Ternary(
        cond: Expr_Variable(
@ -41,6 +47,9 @@ array(
        cond: Expr_Ternary(
            cond: Expr_Variable(
                name: a
+                comments: array(
+                    0: // precedence
+                )
            )
            if: Expr_Variable(
                name: b
@ -48,6 +57,9 @@ array(
            else: Expr_Variable(
                name: c
            )
+            comments: array(
+                0: // precedence
+            )
        )
        if: Expr_Variable(
            name: d
@ -55,6 +67,9 @@ array(
        else: Expr_Variable(
            name: e
        )
+        comments: array(
+            0: // precedence
+        )
    )
    3: Expr_Ternary(
        cond: Expr_Variable(
@ -78,10 +93,16 @@ array(
    4: Expr_BinaryOp_Coalesce(
        left: Expr_Variable(
            name: a
+            comments: array(
+                0: // null coalesce
+            )
        )
        right: Expr_Variable(
            name: b
        )
+        comments: array(
+            0: // null coalesce
+        )
    )
    5: Expr_BinaryOp_Coalesce(
        left: Expr_Variable(
@ -125,4 +146,4 @@ array(
            name: c
        )
    )
-)
+)
--- a/test/code/parser/scalar/constantString.test
+++ b/test/code/parser/scalar/constantString.test
@ -8,8 +8,10 @@ b'';
 b"";
 'Hi';
 b'Hi';
+B'Hi';
 "Hi";
 b"Hi";
+B"Hi";
 '!\'!\\!\a!';
 "!\"!\\!\$!\n!\r!\t!\f!\v!\e!\a";
 "!\xFF!\377!\400!\0!";
@ -40,14 +42,19 @@ array(
        value: Hi
    )
    8: Scalar_String(
-        value: !'!\!\a!
+        value: Hi
    )
    9: Scalar_String(
-        value: !"!\!$!
-    !
-!@@{ "\t" }@@!@@{ "\f" }@@!@@{ "\v" }@@!@@{ chr(27) /* "\e" */ }@@!\a
+        value: Hi
    )
    10: Scalar_String(
+        value: !'!\!\a!
+    )
+    11: Scalar_String(
+        value: !"!\!$!
+    !@@{ "\r" }@@!@@{ "\t" }@@!@@{ "\f" }@@!@@{ "\v" }@@!@@{ chr(27) /* "\e" */ }@@!\a
+    )
+    12: Scalar_String(
        value: !@@{ chr(255) }@@!@@{ chr(255) }@@!@@{ chr(0) }@@!@@{ chr(0) }@@!
    )
 )
--- a/test/code/parser/scalar/docString.test
+++ b/test/code/parser/scalar/docString.test
@ -24,17 +24,26 @@ EOS;
 Test $a and $b->c test
 EOS;

-// comment to force line break before EOF
+b<<<EOS
+Binary
+EOS;
+
 -----
 array(
    0: Scalar_String(
        value:
+        comments: array(
+            0: // empty strings
+        )
    )
    1: Scalar_String(
        value:
    )
    2: Scalar_String(
        value: Test '" $a \n
+        comments: array(
+            0: // constant encapsed strings
+        )
    )
    3: Scalar_String(
        value: Test '" $a
@ -49,6 +58,9 @@ array(
                name: a
            )
        )
+        comments: array(
+            0: // encapsed strings
+        )
    )
    5: Scalar_Encapsed(
        parts: array(
@ -72,4 +84,7 @@ array(
            )
        )
    )
+    6: Scalar_String(
+        value: Binary
+    )
 )
--- a/test/code/parser/scalar/docStringNewlines.test
+++ b/test/code/parser/scalar/docStringNewlines.test
@ -12,7 +12,6 @@ Trailing newlines in doc strings
 <<<'EOF'@@{ "\r\nFoo\r\n\r\n" }@@EOF;
 <<<EOF@@{ "\r\n\$var\r\n\r\n" }@@EOF;

-// comment to force line break before EOF
 -----
 array(
    0: Scalar_String(
--- a/test/code/parser/scalar/encapsedString.test
+++ b/test/code/parser/scalar/encapsedString.test
@ -21,6 +21,7 @@ Encapsed strings
 "$$A[B]";
 "A $B C";
 b"$A";
+B"$A";
 -----
 array(
    0: Scalar_Encapsed(
@ -231,4 +232,11 @@ array(
            )
        )
    )
+    19: Scalar_Encapsed(
+        parts: array(
+            0: Expr_Variable(
+                name: A
+            )
+        )
+    )
 )
--- a/test/code/parser/scalar/float.test
+++ b/test/code/parser/scalar/float.test
@ -53,18 +53,22 @@ array(
        value: INF
    )
    10: Scalar_DNumber(
-        value: @@{ 0xFFFFFFFFFFFFFFFF }@@
+        value: 1.844674407371E+19
+        comments: array(
+            0: // various integer -> float overflows
+            1: // (all are actually the same number, just in different representations)
+        )
    )
    11: Scalar_DNumber(
-        value: @@{ 0xFFFFFFFFFFFFFFFF }@@
+        value: 1.844674407371E+19
    )
    12: Scalar_DNumber(
-        value: @@{ 0xFFFFFFFFFFFFFFFF }@@
+        value: 1.844674407371E+19
    )
    13: Scalar_DNumber(
-        value: @@{ 0xFFFFFFFFFFFFFFFF }@@
+        value: 1.844674407371E+19
    )
    14: Scalar_DNumber(
-        value: @@{ 0xFFFFFFFFFFFFFFFF }@@
+        value: 1.844674407371E+19
    )
 )
--- a/test/code/parser/scalar/int.test
+++ b/test/code/parser/scalar/int.test
@ -10,7 +10,6 @@ Different integer syntaxes
 0xfff;
 0XfFf;
 0777;
-0787;
 0b111000111000;
 -----
 array(
@ -39,9 +38,6 @@ array(
        value: 511
    )
    8: Scalar_LNumber(
-        value: 7
-    )
-    9: Scalar_LNumber(
        value: 3640
    )
 )
--- a/test/code/parser/scalar/invalidOctal.test
+++ b/test/code/parser/scalar/invalidOctal.test
@ -0,0 +1,17 @@
+Invalid octal literals
+-----
+<?php
+0787;
+-----
+!!php7
+Invalid numeric literal from 2:1 to 2:4
+-----
+<?php
+0787;
+-----
+!!php5
+array(
+    0: Scalar_LNumber(
+        value: 7
+    )
+)
--- a/test/code/parser/semiReserved.test
+++ b/test/code/parser/semiReserved.test
@ -13,6 +13,10 @@ class Test {
    public $private;

    const TRAIT = 3, FINAL = 4;
+
+    const __CLASS__ = 1, __TRAIT__ = 2, __FUNCTION__ = 3, __METHOD__ = 4, __LINE__ = 5,
+          __FILE__ = 6, __DIR__ = 7, __NAMESPACE__ = 8;
+    // __halt_compiler does not work
 }

 $t = new Test;
@ -130,6 +134,58 @@ array(
                    )
                )
            )
+            7: Stmt_ClassConst(
+                consts: array(
+                    0: Const(
+                        name: __CLASS__
+                        value: Scalar_LNumber(
+                            value: 1
+                        )
+                    )
+                    1: Const(
+                        name: __TRAIT__
+                        value: Scalar_LNumber(
+                            value: 2
+                        )
+                    )
+                    2: Const(
+                        name: __FUNCTION__
+                        value: Scalar_LNumber(
+                            value: 3
+                        )
+                    )
+                    3: Const(
+                        name: __METHOD__
+                        value: Scalar_LNumber(
+                            value: 4
+                        )
+                    )
+                    4: Const(
+                        name: __LINE__
+                        value: Scalar_LNumber(
+                            value: 5
+                        )
+                    )
+                    5: Const(
+                        name: __FILE__
+                        value: Scalar_LNumber(
+                            value: 6
+                        )
+                    )
+                    6: Const(
+                        name: __DIR__
+                        value: Scalar_LNumber(
+                            value: 7
+                        )
+                    )
+                    7: Const(
+                        name: __NAMESPACE__
+                        value: Scalar_LNumber(
+                            value: 8
+                        )
+                    )
+                )
+            )
        )
    )
    1: Expr_Assign(
@ -321,4 +377,4 @@ array(
            )
        )
    )
-)
+)
--- a/test/code/parser/stmt/class/modifier.test
+++ b/test/code/parser/stmt/class/modifier.test
@ -38,6 +38,11 @@ array(
        stmts: array(
        )
    )
+    1: Stmt_Nop(
+        comments: array(
+            0: // Type in the partial parse could conceivably be any of 0, 16 or 32
+        )
+    )
 )
 -----
 <?php class A { abstract $a; }
--- a/test/code/parser/stmt/declare.test
+++ b/test/code/parser/stmt/declare.test
@ -2,6 +2,8 @@ Declare
 -----
 <?php

+declare (X='Y');
+
 declare (A='B', C='D') {}

 declare (A='B', C='D'):
@ -11,20 +13,13 @@ array(
    0: Stmt_Declare(
        declares: array(
            0: Stmt_DeclareDeclare(
-                key: A
+                key: X
                value: Scalar_String(
-                    value: B
-                )
-            )
-            1: Stmt_DeclareDeclare(
-                key: C
-                value: Scalar_String(
-                    value: D
+                    value: Y
                )
            )
        )
-        stmts: array(
-        )
+        stmts: null
    )
    1: Stmt_Declare(
        declares: array(
@ -44,4 +39,22 @@ array(
        stmts: array(
        )
    )
+    2: Stmt_Declare(
+        declares: array(
+            0: Stmt_DeclareDeclare(
+                key: A
+                value: Scalar_String(
+                    value: B
+                )
+            )
+            1: Stmt_DeclareDeclare(
+                key: C
+                value: Scalar_String(
+                    value: D
+                )
+            )
+        )
+        stmts: array(
+        )
+    )
 )
--- a/test/code/parser/stmt/generator/basic.test
+++ b/test/code/parser/stmt/generator/basic.test
@ -42,6 +42,9 @@ array(
            0: Expr_Yield(
                key: null
                value: null
+                comments: array(
+                    0: // statements
+                )
            )
            1: Expr_Yield(
                key: null
@ -60,11 +63,17 @@ array(
            3: Expr_Assign(
                var: Expr_Variable(
                    name: data
+                    comments: array(
+                        0: // expressions
+                    )
                )
                expr: Expr_Yield(
                    key: null
                    value: null
                )
+                comments: array(
+                    0: // expressions
+                )
            )
            4: Expr_Assign(
                var: Expr_Variable(
@ -112,6 +121,9 @@ array(
                    )
                )
                else: null
+                comments: array(
+                    0: // yield in language constructs with their own parentheses
+                )
            )
            7: Stmt_If(
                cond: Expr_Yield(
@ -179,6 +191,9 @@ array(
                    parts: array(
                        0: func
                    )
+                    comments: array(
+                        0: // yield in function calls
+                    )
                )
                args: array(
                    0: Arg(
@ -192,6 +207,9 @@ array(
                        unpack: false
                    )
                )
+                comments: array(
+                    0: // yield in function calls
+                )
            )
            13: Expr_MethodCall(
                var: Expr_Variable(
@ -259,4 +277,4 @@ array(
            )
        )
    )
-)
+)
--- a/test/code/parser/stmt/if.test
+++ b/test/code/parser/stmt/if.test
@ -81,6 +81,9 @@ array(
            stmts: array(
            )
        )
+        comments: array(
+            0: // without else
+        )
    )
    3: Stmt_If(
        cond: Expr_Variable(
@ -92,4 +95,9 @@ array(
        )
        else: null
    )
+    4: Stmt_Nop(
+        comments: array(
+            0: // without else
+        )
+    )
 )
--- a/test/code/parser/stmt/loop/for.test
+++ b/test/code/parser/stmt/loop/for.test
@ -46,6 +46,9 @@ array(
        )
        stmts: array(
        )
+        comments: array(
+            0: // "classical" loop
+        )
    )
    1: Stmt_For(
        init: array(
@ -74,6 +77,9 @@ array(
        )
        stmts: array(
        )
+        comments: array(
+            0: // multiple expressions
+        )
    )
    2: Stmt_For(
        init: array(
@ -84,6 +90,9 @@ array(
        )
        stmts: array(
        )
+        comments: array(
+            0: // infinite loop
+        )
    )
    3: Stmt_For(
        init: array(
@ -94,5 +103,8 @@ array(
        )
        stmts: array(
        )
+        comments: array(
+            0: // alternative syntax
+        )
    )
 )
--- a/test/code/parser/stmt/loop/foreach.test
+++ b/test/code/parser/stmt/loop/foreach.test
@ -29,6 +29,9 @@ array(
        )
        stmts: array(
        )
+        comments: array(
+            0: // foreach on variable
+        )
    )
    1: Stmt_Foreach(
        expr: Expr_Variable(
@ -123,6 +126,9 @@ array(
        )
        stmts: array(
        )
+        comments: array(
+            0: // foreach on expression
+        )
    )
    7: Stmt_Foreach(
        expr: Expr_Variable(
@ -135,5 +141,8 @@ array(
        )
        stmts: array(
        )
+        comments: array(
+            0: // alternative syntax
+        )
    )
 )
--- a/test/code/parser/stmt/namespace/alias.test
+++ b/test/code/parser/stmt/namespace/alias.test
@ -84,6 +84,9 @@ array(
                alias: A
            )
        )
+        comments: array(
+            0: // evil alias notation - Do Not Use!
+        )
    )
    4: Stmt_Use(
        type: 1
@ -113,6 +116,9 @@ array(
                alias: bar
            )
        )
+        comments: array(
+            0: // function and constant aliases
+        )
    )
    6: Stmt_Use(
        type: 2
@ -159,4 +165,4 @@ array(
            )
        )
    )
-)
+)
--- a/test/code/parser/stmt/namespace/nsAfterHashbang.test
+++ b/test/code/parser/stmt/namespace/nsAfterHashbang.test
@ -0,0 +1,22 @@
+Hashbang followed by namespace declaration
+-----
+#!/usr/bin/env php
+<?php
+
+namespace A;
+-----
+array(
+    0: Stmt_InlineHTML(
+        value: #!/usr/bin/env php
+
+    )
+    1: Stmt_Namespace(
+        name: Name(
+            parts: array(
+                0: A
+            )
+        )
+        stmts: array(
+        )
+    )
+)
--- a/test/code/parser/stmt/namespace/outsideStmt.test
+++ b/test/code/parser/stmt/namespace/outsideStmt.test
@ -19,8 +19,7 @@ array(
                )
            )
        )
-        stmts: array(
-        )
+        stmts: null
    )
    1: Stmt_Namespace(
        name: Name(
@ -34,4 +33,26 @@ array(
    2: Stmt_HaltCompiler(
        remaining: Hi!
    )
+)
+-----
+<?php
+/* Comment */
+;
+namespace Foo;
+-----
+array(
+    0: Stmt_Nop(
+        comments: array(
+            0: /* Comment */
+        )
+    )
+    1: Stmt_Namespace(
+        name: Name(
+            parts: array(
+                0: Foo
+            )
+        )
+        stmts: array(
+        )
+    )
 )
--- a/test/code/parser/stmt/switch.test
+++ b/test/code/parser/stmt/switch.test
@ -49,6 +49,9 @@ array(
        )
        cases: array(
        )
+        comments: array(
+            0: // alternative syntax
+        )
    )
    2: Stmt_Switch(
        cond: Expr_Variable(
@ -56,6 +59,9 @@ array(
        )
        cases: array(
        )
+        comments: array(
+            0: // leading semicolon
+        )
    )
    3: Stmt_Switch(
        cond: Expr_Variable(
--- a/test/code/parser/stmt/tryCatch.test
+++ b/test/code/parser/stmt/tryCatch.test
@ -102,6 +102,9 @@ array(
            )
        )
        finallyStmts: null
+        comments: array(
+            0: // no finally
+        )
    )
    2: Stmt_TryCatch(
        stmts: array(
@ -110,5 +113,8 @@ array(
        )
        finallyStmts: array(
        )
+        comments: array(
+            0: // no catch
+        )
    )
 )
--- a/test/code/prettyPrinter/comments.test
+++ b/test/code/prettyPrinter/comments.test
@ -27,7 +27,6 @@ function justForIndentation()
       Even more text. */
    $foo;
 }
-
 -----
 function justForIndentation()
 {
@ -53,4 +52,16 @@ function justForIndentation()
       More text.
       Even more text. */
    $foo;
+}
+-----
+<?php
+
+function test()
+{
+    // empty
+}
+-----
+function test()
+{
+    // empty
 }
--- a/test/code/prettyPrinter/expr/docStrings.test
+++ b/test/code/prettyPrinter/expr/docStrings.test
@ -0,0 +1,86 @@
+Literals
+-----
+<?php
+
+<<<'STR'
+STR;
+<<<STR
+STR;
+
+<<<'STR'
+A
+B
+STR;
+<<<STR
+A
+B
+STR;
+
+<<<'STR'
+a\nb$c
+STR;
+<<<STR
+a\\nb\$c
+STR;
+
+<<<STR
+a$b
+{$c->d}
+STR;
+
+call(
+    <<<STR
+A
+STR
+    , <<<STR
+B
+STR
+);
+
+function test() {
+    <<<STR
+Foo
+STR;
+    <<<STR
+    Bar
+STR;
+}
+-----
+<<<'STR'
+STR;
+<<<STR
+STR;
+<<<'STR'
+A
+B
+STR;
+<<<STR
+A
+B
+STR;
+<<<'STR'
+a\nb$c
+STR;
+<<<STR
+a\\nb\$c
+STR;
+<<<STR
+a{$b}
+{$c->d}
+STR;
+call(<<<STR
+A
+STR
+, <<<STR
+B
+STR
+);
+function test()
+{
+    <<<STR
+Foo
+STR;
+    <<<STR
+    Bar
+STR;
+}
--- a/test/code/prettyPrinter/expr/intrinsics.test
+++ b/test/code/prettyPrinter/expr/intrinsics.test
@ -0,0 +1,29 @@
+isset, empty, unset, exit, die, clone, eval
+-----
+<?php
+
+isset($a, $a[$b]);
+empty($a);
+empty('foo');
+unset($a, $a[$b]);
+exit;
+exit();
+exit(1);
+die;
+die();
+die('foo');
+clone $foo;
+eval('str');
+-----
+isset($a, $a[$b]);
+empty($a);
+empty('foo');
+unset($a, $a[$b]);
+exit;
+exit;
+exit(1);
+die;
+die;
+die('foo');
+clone $foo;
+eval('str');
--- a/test/code/prettyPrinter/expr/literals.test
+++ b/test/code/prettyPrinter/expr/literals.test
@ -40,21 +40,17 @@ FALSE;
 378282246310005.0;
 10000000000000002.0;

-// strings (normalized to single quoted)
+// strings (single quoted)
 'a';
 'a
 b';
 "a";
 "a\nb";
 'a\'b';
+
+// strings (double quoted)
 "a'b";
 "a\b";
-<<<'STR'
-a\nb$a
-{$b}
-STR;
-
-// strings (normalized to double quoted)
 "$a";
 "a$b";
 "$a$b";
@ -68,9 +64,6 @@ STR;
 "\\{ $A }";
 "{$$A}[B]";
 "$$A[B]";
-<<<STR
-a\nb$a\n{$b}
-STR;

 // make sure indentation doesn't mess anything up
 function foo()
@ -81,6 +74,12 @@ b';
    'a
    b';
 }
+
+// shell exec (similar to double quoted string)
+`foo`;
+`foo$a`;
+`foo{$a}bar`;
+`\`\'\"`;
 -----
 // magic constants
 __LINE__;
@ -101,9 +100,9 @@ FALSE;
 // integers (normalized to decimal)
 0;
 11;
-9;
-17;
-3;
+011;
+0x11;
+0b11;
 // floats (normalized to ... something)
 0.0;
 0.0;
@ -116,19 +115,16 @@ INF;
 1.0E+84;
 378282246310005.0;
 10000000000000002.0;
-// strings (normalized to single quoted)
-'a';
-'a
-b';
+// strings (single quoted)
 'a';
 'a
 b';
+"a";
+"a\nb";
 'a\'b';
-'a\'b';
-'a\\b';
-'a\\nb$a
-{$b}';
-// strings (normalized to double quoted)
+// strings (double quoted)
+"a'b";
+"a\\b";
 "{$a}";
 "a{$b}";
 "{$a}{$b}";
@ -142,14 +138,17 @@ b';
 "\\{ {$A} }";
 "{${$A}}[B]";
 "\${$A['B']}";
-"a\nb{$a}\n{$b}";
 // make sure indentation doesn't mess anything up
 function foo()
 {
-    'a
-b';
+    "a\nb";
    'a
 b';
    'a
    b';
 }
+// shell exec (similar to double quoted string)
+`foo`;
+`foo{$a}`;
+`foo{$a}bar`;
+`\`\\'\\"`;
--- a/test/code/prettyPrinter/expr/shortArraySyntax.test
+++ b/test/code/prettyPrinter/expr/shortArraySyntax.test
@ -6,7 +6,6 @@ Short array syntax
 array(1, 2, 3);
 ['a' => 'b', 'c' => 'd'];
 -----
-!!both {"shortArraySyntax": true}
 [];
-[1, 2, 3];
+array(1, 2, 3);
 ['a' => 'b', 'c' => 'd'];
--- a/test/code/prettyPrinter/expr/yield.test
+++ b/test/code/prettyPrinter/expr/yield.test
@ -0,0 +1,46 @@
+Yield
+-----
+<?php
+
+function gen()
+{
+    yield;
+    yield $a;
+    yield $a => $b;
+    $a = yield;
+    $a = (yield $b);
+    $a = (yield $b => $c);
+}
+// TODO Get rid of parens for cases 2 and 3
+-----
+function gen()
+{
+    yield;
+    (yield $a);
+    (yield $a => $b);
+    $a = yield;
+    $a = (yield $b);
+    $a = (yield $b => $c);
+}
+// TODO Get rid of parens for cases 2 and 3
+-----
+<?php
+
+function gen()
+{
+    $a = yield $b;
+    $a = yield $b => $c;
+    yield from $a;
+    $a = yield from $b;
+}
+// TODO Get rid of parens for last case
+-----
+!!php7
+function gen()
+{
+    $a = (yield $b);
+    $a = (yield $b => $c);
+    yield from $a;
+    $a = (yield from $b);
+}
+// TODO Get rid of parens for last case
--- a/test/code/prettyPrinter/onlyPHP.file-test
+++ b/test/code/prettyPrinter/onlyPHP.file-test
@ -8,4 +8,9 @@ echo 'Bar Foo';
 <?php

 echo 'Foo Bar';
-echo 'Bar Foo';
+echo 'Bar Foo';
+-----
+<?php
+
+-----
+<?php
--- a/test/code/prettyPrinter/stmt/class.test
+++ b/test/code/prettyPrinter/stmt/class.test
@ -2,7 +2,7 @@ Class
 -----
 <?php

-class Foo
+class Foo extends Bar implements ABC, \DEF, namespace\GHI
 {
    var $a = 'foo';
    private $b = 'bar';
@ -17,8 +17,15 @@ class Foo
    public function foo() {}
    abstract static function bar() {}
 }
+
+trait Bar
+{
+    function test()
+    {
+    }
+}
 -----
-class Foo
+class Foo extends Bar implements ABC, \DEF, namespace\GHI
 {
    var $a = 'foo';
    private $b = 'bar';
@ -37,4 +44,10 @@ class Foo
    static abstract function bar()
    {
    }
+}
+trait Bar
+{
+    function test()
+    {
+    }
 }
--- a/test/code/prettyPrinter/stmt/const.test
+++ b/test/code/prettyPrinter/stmt/const.test
@ -0,0 +1,11 @@
+Constant declarations
+-----
+<?php
+
+const FOO = 'BAR';
+const FOO = 1 + 1;
+const FOO = BAR, BAR = FOO;
+-----
+const FOO = 'BAR';
+const FOO = 1 + 1;
+const FOO = BAR, BAR = FOO;
--- a/test/code/prettyPrinter/stmt/declare.test
+++ b/test/code/prettyPrinter/stmt/declare.test
@ -0,0 +1,17 @@
+declare
+-----
+<?php
+
+declare (strict_types=1);
+declare (ticks=1) {
+    foo();
+}
+declare (ticks=2) {
+}
+-----
+declare (strict_types=1);
+declare (ticks=1) {
+    foo();
+}
+declare (ticks=2) {
+}
--- a/test/code/prettyPrinter/stmt/global_static_variables.test
+++ b/test/code/prettyPrinter/stmt/global_static_variables.test
@ -0,0 +1,11 @@
+Global and static variables
+-----
+<?php
+
+global $a, $$a, ${$a[$a]};
+static $a, $b;
+static $a = 'foo', $b = 'bar';
+-----
+global $a, ${$a}, ${$a[$a]};
+static $a, $b;
+static $a = 'foo', $b = 'bar';
--- a/test/code/prettyPrinter/stmt/haltCompiler.file-test
+++ b/test/code/prettyPrinter/stmt/haltCompiler.file-test
@ -0,0 +1,27 @@
+__halt_compiler
+-----
+<?php
+
+echo 'foo';
+__halt_compiler();
+!!!
+???
+-----
+<?php
+
+echo 'foo';
+__halt_compiler();
+!!!
+???
+-----
+<?php
+
+echo 'foo';
+__halt_compiler();
+<?php
+-----
+<?php
+
+echo 'foo';
+__halt_compiler();
+<?php
--- a/test_old/run-php-src.sh
+++ b/test_old/run-php-src.sh
@ -0,0 +1,4 @@
+wget -q https://github.com/php/php-src/archive/php-7.0.5.tar.gz
+mkdir -p ./data/php-src
+tar -xzf ./php-7.0.5.tar.gz -C ./data/php-src --strip-components=1
+php -n test_old/run.php --verbose --no-progress PHP7 ./data/php-src
--- a/test_old/run.php
+++ b/test_old/run.php
@ -43,9 +43,12 @@ if (count($arguments) !== 2) {
 }

 $showProgress = true;
-if (count($options) > 0) {
-    if (count($options) === 1 && $options[0] === '--no-progress') {
+$verbose = false;
+foreach ($options as $option) {
+    if ($option === '--no-progress') {
        $showProgress = false;
+    } elseif ($option === '--verbose') {
+        $verbose = true;
    } else {
        showHelp('Invalid option passed!');
    }
@ -85,6 +88,13 @@ switch ($testType) {
 # pretty print difference due to INF vs 1e1000
 | ext.standard.tests.general_functions.bug27678
 | tests.lang.bug24640
+# pretty print difference due to nop statements
+| ext.mbstring.tests.htmlent
+| ext.standard.tests.file.fread_basic
+# tests using __halt_compiler as semi reserved keyword
+| Zend.tests.grammar.semi_reserved_001
+| Zend.tests.grammar.semi_reserved_002
+| Zend.tests.grammar.semi_reserved_005
 )\.phpt$~x', $file)) {
                return null;
            }
@ -160,11 +170,17 @@ foreach (new RecursiveIteratorIterator(

            if (!$same) {
                echo $file, ":\n    Result of initial parse and parse after pretty print differ\n";
+                if ($verbose) {
+                    echo "Pretty printer output:\n=====\n$code\n=====\n\n";
+                }

                ++$compareFail;
            }
        } catch (PhpParser\Error $e) {
            echo $file, ":\n    Parse of pretty print failed with message: {$e->getMessage()}\n";
+            if ($verbose) {
+                echo "Pretty printer output:\n=====\n$code\n=====\n\n";
+            }

            ++$ppFail;
        }
@ -176,8 +192,10 @@ foreach (new RecursiveIteratorIterator(
 }

 if (0 === $parseFail && 0 === $ppFail && 0 === $compareFail) {
+    $exit = 0;
    echo "\n\n", 'All tests passed.', "\n";
 } else {
+    $exit = 1;
    echo "\n\n", '==========', "\n\n", 'There were: ', "\n";
    if (0 !== $parseFail) {
        echo '    ', $parseFail,   ' parse failures.',        "\n";
@ -201,3 +219,5 @@ echo "\n",
     "\n",
     'Total time:           ', microtime(true) - $totalStartTime, "\n",
     'Maximum memory usage: ', memory_get_peak_usage(true), "\n";
+
+exit($exit);
Author	SHA1	Message	Date
Nikita Popov	47b254ea51	Release PHP-Parser 2.1.0	2016-04-19 15:41:41 +02:00
Nikita Popov	912c0bb9c9	Document that XDebug is super slow	2016-04-19 15:40:08 +02:00
Nikita Popov	9897fa8819	Update CHANGELOG	2016-04-19 15:32:57 +02:00
Nikita Popov	5a6e7dd452	Avoid superfluous newline when printing Nops	2016-04-19 15:10:51 +02:00
Nikita Popov	90eb1165d1	Switch Exit kind attribute to flag To be consistent with everything else.	2016-04-19 14:51:05 +02:00
Nikita Popov	cf9b9e2afa	Switch to 2.1-dev	2016-04-19 14:51:04 +02:00
Nikita Popov	e7869b9f14	Forbid invalid octals in PHP 7 mode	2016-04-18 13:59:18 +02:00
Nikita Popov	4c7ad7e194	Allow nop statements before namespace declaration	2016-04-16 21:39:49 +02:00
Remi Collet	f8a40b3f24	Make autoloader more PSR-4 (#268 )	2016-04-15 15:07:30 +02:00
Nikita Popov	82bb6627c9	Drop superfluous newlines before help text	2016-04-15 14:36:11 +02:00
Remi Collet	d1dd9f5aec	support -h and --help standard options	2016-04-15 14:34:28 +02:00
Graham Campbell	e7f0860d85	Tweaked a phpdoc description	2016-04-10 21:10:55 +09:00
Nikita Popov	954051f240	Run test_old using php -n Xdebug is killing performance.	2016-04-10 21:09:02 +09:00
lvht	39f93f09f9	Add setReturnType() method to function/method builders Also support scalar type hints in existing setTypeHint() method on params.	2016-04-09 18:41:38 +09:00
Nikita Popov	2d0c3b70f8	Fix .travis.yml	2016-04-07 07:39:20 +02:00
Nikita Popov	4252ffa43e	Run test_old against php-src on travis	2016-04-07 06:39:29 +02:00
Nikita Popov	8cacc85913	"Fix" HHVM build	2016-04-07 12:27:08 +09:00
Nikita Popov	588e6a4d4c	Add string kinds and doc string labels Scalar\String_ and Scalar\Encapsed now have an additional "kind" attribute, which may be one of: * String_::KIND_SINGLE_QUOTED * String_::KIND_DOUBLE_QUOTED * String_::KIND_NOWDOC * String_::KIND_HEREDOC Additionally, if the string kind is one of the latter two, an attribute "docLabel" is provided, which contains the doc string label (STR in <<<STR) that was originally used. The pretty printer will try to take the original kind of the string, as well as the used doc string label into account.	2016-04-07 12:09:00 +09:00
Nikita Popov	fa6a17755a	Add two files to test_old skip list The parse after pretty printing differs here, because these tests contain nop (;) statements, which the pretty printer does not preserve.	2016-04-07 11:58:36 +09:00
Nikita Popov	52cb5ecec7	Fix comment reformatting for some cases It did not correctly deal with non-standard comments containing indented lines.	2016-04-07 11:47:40 +09:00
Nikita Popov	1565a2815d	Add --verbose option to test_old/run.php	2016-04-05 19:19:20 +09:00
Nikita Popov	14de71898e	Fix nop statement comment assignment Keep around the start attributes on the lookahead token around in a separate parser property.	2016-04-05 19:19:15 +09:00
Nikita Popov	5fa8493675	Make canonicalization less aggressive	2016-04-05 10:37:45 +09:00
Nikita Popov	b31a973fa7	Another fix for B"" handling The lexer fix also need to account for uppercase B.	2016-04-04 22:07:50 +09:00
Nikita Popov	9ac3592190	Handle uppercase B"" prefix	2016-04-02 21:15:49 +09:00
Nikita Popov	35011d2e4d	Deduplicate attribute assignment code a bit	2016-04-02 08:02:58 +09:00
Nikita Popov	60f01bdd86	Deprecate setters on Comment No point in making it mutable	2016-04-02 07:55:28 +09:00
Nikita Popov	15a2388d75	Add start file offset to Comments	2016-04-02 07:54:01 +09:00
Nikita Popov	68b4c0388a	Fix Serializer/XML test output indent This causes a failure here, but not on another machine, not sure why.	2016-04-02 07:27:01 +09:00
Nikita Popov	eb4bfe1366	Add sanity check for leaveNode() return value	2016-03-27 14:52:15 +02:00
Théo FIDRY	e7ca4b7b04	Better Travis configuration * Enable Travis cache for Composer * Use `--prefer-dist` over `--prefer-source` to download less and make it faster	2016-03-15 17:39:37 +01:00
Nikita Popov	fc36239be5	Remove internal LNumber::parse() method Combine it into ::fromString(), as they both do the same checks.	2016-03-10 13:01:42 +01:00
Nikita Popov	f493219c7d	Move "kind" attribute handling out of lexer Doesn't belong there and will cause issue with multiple assignments.	2016-03-10 12:55:15 +01:00
Nikita Popov	aa199120c7	Add kind attribute for arrays To distinguish array() and [] syntax. The pretty printer respects this attribute. The shortArraySyntax pretty printer option acts as a default in case the attribute is not specified.	2016-03-09 21:31:54 +01:00
Nikita Popov	ae30f97af6	Add "kind" attribute to LNumbers Kind specifies whether the number was formatted as decimal, octal, binary or hex. The pretty printer reproduces the number kind (but not necessarily the exact formatting).	2016-03-09 21:10:55 +01:00
Nikita Popov	47c342a3e4	Add "kind" attribute to Stmt\Exit_ Attribute specifies whether this is a "die" or an "exit" and the pretty printer behaves accordingly.	2016-03-09 20:20:36 +01:00
Nikita Popov	7eac2cfd8b	Introduce Nop statement to collect dangling comments A Nop statement will be inserted into statement lists if there are any trailing comments in the list (which would otherwise not be associated with any node). The pretty printer output currently still contains a superfluous newline.	2016-03-09 19:48:36 +01:00
Nikita Popov	a0c216bf4b	Add dumpComments option to NodeDumper Adding this as an option to avoid breaking people's tests. Some of the test results show pretty clearly that we are incorrectly assigning the same comment multiple times for nested nodes (mentioned in #36).	2016-03-09 19:48:36 +01:00
Nikita Popov	06d9ba42de	Merge pull request #252 from stevemiketa/update-mode Set bin/php-parse file mode to 0755	2016-03-03 20:52:13 +01:00
Steve Miketa	573c7c20c4	Update file mode	2016-03-03 13:06:54 -05:00
Nikita Popov	d5cbf79f2f	Mark 1.x as unsupported	2016-02-28 21:01:06 +01:00
Nikita Popov	ce5be709d5	Release PHP-Parser 2.0.1	2016-02-28 20:48:28 +01:00
Nikita Popov	9829bf69cd	Fix issue #251	2016-02-28 20:28:32 +01:00
Nikita Popov	c8282e6e76	Add badges	2016-02-20 21:53:08 +01:00
Nikita Popov	a73aa7eec1	Pretty printer test coverage Our output for yield / yield from is currently not very nice, but also not easy to change.	2016-02-20 21:49:21 +01:00
Nikita Popov	1fe8f09caa	Fix __halt_compiler() pretty printing edge case We can't strip the <?php at the end of a __halt_compiler() segment in file mode. Fixed by being a bit more explicit in prettyPrintFile() about what we want to do...	2016-02-20 18:24:32 +01:00
Nikita Popov	47509cf927	Next try...	2016-02-20 17:20:16 +01:00
Nikita Popov	5b96a11a1f	Fix .travis.yml	2016-02-20 17:18:19 +01:00
Nikita Popov	7faa1dcab9	Try setting up coveralls	2016-02-20 17:16:46 +01:00
Nikita Popov	65af37f7b0	Doc comment tweaks	2016-02-20 17:06:09 +01:00
Nikita Popov	d6361136e1	Update semi-reserved keyword list Magic constant names have been added after the PHP 7 release. We do not support and likely will not support __halt_compiler here due to lexer limitations.	2016-02-09 13:30:39 +01:00
Nikita Popov	e05ef23743	Merge pull request #244 from pra85/doc Typo fixes in documentation	2016-01-28 17:03:02 +01:00
Prayag Verma	73a9d494fb	typo fixes	2016-01-28 19:31:28 +05:30
Nikita Popov	94f10d3c50	Merge branch '1.x'	2016-01-15 22:05:36 +01:00
Nikita Popov	c4bbc8e236	Support hashbang before namespace declaration Fixes issue #243.	2016-01-15 22:03:42 +01:00
Nikita Popov	6dffb72ce0	Merge branch '1.x'	2016-01-15 22:02:35 +01:00
Nikita Popov	eb73441032	Support hashbang before namespace declaration Fixes issue #243.	2016-01-15 22:01:51 +01:00
Nikita Popov	9a6a147369	Merge pull request #241 from jesseschalken/patch-1 Remove some unecessary @property doc comments	2015-12-29 14:38:17 +01:00
Jesse Schalken	5c2cc50455	Remove some more unecessary @property doc comments	2015-12-29 23:15:00 +11:00
Jesse Schalken	58eb1ea7c3	Remove some unecessary @property doc comments	2015-12-29 23:13:55 +11:00
Nikita Popov	719ca71d4a	Distinguish declare(); and declare(){} It makes semantic difference and the latter form is actually forbidden for strict_types. This sets Declare->stmts to null for the body-less case.	2015-12-07 12:12:00 +01:00
Nikita Popov	f78af2c9c8	Release PHP-Parser 1.4.1	2015-09-19 22:15:08 +08:00