In this patch, the test suite from html5lib validates the tree-construction
steps in the HTML Processor to ensure that they are behaving according to the
HTML specification. This suite of tests is also used by the servo project to
test its html5ever package.
A new test module in the HTML API transforms HTML Processor output to match
the expected tree shape from the external tests. For cases where there are
tests validating behaviors of unsupported HTML tags and constructs, the tests
are marked as skipped. As the HTML API continues to expand its own support,
the number of skipped tests will automatically shrink down towards zero.
Additional tests are skipped through the `SKIP_TEST` array in the test runner.
Fixes#60227.
See #58517.
Props azaozz, costdev, dmsnell, hellofromtonya, jonsurrell, jorbin, swisspidy.
git-svn-id: https://develop.svn.wordpress.org/trunk@58010 602fd350-edb4-49c9-b593-d223f7449a82
This package was removed in WP 6.3, so these exclusion entries are no longer necessary.
Follow-up to [42346], [42665], [49797], [56141], [56667].
See #58831, #58955.
git-svn-id: https://develop.svn.wordpress.org/trunk@56669 602fd350-edb4-49c9-b593-d223f7449a82
Since PHPUnit 9.5.10 and 8.5.21, PHP deprecations are no longer converted to exceptions by default (`convertDeprecationsToExceptions="true"` can be configured to enable this).
Reference: [fac02620f6 Do not convert PHP deprecations to exceptions by default]; [https://github.com/sebastianbergmann/phpunit/blob/9.5.10/ChangeLog-9.5.md#9510---2021-09-25 PHPUnit 9.5.10 changelog].
Let's unpack this:
Previously (PHPUnit < 9.5.10/8.5.21), if PHPUnit would encounter a PHP native deprecation notice, it would:
1. Show a test which causes a deprecation notice to be thrown as **"errored"**,
2. Show the **first** deprecation notice it encountered and
3. PHPUnit would exit with a **non-0 exit code** (2), which will fail a CI build.
As of PHPUnit 9.5.10/8.5.21, if PHPUnit encounters a PHP native deprecation notice, it will no longer do so. Instead PHPUnit will:
1. Show a test which causes a PHP deprecation notice to be thrown as **"risky"**,
2. Show the **all** deprecation notices it encountered and
3. PHPUnit will exit with a **0 exit code**, which will show a CI build as passing.
This commit reverts PHPUnit to the previous behaviour by adding `convertDeprecationsToExceptions="true"` to the PHPUnit configuration. It also adds the other related directives for consistency.
Props jrf, netweb, costdev, SergeyBiryukov.
See #54183.
git-svn-id: https://develop.svn.wordpress.org/trunk@51871 602fd350-edb4-49c9-b593-d223f7449a82
Since PHP 8.1, `readonly` is a reserved keyword and cannot be used as a function name.
In order to avoid PHP parser errors, the `readonly()` function was extracted to a separate file and is now only included conditionally on PHP < 8.1.
This commit also:
* Moves the tests for the `__checked_selected_helper()` function and all the related functions to their own file.
* Switches to named data providers. This makes the output when using the `--testdox` option more descriptive and is helpful when trying to debug which data set from a data provider failed the test.
* Improves the tests in question to make them feature-complete and expand test coverage.
Props jrf, ayeshrajans, haosun, knutsp, swissspidy, SergeyBiryukov.
Fixes#53858.
git-svn-id: https://develop.svn.wordpress.org/trunk@51586 602fd350-edb4-49c9-b593-d223f7449a82
The current config files validate against the PHPUnit XSD schema for config files for PHPUnit 5.7 – 9.2.
The schema was changed in PHPUnit 9.3, and the `filter` and `logging` settings were deprecated in favor of `coverage` and a different format for `logging`.
This commit explicitly sets the schema against which the files currently validate, for clarity.
Props jrf.
See #53363.
git-svn-id: https://develop.svn.wordpress.org/trunk@51583 602fd350-edb4-49c9-b593-d223f7449a82
Now that the tests can run PHPUnit cross-version and Composer will be used to install the test suite in CI, we could switch out the local copies of the [https://github.com/johnkary/phpunit-speedtrap PHPUnit speedtrap] package in favor of using the Composer package, which would prevent us having to make the WP local copies of the class compatible with later PHPUnit versions.
The SpeedTrap test listener was introduced to identify slow tests and take action on these to make them faster.
In practice, however, no notable action was ever taken based on the output of the test listener in all the years it was in place.
With that in mind, it was decided to remove the SpeedTrap test listeners without replacement.
If – at a future date – contributors would want to take action to speed up slow tests anyway, they can:
* Either add the package to their local install and use the output they receive locally to identify slow tests.
* Or use the PHPUnit native `@small` annotations in combination with the PHPUnit `PHP_Invoker` package as described in the PHPUnit documentation to [https://phpunit.readthedocs.io/en/stable/risky-tests.html#test-execution-timeout run tests with time limits].
Follow-up to [35214], [35226], [35767], [44701], [51559-51572].
Props jrf.
See #46149.
git-svn-id: https://develop.svn.wordpress.org/trunk@51573 602fd350-edb4-49c9-b593-d223f7449a82
Since [50441-50442] switched the test workflows to run from `src` instead of `build`, code coverage reporting has stopped working. This was caused by the code coverage configuration continuing to reference `build`. This corrects the configuration so coverage reporting can resume.
This change also introduces the `workflow_dispatch` event to the workflow, which will allow committers to manually run the workflow when desired. For example, to confirm changes to the test suite do not break reporting.
This also adds the `phpunit.xml.dist` and `tests/phpunit/multisite.xml` files to the `paths` list. Since these files are responsible for configuring the test suite and code coverage reporting, any changes to them should verify that no problems were introduced.
Props jrf, johnbillion.
Fixes#52786. See #51734.
git-svn-id: https://develop.svn.wordpress.org/trunk@50592 602fd350-edb4-49c9-b593-d223f7449a82
This makes the configuration consistent with the `WP_RUN_CORE_TESTS` constant being set to `1` in the same config file.
Additionally, update the list to be in line with the current codebase, excluding the plugins and themes bundled with WordPress core from being considered for code coverage.
Note: If you have a local `phpunit.xml` file that changes the `WP_RUN_CORE_TESTS` constant to `0`, you should also - in the same file - change the code coverage setup to use the `src` directory.
Follow-up to [42665], [43311].
Props jrf, mmarco9.
See #50267.
git-svn-id: https://develop.svn.wordpress.org/trunk@48847 602fd350-edb4-49c9-b593-d223f7449a82
All PHP files in `/tests` now conform to the PHP coding standards, or have exceptions appropriately marked.
Travis now also runs `phpcs` on the `/tests` directory, any future changes to these files must conform entirely to the WordPress PHP coding standards. 🎉
See #47632.
git-svn-id: https://develop.svn.wordpress.org/trunk@45607 602fd350-edb4-49c9-b593-d223f7449a82
* Create an abstract `WP_UnitTestCase_Base` class to share between PHPUnit 7.x and older versions.
* Add a speed-trap loader to determine which `SpeedTrapListener` class needs to be loaded for the current PHPUnit version.
* Remove unnecessary `PHPUnit\Util\Test` and `PHPUnit_Util_Getopt` inheritances.
* Update Travis CI config to use PHPUnit 7.x for PHP 7.1, 7.2, and nightly PHP versions.
Props jipmoors, netweb, desrosj, ayeshrajans, soulseekah, SergeyBiryukov.
See #43218.
git-svn-id: https://develop.svn.wordpress.org/trunk@44701 602fd350-edb4-49c9-b593-d223f7449a82
- Adds `WP_REST_Autosaves_Controller` which extends `WP_REST_Revisions_Controller`.
- Autosaves endpoint is registered for all post types except attachment because even post types without revisions enabled are expected to autosave.
- Because setting the `DOING_AUTOSAVE` constant pollutes the test suite, autosaves tests are run last. We may want to improve upon this later.
Also, use a truly impossibly high number in User Controller tests. The number `100`, (or `7777` in `trunk`), could be valid in certain test run configurations. The `REST_TESTS_IMPOSSIBLY_HIGH_NUMBER` constant is impossibly high for this very reason.
Finally, Skip Autosaves controller test for multisite. There's a PHP 5.2 edge case where paths calculated differently, possibly caused by differing version of PHPUnit.
Props adamsilverstein, aduth, azaozz, danielbachhuber, rmccue, danielbachhuber.
Merges [43767], [43768], [43769] to trunk.
See #45132, #45131.
Fixes#45128, #43316.
git-svn-id: https://develop.svn.wordpress.org/trunk@44126 602fd350-edb4-49c9-b593-d223f7449a82
This corrects the attribute needed to allow PHPUnit to generate coverage reports, and also adds a blacklist of files and directories that are excluded from coverage reports.
Props schlessera, jipmoors
Fixes#43067, #43240
git-svn-id: https://develop.svn.wordpress.org/trunk@42665 602fd350-edb4-49c9-b593-d223f7449a82
Some of the files renamed in [41261] are listed in `phpunit.xml.dist`, as they need to be ignore in PHP 5.2.
This followup commit changes `phpunit.xml.dist` to match their new names.
See #41658.
git-svn-id: https://develop.svn.wordpress.org/trunk@41262 602fd350-edb4-49c9-b593-d223f7449a82
Tests in the `ms-excluded` group are now excluded when running tests with multisite enabled, and tests in the `ms-required` group are excluded when running tests without multisite enabled. The end result is a significantly reduced number of skipped tests polluting PHPUnit's output, which means verbose mode can be used to more easily see which tests are skipped or incomplete, and why.
See #40531
git-svn-id: https://develop.svn.wordpress.org/trunk@40520 602fd350-edb4-49c9-b593-d223f7449a82
This reverts [38454] along with its follow-up commits, [38512], [38514], and [38692]. These tests are currently not pass
ing, and maybe they never will. The tests are in a group which does not run by default without a flag, making them quest
ionably useful.
We can re-visit this at a later date.
See #32360
git-svn-id: https://develop.svn.wordpress.org/trunk@38840 602fd350-edb4-49c9-b593-d223f7449a82
Multisite specific tests run as normal with `phpunit.xml.dist` and multisite constants defined. The `ms-files` group causes pollution when mixed with the rest of the tests, so it should be excluded here too.
See #36566.
git-svn-id: https://develop.svn.wordpress.org/trunk@37269 602fd350-edb4-49c9-b593-d223f7449a82
The external-http tests are very slow, and `Wp_Http` functionality is fairly
isolated, so the benefits of skipping these tests by default outweigh the
risks.
A `grunt phpunit:external-http` subtask has been added, to ensure that the
tests are executed during exhaustive runs of the test suite, such as in
continuous integration.
Fixes#30304.
git-svn-id: https://develop.svn.wordpress.org/trunk@30298 602fd350-edb4-49c9-b593-d223f7449a82
A growing number of plugins and other WP-based projects use the core test
tools, such as WP_UnitTestCase, as the basis of their own tests and continuous
integration setups. At the same time, many of these third-party plugins use
custom post types and taxonomies, which are generally registered a single time
during a run of the tests: at 'init', before the testcases have run. Wiping out
these globals between tests will mess with these third-party builds.
Best practice for plugin developers is probably to clean up their own post types
and taxonomies and then reinitialize before each test. But, in the interest of
not breaking everyone's builds, the core test suite will not enforce this.
Fixes#29827.
git-svn-id: https://develop.svn.wordpress.org/trunk@29869 602fd350-edb4-49c9-b593-d223f7449a82
wp-tests-config.php can/should reside in the root of a develop checkout. `phpunit` should be run from the root.
see #25088.
git-svn-id: https://develop.svn.wordpress.org/trunk@25165 602fd350-edb4-49c9-b593-d223f7449a82
