This adjusts the workflow responsible for testing old branches to only test old branches once per month. This more closely resembles the testing practices previously in place on TravisCI.
The latest, supported branch will continue to be tested twice per month.
This also makes adjustments so that the workflow is run whenever changes are made to it. This will help verify changes without having to wait until the next scheduled run.
Fixes#52653.
git-svn-id: https://develop.svn.wordpress.org/trunk@50763 602fd350-edb4-49c9-b593-d223f7449a82
The reorganization of the PHPUnit workflow in [50441] unintentionally caused the tests to be run for every `push` event, even for forks and private mirrors.
Previously, the second job required the first one to pass, and the conditional check on the first prevented both from running. Because the first job is no longer required for the second, both jobs must have the appropriate conditional check.
Fixes#52983.
git-svn-id: https://develop.svn.wordpress.org/trunk@50670 602fd350-edb4-49c9-b593-d223f7449a82
This adds a conditional statement to ensure the workflow that tests old branches only runs on the official `wordpress-develop` mirror. This prevents it from running on forks and private mirrors.
See #52653.
git-svn-id: https://develop.svn.wordpress.org/trunk@50648 602fd350-edb4-49c9-b593-d223f7449a82
When the workflow file is updated, it should always be run to verify changes.
Follow up to [50592].
See #52786.
git-svn-id: https://develop.svn.wordpress.org/trunk@50595 602fd350-edb4-49c9-b593-d223f7449a82
This splits the code coverage workflow into two separate jobs, allowing single site and multisite to run in parallel. This cuts the total workflow run time in half.
Fixes#52923.
git-svn-id: https://develop.svn.wordpress.org/trunk@50594 602fd350-edb4-49c9-b593-d223f7449a82
Since [50285], the WordPress Importer plugin (which is required for the test suite to run successfully) has been installed as part of the `npm run env:install` script.
See #52625.
git-svn-id: https://develop.svn.wordpress.org/trunk@50593 602fd350-edb4-49c9-b593-d223f7449a82
Since [50441-50442] switched the test workflows to run from `src` instead of `build`, code coverage reporting has stopped working. This was caused by the code coverage configuration continuing to reference `build`. This corrects the configuration so coverage reporting can resume.
This change also introduces the `workflow_dispatch` event to the workflow, which will allow committers to manually run the workflow when desired. For example, to confirm changes to the test suite do not break reporting.
This also adds the `phpunit.xml.dist` and `tests/phpunit/multisite.xml` files to the `paths` list. Since these files are responsible for configuring the test suite and code coverage reporting, any changes to them should verify that no problems were introduced.
Props jrf, johnbillion.
Fixes#52786. See #51734.
git-svn-id: https://develop.svn.wordpress.org/trunk@50592 602fd350-edb4-49c9-b593-d223f7449a82
On TravisCI, old branches still receiving security updates were tested on a regular basis. This ensured tests continued to pass as time passed even if updates were not made to these branches.
On GitHub Actions, there is no interface to configure this (TravisCI had a UI), but there is a `schedule` event that can trigger workflow runs on cron that can be used to accomplish the same thing.
This introduces a workflow file that runs twice a month (on the 1st and 15th) to verify the tests within older branches.
Because the `schedule` event only runs within the primary branch, the appropriate workflows in each old branch will be triggered manually through the `workflow_dispatch` trigger using the GitHub REST API. `workflow_dispatch` will need to be added to all workflows in all old branches in order for the event to dispatch successfully.
Fixes#52653.
git-svn-id: https://develop.svn.wordpress.org/trunk@50590 602fd350-edb4-49c9-b593-d223f7449a82
This disables the `fail-fast` option in the GitHub Actions workflow for testing NPM. This allows all jobs in the test matrix to complete, even if one fails, painting a clear picture of where the issue lies.
Follow up to [50435].
See #52625.
git-svn-id: https://develop.svn.wordpress.org/trunk@50579 602fd350-edb4-49c9-b593-d223f7449a82
This also adds inline comments to the end of each action installed using a SHA value to make it easier to determine which version of an action is currently being used.
Follow up to [50474,50476].
Props johnbillion.
See #52625.
git-svn-id: https://develop.svn.wordpress.org/trunk@50486 602fd350-edb4-49c9-b593-d223f7449a82
The NPM test GitHub Actions workflow is currently used to verify that `npm install` and `npm run build` will complete successfully on Windows.
This generalizes the workflow and adds testing for Linux and MacOS to confirm that the scripts used to build WordPress also complete successfully in those environments.
Because MacOS workflow runs consume GHA minutes at a 10x rate, it’s added in a separate job in order to perform a more strict check.
This also adds steps to verify that using `npm run build:dev` and `npm run grunt clean`/`npm run grunt clean —dev` run successfully
Props johnbillion.
Fixes#52658.
git-svn-id: https://develop.svn.wordpress.org/trunk@50485 602fd350-edb4-49c9-b593-d223f7449a82
This change adds a list of paths to check when determining which workflows to run on `pull_request`. This will prevent certain workflows from running unnecessarily when the files updated are not related to the tests and checks being performed.
Props peterwilsoncc, johnbillion.
Fixes#52667.
git-svn-id: https://develop.svn.wordpress.org/trunk@50479 602fd350-edb4-49c9-b593-d223f7449a82
Additionally, the `access_token` input now defaults to `github.token`, so specifying it is unnecessary.
Follow up to [50474].
Props johnbillion.
See #52625.
git-svn-id: https://develop.svn.wordpress.org/trunk@50476 602fd350-edb4-49c9-b593-d223f7449a82
Some GitHub Action scripts require additional permissions to perform the desired operations. This permission is usually given by passing a personal access token (PAT) to the action as an input.
Because PATs grant access to sensitive information about the repository and actions with PATs become trusted actors, 3rd party actions should not be installed by specifying a major or minor version.
Instead, specifying a full length commit SHA will use the 3rd party action as an immutable release, ensuring the workflows within the repository are not affected by upstream security problems should they occur.
Props johnbillion.
See #52625.
git-svn-id: https://develop.svn.wordpress.org/trunk@50474 602fd350-edb4-49c9-b593-d223f7449a82
Using `npm ci` makes better use of the npm cache that's in use, speeding up the test runs by around 30 seconds.
See #52660
git-svn-id: https://develop.svn.wordpress.org/trunk@50446 602fd350-edb4-49c9-b593-d223f7449a82
The `external-http`, `media`, and `restapi` groups combined take roughly the same time as all the other tests combined. Splitting the test run in two speeds up the PHP 5.6 test runs which are the slowest by quite a margin. The net reduction in duration is only seen on PHP versions older than 7.0 where the reduction is greater than the time taken for the job to be set up.
Props peterwilsoncc, desrosj, johnbillion
Special shout out to manzoorwanijk whose article about running steps in parallel started me down this path.
Fixes#52645
git-svn-id: https://develop.svn.wordpress.org/trunk@50444 602fd350-edb4-49c9-b593-d223f7449a82
Some PHPUnit tests were concerned with the state of files in the `build` directory. In order to allow the tests to run without requiring a build to be run first, these have been moved into assertions that run after the build step (and therefore cause it to fail if they do not pass), or into QUnit tests as necessary.
Various other PHPUnit tests implictly depend on built JavaScript files being present. These files are now touched during the test setup to avoid PHP warnings if the `build` files are not present.
The `wp-tests-config-sample.php` file and the GitHub Actions configuration have also been changed so `ABSPATH` uses `src` instead of `build`, therefore allowing the PHPUnit tests to be run without a build having to be run first. This means all new local installations of WordPress will use `src` for PHPUnit testing. If you would like to switch your existing installation over then change the location of `ABSPATH` in `wp-tests-config.php` to point to `src` instead of `build`.
Props peterwilsoncc, iandunn, gziolo, desroj, johnbillion
Fixes#51734
See #45863
git-svn-id: https://develop.svn.wordpress.org/trunk@50441 602fd350-edb4-49c9-b593-d223f7449a82
The `actions/cache` GitHub action uses a `key` input to find the desired cache in storage. Currently, this is defined as a combination of the operating system being used, the software being cached (NPM or Composer), PHP version (Composer only), and a hash of the relevant lock file.
The `cache` action also supports a `restore-keys` input, which is an ordered list of keys to use for restoring a cache if no cache hit occurred for the specified key.
However, this means that every time a lock file is updated, the cache is being recreated by first restoring a previous one (when available), and then installing new versions of dependencies on top of that. This results in old, unused versions of dependencies never being removed from the cache.
In this change, the `restore-keys` are removed. This will force a new, fresh cache to be generated when the desired `key` is not matched. In testing, this results in a ~40% reduction in cache size when compared to the latest successful workflow runs. Since there is no way to manually flush the cache in GHA, the benefits of this change will not be seen until the `package-lock.json` and `composer.lock` files are updated, or the cache at the desired key is evicted.
Props johbillion.
See #52660.
git-svn-id: https://develop.svn.wordpress.org/trunk@50436 602fd350-edb4-49c9-b593-d223f7449a82
In GitHub Actions, there is a `fail-fast` option that will cancel all incomplete jobs within a workflow as soon as one failure is encountered. This defaults to `true`.
This is great for alerting you of a failure faster, but does not give you the full picture of what is failing. For example, it’s possible that there is a failing test on 2 versions of PHP, but not others. If `fail-fast` is enabled, it’s impossible to know this.
Even with `fail-fast` disabled, the workflow will still report a failure if one or more jobs produce a failing result.
Props johbillion, jorbin.
Fixes#52612.
git-svn-id: https://develop.svn.wordpress.org/trunk@50435 602fd350-edb4-49c9-b593-d223f7449a82
Different branches have support for different types of testing based on the tooling that was in place at the time each version was branched. The workflows currently in place have patterns configured to match the relevant `branches` and `tags` for each workflow, but only for `push` events.
This copies the patterns for matching supported `branches` over to the `pull_request` event to prevent workflows from running for a pull request to a branch that does not support that workflow.
Fixes#52643.
git-svn-id: https://develop.svn.wordpress.org/trunk@50432 602fd350-edb4-49c9-b593-d223f7449a82
This group is not specified in the list of excluded groups within the test configuration file, so they already run as part of the main test suite.
Props johnbillion.
Fixes#52608.
git-svn-id: https://develop.svn.wordpress.org/trunk@50413 602fd350-edb4-49c9-b593-d223f7449a82
This updates two published GitHub actions to their latest versions:
- `actions/setup-node` from `v1` to `v2`.
- `styfle/cancel-workflow-action` from `0.5.0` to `0.8.0`.
See #50401.
git-svn-id: https://develop.svn.wordpress.org/trunk@50387 602fd350-edb4-49c9-b593-d223f7449a82
The PHP 8 Docker container for the local WordPress environment now contains xDebug 3.x (the version required for running on PHP 8), so the `xdebug` test group can now be run.
See #50401, #51802.
git-svn-id: https://develop.svn.wordpress.org/trunk@50299 602fd350-edb4-49c9-b593-d223f7449a82
This expands the list of events that triggers automated testing to include tags.
It also refines the matched version ranges for several workflows.
See #50401.
git-svn-id: https://develop.svn.wordpress.org/trunk@50298 602fd350-edb4-49c9-b593-d223f7449a82
The WordPress Importer plugin is now downloaded to the `tests/phpunit/data/plugins` directory when running `npm run env:install`.
This ensures that the PHPUnit test suite will not fail when the plugin is missing.
This also introduces a new `WP_IMPORTER_REVISION` variable to the `.env` file, to control the SVN revision that is checked out.
Props johnbillion.
Fixes#49720.
git-svn-id: https://develop.svn.wordpress.org/trunk@50285 602fd350-edb4-49c9-b593-d223f7449a82
This simplifies the caching of Composer dependencies in the coding standards and PHP compatibility workflows by using a published action. This combines 3 steps into 1 within these workflows.
Because the Composer implementation within the PHPUnit test workflow is a bit specialized (`composer install` is run within the Docker container), caching has been left as is in that workflow. However, the cache key has been changed to include the version of PHP being tested. This will prevent incorrect versions of dependencies being present when they are required on jobs other than PHP 8.
Props jrf.
See #50401.
git-svn-id: https://develop.svn.wordpress.org/trunk@49938 602fd350-edb4-49c9-b593-d223f7449a82
Before the coverage reports were submitted to Codecov.io, HTML coverage reports were compressed into ZIP files and uploaded to the workflow run as an artifact. A weekly schedule was chosen to run this workflow because generating a coverage report is more time consuming, and the resulting reports are quite large (~150-200MB each).
This changes the schedule for the code coverage workflow from weekly to daily and eliminates the ZIP artifacts that were previously generated. This will ensure the code coverage data found at https://codecov.io/gh/WordPress/wordpress-develop is relatively accurate on any given day of the week without needlessly consuming artifact storage.
Props jorbin.
See #50401, #52141.
git-svn-id: https://develop.svn.wordpress.org/trunk@49931 602fd350-edb4-49c9-b593-d223f7449a82
This will help catch issues with Ajax related functionality on multisite.
A few tests have also been marked as `skipWithMultisite()` as they require network admin level capabilities when performing the same operations on a multisite install.
Props garrett-eclipse, netweb.
Fixes#46567.
git-svn-id: https://develop.svn.wordpress.org/trunk@49835 602fd350-edb4-49c9-b593-d223f7449a82
This introduces a new GitHub Action workflow to generate a code coverage report when running the PHPUnit test suite as both a single and multisite install.
The workflow will run once per week on Sunday at 00:00 UTC.
For now, the reports are not submitted anywhere, but they will be uploaded to the workflow run as ZIP file artifacts where they will persist for 90 days.
Making code coverage reports more readily available will hopefully better highlight areas of Core with poor coverage, and encourage more contributors to help increase test coverage of those areas.
Props ocean90, johnbillion.
Fixes#52034.
git-svn-id: https://develop.svn.wordpress.org/trunk@49834 602fd350-edb4-49c9-b593-d223f7449a82
The test reporter uses the hidden `.svn` or `.git` directories to determine which changeset to send to WordPress.org along with the test report.
Because the GitHub Action workflow builds and prepares WordPress in a preceding job to share with each test job, the `.git` directory is missing (it is purposefully not included to limit artifact size).
This re-establishes the directory as a checkout of the `wordpress-develop` repository so that test results can be reported correctly.
See #50401.
git-svn-id: https://develop.svn.wordpress.org/trunk@49786 602fd350-edb4-49c9-b593-d223f7449a82
In the current state, the workflows run regardless of the repository context. This results many needless workflow runs that waste resources.
Workflow runs for private repositories are not free (accounts have a finite allotment of minutes for private repositories). This becomes problematic in private repositories that also mirror the WordPress develop repository, as any workflow runs will draw from a user’s or organization’s allotted action minutes.
Without blanket disabling the workflow manually for all event triggers, or modifying the workflow files in the forked/mirrored repository, there is no way to tune when the workflows run.
This change introduces a conditional statement into all GitHub Action workflows that prevents them from running on forked/mirrored repositories that are not `wordpress-develop`, except when a pull request is being submitted to that repository.
The exception to this is the Welcome workflow that posts a helpful message to first time contributors to `wordpress-develop`. This message is specific to this repository, so should only run when a PR is submitted to that repository.
See #50401.
git-svn-id: https://develop.svn.wordpress.org/trunk@49781 602fd350-edb4-49c9-b593-d223f7449a82
The `install-changed` package records a hash of the `package.json` file locally in a text file and only runs `npm install` when there are changes to account for.
This builds on [47497], which only hashes the `package.json` file after a Grunt task is called. By using `npx install-changed` within the workflow files, the package is hashed before the first Grunt task is run.
Props ocean90.
See #50401, #49594.
git-svn-id: https://develop.svn.wordpress.org/trunk@49369 602fd350-edb4-49c9-b593-d223f7449a82
Because workflow results are reported for each commit, it’s important to let all runs against main and version branches to complete so that the checks are reported accurately.
When considering and reviewing pull requests, the only workflow run that matters is the most recent.
Props ocean90, helen.
See #50401.
git-svn-id: https://develop.svn.wordpress.org/trunk@49244 602fd350-edb4-49c9-b593-d223f7449a82
This change makes better use of the job strategy matrix for workflows. By using `include`, the memcached and test reporting job can be configured more clearly.
Props ocean90.
See #50401.
git-svn-id: https://develop.svn.wordpress.org/trunk@49204 602fd350-edb4-49c9-b593-d223f7449a82
When a contributor opens their first pull request to `wordpress-develop`, the “Welcome” workflow runs and leaves a comment with guidance, helpful information, and resources.
However, because a workflow run triggered by the `pull_request` event runs against the workflow and code from the merge commit, the needed context and permissions to comment on the pull request are missing. By changing the trigger event to `pull_request_target`, the workflow runs against the workflow and code in the base of the pull request and is able to comment on when appropriate.
See #50401.
git-svn-id: https://develop.svn.wordpress.org/trunk@49169 602fd350-edb4-49c9-b593-d223f7449a82
