This sets the following environment variables when running automated performance tests to avoid side effects that can skew performance data when the default development environment variables are used:
* `SAVEQUERIES`: `false`
* `SCRIPT_DEBUG`: `false`
* `WP_DEBUG`: `false`
* `WP_DEVELOPMENT_MODE`: `''`
Props rajinsharwar, desrosj, mukesh27, joemcgill.
Fixes#58825.
git-svn-id: https://develop.svn.wordpress.org/trunk@56313 602fd350-edb4-49c9-b593-d223f7449a82
This restores the GitHub Actions job responsible for automatically retrying a failed workflow once within the E2E testing workflow.
[56198] disabled Slack notifications for this workflow because of the increased number of timeout errors occurring after recent changes until they could be further investigated. Even though the signal-to-noise ration was way too high, there’s still benefit in retrying the workflow once to see if the timeout can be resolved without human intervention. The one retry attempt will not result in any Slack notifications.
Follow up to [56198].
See #58779.
git-svn-id: https://develop.svn.wordpress.org/trunk@56308 602fd350-edb4-49c9-b593-d223f7449a82
Remove Slack notifications of E2E test suite failures while the tests are flakey to improve the signal-to-noise ratio.
Fixes#58779.
git-svn-id: https://develop.svn.wordpress.org/trunk@56198 602fd350-edb4-49c9-b593-d223f7449a82
This also removes the random compat library which is no longer necessary, and adjusts unit tests and CI workflows that no longer need to take PHP 5.6 into account.
Thank you for your service, PHP 5. Onwards!
Props SergeyBiryukov, mukesh27, dingo_d, audrasjb, jrf, costdev, azaozz, JavierCasares, hellofromTonya, samiamnot, spacedmonkey, masteradhoc, knutsp, garyjones, chanthaboune
Fixes#57345
git-svn-id: https://develop.svn.wordpress.org/trunk@56141 602fd350-edb4-49c9-b593-d223f7449a82
This adds a matrix to the e2e test runs on GitHub Actions so the tests run both with and without `SCRIPT_DEBUG` enabled. This allows us to catch scenarios where the minified and non-minified files may differ.
Props Clorith, joemcgill
Fixes#58661
git-svn-id: https://develop.svn.wordpress.org/trunk@56113 602fd350-edb4-49c9-b593-d223f7449a82
Modify the E2E workflow to store the screenshots and HTML files generated during failing tests as an artifact in the workflow.
No artefact is stored for passing test runs.
Props peterwilsoncc, isabel_brison.
Fixes#58596.
git-svn-id: https://develop.svn.wordpress.org/trunk@55991 602fd350-edb4-49c9-b593-d223f7449a82
Update the performance tests to account for the relocation of the theme unit test data repository to the WordPress organisation on GitHub.
Props costdev, kafleg, williampatton, dd32, otto42, poena, jrf, joemcgill, peterwilsoncc, mukesh27.
Fixes#58569.
git-svn-id: https://develop.svn.wordpress.org/trunk@55976 602fd350-edb4-49c9-b593-d223f7449a82
The `permissions` key in a job declares the GitHub permissions that are granted to the token that's used by the job. Restricting the permissions reduces the impact that a vulnerability in the CI system can have.
Props desrosj, johnbillion
See #57865
git-svn-id: https://develop.svn.wordpress.org/trunk@55715 602fd350-edb4-49c9-b593-d223f7449a82
This adds the new performance measurement workflow introduced in [55459] to the one that tests old branches on a regular schedule.
This ensures it continues to work as expected over time, even if the branches with this workflow are not updated for a bit of time.
Props flixos90, joemcgill, desrosj.
See #57687.
git-svn-id: https://develop.svn.wordpress.org/trunk@55507 602fd350-edb4-49c9-b593-d223f7449a82
This updates three GitHub Actions to their latest versions:
- `shivammathur/setup-php`
- `actions/cache`
- `wow-actions/welcome`
The welcome action now uses the `GITHUB_TOKEN` by default, so it no longer needs to be passed manually.
See #57572.
git-svn-id: https://develop.svn.wordpress.org/trunk@55487 602fd350-edb4-49c9-b593-d223f7449a82
This is a follow up to r55459, which fixes the hostname used for logging performance metrics by adding the correct www subdomain.
See #57687.
git-svn-id: https://develop.svn.wordpress.org/trunk@55479 602fd350-edb4-49c9-b593-d223f7449a82
This adds a new GitHub Action workflow that measures a set of performance metrics on every commit, so we can track changes in the performance of WordPress over time and more easily identify changes that are responsible for significant performance improvements or regressions during development cycles.
The workflow measures the homepage of a classic theme (Twenty Twenty-One) and a block theme (Twenty Twenty-Three) set up with demo content from the Theme Test Data project. Using the e2e testing framework, it makes 20 requests and records the median value of the following Server Timing metrics, generated by an mu-plugin installed as part of this workflow:
- Total server response time
- Server time before templates are loaded
- Server time during template rendering
In addition to measuring the performance metrics of the current commit, it also records performance metrics of a consistent version of WordPress (6.1.1) to be used as a baseline measurement in order to remove variance caused by the GitHub workers themselves from our reporting.
The measurements are collected and displayed at https://www.codevitals.run/project/wordpress.
Props adamsilverstein, mukesh27, flixos90, youknowriad, oandregal, desrosj, costdev, swissspidy.
Fixes#57687.
git-svn-id: https://develop.svn.wordpress.org/trunk@55459 602fd350-edb4-49c9-b593-d223f7449a82
This updates the following third-party GitHub Actions to their latest versions.
- `actions/cache`
- `actions/checkout`
- `actions/upload-artifact`
- `actions/setup-node`
- `actions/setup-php`
- `actions/github-script`
See #57572.
git-svn-id: https://develop.svn.wordpress.org/trunk@55152 602fd350-edb4-49c9-b593-d223f7449a82
We're gonna teach 'em how to say goodbye (teach 'em how to say goodbye)
Teach 'em how
To say goodbye
To say goodbye (say goodbye)
Say goodbye (say goodbye)
One last time
Fixes#57228.
git-svn-id: https://develop.svn.wordpress.org/trunk@54995 602fd350-edb4-49c9-b593-d223f7449a82
This improves the speed of the PHPCS scans between workflow runs in GitHub Action by creating a scan cache file for each unique set of arguments passed to `phpcs`.
Props jrf.
Fixes#57148. See #53841.
git-svn-id: https://develop.svn.wordpress.org/trunk@54921 602fd350-edb4-49c9-b593-d223f7449a82
To improve how Composer dependencies are installed and managed within GitHub Actions, the `ramsey/composer-install` third-party action is now used consistently throughout all workflows.
Previously, some workflows manually ran `composer` commands while others already used `ramsey/composer-install`.
The `ramsey/composer-install` action manages caching dependencies across workflow runs internally, which is something that was manually handled before this change.
Props jrf, desrosj.
Fixes#53841.
git-svn-id: https://develop.svn.wordpress.org/trunk@54856 602fd350-edb4-49c9-b593-d223f7449a82
This adds additional inline context as to why the MacOS job is separate from the Windows and Ubuntu ones in the Test npm workflow.
While it is preferable to combine all of these to avoid repeated code, there is currently no way to determine the runner’s OS within the `if` workflow key.
MacOS jobs use GitHub Action minutes at by a multiple of 10. Being more strict about when to run these jobs ensures minutes are not unintentionally consumed within private forks and mirrors.
See https://docs.github.com/en/actions/learn-github-actions/contexts#context-availability for more details.
See #56793.
git-svn-id: https://develop.svn.wordpress.org/trunk@54852 602fd350-edb4-49c9-b593-d223f7449a82
This applies several types of improvements to GitHub Action workflows:
- Updates to inline documentation to ensure accuracy.
- Removal of repetitive or unnecessary debug logging.
- Reorganization of some steps to have configuration steps towards the beginning of jobs.
- Step name updates for consistency across workflows.
Props desrosj, jrf.
See #56793.
git-svn-id: https://develop.svn.wordpress.org/trunk@54851 602fd350-edb4-49c9-b593-d223f7449a82
This updates the `ramsey/composer-install` and `shivammathur/setup-php` actions to their latest versions.
These updates include fixes for the deprecated warnings output when using `set-output` and `save-state` commands.
Props jrf, desrosj.
See #56820, #56882, #56793.
git-svn-id: https://develop.svn.wordpress.org/trunk@54750 602fd350-edb4-49c9-b593-d223f7449a82
When uploading artifacts during GitHub Action workflows, all files uploaded to an artifact name are zipped. When a ZIP file is uploaded, the result is a horrifying state of ZIP-ception.
This changes the workflow to upload all of the needed theme files to the artifact instead of a singular ZIP file created prior to uploading.
This also fixes a bug where ZIP files generated have the desired theme nested within `src/wp-content/themes` instead of just containing the theme files.
Props ndiego, wildworks, desrosj, bgardner, colorful-tones.
Fixes#56898.
git-svn-id: https://develop.svn.wordpress.org/trunk@54740 602fd350-edb4-49c9-b593-d223f7449a82
This introduces a GitHub Action workflow that performs the following steps for default themes:
- Test installing npm dependencies and running the build script (for relevant themes).
- Create ZIP files for the .org Theme directory and uploads them as workflow artifacts.
The workflow will only run on limited occasions, one of those being when Twenty Nineteen, Twenty Twenty, or Twenty Twenty-One is edited. These are the themes with build scripts that need to be tested for issues.
It can also be manually run through the GitHub Actions UI for occasions when the ZIP files are needed for a release.
Props peterwilsoncc, desrosj
Fixes#56898.
git-svn-id: https://develop.svn.wordpress.org/trunk@54699 602fd350-edb4-49c9-b593-d223f7449a82
When a ruleset error is encountered during a PHPCodeSniffer scan, an XML report is not generated and `cs2pr` will exit with a `0`.
In this situation, a workflow run will be marked as passing (even though a failure has occurred) due to the presence of `continue-on-error`.
This adjusts the logic in the Coding Standards and PHP Compatibility workflows to remove the need for the `continue-on-error` option and ensures all failures are accurately reflected within the GitHub Actions UI.
Follow up to [54371].
Props jrf, TobiasBg.
See #55652.
git-svn-id: https://develop.svn.wordpress.org/trunk@54678 602fd350-edb4-49c9-b593-d223f7449a82
This removes the dynamic aspect of the `createWorkflowDispatch()` call that dispatches a Failed Workflow run when another workflow encounters an issue.
By hardcoding `trunk` as the `ref`, the version of the workflow used will always be the latest, most up to date. This ensures older branches receive the bug fixes and improvements made in `trunk` without having to backport them.
See #55652.
git-svn-id: https://develop.svn.wordpress.org/trunk@54674 602fd350-edb4-49c9-b593-d223f7449a82
This updates the third-party action used to post a welcome message to pull requests opened by first time contributors.
This release updates the action to use Node.js version 16 instead of 12, the latter of which support has been deprecated for in GitHub Action runners.
The action has also changed from `bukboo/welcome-action` to `wow-action/welcome`.
See #56793.
git-svn-id: https://develop.svn.wordpress.org/trunk@54651 602fd350-edb4-49c9-b593-d223f7449a82
The `save-state` and `set-output` commands have been deprecated in GitHub Actions. This removes all occurrences of the command within workflow steps.
This will not remove all deprecated notices from workflow summaries (some third-party actions still contain instances of these commands and need to be fixed upstream), but it will fix the notices caused by custom workflow code.
See #56820.
git-svn-id: https://develop.svn.wordpress.org/trunk@54649 602fd350-edb4-49c9-b593-d223f7449a82
This removes the note stating that PHP 8.1 (see [51604]) and 8.2 (see [53922]) are temporarily allowed to fail.
The compatibility issues were resolved and `continue-on-error` was removed in [53922].
See #56009.
git-svn-id: https://develop.svn.wordpress.org/trunk@54507 602fd350-edb4-49c9-b593-d223f7449a82
The latest version of the `actions/github-script` action fixes an issue where passing options to the action would remove any default values not passed (see https://github.com/actions/github-script/pull/293).
This also includes updates to other third-party actions, bringing all third-party versions in Core workflows to their latest versions:
- `actions/cache`
- `actions/setup-node`
- `codecov/codecov-action`
See #55652.
git-svn-id: https://develop.svn.wordpress.org/trunk@54373 602fd350-edb4-49c9-b593-d223f7449a82
When running PHPCS scans (both for checking coding standards and PHP version compatibility), the results are currently only returned silently in a format that GitHub can consume for contextually annotating any code being flagged.
This changes workflows using PHPCS to also display the results of each scan in the GitHub Action log, making it easier to find and understand what is causing failures.
Props jrf.
See #55652.
git-svn-id: https://develop.svn.wordpress.org/trunk@54371 602fd350-edb4-49c9-b593-d223f7449a82
With all known unit test failures now addressed, WordPress 6.1 aims to support PHP 8.1 and 8.2 as much as possible.
While full compatibility with PHP 8.1 and 8.2 is still a work in progress, this commit aims to actively prevent new PHP issues from being introduced in WordPress core.
All remaining known issues are deprecation notices. Please note, a deprecation notice is not an error, but rather an indicator of where additional work is needed for compatibility before PHP 9 (i.e. when the notices become fatal errors). With a deprecation notice, the PHP code will continue to work and nothing is broken.
Follow-up to [49077], [49162], [50299], [51588], [51604], [53922], [54072].
Props jrf, desrosj.
See #55652, #55656, #56009, #56681.
git-svn-id: https://develop.svn.wordpress.org/trunk@54369 602fd350-edb4-49c9-b593-d223f7449a82
This was included in the original pull request that aimed to add support for `octokit/plugin-retry.js` in `actions/github-scripts`, but was actually removed before being merged.
Follow up to [54342].
See #55652.
git-svn-id: https://develop.svn.wordpress.org/trunk@54343 602fd350-edb4-49c9-b593-d223f7449a82
This version adds support for `octokit/plugin-retry.js`, which retries requests automatically when `4xx` or `5xx` response codes are returned.
To start, the feature is configured to retry all `4xx` and `5xx` response codes, unless the server identifies as a teapot.
See #55652.
git-svn-id: https://develop.svn.wordpress.org/trunk@54342 602fd350-edb4-49c9-b593-d223f7449a82
As of version `2.17.0` of the `shivammathur/setup-php` action, the `COMPOSER_NO_INTERACTION` environment variable is configured by default. `—-no-interaction` will always be used.
Props jrf, hellofromTonya, SergeyBiryukov, costdev, desrosj.
Fixes#54695.
git-svn-id: https://develop.svn.wordpress.org/trunk@54313 602fd350-edb4-49c9-b593-d223f7449a82
Because of the scripts that run when `build:dev` is run, it’s more common for this Grunt task to change version-controlled files than when building WordPress to run from `build`.
This moves the `build:dev` tests before the `build` ones in order to detect changes earlier in the workflow.
See #55652.
git-svn-id: https://develop.svn.wordpress.org/trunk@54297 602fd350-edb4-49c9-b593-d223f7449a82
The current `timeout-minutes` value of `20` is a bit too short for MacOS jobs in GitHub Actions, which on occasion take a bit longer.
This bumps that limit to `30` to avoid unnecessarily flagging a job as stuck.
See #55652.
git-svn-id: https://develop.svn.wordpress.org/trunk@54293 602fd350-edb4-49c9-b593-d223f7449a82
Allows to revert changes applied in [54277] - temporary workaround for the failing Test NPM CI check on Windows.
Improvements included:
- generate combined asset files for both production and development
- store in the repository only the production version of the combined assets for packages, we use everything else only in development
- to make unit tests work, ensure that they ignore react fast refresh and use the production version of combined assets that are present in the source code
Props bernhard-reiter, jsnajdr, clorith, wildworks.
Fixes#56615.
git-svn-id: https://develop.svn.wordpress.org/trunk@54289 602fd350-edb4-49c9-b593-d223f7449a82
This changes the Test NPM GitHub Action workflow to temporarily allow the job validating build tools on Windows to fail.
When the `build:dev` Grunt task is run on Windows, the resulting hashes calculated by Webpack are different than other platforms. This seems to be related to how the `remove-accents` dependency is read and processed during the build script.
Since a Windows machine is not used to build WordPress on the build server, this will only affect local development installs for Windows contributors. While this is investigated, this workflow job can be allowed to fail.
Props bernhard-reiter, desrosj, Clorith, gziolo.
See #56615.
git-svn-id: https://develop.svn.wordpress.org/trunk@54277 602fd350-edb4-49c9-b593-d223f7449a82
Previously, it was not possible to use the default `GITHUB_TOKEN` token to create new workflow runs in an effort to prevent accidental recursive workflows.
This has changed, and the `workflow_dispatch` is now one of two exceptions to this rule. Using `GITHUB_TOKEN` is preferred whenever possible to avoid the need for a PAT (personal access token), which expires (when created using the recommended security best practices), and is tied to an individual user.
See https://github.blog/changelog/2022-09-08-github-actions-use-github_token-with-workflow_dispatch-and-repository_dispatch/.
See #55652.
git-svn-id: https://develop.svn.wordpress.org/trunk@54108 602fd350-edb4-49c9-b593-d223f7449a82
This affects the following test groups:
* Ajax tests
* ms-files tests
* External HTTP tests
* Xdebug tests
The tests being run in these particular test groups are passing on PHP 8.1, however, the test runs are still allowed to “continue on error”. This creates the risk that new PHP 8.1 incompatibilities will be introduced without anyone noticing.
By no longer allowing these test runs to “continue on error”, that risk is removed.
Follow-up to [51588], [51604], [53922].
Props jrf.
See #55656, #55652.
git-svn-id: https://develop.svn.wordpress.org/trunk@54072 602fd350-edb4-49c9-b593-d223f7449a82
The correct variable here is `github.run_attempt`, which represents the unique number for each attempt of a particular workflow run in a repository.
The `github.run_number` currently being used represents the unique number for each run of a particular workflow in a repository.
Follow up to [53947].
Fixes#56407.
git-svn-id: https://develop.svn.wordpress.org/trunk@54039 602fd350-edb4-49c9-b593-d223f7449a82
