Week in Test – 1 Nov 2021

Welcome to this week’s edition of Week in Test! This post is a curated list of where you can get involved (i.e. where testers of all skill levels and expertise are needed), learning opportunities, and some reading to keep you informed.

This week focuses on helping with 5.9 new features and enhancements. Remember, feature freeze is 9 Nov.

🙋‍♀️ 🙋 Contributing: Tester Help Needed

Looking for ways to contribute? The following tickets and patches need contributors.

Manual testing help needed

Who? All contributors (not just developers) who can set up a local testing environment, apply patches, and test per the testing instructions.

The following tickets need testers to manual test and provide feedback (test report):

PHPUnit tests help needed

Who? Any QA or PHPPHP PHP (recursive acronym for PHP: Hypertext Preprocessor) is a widely-used open source general-purpose scripting language that is especially suited for web development and can be embedded into HTML. http://php.net/manual/en/intro-whatis.php. developer contributors who can (or is interested in learning how to) build automated PHPUnit tests.

The following tickets need PHPUnit tests build:

  • #15145: Add a wp_list_users() template tag
  • #54331: Add a hook in wp_http_validate_url to control which ports are allowed for remote requests

Reproducing reported issue help needed:

Who? Any contributor.

The following new tickets need testers to attempt reproducing the reported issue and then providing a test report with the results:

  • #54320: There are no more thumbnails for updated PDFs
  • #54430: Twenty Twenty-One: Menu item focus background only works on Chrome (ignored on Safari)
  • #54354: open_basedir warnings on classic post edit screens with TinyMCE plugins present

Reading / Watching

Meetings This Week

  • 2 Nov 2021: Test Triage session starting at 13:00 UTC
  • 5 Nov 2021: Test Scrub starting at 13:15 UTC

Props to @boniu91 for peer review.

#build-test-tools, #core-test

Week in Test – 11 Oct 2021

Welcome to this week’s edition of Week in Test! This post is a curated list of where you can get involved (i.e. where testers of all skill levels and expertise are needed), learning opportunities, and some reading to keep you informed.

🙋‍♀️ 🙋 Contributing: Tester Help Needed

Looking for ways to contribute? The following tickets and patches need contributors.

Manual testing help needed

Who? All contributors (not just developers) who can set up a local testing environment, apply patches, and test per the testing instructions.

The following tickets need testers to manual test and provide feedback (test report):

  • #53801 BlockBlock Block is the abstract term used to describe units of markup that, composed together, form the content or layout of a webpage using the WordPress editor. The idea combines concepts of what in the past may have achieved with shortcodes, custom HTML, and embed discovery into a single consistent API and user experience.-based widgets screen does action wp_footer after each widgetWidget A WordPress Widget is a small block that performs a specific function. You can add these widgets in sidebars also known as widget-ready areas on your web page. WordPress widgets were originally created to provide a simple and easy-to-use way of giving design and structure control of the WordPress theme to the user.
  • #52294: TT1 editor styles broken in RTL
  • #16206: (test patch 16206.8.patch) Comment text not marked as required
  • #54243: Inlined block styles for external assets (images/fonts) and relative URLs not working as expected (patch is upstream in GutenbergGutenberg The Gutenberg project is the new Editor Interface for WordPress. The editor improves the process and experience of creating new content, making writing rich content much simpler. It uses ‘blocks’ to add richness rather than shortcodes, custom HTML etc. https://wordpress.org/gutenberg/)

PHPUnit tests help needed

Who? Any QA or PHPPHP PHP (recursive acronym for PHP: Hypertext Preprocessor) is a widely-used open source general-purpose scripting language that is especially suited for web development and can be embedded into HTML. http://php.net/manual/en/intro-whatis.php. developer contributors who can (or is interested in learning how to) build automated PHPUnit tests.

The following tickets need PHPUnit tests build:

  • #47642: Order by comment count – posts list tables
  • #49985: REST APIREST API The REST API is an acronym for the RESTful Application Program Interface (API) that uses HTTP requests to GET, PUT, POST and DELETE data. It is how the front end of an application (think “phone app” or “website”) can communicate with the data store (think “database” or “file system”) https://developer.wordpress.org/rest-api/.: Using _embed and _fields query parameters in the same query
  • #52252: PHP Notice when monthnum query var is set without the year QV

Reproducing reported issue help needed:

Who? Any contributor.

The following new tickets need testers to attempt reproducing the reported issue and then providing a test report with the results:

  • Currently no tickets in the queue are ready for testing

Calls for Testing

Reading

Meetings This Week

Props to @boniu91 for peer review.

#build-test-tools, #core-test

5.9 End-to-End (e2e) Strategy Session Summary

A working session was held today to discuss big picture goals and what to accomplish during the 5.9 cycle.

Where e2e tests live: GutenbergGutenberg The Gutenberg project is the new Editor Interface for WordPress. The editor improves the process and experience of creating new content, making writing rich content much simpler. It uses ‘blocks’ to add richness rather than shortcodes, custom HTML etc. https://wordpress.org/gutenberg/ vs. CoreCore Core is the set of software required to run WordPress. The Core Development Team builds WordPress.

Consensus reached as to where e2e tests will live within each repository:

  • Core: non-Gutenberg features, UIs, and UXUX UX is an acronym for User Experience - the way the user uses the UI. Think ‘what they are doing’ and less about how they do it. including:
    • new stuff being built during a release cycle
    • existing interfaces such as CustomizerCustomizer Tool built into WordPress core that hooks into most modern themes. You can use it to preview and modify many of your site’s appearance settings., login, pluginPlugin A plugin is a piece of software containing a group of functions that can be added to a WordPress website. They can extend functionality or add new features to your WordPress websites. WordPress plugins are written in the PHP programming language and integrate seamlessly with WordPress. These can be free in the WordPress.org Plugin Directory https://wordpress.org/plugins/ or can be cost-based plugin from a third-party and theme installation/activation, interaction with list tables, settings page, etc.
  • Gutenberg: its functionality being built within its repository

5.9 Core e2e focus areas

All recent major changes to Core with user facing functionalities are top priorities to receive e2e tests as part of WordPress 5.9. These areas are:

  • The application passwords feature
  • The uploading of new versions of plugins/themes feature
  • The plugin dependency project
  • The updater project
  • Twenty Twenty-Two theme

Porting tests from Gutenberg to Core

tl;dr:

  • Tests that test Gutenberg’s functionality will remain in its repository and not be ported to Core
  • Core specific tests (such as login) can be moved to Core

Along with the previous areas to test, the following priority would be to port some Core tests that are currently implemented in Gutenberg (e2e-test-utils package). Concerned tests are those related to the login feature.

The next step after this would be to improve the login tests to make them more performant. This includes for instance cookie based authentication across all tests in Core and Gutenberg.

Other Roadmap Items

These items are not necessarily part of 5.9 Goals. However, these are part of the bigger picture Test Roadmap.

  • Documentation: The goal is to help contributors quickly contribute to testing.
    • Test Handbook: Clearly document multiple workflows for folks to pick their onramp into testing
    • README: #53550 get its PR reviewed and merged
  • Visual Regression: #49606 is an experiment to allow local vision regression testing.
    • From these learnings, plans can be crafted for how to build it into an automated CI process.
    • The challenges for the CI are storage of the artifacts and unreliability of testing these across different environments. A third party service may be possibility to explore in the future.
  • Is Core a good experimental sandbox for Playwright?

Proposal to migrate to Playwright for e2e tests

tl;dr:

  • Not a blocker to build Core’s e2e tests
  • Requires changes in e2e-test-utils package (maintained by Gutenberg)
  • Needs a migrationMigration Moving the code, database and media files for a website site from one server to another. Most typically done when changing hosting companies. plan

All agreed Playwright for e2e tests would indeed have a lot of advantages for both Core and Gutenberg. However, consideration must be given for breaking changes and impacts for extenders using the test utils. Kai noted the utils could be made compatible with both Puppeteer and Playwright, meaning no breaking changes.

Also, a migration plan will need to be discussed on developed once the test utils are ready for Playwright. Part of this plan is to figure out how to change Gutenberg and Core as well as how to upgrade existing, not yet committed patches/PRs. The how and when is yet to be determined.

As there is still work to do to prepare for Playwright, the team agreed to continue with the roadmap to build e2e tests in Core.

Props @hellofromTonya for peer review and proofreading.

#build-test-tools, #core-test, #summaries

Week in Test – 4 Oct 2021

Welcome to this week’s edition of Week in Test! This post is a curated list of where you can get involved (i.e. where testers of all skill levels and expertise are needed), learning opportunities, and some reading to keep you informed.

🙋‍♀️ 🙋 Contributing: Tester Help Needed

Looking for ways to contribute? The following tickets and patches need contributors.

Manual testing help needed

Who? All contributors (not just developers) who can set up a local testing environment, apply patches, and test per the testing instructions.

The following tickets need testers to manual test and provide feedback (test report):

  • #53275: Wrap options on pluginPlugin A plugin is a piece of software containing a group of functions that can be added to a WordPress website. They can extend functionality or add new features to your WordPress websites. WordPress plugins are written in the PHP programming language and integrate seamlessly with WordPress. These can be free in the WordPress.org Plugin Directory https://wordpress.org/plugins/ or can be cost-based plugin from a third-party page to second line Done ✅
  • #53801 BlockBlock Block is the abstract term used to describe units of markup that, composed together, form the content or layout of a webpage using the WordPress editor. The idea combines concepts of what in the past may have achieved with shortcodes, custom HTML, and embed discovery into a single consistent API and user experience.-based widgets screen does action wp_footer after each widgetWidget A WordPress Widget is a small block that performs a specific function. You can add these widgets in sidebars also known as widget-ready areas on your web page. WordPress widgets were originally created to provide a simple and easy-to-use way of giving design and structure control of the WordPress theme to the user.
  • #52241: Windows OS specific – infinite loopLoop The Loop is PHP code used by WordPress to display posts. Using The Loop, WordPress processes each post to be displayed on the current page, and formats it according to how it matches specified criteria within The Loop tags. Any HTML or PHP code in the Loop will be processed on each post. https://codex.wordpress.org/The_Loop. in clean_dirsize_cache() Done ✅

PHPUnit tests help needed

Who? Any QA or PHPPHP PHP (recursive acronym for PHP: Hypertext Preprocessor) is a widely-used open source general-purpose scripting language that is especially suited for web development and can be embedded into HTML. http://php.net/manual/en/intro-whatis.php. developer contributors who can (or is interested in learning how to) build automated PHPUnit tests.

The following tickets need PHPUnit tests build:

  • #47642: Order by comment count – posts list tables
  • #52241: Windows OS specific – infinite loop in clean_dirsize_cache() Done ✅
  • #50567: Set $post in update_post_cache()
  • #49985: REST APIREST API The REST API is an acronym for the RESTful Application Program Interface (API) that uses HTTP requests to GET, PUT, POST and DELETE data. It is how the front end of an application (think “phone app” or “website”) can communicate with the data store (think “database” or “file system”) https://developer.wordpress.org/rest-api/.: Using _embed and _fields query parameters in the same query
  • #52252: PHP Notice when monthnum query var is set without the year QV

Reproducing reported issue help needed:

Who? Any contributor.

The following new tickets need testers to attempt reproducing the reported issue and then providing a test report with the results:

  • #54192get_header_image_tag doesn’t add srcset/sizes Needs reporter feedback ✅
  • #54205 – jqxhr is undefined inside of deferred.done() when using wp.media to add a custom image upload Needs reporter feedback ✅
  • #54211 – Small css bug when using customize-controls in customizer.php Done ✅

Reproducing the reported issue is the first step in a new defect ticket’s lifecycle. Why? In order to fix a bug, first step is confirm the bug is reproducible and is due to WordPress CoreCore Core is the set of software required to run WordPress. The Core Development Team builds WordPress. itself (and not a third party like a plugin or theme).

Calls for Testing

Reading

Meetings This Week

Props to @boniu91 for peer review.

#build-test-tools, #core-test

Test Team Chat Agenda for 28 Sep 2021

Here is the agenda for the upcoming Test Team Chat scheduled for 2021-09-28 13:00.

This meeting is held in the #core-test channel in the Making WordPress slackSlack Slack is a Collaborative Group Chat Platform https://slack.com/. The WordPress community has its own Slack Channel at https://make.wordpress.org/chat/..

Highlights

Agenda

  • Changes to coreCore Core is the set of software required to run WordPress. The Core Development Team builds WordPress.’s PHPPHP PHP (recursive acronym for PHP: Hypertext Preprocessor) is a widely-used open source general-purpose scripting language that is especially suited for web development and can be embedded into HTML. http://php.net/manual/en/intro-whatis.php. test suite -> What does this mean for test?
  • 5.9 Test Roadmap
  • Focal group updates
  • Any blockers?
  • Open floor

Open Floor

Do you have something to propose for the agenda? Please leave a comment below.

Can’t make the meeting? Share anything relevant for the discussion in a comment below.

Props: @boniu91 for peer review.

#agenda, #build-test-tools

Week in Test – 27 Sep 2021

Welcome to this week’s edition of “Week in Test”, a curated list of where you as a contributor can get involved (testers needed), learning opportunities, and some reading to keep you informed.

🙋‍♀️ 🙋 Contributing: Tester Help Needed

Looking for ways to contribute? The following tickets and patches need contributors.

Manual testing help needed

Who? All contributors (not just developers) who can set up a local testing environment, apply patches, and test per the testing instructions.

The following tickets need testers to manual test and provide feedback (test report):

  • #52224: RSS WidgetWidget A WordPress Widget is a small block that performs a specific function. You can add these widgets in sidebars also known as widget-ready areas on your web page. WordPress widgets were originally created to provide a simple and easy-to-use way of giving design and structure control of the WordPress theme to the user.: allow removing the feed icon link
  • #48787: Classic Editor user interface CSSCSS CSS is an acronym for cascading style sheets. This is what controls the design or look and feel of a site. inconsistencies when toggling “Enable full-height editor …”
  • #52241: Windows OS specific – infinite loopLoop The Loop is PHP code used by WordPress to display posts. Using The Loop, WordPress processes each post to be displayed on the current page, and formats it according to how it matches specified criteria within The Loop tags. Any HTML or PHP code in the Loop will be processed on each post. https://codex.wordpress.org/The_Loop. in clean_dirsize_cache()

PHPUnit tests help needed

Who? Any QA or PHPPHP PHP (recursive acronym for PHP: Hypertext Preprocessor) is a widely-used open source general-purpose scripting language that is especially suited for web development and can be embedded into HTML. http://php.net/manual/en/intro-whatis.php. developer contributors who can (or is interested in learning how to) build automated PHPUnit tests.

The following tickets need PHPUnit tests build:

  • #47642: Order by comment count – posts list tables
  • #52241: Windows OS specific – infinite loop in clean_dirsize_cache()
  • #50567: Set $post in update_post_cache()
  • #49985: REST APIREST API The REST API is an acronym for the RESTful Application Program Interface (API) that uses HTTP requests to GET, PUT, POST and DELETE data. It is how the front end of an application (think “phone app” or “website”) can communicate with the data store (think “database” or “file system”) https://developer.wordpress.org/rest-api/.: Using _embed and _fields query parameters in the same query
  • #52252: PHP Notice when monthnum query var is set without the year QV

Reproducing reported issue help needed:

Who? Any contributor.

The following new tickets need testers to attempt reproducing the reported issue and then providing a test report with the results:

  • #54147 – XMLRPC APIAPI An API or Application Programming Interface is a software intermediary that allows programs to interact with each other and share data in limited, clearly defined ways. ignores empty terms_names array
  • #54161 – WordPress destroy IIS web.config when using “location” config
  • #54169GutenbergGutenberg The Gutenberg project is the new Editor Interface for WordPress. The editor improves the process and experience of creating new content, making writing rich content much simpler. It uses ‘blocks’ to add richness rather than shortcodes, custom HTML etc. https://wordpress.org/gutenberg/ issue – when columns are set to Full width they extend beyond editor window bounds
  • #54181 – Terms management screen shows bulk actions dropdown even if there are no terms

Reproducing the reported issue is the first step in a new defect ticket’s lifecycle. Why? In order to fix a bug, first step is confirm the bug is reproducible and is due to WordPress CoreCore Core is the set of software required to run WordPress. The Core Development Team builds WordPress. itself (and not a third party like a pluginPlugin A plugin is a piece of software containing a group of functions that can be added to a WordPress website. They can extend functionality or add new features to your WordPress websites. WordPress plugins are written in the PHP programming language and integrate seamlessly with WordPress. These can be free in the WordPress.org Plugin Directory https://wordpress.org/plugins/ or can be cost-based plugin from a third-party or theme).

Reading

Props to @boniu91 for peer review.

#build-test-tools, #core-test

5.9 End-to-End (e2e) Working Strategy Session Agenda – RESCHEDULED for 6 Oct 2021

With efforts in progress to introduce end-to-end (e2e) testing in CoreCore Core is the set of software required to run WordPress. The Core Development Team builds WordPress., this working session brings folks together to collectively define the big picture goals and what to accomplish during the 5.9 cycle (i.e. roadmap). These goals can then serve as the North Star 🌟 to guide contributors for maximizing impact.

When: RESCHEDULED for 2021-10-06 10:00
Where: Zoom call (link will be shared in the Making WordPress slackSlack Slack is a Collaborative Group Chat Platform https://slack.com/. The WordPress community has its own Slack Channel at https://make.wordpress.org/chat/. #core-test channel prior to the start of the session)
Topic: Strategize e2e goals for the 5.9 release cycle

Agenda

  • Define big picture goals of what to accomplish during the 5.9 cycle
  • Identify and prioritize areas of Core that could most benefit from e2e tests
  • Discuss if any GutenbergGutenberg The Gutenberg project is the new Editor Interface for WordPress. The editor improves the process and experience of creating new content, making writing rich content much simpler. It uses ‘blocks’ to add richness rather than shortcodes, custom HTML etc. https://wordpress.org/gutenberg/ tests can be ported over to Core
  • Strategize stepping stones
  • Identify key areas of contributor needs and responsibilities

Resources

Props to @justinahinon for proofreading.

#agenda, #build-test-tools

Week in Test – 20 Sep 2021

Welcome to this week’s edition of “Week in Test”. This post highlights where you as a contributor can get involved (testers needed), learning opportunities, and some reading to keep you informed.

🙋‍♀️ 🙋 Contributing: Tester Help Needed

Looking for ways to contribute? The following tickets and patches need contributors.

Manual testing help needed

Who? All contributors (not just developers) who can set up a local testing environment, apply patches, and test per the testing instructions.

The following are tickets and patches that need testers to manual test and provide feedback (test report):

  • #52241: Windows OS specific – infinite loopLoop The Loop is PHP code used by WordPress to display posts. Using The Loop, WordPress processes each post to be displayed on the current page, and formats it according to how it matches specified criteria within The Loop tags. Any HTML or PHP code in the Loop will be processed on each post. https://codex.wordpress.org/The_Loop. in clean_dirsize_cache()
  • #53429: Twenty Twenty-One dark mode (for 5.8.2) Done

PHPUnit tests help needed

Who? Any QA or PHPPHP PHP (recursive acronym for PHP: Hypertext Preprocessor) is a widely-used open source general-purpose scripting language that is especially suited for web development and can be embedded into HTML. http://php.net/manual/en/intro-whatis.php. developer contributors who can (or is interested in learning how to) build automated PHPUnit tests.

The following tickets need PHPUnit tests build:

  • #47642: Order by comment count – posts list tables
  • #52241: Windows OS specific – infinite loop in clean_dirsize_cache()
  • #50567: Set $post in update_post_cache()
  • #49985: REST APIREST API The REST API is an acronym for the RESTful Application Program Interface (API) that uses HTTP requests to GET, PUT, POST and DELETE data. It is how the front end of an application (think “phone app” or “website”) can communicate with the data store (think “database” or “file system”) https://developer.wordpress.org/rest-api/.: Using _embed and _fields query parameters in the same query
  • #52252: PHP Notice when monthnum query var is set without the year QV

Reproducing reported issue help needed:

Who? Any contributor.

Since last week, there are 10 new tickets which need testers to attempt reproducing the reported issue and then providing a test report with the results.

Reproducing the reported issue is the first step in a new defect ticket’s lifecycle. Why? In order to fix a bug, first step is confirm the bug is reproducible and is due to WordPress CoreCore Core is the set of software required to run WordPress. The Core Development Team builds WordPress. itself (and not a third party like a pluginPlugin A plugin is a piece of software containing a group of functions that can be added to a WordPress website. They can extend functionality or add new features to your WordPress websites. WordPress plugins are written in the PHP programming language and integrate seamlessly with WordPress. These can be free in the WordPress.org Plugin Directory https://wordpress.org/plugins/ or can be cost-based plugin from a third-party or theme).

Reading

Props to @boniu91 for peer review.

#build-test-tools, #core-test, #testing-plugins

Test Team Chat Summary: 14 September 2021

The meeting started on SlackSlack Slack is a Collaborative Group Chat Platform https://slack.com/. The WordPress community has its own Slack Channel at https://make.wordpress.org/chat/. here.

Short explanation for first time attendees

@hellofromtonya reminded, that this bi-weekly meeting is where people who test WordPress CoreCore Core is the set of software required to run WordPress. The Core Development Team builds WordPress. gather to discuss important things regarding test contribution. Testing includes manual testing, attempting reproduce reported issues, and automated testing.

You can find the agenda meeting here.

Week in Test introduction

@hellofromtonya described the purpose of this weekly publication, in short, it includes:

  • Parts of the core where testing help is needed this week
  • Learning and reading opportunities

Team agreed that it’s very helpful.

Last week’s Team update introduction

@hellofromtonya reminded that it’s a Team update, an overview of what has happeed last week. It also includes stats that are related to our team.

The Team agreed that the queue of the items that need testers’ attention is long.

Focal group updates

@hellofromtonya explained that it’s a time for testers to share updates for testing project initiatives. This section has started with:

  • PHPPHP PHP (recursive acronym for PHP: Hypertext Preprocessor) is a widely-used open source general-purpose scripting language that is especially suited for web development and can be embedded into HTML. http://php.net/manual/en/intro-whatis.php. 8.1 fixes are underway
  • Modernization to Latest PHPUnit project is nearly finished. Backporting is next, then dev note
  • PHPUnit Test modernization is ongoing

@hareesh added:

  • The latest version of jQuery UIUI UI is an acronym for User Interface - the layout of the page the user interacts with. Think ‘how are they doing that’ and less about what they are doing. was recently merged to Core. Need a lot of testing.

Documentation strategy

@francina opened this discussion, asking what instructions we want to give. The main reason was related to setting up environment.

@hellofromtonya proposed to create a docs empowering everyone to contribution, we need clear entry points for:

  • Local machine setups
  • How to do different types of testing
  • How to create test reports

Team agreed that this documentation should be present in Make Test website, Make Core should only refer to it.

@hellofromtonya mentioned, that PHPUnit docs page lacks a reference to where contributors can set up their local machines. That should be a priority.

@francina @juhise @hellofromtonya agreed that the Team should start documenting the steps for Docker testing environment and later repeat the same pattern for other solutions. On of the concerns is that screenshot are getting outdated very quickly.

@mkaz shown blockBlock Block is the abstract term used to describe units of markup that, composed together, form the content or layout of a webpage using the WordPress editor. The idea combines concepts of what in the past may have achieved with shortcodes, custom HTML, and embed discovery into a single consistent API and user experience.-editor’s development site setup instructions based on wp-env. Team agreed that all projects should be pointing to one place where everything related to development environment is stored and maintained.

@hellofromtonya proposed a step-by-step instrctions of creating documentation for different types of test environments:

  • Start with Docker
  • Setups on Make Test with reference in Make Core
  • Figure out the doc info structure / strategy
  • Doc for different OS: Windows, Mac, and Linux
  • Link to tooling’s official docs
  • Get feedback from contributors
  • Refine
  • Repeat

@francina proposed working session with setting up environments. It’s going to happen on 2021-09-23 14:00

People with different OS will be essential for the successful meeting.

@boniu91 shared a reminder that on 2021-09-17 13:15 the Test Scrub is going to happen taking care of the following tickets:

#build-test-tools, #core-test, #test

Test Team Chat Agenda for 14 Sep 2021

Here is the agenda for the upcoming Test Team Chat scheduled for 2021-09-14 13:00.

This meeting is held in the #core-test channel in the Making WordPress slackSlack Slack is a Collaborative Group Chat Platform https://slack.com/. The WordPress community has its own Slack Channel at https://make.wordpress.org/chat/..

Agenda

  • Week in Test: where you can contribute and learn this week
  • Review last week’s Team update
  • Focal group updates
  • Open floor

Open Floor

Do you have something to propose for the agenda? Please leave a comment below.

Can’t make the meeting? Share anything relevant for the discussion in a comment below.

Props: @boniu91 for peer review.

#agenda, #build-test-tools