Core Privacy Agenda – 14 Nov 2018

When/Where: Join us in #core-privacy on Making WordPress Slack from 1500-1600 UTC

Agenda:

  • New time for core privacy weekly chat starting next week (@desrosj)
  • WP 5 drops Tue Nov 27 – do we need another quick round of WP 5 compatibility testing with privacy features? (@allendav)
  • Tickets
    • Update on moving core files from @allendav (core:43895) – developing a plan here – https://docs.google.com/document/d/1naG6Bs2RK7j1PkKXmCdflY3ei2ZgZtV6zHCWeRz01s0/edit
  • Needs-privacy-review tag
    • Update from @garrett-eclipse (meta:3896) – “wrote the patch and it’s now awaiting a commit. There’s two additional items after commit which is including links but we’ll hold on that until the tag starts to get some usage”
  • Raising awareness of core privacy work and meetings
    • Update from @riankinney re marketing newsletter — getting the word out about Office Hours, our need for developers and end-user feedback
  • “What is privacy in our context?”
    • What is privacy in the sense of what areas do we look at in our work @idea15
    • Separating the theory from the practice (e.g. our dev guidelines)
    • Update on working with Chris Tiezel for cross-platform definitions @idea15

Items postponed to next week:

  • WP mobile app permissions and tracking (@allendav)
  • Follow up on 3rd party code (Twitter, Facebook, Quantcast?) on WP.org footers (@allendav)

What the Cool Privacy Kids Are Reading / Trying This Week:

  • 60 Minutes on GDPR – https://www.cbsnews.com/news/gdpr-the-law-that-lets-europe-take-back-their-data-from-big-tech-companies-60-minutes/ props @idea15
  • ICO updates https://ico.org.uk/for-organisations/guide-to-the-general-data-protection-regulation-gdpr/security/passwords-in-online-services/ https://ico.org.uk/for-organisations/guide-to-the-general-data-protection-regulation-gdpr/security/ https://ico.org.uk/for-organisations/guide-to-the-general-data-protection-regulation-gdpr/accountability-and-governance/data-protection-by-design-and-default/ props David Orf
  • WP GDPR Compliance Plugin Privilege Escalation Flaw https://www.wordfence.com/blog/2018/11/privilege-escalation-flaw-in-wp-gdpr-compliance-plugin-exploited-in-the-wild/ props @garrett-eclipse

Upcoming WordCamp (and other Event) Privacy Talks:

  • Recent talks
    • @chrisweigman Encrypt All the Things – WordCamp Orlando https://2018.orlando.wordcamp.org/schedule/sessions/
    • @riankinney Ethics on the Web – WordCamp Orlando https://2018.orlando.wordcamp.org/schedule/sessions/
    • Chris Teitzel – With Great Power Comes Great Responsibility – WordCamp Seattle – https://2018.seattle.wordcamp.org/session/with-great-power-comes-great-responsibility/
  • Planned talks
  • Opportunities

Helpful links:

  • Core Privacy Component Home Page https://make.wordpress.org/core/components/privacy/
  • Core Privacy Posts https://make.wordpress.org/core/tag/core-privacy/
  • Core Privacy Roadmap https://make.wordpress.org/core/roadmap/privacy/
  • Plugin Privacy Handbook https://developer.wordpress.org/plugins/privacy/
  • Open Privacy Component Tickets https://core.trac.wordpress.org/query?status=accepted&status=assigned&status=new&status=reopened&status=reviewing&component=Privacy&col=id&col=summary&col=status&col=owner&col=type&col=priority&col=milestone&col=component&order=priority

#core-privacy, #privacy

JavaScript Chat Summary – November 13th

Below is a summary of the discussion from this week’s JavaScript chat (agendaSlack transcript).

Have a topic for discussion for the next meeting? Leave a suggested edit on next week’s agenda.

Npm packages publishing workflow

Discussion: When working on the WordPress 5.0 release cycle we noticed that our workflow for publishing packages to npm isn’t ideal. Using Lerna helps a lot but it doesn’t solve all issues. We should work towards improving the workflow and address the items listed in this GitHub issue.

Considerations:

  • At the moment we defer to Lerna performing version increments for all packages. We only express our intent for the change inside of the corresponding changelog files when working on PRs. The biggest downside of this approach is that we need to scan all existing changelog files during publishing and re-apply manually the proper version bump for each package. It would be a nice improvement if we could update both changelog and package.json with a new version at the same time when PR gets merged. Lerna could use that and skip the step to ask what version should be applied.
  • In the past, we had publish failures caused by 2FA timeout where we had more packages to publish than it could be handled in 30 seconds. There is an open issue where improved 2FA Support is discussed. However, 2FA isn’t the only thing that could go wrong in the process so it would be nice to see publish re-tries in the flow and prompts for new OTP token when the current one expires.
  • It would be nice to see is a way to verify if published packages are healthy. As mentioned above, we had 2 cases where 2FA timeout caused some issues which led to having broken package.json files on npm. It might happen again, so we should be at least able to detect it immediately and take proper actions.

Action Items:

  • Share what we discussed today with Lerna team in RFC: Improving lerna publish issue in their repository and figure out how we can work together to improve our workflow.
  • Continue discussion next week.

i18n Core Merge Update

We have an update on the i18n front in that support for JavaScript localization has been committed for WordPress 5.0 and support added to translate.wordpress.org. You can read more at the following links:

The major functionality has launched and only bug fixes are likely to remain. It’s safe to say that plugin and theme support can be added after 5.0 release.

Open discussion

Question: @nerrad asked whether the @wordpress package namespace on npm is intended for only things used specifically by the WordPress team or project. Are they also things we think contribute to the larger JavaScript ecosystem that might use WordPress?

@aduth shared that we should be spending resources most toward things which are actively used in the development of WordPress. If someone wants to contribute enhancements or fixes, that on its own would be fine. We also don’t need to pull projects no longer used.

It was also discussed that the deprecation feature offered by npm should work well for cases when we know up front that a package won’t be maintained anymore.

5.0 / Gutenberg Status Update – Nov 13

Current open PRs for review: 44 in 4.4 Milestones (+9)
Current open issues milestoned RC: 148 (-14)
Current open issues milestoned 5.0.0: 3 (+1)
Current open issues milestoned 5.0.1: 2
Current open issues milestoned 5.0.x (fast follow): 19 (+4)
Current beta: Beta 4, released today.
Current plugin: 4.3, released yesterday.
Next beta: Beta 5, planned for Thursday.

Note: this is a daily update on progress towards WordPress 5.0, as outlined in this post. This status update is also shared in the #core-editor channel in Slack.

What’s new in Gutenberg? (12th November)

This update gathers a big number of fixes and improvements. It also completes many accessibility tasks — the original milestone for blockers is now closed.

Version 4.3 is fully released and Beta 4 will be coming later in the day.

Showing integration with the 2019 theme

4.3 / WordPress 5.0 (Beta 4) 🐺

Bug Fixes

Other Changes

Testing

Documentation

Mobile Apps

Deprecations removed with this version.

Status Update

  • Current PRs for review: 35 in 4.4 Milestone.
  • Open issues in 5.0 RC: 162.
  • Open issues in 5.0.0: 2.
  • Open issues in 5.0.1: 2.
  • Open issues in 5.0.x (fast follow): 15.
  • Current beta: Beta 3, November 5.
  • Current plugin: 4.3, released today.
  • Next beta: Beta 4, planned for today.

#core-editor, #editor, #gutenberg

Update on 5.0 Release Schedule

As discussed during the Core devchat this week, the initial November 19th target date is looking a bit too soon for a release date. After listening to a lot of feedback — as well as looking at current issues, ongoing pull requests, and general progress — we’re going to take an extra week to make sure everything is fully dialed in and the release date is now targeted for Tuesday, November 27th (within the space that was originally outlined).

Taking a step back, it’s great to see all the progress since the days of the prototypes and how full theme integration, like with the 2019 theme, can make the experience come together…

It has also proven difficult to align plugin releases (and their release candidates) with beta releases of core, process wise, so here’s the outline for the final stretch:

  • Core betas to go out on Monday, November 12th (beta 4) and Thursday, November 15th (beta 5) aligning with plugin releases (4.3 and 4.4).
  • Work to be organized and prioritized around three milestones: 5.0 Release Candidate, 5.0 Launch, 5.0.1, and 5.0.x (planning for +0.0.1 every two weeks following 5.0).
  • 5.0 RC1 targeted for Monday, November 19th.
  • Daily high-level updates on current status (open pull requests to be reviewed, outstanding bugs, etc) in the core-editor channel.

The last part of delivering a project of this scale is always hard and emotions run high. I’d like to take a step back and truly thank everyone for the staggering amount of work, attention, and care being put into preparing this release. I am grateful for the hard work of all contributors — not just those submitting code — including those who have given candid and valuable feedback throughout. It is all helping us move towards a better product.

#5-0

PHP Meeting Recap – November 5th

This recap is a summary of our previous PHP meeting. It highlights the ideas and decisions which came up during that meeting, both as a means of documenting and to provide a quick overview for those who were unable to attend.

You can find this meeting’s chat log here.

Chat Summary

  • Note that, in order to maintain the original time in most parts of the world after the end of daylight saving time, the meeting time has been adjusted from 15:00 UTC to 16:00 UTC.
  • Due to the focus on Gutenberg and the resulting lack of activity in other areas, this and the following meetings should preferably be used for some “housekeeping”, reviewing the project roadmap and planning the next steps after 5.0.
  • @flixos90 asked to review the Servehappy feature project page, since that has not been updated in a long time. The page would later be updated to reflect some of the additional requirements that came up mid-2018 and to show the currently planned timelines for all features in the project’s scope, plus links to relevant resources and tickets.
  • @afragen brought up #44619 which had not been tagged with the “servehappy” keyword before, mentioning that the ticket is close to ready. @sergeybiryukov pointed out that it should receive design feedback first.
  • @joyously asked about the plans for themes requiring certain PHP versions. While nothing has been developed in this regard yet, themes should eventually be able to specify a minimum required PHP version, just like plugins. This information should be available through the Themes API, and WordPress core should implement techniques to restrict installation, updates and activation of such themes if the requirements are not met, again just how it has been worked on for plugins for the past couple months. However proceeding with this is currently blocked by the lack of theme readme support for wordpress.org, since the minimum requirements should be specified via the readme.
  • The Tide project could in the future be used as an additional means to determine PHP compatibility of plugins and themes.

Next week’s meeting

  • Next meeting will take place on Monday, November 12th, 2018 at 16:00 UTC in #core-php.
  • Agenda: Review future proceedings for the project and refine roadmap and priorities.
  • If you have suggestions about this but cannot make the meeting, please leave a comment on this post so that we can take them into account.

#core-php, #php, #servehappy

5.0: JavaScript language packs are here 🎉

News from the JS package inclusion focus: JavaScript language packs are here! Thanks to AWESOME work by @herregroen, @ocean90, @swissspidy, @nerrad, @atimmer and @schlessera we can now translate strings in JavaScript files and distribute them via https://translate.wordpress.org. This functionality will soon be expanded to also work for plugins and themes. This is a major milestone for JavaScript development in WordPress and completes the JavaScript package inclusion focus.

How does it work?

In short, to make it work, you need to

  • use the @wordpress/i18n package for making strings in your JavaScript translatable.
  • let WordPress know that a script has translations by calling wp_set_script_translations( 'my-handle', 'my-domain' ) after you register a script.

Read more about it in the JS i18n devnote.

Can I have a look at the tickets involved?

Sure thing! Here’s an overview of all the work we’ve done in the last few weeks to get to this point:

  1. WP-CLI: Make sure wp i18n CLI command includes JavaScript translations when generating Potfiles. Relevant PRs: https://github.com/wp-cli/i18n-command/pulls?q=is%3Apr+is%3Aclosed+author%3Aherregroen
  2. Meta: generate translations on translate.wordpress.org for core, themes and plugins. Ticket: https://meta.trac.wordpress.org/ticket/3748
  3. Meta: generate and serve JS language packs from translate.wordpress.org for core, themes and plugins. Ticket: https://meta.trac.wordpress.org/ticket/3876
  4. Core: Logic to load JS translations. Ticket: https://core.trac.wordpress.org/ticket/45103
  5. Core: API to register JS translations for a script. https://core.trac.wordpress.org/ticket/45161

What’s next?

With JS language packs we have concluded the package inclusion focus. In the remaining time we will keep focus on two things:

  • Keep the packages up-to-date until 5.0 ships.
  • @herregroen will shift focus towards generating the core JS code documentation on Devhub (https://developer.wordpress.org) and making it searchable. Since this pretty much only involves meta, this is not a release blocker. But with the heaps of JavaScript that 5.0 adds to WordPress, it would be nice to have its documentation searchable and discoverable through Devhub. At the same time it would also make WordPress’ legacy JavaScript (of which a lot was documented in the last years in the JS documentation initiative) more accessible to the broader community. Work on this had already started as part of the JS docs initiative.

#i18n

New! JavaScript i18n support in WordPress 5.0

For years, internationalization (i18n) has been a thing that has been pretty well supported in WordPress when it comes to PHP development. For PHP, WordPress already provides all the tools necessary to make it as easy as possible to localize WordPress core, themes and plugins to any language. Today we are bringing the same capabilities to JavaScript development for WordPress.

How does it work?

When registering your scripts you can add wp-i18n as a dependency to allow you to add translatable strings as you would in PHP:

wp_register_script( 'my-handle', plugins_url( '/js/my-file.js', MY_PLUGIN ), array( 'wp-i18n' ) );

Inside your scripts you will then be able to use wp-18n as follows:

const { __, _x, _n, _nx } = wp.i18n;

__( '__', 'my-domain' );
_x( '_x', '_x_context', 'my-domain' );
_n( '_n_single', '_n_plural', number, 'my-domain' );
_nx( '_nx_single', '_nx_plural', number, '_nx_context', 'my-domain' );

These functions mirror their PHP counterparts and can be used in exactly the same manner.

The final step is to tell WordPress your script contains translations and of which domain, this is to allow WordPress to selectively load only the necessary translations to ensure everything is as fast as can be:

wp_set_script_translations( 'my-handle', 'my-domain' );

Advanced usage

Right now it’s already possible to ship your own translations using the load_textdomain function and passing your own MO file. This is also possible using wp_set_script_translations which accepts an optional third path argument that allows you to tell WordPress to first look elsewhere for translations:

wp_set_script_translations( 'my-handle', 'my-domain', plugins_url( '/languages', MY_PLUGIN ) );

If passed WordPress will first check if a file in the format of ${domain}-${locale}-${handle}.json exists in the given path and use it as the source of translations if so. Alternatively it will also first check the given path for the md5 filename before defaulting to the WordPress languages directory.

If you want to ship your own translation files these should be in the JED 1.x ( .json ) format. GlotPress is able to create these along with other tools such as po2json. Ideally these files should only contain translations that occur within their respective JS files.

Behind the screens

When you upload your plugin or theme to wordpress.org all JS files will automatically be parsed the same as is already being done for PHP files. Any detected translations will be added to translate.wordpress.org to allow the community to cooperate to ensure WordPress, plugins and themes are available in as many languages as possible.

In order to parse all JS files the i18n-command for wp-cli is used. This replaces makepot.php to not only allow picking up translations in JS files but also to audit strings, parse strings only of a specific text domain and even pick up a few strings that weren’t detected by makepot.php. This command is freely available and open-source as makepot.php was and it’s recommended that anyone using makepot.php transition over to this much improved replacement.

Based on these parsed translations Language Packs are generated. Traditionally these used to only contain PO and MO files, one pair for each locale. In order to selectively load only the necessary translations regardless of whether it’s used or not a few more files are being added, one JSON file for every JS file that contains translations per locale.

When parsing JS files for translations we don’t know which handle is used to register that file so we’ve had to use an alternate mechanism to find the translations belonging to each file. To do this we’re using the md5 of the relative path of each file. This is appended to the usual name of ${domain}-${locale} in the form of ${domain}-${locale}-${md5}.json.

When you set script translations for a handle WordPress will automatically figure out the relative md5 hash of your source file, check to see if a translations file exists and if so ensure that it’s loaded into wp.i18n before your script runs.

Plugin and theme support

Translation and Language packs support for plugins and themes that are hosted on the repo is expected in the upcoming weeks. The patches are ready and waiting for commit. Plugin and theme authors are encouraged to start using wp-i18n in their JavaScript projects.

Credits

This API wouldn’t have been possible without the long standing efforts of @ocean90, @swissspidy, @nerrad, @atimmer, @schlessera and more recently @herregroen. Thanks a ton for the incredible work you’ve all put into making this a reality! Another thanks to @herregroen for providing the necessary input for this devnote.

#i18n

Dev Chat Summary: November 7th (5.0 Week 6)

This post summarizes the dev chat meeting from November 7th (agenda, Slack archive). Note that the timing for this meeting will move to 2100 UTC starting Nov 14.

5.0 Planning and Updates

  • We have a lot of PRs that need review, so please pick up one or two if that’s something you excel at.
  • Beta 3 came out on Monday. Notable updates were also published about Classic Editor plugin support and meta box compatibility.
  • Patches have been issued for Gutenberg 4.0 through 3.6 so that they automatically disable the plugin early enough in the 5.0 upgrade process.
  • For the upgrade process, core will focus on messaging introducing the new editor on the post-upgrade page (aka the About page), while the Classic Editor plugin will be responsible for any communication encouraging users to give the new editor a try.
  • From the REST API desk, we are in need of some eyes/contributions on both https://github.com/WordPress/gutenberg/issues/11280 and https://github.com/WordPress/gutenberg/issues/9151
  • From the JS desk, focus has shifted to generating the core JS code documentation on developer.wordpress.org and making it searchable. Also, the JS language packs are almost here!
  • Feature freeze on Twenty Nineteen has started, which puts all focus on these bug fixes and issues.
  • There are a number of PHP related issues on the milestone for contributors who don’t feel comfortable with React. For anyone able to test, review, or work on core PHP issues, check this report: https://core.trac.wordpress.org/report/6
  • We still have a few more bug scrubs coming up this week, and could use some contributors there. A new set will be posted soon, too.

Component and Focus Updates

  • Core PHP is continuing work on Servehappy, and WSOD protection. https://make.wordpress.org/core/2018/11/05/php-meeting-recap-october-29th/
  • Core Javascript meeting covered a lot of ground (Node LTS 10.13.0, back compat, e2e tests, ). Recap is available here: https://make.wordpress.org/core/2018/11/07/javascript-chat-summary-november-6th/

Announcements and Open Floor

Next meeting

The next meeting will take place on Wednesday, November 14 21:00 UTC in the #core Slack channel. Please feel free to drop in with any updates or questions. If you have items to discuss but cannot make the meeting, please leave a comment on the upcoming agenda post so that we can take them into account.

#5-0#a11y#core#core-editor#core-js#core-media#core-php#core-restapi#dev-chat#gutenberg#summary#team-reps

JavaScript Chat Summary – November 6th

Below is a summary of the discussion from this week’s JavaScript chat (agendaSlack transcript).

Have a topic for discussion for the next meeting? Leave a suggested edit on next week’s agenda.

Node LTS 10.13.0

With the tagging of a new Node LTS 10.13.0 last week (from 8.x), both Gutenberg and Core builds are now running the latest version.

Data Module Generic Store API

refactoring of the data module has been merged, and it now exposes a new optional lower-level API for non-Redux stores, or more personalized instances of a Redux store.

Refer to the documentation for more information.

JavaScript Future Backwards Compatibility (and React)

Discussion: How can WordPress continue its commitment to backwards-compatibility while also not perpetually lagging behind the current version of dependencies (specifically, but not exclusively React).

Considerations:

  • Explore what it could look like to version WordPress scripts
  • Targeting the abstraction (wp.element) an intentional point in ensuring consistent version use.
  • Provide and/or surface codemods if it comes to the point of being forced to introduce a breaking change (Facebook does similarly with React)
  • Gutenberg’s deprecation pattern is not something which would necessarily continue post-5.0, but could serve as precedent for a future deprecation model

Action Items:

  • Compile a document which outlines expectations for a plugin author to anticipate and account for future compatibility, how and when a breaking change would be introduced

Implementing End-to-End Tests in Core

Continuation of discussion from last week.

Problem: How can Gutenberg’s end-to-end test suite be brought into the core development workflow?

Considerations:

  • Not blocking for a 5.0 release
  • Tracked at #45165
  • Having an end-to-end testing solution is useful for more than just the editor, and useful for more than just core development

Solutions:

Make some core part of the end-to-end solution available in @wordpress/scripts.

i18n Core Merge Update

@omarreiss shared the following update:

WP-CLI: Make sure wp i18n CLI command includes JavaScript translations when generating potfiles. [Done] Relevant PRs: https://github.com/wp-cli/i18n-command/pulls?q=is%3Apr+is%3Aclosed+author%3Aherregroen .
Meta: generate translations on translate.wordpress.org for core, themes and plugins. [MVP Done] Ticket: https://meta.trac.wordpress.org/ticket/3748
Meta: generate and serve JS language packs from translate.wordpress.org for core, themes and plugins. [Has patch] Ticket: https://meta.trac.wordpress.org/ticket/3876.
Core: Logic to load JS translations. [Done] Ticket: https://core.trac.wordpress.org/ticket/45103
Core: API to register JS translations for a script. Needs meta side of things to be finished. [Has patch] Ticket: https://core.trac.wordpress.org/ticket/45161
Make: Publish an update and a devnote on how the JS translations work. [todo @omarreiss / @herregroen]

@herregroen shares that for most plugin authors who host their plugins through the plugin repository, the change in workflow should be a matter of using wp_set_script_translations in place of where traditionally one would use wp_localize_script.

Reflecting on @wordpress/hooks

Prompt: As a retrospective of how wp.hooks has been used in the development of core, what feedback do developers have to share?

Feedback:

  • @youknowriad : “In general I’m thinking JavaScript is not a good fit for our hooks because JavaScript is a real time UI not like PHP where the flow is linear. The UI has to be Reactive and it’s very hard to achieve with hooks without pitfalls.”
  • @omarreiss : “I think it might also not be the pattern we want to teach the WP community.”
  • @herregroen : “I would give some thought to deprecating before release to send a clear message going forward. I think it would be confusing to have this familiar API and then never expand it and soon after start deprecating it.”
  • @aduth : “My general experience with them is that they’re not completely avoidable, but in the majority of cases I’ve seen them proposed, it’s been the wrong way to go about a problem with some serious non-obvious consequences and I’ve thus looked to them as a “footgun” (a tempting tool too easy to use wrongly)”
  • @adamsilverstein : “i’ve mainly tried using the withFilters HOC which seemed to work well, i’m not really sure where else hooks have been used in Gutenberg. I agree they can easily be misused and better approaches are often available.”
  • @schlessera : “It’s a procedural pattern that quickly breaks down once you go past procedural code, even in PHP.”

Takeaways:

There’s some general consensus that the pattern is not a good fit for how Gutenberg has been architected, and it’s assumed that this will apply as generally true for core features written in a similarly JavaScript-oriented approach.

It’s also true that there is usage of JavaScript hooks within Gutenberg for which a better alternative has surfaced. Thus, they will likely remain a tool, albeit one we should consider to avoid as much as possible, and limit exposure and advocation.

Action Items:

Consider a communication plan for the role of hooks (future agenda item).

#javascript