Dev Chat Summary: September 6th (4.9 week 6)

This post summarizes the dev chat meeting from September 6th (agendaSlack archive).

4.9 schedule review

  • 3 weeks until the feature project merge deadline, 4 weeks until Beta 1
  • Customizer improvements for merging Changeset drafting and scheduling has yet to kick off development, designs are nearing completion (see: #39896 and #28721)
  • Gallery widget is still under development but it seems to have stalled, TODO’s noted on related GitHub PR
    • @joemcgill to look into avoiding serializing attachments data in the widget this week
  • @obenland working on wrestling the widget mapping issue when switching themes (see: #39693)
  • Page on Front progressing slowly, likely not ready for dev before Feature Merge
  • Theme switching issue for nav menu mapping has already been merged in trunk (see: #39692)
  • CodeMirror feature plugin (aka Better Code Editing) needs testing and a few outstanding issues that would benefit from contributors. Plan is to merge this week.
  • @psykro to look into #9757
  • “Add Media” button in the Text widget great opportunity for new contributors
  • #35827 could use an owner and remaining items in 4.9 Goals post could use contributors to help land in the release

Editor update

Iterating in trunk

  • @matt: I’m fine with more iteration happening in trunk vs how we’re bouncing patches around Trac so much
  • @matt: I’m okay with parts of trunk being broken as we iterate in this phase of dev
  • @desrosj: Do we have an established process for reverting things that break?
  • @obenland: I think we’re not talking about “PHP fatals”-broken, but rather a feature maybe not fully functional

HTML5 input types for validation

  • @afercia: any thoughts about relying on HTML5 input types browsers built-in validation only?
  • @azaozz: used to be buggy, seems to be working properly now
  • @afercia: seems to me still premature to rely on required for validation
  • @afercia: looking to leads to make a decision as new browsers support policy
  • @asaozz: Worth some testing, especially on the “lower end”, IE11
  • @afercia: there are still CSS rules in ie.css for Internet Explorer 6 (and 7, and 8). Can they just be dropped?
  • @azaozz: no need of ie.css in my honest opinion
  • @azaozz: intention is not to completely break old browsers if they still work, but to stop testing in them
  • @clorith: concerned about users locked into older browsers, like IE8, and keeping option for them to enqueue scripts relevant to their browser
  • @afercia: I wanted to start the discussion about this as it relates to the new browsers support policy

General announcements

#4-9, #core, #core-customize, #core-editor, #dev-chat, #gallery, #gutenberg, #html5, #summary, #trunk, #widgets

Dev Chat Summary: June 7th (4.8 week 6)

This post summarizes the dev chat meeting from June 7th (agendaSlack archive).

4.8 timing recap and Pre-Final Release & Dry Run checklist items

  • Beta 1 went out on Friday, May 12th; Beta 2 went out on Monday, May 22nd
  • RC1 went out on Thursday, May 25th; RC2 went out on Thursday, June 1st
  • 4.8 is scheduled for June 1, 2017 at 9am EDT
  • Events widget looks ready
  • Credits API to be updated by @ocean90 tomorrow morning
  • About page update for responsive, CDN-hosted images coming from @melchoyce
  • Announcement post draft is ready to go; @jorbin & @ocean90 to help provide contributor & language count as input
  • Announcement email being drafted by @matt
  • Codex page to be updated by @jbpaul17
  • Agreed to remove “partial back to IE8” from Browser support page in Design handbook
  • tinymce/plugins/wpembed to be added to $_old_files by @ocean90
  • No new default theme, so $_new_bundled_files is fine
  • Updates to default themes and submission to repo to be done by @davidakennedy, committed by @ocean90
  • Hosts email to be drafted by @jbpaul17, email to be reviewed & sent by @jorbin
  • Systems to be covered by @vnsavage
  • grunt prerelease check for tests & standards to be run by @jorbin

4.8 Bug Scrub

  • Reviewing four tickets in Defects Awaiting Review, reported against trunk section from Report 40
  • #40929: relates to improved translator docs, punting to next minor release (4.8.1)
  • #40932: moved to Future Release
  • #40927: not a regression, moved to Future Release
  • #40906: marked as a dupe of #40685, not a blocker for 4.8

Other News

  • Customize: looking for a new contributor to work on the HTML/Code widget, a good-first-bug, please chat in #core-customize if you’re interested
  • Editor: working to get the plugin in the plugin repo so that more people can review it and provide feedback. Goal is this week.
  • Devchat coordination: will be covered in upcoming devchat

#4-8, #core, #core-editor, #dev-chat, #summary

Dev Chat Summary: May 3rd (4.8 week 1)

This post summarizes the dev chat meeting from May 3rd (agendaSlack archive).

4.8 Timing

  • 4.8 schedule page has been published
  • Beta 1 is next Friday, May 12th; Release Candidate is Thursday, May 25th; target launch is Thursday, June 8th; and WCEU quickly follows June 15-17
  • WordPress 4.8 will be the first “major” release of 2017 and ideally includes the TinyMCE inline element / link boundaries, new media widgets, WYSIWYG in text widget, and the WordCamp / meetup dashboard upgrade to the “news” section
  • Assuming all goes as currently planned, we have just over a week to commit any new enhancements or feature requests
  • Concerns were raised to the compressed timeline for 4.8 and the stresses of getting a major release out in ~5 weeks
  • Recommendations were made to the effect of allowing 4.8 to progress more gradually and target a July/August timeframe for launch
  • Update since devchat: Confirmed existing 4.8 timeline with @matt, plan is to proceed with the enhancements that are ready now, anything that’s not ready will wait for an upcoming release

4.8 Bug Scrubs

  • Will publish times for upcoming scrubs to review items in the 4.7.5 and 4.8 milestones in a separate Make/Core post
  • @jbpaul17 & @desrosj will run general scrubs, @flixos90 will deal with multisite tickets in the multisite-specific bug-scrub next Monday
  • Please reach out to @jbpaul17 if you have availability to run a general or focused scrub on tickets in the 4.7.5/4.8 milestones over the next wee

4.8 Dev Notes / Field Guide

  • Regardless of final decision on 4.8 timeline, we need to get started on creating Dev Notes and assembling the Field Guide
  • Several comments were made about potential improvements to the Dev Notes / Field Guide during the 4.7 Retrospective
  • In summary: have more than one person own the Field Guide coordination and Dev Notes creation; more Dev Notes are better than less; create component-specific Dev Notes where feasible
  • Updated listing of Dev Notes needed and those responsible:
    • 1) Editor: TinyMCE inline element / link boundaries – @iseulde & @azaozz
    • 2) Editor: TinyMCE version 4.6 – @iseulde & @azaozz
    • 3) Editor: Edge fixes – @iseulde & @azaozz
    • 4) Customize: Media widgets (#32417)
    • 5) Customize: Visual text widget (#35243)
    • 6) Customize: Dynamically-resized controls pane (#32296)
    • 7) WordCamp / meetup dashboard upgrade to the “news” section
  • If you can help write one of the unassigned Dev Notes above or have others that you feel should be written, please comment on this post or let @jbpaul17 know
  • Per the Releasing Major Versions page, we should aim for Dev Notes around Beta 1 and the Field Guide by the Release Candidate; so please get started writing those Dev Notes, thanks!

Multisite Update

  • Discussed yesterday the idea of introducing ms-site.php and ms-network.php files for the real site/network APIs we’ve been introducing over the past couple releases (and will continue to introduce further functions)
  • All functions currently reside in ms-blogs.php which is getting rather cluttered
  • We propose introducing those two files and moving some functions in there and are looking for concerns from the team
  • Another question was whether they would need to be included from ms-blogs.php (for backwards compatibility) or whether we’re fine including them in wp-settings.php
  • Relates to: #40647 (Introduce ms-site.php and ms-network.php files)
  • Feedback would ideally be left on the ticket above

#4-8, #core-customize, #core-editor, #dev-chat, #summary

First Quarter Check-in

Just wanted to give folks my perception and feelings on of how we’re doing thus far with the core foci:

Writing: I’m really happy with the progress. It has had some slower weeks here and there the past few months, but by and large the technical prototypes we implemented have been successful and we’re ready to move into the next phase. We have a Chrome fix we have to get in the next minor release, and the link boundary improvements will be going into TinyMCE core and could be great for an interim +0.1 release.

Customization: Doing well. Remember: The plan is for the larger block-driven customization work to kick off in June. Prior to that, we’re focusing on widgets and other low-hanging fruit. Lack of developers slowed us down last few months, now doing better but could still use more help there. Media widgets + WYSIWYG on text widget seem simple but will have a big user impact.

REST API: There has been little to no perceivable progress on having any parts of wp-admin powered by the REST API.

Considering 4.8: The TinyMCE inline element / link boundaries, new media widgets, WYSIWYG in text widget, and perhaps something else small like the WordCamp / meetup dashboard upgrade to the “news” section, would comprise a nice chunk of new functionality for a +0.1 “major” release. I’m hopeful their progress over the remainder of April will allow us to kick off a process to do a nice 4.8 update in the May / June timeframe, without drawing too much focus away from the Big Changes in the next-generation editor that is still the top priority.

#4-8

Continuing inline docs improvements adjacent to 4.8

As we’re now into the full throes of the 4.8 cycle, the uncertainty that comes with not releasing “until it’s ready” inevitably creates a lull in areas other than the three focuses. Areas like maintaining our inline documentation, which populates the official Code Reference.

In the past, the freshness of core’s inline documentation relied almost entirely on a regular, major release schedule. And due to a preference for keeping the number of changed files low, inclusion of docs fixes in minor releases has previously been a rare occurrence.

Until now.

I’ve spoken with @matt, and the decision has been made to go ahead and prioritize some inline docs fixes for inclusion in minor releases going forward.

As with any decision, there are certainly pros and cons. Here are some of them:

Pros:

  • Ability to continue our ongoing inline docs maintenance adjacent to the 4.8 major release
  • Ability to address some glaring docs errors that we’ve been fixing manually in the Code Reference
  • Continue forward progress in documenting core JavaScript
  • Prioritize docs improvements for existing functionality in the three focus areas ahead of the 4.8 release, freeing up resources for documenting new functionality

Cons:

  • Number of changes and changed files in minor releases will increase (within reason)
  • All changes pushed to trunk will also need to be backported to the 4.7 (or current stable) branch

It’s worth noting that the reason the number of changed files has traditionally been kept low is to reduce the number of automatic update failures. The hope is that since we’ve been pushing automatic updates for 10 major versions now, reliability is less of a factor now than it has been previously.

It’s also worth noting that we shouldn’t expect a downtick in activity for core team resources focused on the three areas following this decision. As always, inline docs contributors will be focused on major release priorities before minor release ones.

This decision simply maintains the inline docs team’s ability to ensure the usefulness of core’s source documentation for the thousands of users and developers who rely on it every day.

#inline-docs, #minor-releases, #release-process

Dev Chat Summary: March 1st (4.7.3 week 5)

This post summarizes the dev chat meeting from March 1st (agendaSlack archive).

4.7.3 Schedule

  • Reminder of plan to release 4.7.3 as bugfix and maintenance release on Monday March 6, 2017
  • RC is available so please test

Community Summit

  • Working to review submissions on Planning for Community Summit 2017 post on Make/Core as well as submissions to the Make/Summit team via the Community Summit 2017: Sign-up Request post
  • Between now and Friday, March 5th the Core team needs to come up with:
    • 1) a list of topics for the summit
    • 2) A list of representatives to attend the Community Summit
    • 3) One or two contributors who are willing to help with the organization of the event
  • “participating” generally means being physically present for the discussions in Paris, France days prior to WCEU this summer for the Community Summit
  • Each topic facilitator will do both a pre-summit and post-summit Make/Core post. @jbpaul17 to confirm timelines with @_dorsvenabili to help prep those facilitators for those post timings.
  • Javascript in core [will submit to CS]
    • “what we hope and imagine for the future with the REST API, and how we hope to get there… what we have in core now and how we can improve it and how we can attract more JavaScript first developers to build on WordPress and especially contribute to core… How the REST API relates to wp-admin.” Submitted by @adamsilverstein to attend and volunteer to help in whatever role is most helpful.
    • “REST API admin usage: Where we can start moving things to using the API (and maybe even get a couple of them done at the summit)” Related submission from @chriscct7, recommended to include @rmccue
    • @kadamwhite: A heavy dependency on “the future of JS in core” and that discussion should originate from the broader WP community, not be mandated by the REST API group
  • Technology version support policies [will submit to CS]
    • @jorbin: (versions of PHP, MySQL, Browsers, Screen Readers, other AT, etc.) Let’s come up with some concrete plans for when we intend to deprecate things and how we want to handle it. People Who would be good to have in this discussion: @dd32 (to help with stats) @pento (to help with messaging) @afercia and @rianrietveld ( to help formulate AT support policies if they don’t exist already), @westonruter ( as maintainer of the largest JS component) @azaozz ( as maintainer of tinyMCE component) @matveb ( as dev lead of new editor)
    • @getsource, @boonebgorges, and @matt as additional reps for this topic
  • Improved management of contributors with time to spare [will submit to CS]
    • @johnbillion: This topic is particularly focused on pre-existing contributors who are paid to contribute to WordPress (eg. those whose time is sponsored by their employers), but also pre-existing contributors who aren’t sponsored but who do want to contribute a significant and/or consistent amount of time, and also potential contributors in a similar position.
      As a project, we need to manage these people’s time much better. These people need to be project managed in one way or another to avoid repeats of situations we’ve had in the past where a contributor is literally being paid to fix things in WordPress and the project is failing to enable them to do so effectively, or even at all. I’d (@johnbillion) like to attend the summit, and I’d be happy to jointly lead this discussion with someone who has good project management experience and some ideas about how WordPress might be able to better manage contributors, but at the same time do it in such a way that retains the fun and interesting aspects of contributing without turning it into something that too closely resembles “work”. [Side note from John: Worth noting that this doesn’t only apply to core, but it’s a good place to start.]
    • @helen did a survey of time availability a while ago, sent list to John to use for this topic
    • @aaroncampbell, @getsource, @jorbin, @boonebgorges, and @logankipp as additional reps for this topic
  • On-boarding experience for new contributors [will submit to CS]
    • @joemcgill: Lots of people who want to get involved have no idea where to focus their efforts.
    • @kadamwhite: Speaking for myself this is hugely related to the future of JS in core and the REST API, since those pieces really need the energy new contribs would bring
    • @getsource: I am willing to participate or lead, although I don’t know what leading it means besides guiding conversation at this point. @aaroncampbell also willing to lead.
    • @peterwilsoncc, @flixos90, @logankipp, @jorbin, @johnbillion, and @stevenkword as additional reps for this topic
  • Communicating changes to WordPress Core [will NOT submit to CS]
    • @jorbin: For the past few years, core has produced a field guide and worked with the meta and plugins team to email plugin others about changes to core. Each release though triggers a number of people who don’t know about changes until after the release. Challenge: How can we help ensure changes that aren’t worthy of user marketing promotion are known by a far greater percentage of WordPress developers?
      Might also impact or benefit from input from +make.wordpress.org/plugins +make.wordpress.org/themes +make.wordpress.org/marketing +make.wordpress.org/meta.
      Even when we get the field guide out on time, issues come up post release.
      two ideas:
      1) Translating the field guide (is this reasonable if the posts that it links to aren’t translated?) Also means polyglots should be in the discussion
      2) Using the new release email mailing list to announce RC
    • @helen: I think it’s worth at least starting the conversation earlier, even if it ends up still being valuable to continue something in person.
    • @desrosj: There may also be some great ideas from people who cannot attend in person. It would be a great opportunity for them to have their ideas heard and contribute, even if they are not able to follow through with the discussion in person at the summit.
    • @jorbinI’m going to withdraw the communication topic as my proposal for the summit with the note that I might want to resubmit it depending on how the virtual discussion goes
    • @azaozz and @sergey as additional reps for this topic
  • Security [will submit to CS]
    • @chriscct7: The process of a security ticket from report through triage through disclosure. Aaron Campbell (security czar) has made it clear this needs to be discussed at some point and I feel like the community summit would provide a good venue as many of those on the team will be there in person and we can mirror the conversation easily for those who are not. Recommend including @aaroncampbell
    • @aaroncampbell: This is actually a good idea, although I don’t think it’s because “those on the team will be there” but rather because I’d love to get input from some other people too, and security is generally sensitive enough that a place like the summit seems useful
    • @rmccue, @kadamwhite, @matveb, @joen, @westonruter, @melchoyce as additional reps for this topic
  • Collection of Anonymous data [will NOT submit to CS]
    • @chriscct7: If core is interested in doing it, I think my experience with doing it for a trac ticket (settings reduction) might prove to be useful to add to the discussion. Recommend including @drewapicture
    • General agreement to NOT include this topic since this is currently opt-in and the issue is finding an owner of this topic
  • Bootstrap/Load [will NOT submit to CS]
    • @schlessera: Opening up the WordPress Core Architecture to make it flexible enough as a platform so that it can:  * serve both novice end-users as well as large-scale enterprise installations in an optimized way;  * quickly adapt to changing external requirements, to keep up with the accelerating pace of the web. Recommend including @rmccue
    • General agreement to NOT include this topic since it does not need to happen in-person, already has discussions underway, and should be scheduled in next couple of weeks
  • Code editor [will NOT submit to CS]
    • @georgestephanis: Code Syntax Highlighting implementation and accessibility concerns — how we can get CodeMirror or whatever better library there is implemented and rolled out for both Customizer Custom CSS, Theme/Plugin Editor, and Content Blocks. Recommend including @afercia @westonruter
    • General agreement to NOT include this topic since it does not need to happen in-person and should happen sooner than the CS.
  • REST API authentication [will NOT submit to CS]
    • @georgestephanis: Third-party authentication with the REST API.    Between OAuth 1.0a, OAuth2, central application brokers, Application Passwords, or some other system — there’s a lot of possibilities here, and it’d be really nice if Core could pick something and move forward with it before folks start spoofing cookie authentication in applications to integrate with core.
    • Relevant chat summary from the last time we had one
    • This really needs an owner, otherwise it’ll continue to be punted. There’s fundamental differences on what the direction should be.
    • @samuelsidler: I don’t think core can decide until someone has documented the possible options, along with their strengths and weaknesses, then had some discussions on what would be best for core and why.
    • @georgestephanis, @rmccue, @logankipp volunteered post on Make/Core to move this topic along
    • We will table this idea and maybe propose it for the summit based upon how the near term discussions go
  • Front-end Editing [will NOT submit to CS]
    • @westonruter: Frontend editing powered by bootstrapping the customizer onto the frontend, with inline direct manipulation of elements on the page and the controls sidebar being lazy loaded to slide in from the left as needed. Editable elements include post content and site configuration (sidebars, menus, options, etc). Recommend including @celloexpressions
    • General agreement to NOT include this topic since it depends on too many other things we won’t know by then, so we will pass on that topic (at least for now).
  • Nextgen Widgets [will NOT submit to CS]
    • @westonruter: Next generation of widgets which harmonize with content blocks in the editor.
    • General agreement to NOT include this topic for the CS, but good conversation for the contributor day.
  • Feedback on Core focuses [will NOT submit to CS]
    • @georgestephanis: Six months in, how are we feeling about shifting away to a more top-directed set of focuses for the year?
    • General agreement to NOT include this topic as it’ll be hard to say until/unless we’ve shipped a core release by then (we likely won’t) and is a conversation that should happen in public.
  • Complete list of representatives nominated to attend the Community Summit: @matt, @nacin, @adamsilverstein@rmccue@kadamwhite@chriscct7, @dd32@pento@afercia@rianrietveld@westonruter@azaozz@matveb, @getsource, @boonebgorges@aaroncampbell, @jorbin, @logankipp, @peterwilsoncc, @flixos90, @johnbillion, @stevenkword, @azaozz, @sergey, @karmatosed, @joen, @westonruter, @melchoyce, @jnylen0, @ipstenu, @joemcgill, @joehoyle, @rachelbaker, @michael-arestad, @petya, @danielbachhuber, @ocean90, @samuelsidler, @afercia@desrosj, @iseulde, @jjj@celloexpressions
  • We’re still searching for 1-2 contributors who are willing to help with event organization, so please comment here or reach out to @jbpaul17 if you’re interested
  • @jbpaul17 will send the Core team responses to the Community Summit team by Friday, March 3rd.

Browser support

  • Please take a look at @desrosj’s post: The New Editor and Browser Support
  • This will be a topic of discussion at next week’s devchat.
  • Please leave your thoughts there as comments, and bring them along next week as well.

#4-7, #4-7-3, #community-summit, #core, #core-customize, #core-editor, #core-restapi, #dev-chat, #summary

Editor Technical Overview

As we start looking at the editor from a technical perspective it’s important we identify the main obstacles and requirements we face before we start conjecturing solutions. As @matt wrote before, the editor focus aims to make writing rich posts effortless. This has taken the path of treating a post as being composed of distinct pieces of content called blocks. These pieces should be easy to insert and manipulate, providing rich and contextual interfaces to interact with as you craft a post.

So how do we go about turning this into a reality? Content in WordPress is, fundamentally, HTML-augmented text; that is to say, it has no inherent data structure. This has been a very important aspect of WordPress and a force for the open web—it speaks to the sense of ownership and freedom WordPress gives you, since it’s always easy to get the full content of your publications—yet the lack of structure gets in the way of the goal to treat content as composed from individual pieces. (This reality also became an issue for the development of post formats a few years ago, but I digress.)

It’s relatively easy to add structure, but it’s not trivial to do so in a way that doesn’t harm data integrity, portability, and the cohesiveness of the post_content. So let’s lay out a first requirement:

① Shape of the Data: Portable Text

To ensure we keep a key component of WordPress’ strength intact, the source of truth for the content should remain in post_content, where the bulk of the post data needs to be present in a way that is accessible and portable, while still providing additional structure on top of HTML semantics for our editing tools. Data needs to be praised and respected. This additional structure would hopefully be invisible to the document’s display context, as it ensures the rendered content is viewable in situations that may not be aware of blocks at all. (Think of email clients, RSS, older editor versions, mobile editors, database dumps, etc.)

Storing things separately means post_content becomes either a pointer or duplicated data, which fragments the source of truth since they can get out of sync easily. (A few content block plugins do this by storing structured data in postmeta and pure data in post_content.) On the other hand, storing things together means structure can become gibberish if it’s not formatted properly before display.

How can we then offer users a great experience when creating or manipulating content without sacrificing the spirit of integrity and data reliability that is expected from WordPress? Good representations of the data would also make it easier to develop robust collaboration tools in the future by allowing us to lock things in a more fine grained way. I believe this is important to figure out soon to allow us to prototype quickly, so I’ll follow up with an initial proposal by the end.

Honouring HTML leads to a second requirement:

② Simplicity and Semantics of Representation

Unless we are improving the semantics of the document we should minimize what markup we add to identify a block; for example, avoid adding extra DOM elements or attributes, both for simplicity and standards sake. WordPress has been a champion of web standards, and we should not venture away from this quality. How can we add structure in a way that remains invisible to the output (as meaningful content as possible) but gives the necessary hooks to infer a structured view for editing purposes?

While we discuss how to structure the content to include invisible demarcation of blocks, the aspect of their nature leads to a third requirement:

③ Static & Dynamic Blocks

Blocks can be either static or dynamic. That is to say, some blocks can be stored with all the necessary aspects needed for their display, while others need to be processed before generating their final output (shortcodes, embeds, widgets, etc). This distinction is important because the most common two blocks people will naturally use are text and images. We should not break their clarity as we treat them as blocks.


 # Static
Here’s some text.

# Dynamic
[text id=123] // Pulls "Here's some text" from somewhere.

This conceptual separation of blocks is useful for designing our project, yet they generate abstract complexity which users should not be exposed to, leading to a fourth requirement:

④ Consistent Experience

One of the biggest benefits of blocks is that composing a post becomes more intuitive and reliable. Everything is inserted under the same assumptions; discovering what can be inserted is a natural part of inserting content. To the user all blocks behave in a consistent and familiar way, even if they provide tailored UIs for their controls.

The user should also be able to edit a post in a different system (mobile, REST, older version of core, apps like Mars Edit, etc), even if they lose the advantages of block editing. That’s another reason why post_content as source of truth matters, compared to storing a JSON structure in postmeta. Having things in two places means they can get out of sync depending on what tool you used to edit.

These nuances of data, UI, and display lead to a final and more general requirement, which is understanding the system we’ll be crafting:

⑤ The System

The editor experience has three fundamental aspects to its system: the UI used to manipulate a block; the demarcation of the block; the rendered output of the block. These are all separate concerns, from the tools we craft to edit a post, to the document syntax that holds the data structure, to the way the final output is generated to be displayed as HTML. (With static blocks that last aspect may be of no significant concern since the document doesn’t care about the presence of static blocks, it just displays them.)

Picturing these concerns as connected but fundamentally separate would help us figure out the best design and technical solutions for each stage, while avoiding us taking aggressive moves by coupling expectations. For instance, JavaScript is a natural technology to look at when it comes to the ability to manipulate and interact with content blocks, yet it may not be the best at all when it comes to rendering the final post to a viewer. Avoiding painting the whole flow under the same light should allow us to focus efforts, because we don’t have to change everything.

❶ Coda

As a final coda, and following @joen‘s design exploration, let’s keep in mind that our first goal should be to set up a reliable foundation to allow us to iterate quickly and test assumptions. I propose we focus initially on a few static blocks (text, image with and without caption, quote) to limit scope of the project.

In which ways can we fulfil ① and ② from the above requirements?

Shortcodes fall short in that they are not invisible, they are opaque, not standing to the scrutiny of semantics, and are also painful to parse. Alternatives could be data-* attributes in the HTML elements or custom elements (paving the way for web components, perhaps), yet we need to be careful with adding cruft.

One other possibility is to look at HTML comments as a way to provide explicit demarcation to post_content without affecting the node structure of the document for something that is inherently spurious to the HTML semantics. WordPress already uses comments for the more tag (<!--more-->) and splitting content into pages in a way that has proven to be quite robust.

It could look something like this:

<!-- wp:image -->
<figure>
  <img src="/">
  <figcaption>A picture is worth a thousand words</figcaption>
</figure>
<!-- /wp -->

There are drawbacks, benefits, and implications with each that we should discuss separately. Are there any other possible solutions?

Please, join us in #core-editor if you are interested. We’re holding weekly meetings in Slack, Wednesdays at 19:00 CET.

#editor

Dev Chat Summary: December 21st (4.7.1 week 2)

This post summarizes the dev chat meeting from December 21st (Slack archive).

4.7 Retrospective

  • Reviewing comments on 4.7 Retrospective post on Make/Core
  • We will go through comments and discuss if there are changes to our process that we should recommend
  • Goal is not to second-guess decisions that were made, the goal is to figure out if the process can be improved in future releases
  • Things to start doing:
    • “We failed at getting the field guide and email to plugin dev out early enough. We have aimed to have that out around beta 2 and usually end up getting it out around RC the last few releases. This time it came out the day before (field guide) and day after the release (email).”
      • Coming up with some documentation and ensuring that it’s not just owned by one person is a good way to improve it
      • We should also ensure it is included in the release checklist
    • “The posts explaining new features and changes are helpful, but I think that we need more of them. I follow the trac feed, and sometimes I know that as a plugin developer a particular ticket is important for me to take note of, but it can be difficult to unravel exactly what the final decision was just based on the changesets. An example of something that is going on right now is the a11y team’s work on removing excessive content from headings on admin screens. Often API changes get documented and UI changes don’t, but I’m a perfectionist and I like to stay up to date on the latest design/a11y evolutions as well. I can usually figure out what changes I might want to make in my plugins based on the changes in core, but I’m sure that often most plugin developers don’t even know that there was a change, if they don’t read every ticket.”
      • Request here is to have more Dev Notes and explanations about what is changing
      • It would help to spread the load of writing Dev Notes a bit, sometimes they are time consuming, especially if you’re not much for writing this sort of contextual summary
      • Some components generate a component summary dev note which is a good practice
      • Should we also maybe reach out to the docs team to see if they want/can help?
      • Anyone have ideas for how we get people involved with drafting the note even if they aren’t leading developers on a feature/component?
      • We do list out every change in the “this week in core posts” (shout out to the team that works on those!) already, so there isn’t anything that goes unannounced
      • The solution suggested is more posts, but the problem appears to be that people aren’t seeing changes that they think might affect them
      • Getting to Trac and subscribing to whatever feed is a little hidden. Even Slack notifications is hidden. A more public place would be good.
      • The solution could be to push people to the Week in Core posts, they already list every change categorized by components
      • Someone willing to lead a discussion (likely on make/core) on how to improve this?
      • Action Item: wait and see how lack of timed release cycles plays out
    • “We need to collectively review the “feature plugin merge guidelines” listed here. While development in plugins has become less prominent, most of the bigger projects merging into core in 4.7 (I would exclude the REST API since that’s less user-facing) skipped many of the steps here. A lot of the points don’t apply to most projects – to the point that the checklist is often forgotten entirely. But there remains a need for better quality control and an updated checklist or recommended merge considerations for larger projects should be created accordingly. These written guidelines can better inform merge decisions and assess readiness.”
      • Can we reasonably make full test coverage (covering basic use cases at least) a requirement there?
      • This may be null as feature plugins may not play a significant, or any, role in the future
      • More “wait and see how new process/focus shakes out”
      • Action Item: No more feature plugins
    • “On a related note, clearer procedures about backing out merged features are needed. Particularly if a feature goes through an extensive process to demonstrate readiness and is approved for merge, input on removing the feature during beta/RC should be solicited publicly via make/core posts and scheduled meetings, similarly to the initial merge decision. Otherwise, the decision to remove a feature can seem to ignore the value of the opinions that went into making the decision initially and may not be fully informed about the broader implications with respect to related aspects of a component. If work on a feature seems to stagnate as bugs accumulate during beta and a lead is considering pulling it due to lack of attention, contributors working on the feature should be notified so that they can address the issues or recommend removing the feature based on availability. Perhaps putting more trust in the feature teams and component maintainers that are most intimately familiar with a given project could help ensure that decisions are more broadly considered.”
      • Still a question of who really owns final decision/veto power; @matt as product owner likely
      • Whomever is leading the release has final decision. That’s why they’re a lead. That much should be clear.
      • Action Item: continue to communicate changes clearly and early
      • Release leads and core leads need to be trusted to prioritize based on goals for the release
      • When somebody is unable to solicit feedback, we need to have honest conversations about why this is happening
    • “Add automated acceptance testing for the user flows. If we add these for new features added, we can ensure they work across browsers. And in future releases, these tests can ensure that we don’t break existing flows. Run tests on BrowserStack. See #34693.”
      • Any volunteers to help work on this?
      • Anyone think automated acceptance testing for user flows is a bad idea?
      • It could be difficult to maintain
      • This is done at Automattic: https://github.com/Automattic/wp-e2e-tests
      • Action Item: keep exploring in the ticket
    • “more focus on Trac and tickets, every committer should try to follow Trac on a daily basis. Not to know 100% of the details of each ticket but at least to get a sense of what’s going on. Also, the number of tickets on Trac is increasing more and more, there’s the need of some serious ‘Trac Gardening'”
      • A big ask for every committer following Trac on a daily basis
      • Especially since the vast majority of committers aren’t being donated anywhere near full time (and a large number are 100% volunteer)
      • This is why there’s component maintainers, so that we don’t overburden each person
      • Trac Gardening is something anyone is welcome to do, you don’t need to be a committer
      • Trac Gardening would benefit from some mentorship to be more effective
      • If there could be some mentoring for this – an initial joint meeting to get people started might we get some more interest?
      • We could benefit from improved workflows for managing the resources we do have and to reduce the uncertainty in gardening/contributing in non-code ways
      • Trac Gardening can be a thankless task to a novice who comments on tickets, asks for dev-feedback and then nothing further happens for months. Perhaps the dev-feedback tag needs watching more rather than all tickets.
      • Action Item: generalize 4.7 Bug Scrubs page “to run a bug scrub, announce it here, talk to these people, look at this report in Trac, then ping people on the tickets listed”
  • Things to continue doing:
    • “The combination of a Git startup phase and Slack is excellent. At least for the Twenty Seventeen theme.”
      • GitHub likely helps get new contributors involved, but not sure they stick around
      • GitHub is easier to follow along, post mockups, get feedback, review code
      • GitHub better with searching, labelling, organizing, looking at PRs, realtime updates, making branches and then submitting PRs from branches, plus its app
      • Our current code review process is sub-optimal because there are no workflows to support it (e.g., line-by-line comments on changes)
      • It would be good to separate what is the project management tool vs. version control method
      • GitHub is sub-optimal when iterating on PRs. In Trac, you can make minor changes to a patch and upload it to a ticket. In Github, depending on permissions patch iteration is not straightforward.
    • “Weekly design meetings.”
    • “On the upside, having clear deadlines for when enhancements need to be merged into core is very helpful for prioritizing time and focussing resources. I hope we will continue some form product calendar in the spirit of “deadlines aren’t arbitrary,” even if they take a different rhythm.”
    • “increase the effort to involve different teams in collaborating on features development, where different skills and knowledge are needed, of course.”
  • @jorbin to work on a summary of what was discussed here and post it on Make/Core

General Announcements

  • Uncertain if anyone is planning on running a core dev chat next week (or any weeks going forward), so watch for agendas on Make/Core or other notifications in #core

#4-7, #4-7-1, #dev-chat, #summary

Merge Proposal Discussion: REST API Content Endpoints

There are discussion meetings and office hours in #core-restapi at 2016-10-14 14:00UTC and 2016-10-14 19:00UTC on Friday the 14th. Our next team meeting is on 2016-10-17 14:00UTC. Please attend some of all of these, because

We are meeting at 2016-10-18 01:00 UTC to make a decision on this merge proposal!

To that end, the below discussion points will be updated regularly, please leave comments on this post or join the conversation in #core-restapi.

Yesterday at the dev chat the API Team proposed the Content API Endpoints for merge in WordPress 4.7. There was popular support for this feature but as @jorbin and @helen noted that the lack of dissent suggested additional review is needed, so the API Team is continuing to seek thorough review & constructive criticism of the content endpoints, including the open questions previously shared on the week 7 and week 8 API team updates.

The merge proposal also engendered follow-up discussion in the #core-restapi channel about the benefit content endpoints bring to core, whether having such endpoints built in is quantifiably more beneficial than having them as a plugin, whether moving development from a plugin to core would slow development, and whether the endpoints as-proposed have been sufficiently reviewed for security and performance. We attempt to capture those questions & concerns (and the perspectives on them) below.

Security

Have the content API endpoints been thoroughly reviewed for security?

  • The REST API plugin has been on HackerOne for over a year with paid bounties for bugs
  • @barry has begun a security review

Performance

How does the API measure up against alternatives? Are there concerns about how the API could impact the servers to which it is deployed?

  • DeliciousBrains did a performance comparison with Admin AJAX and found the REST API to have a performance improvement (These tests have not yet been independently verified)
  • @mikeschroder notes in the comments that using the REST API in place of Admin-Ajax will also bring speed benefits by permitting many previously-uncacheable requests to be cached.

User Feedback

Are the content endpoints sufficiently well-tested & vetted by the community?

  • @matt questions whether feedback is coming too late in development for concerns to be acted upon
    • @rmccue notes that the v2 endpoints were created based on user feedback; REST API endpoints are being deployed by plugins and running on VIP, and the content endpoints have been in wide use across a variety of sites, leading to 90+ code contributors and far more developers’ support & feedback on the endpoints
  • @rmccue has also reached out to Phil Sturgeon for feedback and will follow up

Do Content Endpoints Benefit Core Development?

Will having these endpoints in core improve future core development, or solve any immediate problems?

  • @bradyvercher suggested that the content API endpoints would remove the need to write a variety of one-off ajax callbacks when developing future WordPress Core AJAX functionality
  • @westonruter notes that the customizer could dynamically create settings for posts and other kinds of content without having to wire up new admin-ajax handlers

Will Merging Negatively Impact API Development?

Will having to work through trac instead of GitHub cause development to atrophy?

  • @jjj argues that merging will slow development, because GitHub-hosted plugins are not bound to WordPress release cycles and have less overhead for features to be developed and deployed for testing. @jjj requested a plan for how the REST API will be developed going forward after the merge of these endpoints that would account for the added friction.
  • @krogsgard countered that core increases the visibility of a project like the content endpoints
    • The number of new contributors in this Slack discussion suggests that this merge proposal is bringing in new voices; whether this supports Brian’s point or not, the team is grateful for the breadth of perspectives being shared -Ed.
  • @rachelbaker suggested that the API endpoints are sufficiently inter-dependent with core WordPress code that maintaining the plugin separately amounts to maintaining a fork, and that such separated development is untenable long-term.
  • @matt hopes that a merge of these endpoints would slow release speed, but not development speed; @rmccue feels that development speed will stay the same or increase, and that tying releases to WordPress Core increases the stability and predictability of the API endpoints.
  • The versioning of the API endpoints supports forward compatibility

Do Content Endpoints Belong on Every WordPress Site?

What are the pros and cons to having every WordPress site have content API endpoints?

  • @rmccue suggests the API has network effects that can only be realized with a large install base. @krogsgard draws a comparison to RSS, the widespread availability of which enables everything from podcasting from WP to the use of apps like Feedly.
  • @matt suggests that the Atom API is a better analogue than RSS, which is an independent and pre-existing standard, and that network effects could be tested through inclusion in Jetpack
  • @joostdevalk notes that many plugins, like Yoast, have data tied to existing content such as posts and pages; either they expose the content through their own endpoints, or core does. If Core exposes content types through the API then plugins may build on top of that shared foundation, not independently reinvent the wheel. “if this doesn’t end up in core, we’ll start rolling our own API for stuff. Others will too. Interoperability won’t be there, for even the most basic stuff. I think this isn’t like RSS, I think this is much more like all of us using the same table space in MySQL.”
    • @shelob9 and @masonjames agree that merging the endpoints would create a consistent and reliable open “standard” that WordPress developers can use instead of continually reinventing how to read and edit post data over HTTP.
    • In response to the question “what prevents you from building on the endpoints in their plugin form,” @joostdevalk went on to note that plugin dependencies would make that a viable option, but that burden currently lies on the user. Plugin installation is not frictionless.
  • Can these endpoints be bundled? short takeaway: no
    • Woo bundled the API infrastructure before it was merged; doing so for content endpoints would require bundling prohibitively large amounts of endpoint code.
    • @nerrad worries that if plugins bundle different versions of the endpoints plugin, then those plugins may conflict if all bundled copies are not kept in sync.
      • @nerrad clarifies in the comments below that these worries also encompass the additional risk of conflicts when plugin authors each build their own versions of these content endpoints, instead of leveraging a shared standard: if two plugins each expose their own REST collection for posts, a developer working on a site with multiple such endpoints will need to decide which to extend, and will then have their extension tied to that specific plugin rather than to a core API.
  • @schrapel and @jorbin discussed that these content endpoints make it much easier to crawl a site, which also brings some potential performance concerns: no new content is exposed, but the process of aggregating it is easier and more easily automated.
  • In the  comments below @foliovision believes that merging the endpoints will be the best way to assert the level of back-compatibility that mid-size agencies need in order to confidently utilize the endpoints.

Please leave thoughts, questions, concerns, comments & experience in the comments below. Thank you!

Edited 2016-10-16 to include the below comments into the body of the post

#4-7, #rest-api

REST API Team Update, 4.7 Week 8

Summary: Beta 15 has been released, there are open questions that would benefit from your feedback, and the Content API Endpoints and OAuth Server are being proposed for merge as distinct, separate enhancements to the existing WordPress REST API infrastructure.

REST API v2 Beta 15 released

The 15th beta release of the REST API content endpoints plugin was released on October 7. This release builds on top of the recent Beta 14 to…

  • Add support for Post Meta, Term Meta, User Meta and Comment Meta within their parent endpoints
  • Introduce a settings endpoint to allow key site setting values to be retrieved & modified using the API
  • Introduce query parameters to query for posts that are NOT IN one or many terms of specific taxonomies
  • Resolve bugs, including bad comparison logic when updating comments.

Please try it out and report any outstanding issues; the REST API project gained its 90th code contributor this week and the team is deeply grateful for the energy and support of the broader WordPress community in testing out this merge-candidate plugin!

New Questions & Discussion Items

Items which have arisen through final ticket triage & review on which the team seeks feedback:

  1. Should the `filter` shim should be removed prior to merge? It is the majority position of the API team that `filter` be deprecated to dramatically improve the simplicity and consistency of API query functionality
  2. How should comments be handled for password-protected posts? Should the password be passed as a query parameter with the PUT/POST request, or is there a better option?
  3. Should the API match core’s logic when users with the `unfiltered_html` capability are creating or updating Posts or Comments?

Meeting Notes

At the weekly team meeting on October 10 the group reviewed open issues in the 2.0 milestone, which represents the candidate for our merge proposal shared last week.

Meeting attendees agreed to review open issues and pull requests individually, and to reconvene on Tuesday at 1500UTC to ensure all priority tickets had an owner.

At that meeting on October 11, the team reviewed the incoming feedback around the OAuth plugin (linked above). While the API team feels that having a built-in authentication solution provides a much-needed service, particularly to developers building mobile and desktop applications, the design and usability feedback we have received does indicate that the plugin needs more work.

OAuth’s place in the Merge Proposal

The API Team believes that the identified issues are resolvable, that the OAuth plugin is on track and that it should still be considered for merge in 4.7. However, after discussion within the team, input from @matt, and advice from @aaroncampbell and other core committers, we have edited our merge proposal to submit the Content API Endpoints and OAuth server as separate merge candidates. The API Team proposes both components for merge, but we submit the content endpoints for consideration independently of the OAuth1 server.

Content Endpoints Without OAuth

The Content API endpoints are stable, well-tested, and in wide production use across a variety of applications. Theme and plugin developers will benefit from having canonical, well-tested API endpoints in core, which may be used to query WordPress both from PHP code and from JavaScript applications running on the front-end or admin of WordPress. Sharing the endpoints for core data types enables increased consistency of what data is exposed and how it is persisted across different plugins, improving consistency and shortening development time by using . These themes and plugins have full read and write access to the API using the existing cookie & nonce authentication.

Mobile and desktop applications can leverage these same endpoints in a read-only capacity to create a variety of powerful reader-oriented applications and tools that expand the capability of what WordPress can do today, such as a unified reader for Make WordPress blogs and other experiments hypothesized by @jorbin.

Should OAuth 1 not be accepted for 4.7, secure write access for these external applications would still be only a plugin install away; and while having an OAuth server in core will provide a canonical approach for authenticating from remote applications, depending on the needs of a specific site or specific client application other authentication schemes may actually be preferable. Plugins exist for JWT Authentication and of course OAuth 2, and should OAuth 1 not be accepted for 4.7 these plugins may still be installed to enable an external application to opt-in to secure write access to your WordPress site.

In Summary

The API team submits for 4.7 merge consideration two enhancements to the REST API infrastructure: the Content API Endpoints for core WordPress datatypes, and an OAuth server which will reduce the setup time needed to securely interact with those endpoints from outside of WordPress. We believe these enhancements are each individually sufficiently tested and mature to meet the quality and security standards of WordPress Core, and each individually provides wide-reaching benefit to WordPress developers, and through them to the authors, readers & publishers of the web.

#4-7, #rest-api