Hallway Hangout: Data Liberation Discussion and Brainstorm

Welcome!

The WordPress coreCore Core is the set of software required to run WordPress. The Core Development Team builds WordPress. development team builds WordPress! Follow this site for general updates, status reports, and the occasional code debate. There’s lots of ways to contribute:

Found a bugbug A bug is an error or unexpected result. Performance improvements, code optimization, and are considered enhancements, not defects. After feature freeze, only bugs are dealt with, with regressions (adverse changes from the previous version) being the highest priority.? Create a ticket in the bug tracker.

Want to contribute? Get started quickly with tickets marked as good first bugs for new contributors or join a bug scrub. There’s more on the reports page, like patches needing testing, and on feature projects page.

Other questions? Here is a detailed handbook for contributors, complete with tutorials.

Data Liberation is a major focus for the WordPress Project in 2024 – furthering the mission of Democratizing Publishing by making it easier for people to get their content in and out of WordPress.

This is an exciting area of work, but also very broad. It is important to make sure the initiatives that make up this work are best suited to be impactful and meet the needs of real users.

Come and join an open discussion on this area of work – and help shape the future of data liberation in WordPress.

In this informal chat we may touch on:

Challenges of migrating from third-party platforms to WordPress
The good, bad, and ugly of exporting WordPress content
The potential of interoperability between BlockBlock Block is the abstract term used to describe units of markup that, composed together, form the content or layout of a webpage using the WordPress editor. The idea combines concepts of what in the past may have achieved with shortcodes, custom HTML, and embed discovery into a single consistent API and user experience. libraries and page builders
What work will make data liberation, and who should work on it

Join us on Zoom at Wednesday, April 3, 2024, at 7pm. Add this event to your Calendar.

If you’d like to do some reading in advance I can recommend checking out:

#hallway-hangout

Hallway Hangout Notes

Disappointingly, the recording of the session failed for unknown reasons – but we had a fantastic session with about 10 folks in attendance. I’ve included my notes below for anyone who would like to catch up on what we chatted about, and share further thoughts in the comments.

Third Party Platforms -> WordPress

Doing these migrations at scale is hard – dealing with 10s or 100s of thousands of posts.

Much of the work migrating from third parties theoretically relies on APIAPI An API or Application Programming Interface is a software intermediary that allows programs to interact with each other and share data in limited, clearly defined ways. access that may not be available.

Common pathway mentioned was having HTMLHTML HyperText Markup Language. The semantic scripting language primarily used for outputting content in web browsers. to import into WP content and hoping it can be transformed into blocks effectively.

Discussed the challenges of mapping content of the source site into the correct fields (and ‘type’ of data/field)

Because GutenbergGutenberg The Gutenberg project is the new Editor Interface for WordPress. The editor improves the process and experience of creating new content, making writing rich content much simpler. It uses ‘blocks’ to add richness rather than shortcodes, custom HTML etc. https://wordpress.org/gutenberg/ blocks are defined in JavascriptJavaScript JavaScript or JS is an object-oriented computer programming language commonly used to create interactive effects within web browsers. WordPress makes extensive use of JS for a better user experience. While PHP is executed on the server, JS executes within a user’s browser. https://www.javascript.com/. – all validation happens client side so it’s hard to process and validate migrationMigration Moving the code, database and media files for a website site from one server to another. Most typically done when changing hosting companies. to blocks.

We need more robust handling of malformed html – as source sites won’t always have it correct.

Excitement for the Fields API and options that brings for mapping content.

Exporting from WordPress

Discussed the fact that XML export isn’t always useful.

Talked about the challenge of shortcodes and other dynamic content within sites. When exporting we rely on the destination having the same set up.

What would it look like to have an export of content with dynamic content rendered? It would be a snapshot in time – but would be more human readable.

Blocks/Builders/HTML

Discussed the challenge of so many Page Builders in the ecosystem – are we hoping to build pathways/solutions for all of them? Or do we prioritise (which comes with its own challenges).

Agreement that the blockBlock Block is the abstract term used to describe units of markup that, composed together, form the content or layout of a webpage using the WordPress editor. The idea combines concepts of what in the past may have achieved with shortcodes, custom HTML, and embed discovery into a single consistent API and user experience. directory idea could be helpful with this.

Having better definition (and documentation) of blocks might be a good first step to make that transform possible in general terms, and then specific solutions can be worked on.

WordPress > WordPress migration

Discussed the challenges of media migration.

OK when environments are the same – or you move the whole site (which is what most third-party plugins do).

Touched on staging sites – and the evergreen challenge of database syncing, and which is the ‘source of truth’.

A great quote from one of the attendees – “I’d hope we start with better generic APIs (like the HTML API improvements), and then the individual implementations in parallel. If we can parse the data, the rest is just mapping it”

Other notes

It was great to hear folks with a range of use cases, from small business sites through to agencies with the challenges of doing this work at scale – and all the edge and corner cases.

There’s a sense in which having better handling of data closer to the WP side of things may give a more common ‘point’ for those migrations to aim for, meaning agencies/developers only need to handle the getting it to that point – not all the way into WP. The question is where in the process does a common ‘transform’ take place?

There was a great suggestion to set up label for data liberation related work within the Gutenberg GithubGitHub GitHub is a website that offers online implementation of git repositories that can easily be shared, copied and modified by other developers. Public repositories are free to host, private repositories require a paid subscription. GitHub introduced the concept of the ‘pull request’ where code changes done in branches by contributors can be reviewed and discussed before being merged be the repository owner. https://github.com/ Repo.

—-
Thanks again to all that attended!

Sallie Goetsch 1:58 am on April 4, 2024

Heard about this at the last minute and was able to attend. I’ve now joined the SlackSlack Slack is a Collaborative Group Chat Platform https://slack.com/. The WordPress community has its own Slack Channel at https://make.wordpress.org/chat/. channel. I don’t have the technical chops to build any of this, but will be happy to test things, as imports into WordPress or from page builders to GutenbergGutenberg The Gutenberg project is the new Editor Interface for WordPress. The editor improves the process and experience of creating new content, making writing rich content much simpler. It uses ‘blocks’ to add richness rather than shortcodes, custom HTML etc. https://wordpress.org/gutenberg/ are definitely things I end up doing quite a lot of in real life.
jordesign 1:30 am on April 5, 2024
Hallway Hangout Notes

Disappointingly, the recording of the session failed for unknown reasons – but we had a fantastic session with about 10 folks in attendance. I’ve included my notes below for anyone who would like to catch up on what we chatted about, and share further thoughts in the comments.

Third Party Platforms -> WordPress
- Doing these migrations at scale is hard – dealing with 10s or 100s of thousands of posts.
- Much of the work migrating from third parties theoretically relies on APIAPI An API or Application Programming Interface is a software intermediary that allows programs to interact with each other and share data in limited, clearly defined ways. access that may not be available.
- Common pathway mentioned was having HTMLHTML HyperText Markup Language. The semantic scripting language primarily used for outputting content in web browsers. to import into WP content and hoping it can be transformed into blocks effectively.
- Discussed the challenges of mapping content of the source site into the correct fields (and ‘type’ of data/field)
- Because GutenbergGutenberg The Gutenberg project is the new Editor Interface for WordPress. The editor improves the process and experience of creating new content, making writing rich content much simpler. It uses ‘blocks’ to add richness rather than shortcodes, custom HTML etc. https://wordpress.org/gutenberg/ blocks are defined in JavascriptJavaScript JavaScript or JS is an object-oriented computer programming language commonly used to create interactive effects within web browsers. WordPress makes extensive use of JS for a better user experience. While PHP is executed on the server, JS executes within a user’s browser. https://www.javascript.com/. – all validation happens client side so it’s hard to process and validate migrationMigration Moving the code, database and media files for a website site from one server to another. Most typically done when changing hosting companies. to blocks.
- We need more robust handling of malformed html – as source sites won’t always have it correct.
- Excitement for the Fields API and options that brings for mapping content.
Exporting from WordPress
- Discussed the fact that XML export isn’t always useful.
- Talked about the challenge of shortcodes and other dynamic content within sites. When exporting we rely on the destination having the same set up.
- What would it look like to have an export of content with dynamic content rendered? It would be a snapshot in time – but would be more human readable.
Blocks/Builders/HTML
- Discussed the challenge of so many Page Builders in the ecosystem – are we hoping to build pathways/solutions for all of them? Or do we prioritise (which comes with its own challenges).
- Agreement that the blockBlock Block is the abstract term used to describe units of markup that, composed together, form the content or layout of a webpage using the WordPress editor. The idea combines concepts of what in the past may have achieved with shortcodes, custom HTML, and embed discovery into a single consistent API and user experience. directory idea could be helpful with this.
- Having better definition (and documentation) of blocks might be a good first step to make that transform possible in general terms, and then specific solutions can be worked on.
WordPress > WordPress migration
- Discussed the challenges of media migration.
- OK when environments are the same – or you move the whole site (which is what most third-party plugins do).
- Touched on staging sites – and the evergreen challenge of database syncing, and which is the ‘source of truth’.
- A great quote from one of the attendees – “I’d hope we start with better generic APIs (like the HTML API improvements), and then the individual implementations in parallel. If we can parse the data, the rest is just mapping it”
Other notes
- It was great to hear folks with a range of use cases, from small business sites through to agencies with the challenges of doing this work at scale – and all the edge and corner cases.
- There’s a sense in which having better handling of data closer to the WP side of things may give a more common ‘point’ for those migrations to aim for, meaning agencies/developers only need to handle the getting it to that point – not all the way into WP. The question is where in the process does a common ‘transform’ take place?
- There was a great suggestion to set up label for data liberation related work within the Gutenberg GithubGitHub GitHub is a website that offers online implementation of git repositories that can easily be shared, copied and modified by other developers. Public repositories are free to host, private repositories require a paid subscription. GitHub introduced the concept of the ‘pull request’ where code changes done in branches by contributors can be reviewed and discussed before being merged be the repository owner. https://github.com/ Repo.
—-
Thanks again to all that attended!

Comments are closed.

Welcome!

Communication

Share this: