Zack Krida 11:41 pm on August 22, 2022

Openverse Priorities Brainstorm

This call for feedback and discussion will be open until September 5th.

The OpenverseOpenverse Openverse is a search engine for openly-licensed media, including images and audio. Find Openverse on GitHub and at https://openverse.org. community held a prioritization meeting on August 10th to discuss next steps for upcoming projects. You can read the notes from that conversation here. Since that meeting, we’ve completed a number of the projects discussed:

APIAPI An API or Application Programming Interface is a software intermediary that allows programs to interact with each other and share data in limited, clearly defined ways. Stability & Performance Mitigation
Re-enable data refreshes
Catalog Operational Improvements

I (@zackkrida) would like to take some time now to discuss what our next projects should be. To kickoff the conversation I’ll start with some ideas, proposing projects grouped by the primary repository in which work will take place.

I’ve tried to strike a balance between operational improvements and implementing new features. We still need to do more work on getting documentation for these projects live on this Make site and handbook, but can do so after we’ve reached consensus on next steps.

Here are my project ideas, by repository:

Frontend

Copy Improvements. Make text-only edits across the Openverse experience to clarify user flows, actions, and the value Openverse brings to the community.
iFrameiframe iFrame is an acronym for an inline frame. An iFrame is used inside a webpage to load another HTML document and render it. This HTML document may also contain JavaScript and/or CSS which is loaded at the time when iframe tag is parsed by the user’s browser. MigrationMigration Moving the code, database and media files for a website site from one server to another. Most typically done when changing hosting companies.. Migrate Openverse away from the current iframe-ed solution and switch to a dedicated domain name.

API

Result Sorting Options. Allow users to use the current ‘Recommended’ algorithm, but also a new ‘most recent’ sort option. This would require accompanying frontend work. It would be a nice introduction for our our developers on how the results are currently sorted while providing a useful feature to users.

Catalog

Provider Script Refactors. This operational improvement will allow us to make bulk changes to the base class in the future and embrace a standardized approach to writing provider scripts.
3D Model Support. While 3D model support will take place across all three of our repositories, work can begin in the catalog to build relationships with providers; start writing provider scripts; and start collecting 3D model metadata.

I’m looking forward to these ideas being a launch point for additional discussion, specifically around who would like to volunteer to lead these project(s) and other project ideas you think take priority.

Francisco 10:25 pm on August 25, 2022

I bend on the “Copy improvements” task for our next milestone. It is a quick win as the amount of work seems lower than the other points while the experience improvement is significant. In that vein, I would like to lead that initiative. I will need some coding help, yet it is something where I can rely on my basic knowledge.

On the other hand, the “3D model support” is a goal that can wait for the following year. It was not as voted as the other ideas shared, and adding a new content type sounds like a considerable workload that will require a design stage before starting its implementation.

I am curious about the “Result Sorting Options” idea. It sounds interesting but I would like to dive into the details and what use cases we envision for it.

And finally, adding that for the iFrameiframe iFrame is an acronym for an inline frame. An iFrame is used inside a webpage to load another HTML document and render it. This HTML document may also contain JavaScript and/or CSS which is loaded at the time when iframe tag is parsed by the user’s browser. migrationMigration Moving the code, database and media files for a website site from one server to another. Most typically done when changing hosting companies. point, we will need to solve the headerHeader The header of your site is typically the first thing people will experience. The masthead or header art located across the top of your page is part of the look and feel of your website. It can influence a visitor’s opinion about your content and you/ your organization’s brand. It may also look different on different screen sizes. and homepage proposals before. The header (#1669) is open for feedback, while the homepage (#1433) needs some tweaks based on the comments posted.

Thanks for wrapping up the session and looking forward to what comes next.
Dhruv Bhanushali 9:47 am on August 26, 2022

The thing I’m most excited about on the list is the `iframeiframe iFrame is an acronym for an inline frame. An iFrame is used inside a webpage to load another HTML document and render it. This HTML document may also contain JavaScript and/or CSS which is loaded at the time when iframe tag is parsed by the user’s browser.` migrationMigration Moving the code, database and media files for a website site from one server to another. Most typically done when changing hosting companies. because it will unblock us to implement so many improvements in the areas of SEO and OG metadata. It also allows us to delete a bunch of glue code in the theme that loads the site inside the `iframe`. Also @fcoveram‘s new designs for the headerHeader The header of your site is typically the first thing people will experience. The masthead or header art located across the top of your page is part of the look and feel of your website. It can influence a visitor’s opinion about your content and you/ your organization’s brand. It may also look different on different screen sizes. look absolutely amazing. I would love to take the initiative to build some of those new base components (once the design iterations have completed, of course).

I’m also looking forward to supporting 3D models but this seems to be in the distant future. We can collect some data about different providers and the details they can provide while @fcoveram gets the time to design the UXUX UX is an acronym for User Experience - the way the user uses the UI. Think ‘what they are doing’ and less about how they do it. around it but the actual implementation will be blocked on these fronts.
- Francisco 1:08 pm on August 26, 2022
  
  I would love to take the initiative to build some of those new base components (once the design iterations have completed, of course).
  
  I have already made personal iterations so looking forward to seeing the team’s feedback. If we do not have further comments, then we are ready to start its implementation.
  
  the time to design the UXUX UX is an acronym for User Experience - the way the user uses the UI. Think ‘what they are doing’ and less about how they do it. around it but the actual implementation will be blocked on these fronts.
  
  Fortunately, we are on a good base. Issue #1432 iterates and solves most use cases shared, but probably new things will come up once we start facing the content support.
sarayourfriend 4:27 pm on August 26, 2022
Thanks for writing this up, Zack. I spoke with you privately a bit about my thoughts and then ended up discussing it with Madison and Staci over coffee later yesterday as well (was a pleasure to see them in person). This helped solidify some general and specific thoughts that I’ve been having about how we’re currently making prioritisation decisions.

The TL;DR of my sentiments here are that I think we’re mostly prioritising based on two things:

1. Guesses about what users want
2. Kind of what we personally want to work on and what seems most immediately achievable

The second one I don’t think is necessarily a bad thing. Having folks motivated and personally excited about a feature tends to produce better collaboration and more thoughtful implementations. The first, however, is a bit concerning and I think easy to evidence based on one big problem that exists: we currently have essentially _zero_ “business intelligence” about how OpenverseOpenverse Openverse is a search engine for openly-licensed media, including images and audio. Find Openverse on GitHub and at https://openverse.org. is used, even on the most basic and fundamental questions. While we can take guesses about what users want, it’s not currently possible to know whether anything we do positively or negatively affects them. The reasons for this are pretty simple:

1. We have not recently and are not currently interviewing actual users about their experience with Openverse.
2. We do not have any insight into even the most basic “metrics” that we could think of for Openverse’s frontend, like the click-through rate of searches.

The first is one that is difficult to solve right now because of time constraints (I am guessing). It’s definitely important for us to be able to do again eventually, but I’m not as worried about it, even though it means we’re operating with less information than we could otherwise have.

The second, on the other hand, is more concerning to me because it’s the first flashlight we could use to illuminate some of our way forward and to give us immediate and clear feedback about whether our changes are having a positive effect or not. For example, result relevancy, iframeiframe iFrame is an acronym for an inline frame. An iFrame is used inside a webpage to load another HTML document and render it. This HTML document may also contain JavaScript and/or CSS which is loaded at the time when iframe tag is parsed by the user’s browser. migrationMigration Moving the code, database and media files for a website site from one server to another. Most typically done when changing hosting companies., and copy improvements (all listed as suggested priorities here, more or less) would all most likely have a direct measurable effect on observations we make about how useful Openverse is for our users.

I’ll quickly not here to acknowledge that our APIAPI An API or Application Programming Interface is a software intermediary that allows programs to interact with each other and share data in limited, clearly defined ways. is a significant traffic target and we have some large API users. I have some ideas for how to measure things here, but I’m not sure if it would be as fruitful as measuring our own browser frontend to our data.

My sense right now is that even very basic analytics would open up the path for helping us prioritise projects that we think will help them and, additionally, assist in knowing whether those projects had the effect we presumed they would have.

For example: if we started to measure the click-through rate of search queries (how many times a search query results in the user clicking through to the provider), our copy changes to what is currently the “view” or “download” button (depending on media type) would be easy for us to measure the effectiveness of. We have good guesses about what type of language could improve that, but under current circumstances we will not have any idea if the situation is actually improved. Aside from stochastic feedback we sometimes get from community members (which is valuable and thoughtful, but essentially anecdotal) we just don’t have visibility into how our changes effect the usefulness of Openverse’s frontend.

I share all of this essentially to say that I think we should not currently prioritise 3D Models in the catalogue. Here are my reasons for saying this:

1. 3D Model support, as noted in this post, will be a cross-repository effort. My personal estimate for this work, based on how long it’s taken to get audio into a good spot (from planning to “out of betaBeta A pre-release of software that is given out to a large group of users to trial under real conditions. Beta versions have gone through alpha testing in-house and are generally fairly close in look, feel and function to the final product; however, design changes often occur as part of the process.”), is at least a year, from catalogue to frontend. I suspect 3D Models might be even longer because I am slightly afraid of problems we’ll run into with model viewer interactions, but I’ll be conservative and just match it with our current experience with audio.
2. 3D Models are cool and it improves one visible metric (number of media types supported), but they don’t in and of themselves produce a better experience for our users… at least not provably. Without analytics, we won’t know.
3. Without analytics, we don’t know how useful our current media types are. If we add 3D Models but our click-through rate is really low, for example, then we’ve just spent a bunch of time adding some cool stuff that very few people will actually use. On the other hand, if we spend the same time measuring the broad user-story metrics that we are able to measure now combined with researching and improving relevancy of our existing media types, we would make it far more likely that users come back and continue to use Openverse. That will benefit the existing and future media types because users will believe that Openverse is actually useful and important. I don’t think analytics would take a year to implement, but even if it did, it would unlock our ability to prioritise in a pretty significant way: we’d actually know if the changes we’re making are improving or hurting the user experience.
4. The catalogue has significant infrastructural hurdles that continue to grow. Staci mentioned to me that iNaturalist will add 15% more images to our catalogue. If Elasticsearch indexing scales linearly (which would be lucky) that would extend the current data refresh time by 15% as well. It’s already a long process. The more data we add, those processes will just take longer and longer and longer. If we add a new media type and continue ingesting new providers in images and ingesting more and more audio, what happens if all together our data refresh lasts longer than a week? Eventually we’d run into a situation where data is being ingested but the length of time between ingestion and indexing into Elasticsearch gets bigger and bigger and bigger (because it takes longer to index than to ingest). There are some potentially easy wins here. Some ideas Madison, Staci and I came up with looking into are: using the `UPDATE_INDEX` action on the ingestion server instead of completely rebuilding indexes from scratch; eliminating all indexes except for the identifier index on the API database, using Elasticsearch as the primary datastore for the API results and switching to use something like `pg_bulkload` to copy newly ingested records into the API database _if they’re even needed there at all_ (can we _just_ use ES on the API side?). Additionally, we will need to start digging into computing scores for individual results. Some score components may not come directly from provider metadata and may only be able to be accurately or consistently pulled from the media files themselves. There are a handful of things we’ve already discussed pulling files down for like perceived image quality and audio bitrate and waveform generation. These will increase the length of time it takes to ingest a single result fully and get it ready for indexing. All of these things together means that we really do need to consider our current high-latency approach a long-term risk that will likely get harder to solve as we get more data because retroactive score calculation, especially for file-based score components, will be a heavy, lengthy, and complicated process all by itself.
5. In addition to these catalogue issues, we also know about and have some good starting ideas for how to better deal with dead links in the longer term in ways that will improve API efficiency and overall data quality in the catalogue.

With all of this in mind, I would suggest the following things to prioritise. It’s not a huge modification to the original list but I do think it’s significant. I’ve added justifications for each thing.

Frontend:
- Copy improvements: We’ve received direct feedback that we can address and we are also making things more consistent across media types. These require a lot of forethought and collaboration but we have a lot of people participating in this activity in various different areas and we have lots of enthusiastic contributors on the frontend who are excited to work on things. Copy improvements are a great way to ease more people into contributing to the frontend.
- iFrame Migration: We’ve already dedicated ourselves to this and, as mentioned by Dhruv, it will unlock the ability to work on some obvious SEO improvements that have already been identified. These are objective things we know will improve Openverse’s SEO. This project actually has several sub projects that need to be planned: (a) infrastructural changes necessary (maybe just a CNAME record, probably not _just_ that though), (b) new headerHeader The header of your site is typically the first thing people will experience. The masthead or header art located across the top of your page is part of the look and feel of your website. It can influence a visitor’s opinion about your content and you/ your organization’s brand. It may also look different on different screen sizes. planning, prototyping and implementation, (c) design iterations and subsequent planning, prototyping and implementation.
- Plan and implement analytics tooling: See other justifications below for this work, this part complements the heavier analysis work on the API side (maybe on Catalogue too, if we use Airflow to do analysis of behaviours we see).
Overall, the frontend has the largest number of active contributors which gives us a lot of freedom. On the other hand, the frontend has a ton of GitHubGitHub GitHub is a website that offers online implementation of git repositories that can easily be shared, copied and modified by other developers. Public repositories are free to host, private repositories require a paid subscription. GitHub introduced the concept of the ‘pull request’ where code changes done in branches by contributors can be reviewed and discussed before being merged be the repository owner. https://github.com/ issues so I don’t think we would run out of work here. If we did, that’d be an excellent problem to have and we’d get to celebrate having met all of our commitments (and anticipate to dreaming a little bigger next time).

API:
- Plan and implement analytics infrastructure: Result sorting options could prove to be useful and popular, but we’d never know because we don’t actually measure any usage right now. Planning and implementing user-story level analytics from the frontend (and anything we can glean from repeated requests from API request sessions) would give us the ability to actually understand the effect of our changes.
- Plan and implement getting dead links back up to the catalogue: This is a bit of API + Catalogue planning. It is mentioned on the catalogue list as well. This should be planned together as what the catalogue can do with the dead links may dictate what the API needs to serve up (or whatever the API ends up doing)
Catalogue:
- Provider script refactors: These are well justified refactors.
- Research and identify data pipeline improvements: This isn’t just a catalogue project as it does require looking into improvements for the API (can we use ES as the sole source for result storage, could we drop indexes on the API DB, etc) but I think the folks working on the catalogue are the most generally knowledgeable about data infrastructure things and the entire data pipeline probably informs how all of this would fit together.
- Plan and implement how to use dead link information: The catalogue side planning of the API’s dead link stuff. Same justification.
When I look at this list it does look like we’d be stretching ourselves even with this “basic” set of prerequisites for getting more insight into whether our changes even positively impact Openverse’s users. I’m also assuming that we’d continue ongoing perpetual work like adding new image and audio providers, improving infrastructure reliability, ease of deployment and service observability, bug fixes, etc.

In summary: I think we should consider prioritising planning and wiring up/implementing our ability to analyse current usage patterns and make improvements to our data quality and gain insight into how we can measure the effects of changes to result scoring.

The prioritisation I suggest above has the following qualities:

1. It sets us up to be able to make better decisions about future prioritisation by giving us insight into how past projects and hypotheses play out
2. It has a couple flashy and exciting frontend changes that are easy to market Openverse around (iframe migration includes several significant redesigns like the impressive new header design)
3. It gives us clear business intelligence that helps justify the long-term maintenance and investment in Openverse
4. It makes Openverse’s catalogue and API infrastructure sustainable long term. Right now our current approach has a hard limit before it essentially gets away from us (1 week of data refreshing).

These are just my opinion, of course. I’m not going to be around to continue in this discussion for the next few weeks, but I trust all of you to consider what I’ve said and make the best decision, regardless of whether that includes any of my proposed ideas. Openverse is lucky to have a dedicated and passionate set of intelligent and thoughtful folks working on it.
- Zack Krida 11:10 pm on August 29, 2022
  
  I am working on a broader post that addresses some of your other thoughts on prioritization, but I wanted to point you towards this comment about some of our past analytics code and what we collected.
stacimc 7:06 pm on August 26, 2022

I’ll weigh in briefly on 3D models: from the Catalog side, I believe this work is achievable and something we could prioritize relatively easily. I’m not familiar with the frontend side, but trust teammates’ evaluation that it will take considerably longer to implement there. We could do the work on the Catalog side and plan on delaying the frontend work until prioritization allows; my concern here is that no matter how this is timed, I anticipate that necessary changes to the data structure will be discovered while working on the frontend side. However, this will be much more difficult for Catalog folks if a longer time has passed since it was implemented on the backend, and if a large amount of provider data has already been ingested. That being said, I’d be more interested in what folks who worked on the addition of Audio have to say here about whether it would be useful to do the Catalog work early.

Regarding analytics as described by Sara: this is something I’d be really interested to work on, and even more so to have available to us. I’m especially interested to see if this work would be much quicker to implement than, say, 3D models. Even a minimal set of metrics around click-through rate would be a huge asset. I don’t feel especially qualified to say how complex that project would be, but I think this is definitely worth discussing. Being able to accurately track the impact of changes could also help to reveal some of these smaller frontend proposals to be the big wins we hypothesize them to be.

About the other Catalog priorities: I will add that in addition to refactoring the provider scripts, we also need to reenable the reingestion workflows. I do think this is critical, because it allows us to (a) backfill dated providers, and (b) ingest updates to old data from dated providers. Some of our largest providers, including Flickr, are dated DAGs. Much of this work is in progress and will wrap soon, so maybe it should be excluded from this longer-term prioritization, but I wanted to mention it!

A lot of work has been done on the Catalog to get old workflows simply up and running, and I think we’re very close to getting it to a relatively ‘stable’ state with the completion of the provider refactors and reingestion flows. Maybe we will finally be at a point where larger changes to the pipeline can be considered seriously. Notably, the ideas that Madison, Sara, and I discussed and which Sara described above wouldn’t just be refactors that improve code quality or make contributing easier (although those are obviously still valuable goals) — they would aim to significantly move the needle on one of our few observable metrics, which is the time from ingestion to availability on the frontend for an individual record. At the moment this is maximum ~10 days for an image (if it is ingested shortly after an image data refresh begins. It must wait 7 days for the next image refresh to start, and another 3 days for it to complete). If we do nothing, this number will also grow, as the (exciting) addition of new images increases the length of the image refresh.

Final thoughts: I agree very strongly with the goal Zack outlined in the parent post, `a balance between operational improvements and implementing new features`. As estimates for the amount of time 3D models will take appear to be very long, I’m becoming more excited by the prospect of adding some minimal analytics, and working on improving the data refresh. I think both ideas require a lot more RFC/scoping work to verify the assumptions about how long they will take, however. The iframeiframe iFrame is an acronym for an inline frame. An iFrame is used inside a webpage to load another HTML document and render it. This HTML document may also contain JavaScript and/or CSS which is loaded at the time when iframe tag is parsed by the user’s browser. migrationMigration Moving the code, database and media files for a website site from one server to another. Most typically done when changing hosting companies. and new headerHeader The header of your site is typically the first thing people will experience. The masthead or header art located across the top of your page is part of the look and feel of your website. It can influence a visitor’s opinion about your content and you/ your organization’s brand. It may also look different on different screen sizes. designs bring some exciting user-facing frontend changes. Although I would love to have analytics already in place before adding something like the Result Sorting Options, I also think that could be a great project to add quickly so we can continue providing user features if the analytics component is more complex than assumed.
beccawidom 10:38 pm on August 26, 2022
I love everything here about adding analytics, and would love to help with that if I can. I wanted to also propose another piece of work.

I took a look at the openverse roadmap, and I’m inspired by goal #3: “Improve result quality and relevancy: OpenverseOpenverse Openverse is a search engine for openly-licensed media, including images and audio. Find Openverse on GitHub and at https://openverse.org. will make significant improvements towards showcasing high-quality, relevant results, in order to increase adoption by new users and encourage existing users to come back.”

Without getting into expert elasticsearch configuration or using image recognition algorithms, I do think that there are some things we could do – and that I would be interested in volunteering to work on – that could potentially improve the user experience.

For context, I’d like to start with my experience of using Openverse to search for artwork by Kiyohara Yukinobu, an artist I came across while working on the met museum refactor.
- First, I just typed Kiyohara Yukinobu into the search box – no punctuation, no additional check boxes – and I got three results: 1 from Wikimedia and 2 from flickr, but none from the Met Museum.
- Next, I scrolled down on the sidebarSidebar A sidebar in WordPress is referred to a widget-ready area used by WordPress themes to display information that is not a part of the main content. It is not always a vertical column on the side. It can be a horizontal rectangle below or above the content area, footer, header, or any where in the theme. to find the Creator checkbox, did the same (no punctuation), and got 68 results. Not all of them were relevant.
- Finally, I put the artist name in quotes and kept the creator checkbox on, and I got 15 results, all of them seem to be relevant, but visually it was really 10 distinct images. For the particular Met Museum images I expected, Wax wings and cherry blossoms appeared once from wikimedia, Bird and Flowers appeared once from wikimedia, Quail and Millet appeared 4 times, including from the met, And none of the original three distinct and relevant images from wikimedia and flickr were on the page.
Maybe for the Met Museum in particular, as part of the refactor, I should just append by {artist} to the end of the title, where there is a meaningful artist name, since we know that images from the Met Museum will be fine art and that people might want to search by artist name.

That said, I also want to create clarity on our metadata transformations across the stack, e.g. do we need to add something somewhere to set a quality level on tags by provider based on this? Is the front end or the APIAPI An API or Application Programming Interface is a software intermediary that allows programs to interact with each other and share data in limited, clearly defined ways. doing some kind of transformation to put together “description” for elastic search? If it’s present, where would the code for that exist?

And I guess bigger picture – if we’ve decided that we are a more bespoke, open license, relationship with providers kind of search engine (as opposed to a scaled algorithm driven one), then I think we’re obligated to treat the data we get from providers with a lot of care, and document what we mean by title and creator and tags and so on, and how we translate from their datasets to ours.

Perhaps this is something we should prioritize over 3D models, since leveraging our existing data could give us much better result relevancy, and adding new datasets without those understandings may not be as impactful for users.
Madison 1:14 am on August 27, 2022
I appreciate the thoughtfulness and intentionality behind everything that folks have discussed so far! Sara’s suggestions around analytics are well-put and reasonable. The metrics suggested would give us some crucial insight into “is what we’re doing working”.

I’ve spent the last week trying to unpack what my motivation was for wanting to pursue 3D models; why was I excited about it and why did I want to continue to push for it over other priorities? For myself, I realized it had to do with the following:
- As previously mentioned, 3D models (once they make it to the frontend, or even the APIAPI An API or Application Programming Interface is a software intermediary that allows programs to interact with each other and share data in limited, clearly defined ways.) are a flashy and obvious value add that we can show off. It becomes a very easy thing to point at as an accomplishment.
- My view of the 3D model project starts at the catalog and ends at the ingestion server. When I’m thinking about “what it would take to add 3D models”, I’m not thinking about the tremendous effort this would take on the frontend.
- Having 3D models in the catalog, with regularly running provider ingestion scripts, ensures that we can have plenty of data present when we move to the API/frontend implementation and guarantees that the catalog isn’t a blocker.
- We’ve already sketched out much of the work for 3D models on the catalog/API end, so it’s relatively known territory.
That being said, other folks have mentioned points that work counter to the above (respectively):
- While this would be an obvious add, what’s not obvious in our current setup is how useful it would actually be to users.
- Implementing 3D models is going to be a significant effort on the frontend. When thinking about adding 3D models, I have to consider this aspect more often.
- Other folks have pointed out that we will almost certainly have changes we’ll need to iterate on, and that will be harder if we’ve accumulated a large amount of data that may need to be re-ingested.
- Just because something may be tactically easier to approach doesn’t mean that we should do it; it may be more useful to take a step back and plan more strategically. I say this mostly for myself – new projects and unknown fields can be daunting and difficult to navigate, and I often find that I’m much more drawn to clear plans with discrete tasks. 3D models was filling this space for the near future, but a step back and a new plan may be a better move.
I agree with others that our priorities (for the API and the catalog) should probably be along the line of:
- Finishing the provider refactor
- Getting reingestion workflows online
- Analytics & metrics infrastructure
- Search result improvement
With that last point, I think there’s a need to learn more about the current setup. I suspect there are small improvements we could make that would have a significant impact on search results. Some to potentially explore are:
- Verifying tag search behaves as expected
- Implementing “authority” document score boosting
- Collecting more descriptions
- Adjusting how great a factor age is in document score
The above seems like plenty of work, and all of it sets us up to be more prepared for 3D models when the times comes. As long as we do have changes we can showcase to the community as visible or tangible improvements of OpenverseOpenverse Openverse is a search engine for openly-licensed media, including images and audio. Find Openverse on GitHub and at https://openverse.org., I think we’ll be in a good place!
- kgodey 8:19 pm on August 29, 2022
  
  I don’t have any opinions about what to prioritize from the list above, but I think that talking to users and/or collecting some usage analytics would be helpful in prioritizing your roadmap. I remembered writing up a list of useful metrics to help us prioritize work at CC, I dug up the issue in case that’s helpful. I’m not sure if we implemented these or not, we certainly didn’t have time to dig into them and use them.
  - Zack Krida 11:02 pm on August 29, 2022
    We had successfully implemented four of these analytics events, and yes, never actually had the opportunity to analyze the data:
    
    1. sendSearchQueryEvent – Send the query the user searched
    2. sendResultClickedEvent – Send the query, the uuid of the clicked on result, and the rank of the result (i.e, it’s index in the list of results).
    3. sendDetailPageEvent – The uuid of the result navigated to
    4. sendSearchRatingEvent – A thumbs up / down asking ‘are these results relevant?’ on the search result pages
    
    For anyone else following along, the analytics server was removed earlier this year because we wanted to make in opt-in privacy-wise and evaluate alternative backends:
    
    https://github.com/WordPress/openverse-api/pull/618
    
    https://github.com/WordPress/openverse-frontend/pull/1084
Tony G 11:26 pm on August 27, 2022

Random observer notes here:

RE: Guesses about what users want, metrics/analytics, etc : It’s my impression that the WP community at large is simply not aware of OpenverseOpenverse Openverse is a search engine for openly-licensed media, including images and audio. Find Openverse on GitHub and at https://openverse.org.. There have been a few announcements and mentions, but it hasn’t yet reached the common admin/developer who is looking for images to add into content – I came back here specifically because I’m in a forum discussion where that’s evident, and I wanted to check the current status.
I believe this will change radically with integration with WP CoreCore Core is the set of software required to run WordPress. The Core Development Team builds WordPress., and easy retrieval of assets into posts and the media library. I believe cultivation of pluginPlugin A plugin is a piece of software containing a group of functions that can be added to a WordPress website. They can extend functionality or add new features to your WordPress websites. WordPress plugins are written in the PHP programming language and integrate seamlessly with WordPress. These can be free in the WordPress.org Plugin Directory https://wordpress.org/plugins/ or can be cost-based plugin from a third-party developers to use the APIAPI An API or Application Programming Interface is a software intermediary that allows programs to interact with each other and share data in limited, clearly defined ways. will foster a base of Openverse users who can at least comment on it from that angle, perhaps before it’s available in Core. When people are actually using it, metrics and feedback will be overwhelming. To prepare for that, ongoing #marketing will reach more people and drum up interest, in announcements in Core update Welcome page, and the WP Events and News widgetWidget A WordPress Widget is a small block that performs a specific function. You can add these widgets in sidebars also known as widget-ready areas on your web page. WordPress widgets were originally created to provide a simple and easy-to-use way of giving design and structure control of the WordPress theme to the user..

RE: 3D – I know nothing of the effort on this so far and gladly accept that all of my notes here might be completely invalid.
I’m a total nerd, did lots of interactive VRML, have used 3D assets, and have an ongoing side interest in such things. But this is not something that is discussed in common WP circles. It’s just not on the radar yet.
So as cool and forward thinking as it is, I personally believe the Openverse initiative would benefit more with the other focuses (API, integrations, blocks…) for media that is in the most common use right now (images, video, audio). I’m not saying I think this should not be pursued with some resources, now or in the future. I’m saying I don’t see effort put into 3D right now as being key to getting Openverse off the ground for WordPress users, even into the coming year.
That hurts. I like to push the envelope too, to create the future so that it’s ready when people ask for it. But at this moment, I don’t even see 3D that close on the horizon (in terms of demand) to compare its priority with with other projects that really need to be worked on sooner – it’s a matter of need vs want for this specific project at this specific time. Sorry.
Maybe that’s a good thing. Maybe the initiative to support 3D assets has a rare gift, that development can be deferred until there is adequate demand on the horizon, that there is time to prepare for this to be a major feature rollout just when the audience is ready.

Sincere appreciation for all efforts on Openverse.
Krystle Salazar 10:17 pm on August 29, 2022
Thank you for writing this to continue the discussion. I’ll share my ideas about projects prioritization.

Frontend
I agree with the priorities listed here for the rest of the year. Perhaps we may add a project/milestone to manage the efforts of migrating to Nuxt 3 or move away in favor of a setup with VueVue Vue (pronounced /vjuː/, like view) is a progressive framework for building user interfaces. https://vuejs.org/. 3 & Vite.

APIAPI An API or Application Programming Interface is a software intermediary that allows programs to interact with each other and share data in limited, clearly defined ways. / Catalog
The important thing I would really prefer to start before the 3D models is to make a review of our data stores. Some gaps in knowledge of old processes have resulted in duplicated work in different parts of the stack (e.g., license_url), this situation is confusing and I consider in part it can be easily solved, but need some careful coordination between the two (or even three) repositories. I will be glad to collaborate on this effort and even lead if folks want to pass on that role.
- Data normalization: I’d like us to include things like creating guidelines for selecting the URLURL A specific web address of a website or web page on the Internet, such as a website’s URL www.wordpress.org when there are different image sizes available, authors, etc., for provider scripts. See the list in the DAG Status Information. Maybe also include the database unification.
- Improve dead link handling and sanitation
- Migrate the Ingestion process to a DAG
- Reactive stopped provider DAGs: not sure if this is a goal of the current Catalog milestone (Provider DAG refactors) but since we’re already promoting this content we should reactive those soon
- Flag mature content on ingestion by tags: this is a new one I proposing. I thought recently that we could implement a naive filterFilter Filters are one of the two types of Hooks https://codex.wordpress.org/Plugin_API/Hooks. They provide a way for functions to modify data of other functions. They are the counterpart to Actions. Unlike Actions, filters are meant to work in an isolated manner, and should never have side effects such as affecting global variables and output. just by inspecting the tags against a list of flagged tags. This could make search results much safer.
Infrastructure
These projects have been more or less decided and one is already ongoing but I mention them here again to take them into account when weighing the priority of other projects based on time and team availability constraints.
- MigrationMigration Moving the code, database and media files for a website site from one server to another. Most typically done when changing hosting companies. to ECS
- Monitoring & Analytics
- Improvements to the thumbnails server and the waveform generation
Zack Krida 11:29 pm on August 29, 2022
I’m really excited by the discussion here and going to try and grab some common ideas I’m seeing from multiple folks.

On 3D models

While there’s consensus that the catalog work would be straightforward, there’s a lot of folks indicating that it would be easiest to work on the entire feature end-to-end and that there is other work that would be better to prioritize. Work that would help us quantify the value-add of 3D models in the first place. I’m inclined to agree with assessment. Perhaps we could revisit showing video and 3d models as ‘coming soon’ in the frontend UIUI UI is an acronym for User Interface - the layout of the page the user interacts with. Think ‘how are they doing that’ and less about what they are doing., add them to the content switcher, and look at page views on those media types as one metric to use when considering implementing them. 3D models are cool and flashy and exciting but can wait.

Improving result quality and relevancy

I really admire @stacimc‘s point, which was something I hadn’t considered:

“…one of our few observable metrics, which is the time from ingestion to availability on the frontend for an individual record.”

This is an excellent, measurable metric. I also think, unfortunately, it doesn’t mean much to users right now, because they’re not able to easily see the newest images in OpenverseOpenverse Openverse is a search engine for openly-licensed media, including images and audio. Find Openverse on GitHub and at https://openverse.org.. This is another reason I would really like to prioritize this project:

Result Sorting Options. Allow users to use the current ‘Recommended’ algorithm, but also a new ‘most recent’ sort option. This would require accompanying frontend work. It would be a nice introduction for our our developers on how the results are currently sorted while providing a useful feature to users.

I’m seeing a lot of agreement that result quality has a lot of room for improvement, and doing this would have several benefits:

1. It gives users a clear and direct way to change results for a search, see images they haven’t seen previously, and be ‘in control’ of the results they are viewing.
2. It adds a new and beneficial feature to the user interface, which we haven’t really done since launching our redesign.
3. Completing the work would be a strong opportunity to document and better understand our _existing_ search ranking.

I am very sympathetic to the idea that it would be great to have analytics tracking before implementing this. I also think that’s true of any significant feature. As the core contributorsCore Contributors Core contributors are those who have worked on a release of WordPress, by creating the functions or finding and patching bugs. These contributions are done through Trac. https://core.trac.wordpress.org. to a project sometimes we _do_ have to make a best-guess effort at what users will value, and I think a ‘sort by’ option for a search engine is a quite common and standardized enough feature that it would be immediately useful, and that we could add tracking to it as soon as we’ve built that functionality. The opportunity to better understand our search ranking as it is now should also not be overlooked. I see that as a prerequisite to many of the other ideas surrounding search relevancy.

The frontend work here is also quite small, we only need a piece of UI that adds an additional query param to search responses, and perhaps stores the value as a default for future visits. This could be a dropdown at the top of the search results, in the filterFilter Filters are one of the two types of Hooks https://codex.wordpress.org/Plugin_API/Hooks. They provide a way for functions to modify data of other functions. They are the counterpart to Actions. Unlike Actions, filters are meant to work in an isolated manner, and should never have side effects such as affecting global variables and output. bar, or somewhere else, but would be a single PR and maybe a few hours of work at most.

On quick frontend wins

These two discussed items:
- Frontend copy improvements.
- iFrameiframe iFrame is an acronym for an inline frame. An iFrame is used inside a webpage to load another HTML document and render it. This HTML document may also contain JavaScript and/or CSS which is loaded at the time when iframe tag is parsed by the user’s browser. migrationMigration Moving the code, database and media files for a website site from one server to another. Most typically done when changing hosting companies..
These are two things that immediately benefit users, by improving the UXUX UX is an acronym for User Experience - the way the user uses the UI. Think ‘what they are doing’ and less about how they do it. and making it easier to understand the _value_ Openverse provides. I think we could kick these off ASAP with @fcoveram leading the copy changes and @dhruvkb leading the iframe migration. The iframe migration will include some user interface improvements as well, like the headerHeader The header of your site is typically the first thing people will experience. The masthead or header art located across the top of your page is part of the look and feel of your website. It can influence a visitor’s opinion about your content and you/ your organization’s brand. It may also look different on different screen sizes. proposal. I don’t think the copy changes need to be perfect, either…a first-pass at some improvements would go a long way to helping future iterations. Related to folks interest in user testing it’s also quite reasonable to assume we’d want to A/B test copy changes in the future instead of just making “Guesses about what users want” as mentioned previouisly.

The catalog

Finally, on the catalog side, pausing 3D model progress should leave room for some of the other ideas discussed, perhaps:
- Reenabling reingestion workflows
This would go a long way to improve result quality, particularly around Flickr. We currently have indexed some content from Flickr that is no longer hosted avaliable or was removed, and this content is often explicit or reported for other reasons.

Analytics & metrics infrastructure

I think this is a great idea, and something we could get up quite quickly. There’s some prior art on this see [here](https://make.wordpress.org/openverse/2022/08/22/openverse-priorities-brainstorm/#comment-87) and the main pieces of work would be:
- Adding opt-in UI in the frontend (or documentation about heavily-anonymized data)
- Creating a new standalone microservice to collect and store the events
Loving this conversation so far and feel like we’re coalescing on some great ideas. Thanks all!
- Madison 11:37 pm on August 31, 2022
  
  I like and appreciate your idea of using result sorting as a relatively straightforward way to dive deep into our search infrastructure in order to understand it better 😄 Certainly it would be nice to have some way of approaching it that has clear completion criteria and would help us pick apart some of the moving parts!
- sarayourfriend 1:29 am on September 19, 2022
  
  > Creating a new standalone microservice to collect and store the events
  
  We could make use of something like GoatCounter which is privacy respecting, highly configurable, and would require no additional infrastructure on our side to maintain. It is currently free to use and I think with our low usage we would fall under the free-to-use “small to medium business” levels of traffic (not millions of pageviews). I’ve been using it on https://emdr-lightbar.org and it’s great.
  
  We could also self-host it. It doesn’t look too hard to host.
  
  The main thing I want to push back a bit on is that we have to create a new service from scratch to collect and store the events. We definitely do not.
Olga Bulat 7:13 pm on September 2, 2022
There are a lot of thoughtful comments in this thread!

Here are my thoughts on priorities:
Frontend
1. Copy improvements – this project is already being worked on, and it a quick win for the UXUX UX is an acronym for User Experience - the way the user uses the UI. Think ‘what they are doing’ and less about how they do it..
2. IFrameiframe iFrame is an acronym for an inline frame. An iFrame is used inside a webpage to load another HTML document and render it. This HTML document may also contain JavaScript and/or CSS which is loaded at the time when iframe tag is parsed by the user’s browser. migrationMigration Moving the code, database and media files for a website site from one server to another. Most typically done when changing hosting companies. on the front end, starting with the new headerHeader The header of your site is typically the first thing people will experience. The masthead or header art located across the top of your page is part of the look and feel of your website. It can influence a visitor’s opinion about your content and you/ your organization’s brand. It may also look different on different screen sizes.. This can be followed by all the SEO issues that our wonderful community members had created.
3. Full conversion to TypeScript. This will help us avoid the problem we sometimes encounter when JSDoc types don’t match or don’t work together well with TS types, probably due to some VueVue Vue (pronounced /vjuː/, like view) is a progressive framework for building user interfaces. https://vuejs.org/. / Vue 2 compilation details.

APIAPI An API or Application Programming Interface is a software intermediary that allows programs to interact with each other and share data in limited, clearly defined ways.

Result Sorting Options. Allow users to use the current ‘Recommended’ algorithm, but also a new ‘most recent’ sort option. This would require accompanying frontend work. It would be a nice introduction for our our developers on how the results are currently sorted while providing a useful feature to users.

This idea actually made me think that we could create ‘/all’ endpoints that return all media results available in the API database, without passing through ElasticSearch. This could be a good interim solution to providing users with access to the whole database without breaking our API and without generating a data dump in parquet or some other format.

Catalog
- Provider script refactors to use the ProviderDataIngester class.
- Add loading of the image dimensions to the script (using additional requests).
- Change the structure of the upstream database and clean up the available data.
- Zack Krida 1:00 pm on September 7, 2022
  
  This idea actually made me think that we could create ‘/all’ endpoints that return all media results available in the APIAPI An API or Application Programming Interface is a software intermediary that allows programs to interact with each other and share data in limited, clearly defined ways. database, without passing through ElasticSearch.
  
  Really interesting idea. How would that work? That seems like an impossible amount of data to process over an httpHTTP HTTP is an acronym for Hyper Text Transfer Protocol. HTTP is the underlying protocol used by the World Wide Web and this protocol defines how messages are formatted and transmitted, and what actions Web servers and browsers should take in response to various commands. endpoint. Or would that endpoint be paginated?

Welcome!

Frontend

API

Catalog