Dhruv Bhanushali 12:16 am on April 3, 2023
Tags: openverse ( 55 ), week-in-openverse ( 46 )

A week in Openverse: 2023-03-27 – 2023-04-03

openverse

Merged PRs

#1113: Bump ipython from 8.11.0 to 8.12.0 in /api
#1104: Pass actor for staging deploys with the `-f` flag
#1103: Add `GITHUB_TOKEN` to GitHubGitHub GitHub is a website that offers online implementation of git repositories that can easily be shared, copied and modified by other developers. Public repositories are free to host, private repositories require a paid subscription. GitHub introduced the concept of the ‘pull request’ where code changes done in branches by contributors can be reviewed and discussed before being merged be the repository owner. https://github.com/ CLICLI Command Line Interface. Terminal (Bash) in Mac, Command Prompt in Windows, or WP-CLI for WordPress. step
#1098: Update other references of media count to 700 million
#1067: Fix typo in docs building on `main`
#1065: Restore Django Admin views
#1063: Use label.yml to determine required labels
#1058: Fix issues in the workflow simplifications of #1054
#1054: Simplify CI + CD workflow
#1051: Pin pnpm version in frontend `Dockerfile`
#1048: Bump boto3 from 1.26.84 to 1.26.100 in /ingestion_server
#1047: New issue automation fix: convert the jobs into steps to share env variables
#1044: Bump bottle from 0.12.24 to 0.12.25 in /ingestion_server
#1042: Bump boto3 from 1.26.97 to 1.26.99 in /api
#1041: Bump filelock from 3.9.0 to 3.10.7 in /ingestion_server
#1040: Bump pytest-order from 1.0.1 to 1.1.0 in /ingestion_server
#1039: Bump aws-actions/configure-aws-credentials from 1 to 2
#1038: Use `ACCESS_TOKEN` for the Project automation
#1034: Dispatch workflows instead of regular reuse to show deployment runs
#1031: Use the `issue.node_id` for GraphQL APIAPI An API or Application Programming Interface is a software intermediary that allows programs to interact with each other and share data in limited, clearly defined ways.
#1029: Defer the `tags_list` for media models
#1028: Absorb `actionlint` into pre-commit
#1026: Add stack label to dependabot & sync label PRs
#1007: Fix Re-running failed Playwright tests leaves failure comment
#990: Retry `up` recipe in case port is occupied
#904: Save cleaned up data during the cleanup step

Closed issues

#1099: Can't run linting locally
#1064: SlackSlack Slack is a Collaborative Group Chat Platform https://slack.com/. The WordPress community has its own Slack Channel at https://make.wordpress.org/chat/. notification when CI/CD fails on main
#1061: httpHTTP HTTP is an acronym for Hyper Text Transfer Protocol. HTTP is the underlying protocol used by the World Wide Web and this protocol defines how messages are formatted and transmitted, and what actions Web servers and browsers should take in response to various commands. in api response
#1033: Deployment workflow runs do not show in workflow run history
#999: Links in CONTRIBUTING.md are not working
#950: Configure TS to treat project as composite
#902: Phylopic images are broken
#896: Absorb `actionlint` into pre-commit
#884: Use label.yml to determine required labels
#864: Re-running failed Playwright tests leaves failure comment
#861: Add additional logging to the cleanup process
#654: Add a list of domains that are known to support TLS to the cleanup step
#337: Configure isort to include source paths
#734: Incorrectly formatted OAuth requests cause uncaught errors
#634: Investigate alternatives to Google Analytics

openverse-catalog

Merged PRs

#1074: Create DAG to fix PhyloPic's `foreign_identifier` column
#1072: Offset iNaturalist DAG from monthly by one day
#1071: 🔄 synced file(s) with WordPress/openverseOpenverse Openverse is a search engine for openly-licensed media, including photos, audio, and video. Openverse is also the name for the collection of related code repositories that make up the project. find Openverse at https://openverse.org.
#1070: Update pgcli version to 3.5.0
#1069: 🔄 synced file(s) with WordPress/openverse
#1068: Update Freesound to quarterly, extend timeout
#1067: 🔄 synced file(s) with WordPress/openverse
#1064: Bump apacheApache Apache is the most widely used web server software. Developed and maintained by Apache Software Foundation. Apache is an Open Source software available for free.-airflow[amazon,http,postgres] from 2.5.1 to 2.5.2
#1060: Update PhyloPic DAG to use API v2
#1058: Log last query_params hit before AirflowTaskTimeout
#1054: Add isort configuration file
#1047: Update Flickr large batch handling

Closed issues

#1073: Update PhyloPic's `foreign_identifier` field
#1025: Delay iNaturalist from `@monthly`
#998: Update Phylopic to use v2 API

openverse-infrastructure

Merged PRs

#449: Update actor input for staging
#447: Accept actor as input for dispatch and call
#441: Add SENTRY_DSN to ECS API
#435: Update env vars to fix URLURL A specific web address of a website or web page on the Internet, such as a website’s URL www.wordpress.org scheme for related endpoints
#433: Bump catalog-airflow to v1.5.2
#432: 🔄 synced file(s) with WordPress/openverse
#431: Point staging frontend to staging API
#430: 🔄 synced file(s) with WordPress/openverse
#429: Add documentation for running the staging data refresh
#427: Remove unnecessary branch checks
#423: Remove openverse-api modules from legacy environments

Closed issues

#439: Add Sentry DSN to API ECS configuration
#393: Decommission legacy API module
#390: Decommission `api-dev.openverse.engineering`
#317: Update the list of modules

#openverse, #week-in-openverse

Olga Bulat 3:43 pm on March 30, 2023
Tags: openverse-weekly-community-meeting ( 49 )

Community Meeting Recap (28 March 2023)

[Meeting start]

This post only summarizes the discussion of the agenda items. To keep up with the progress of OpenverseOpenverse Openverse is a search engine for openly-licensed media, including photos, audio, and video. Openverse is also the name for the collection of related code repositories that make up the project. find Openverse at https://openverse.org., you can check out the “Week in Openverse” posts.

📒 Agenda

We announced the release of WordPress 6.2 “Dolphy” on March 29, 2023, with built-in Openverse browsing in the “media” tab of the blockBlock Block is the abstract term used to describe units of markup that, composed together, form the content or layout of a webpage using the WordPress editor. The idea combines concepts of what in the past may have achieved with shortcodes, custom HTML, and embed discovery into a single consistent API and user experience. inserter.

The second item on the agenda was whether we should require branches to be up-to-date before merging (the related discussion is in this issue)

A concern was voiced by @sarayourfriend and @aetherunbound that merging PRs would require a lot of wait time if several people are trying to do so at once. It seems like that case would be irrelevant if we also added merge queues. The queue would automatically handle rebasing and ensuring CI/CD passes before merging the code, meaning maintainers would not need to babysit merges if having an updated copy was required. The teams agrees on adding support adding required update if merge queues also accompanied it!

And lastly, a great comment by @sarayourfriend in the Content safety flow design might expand the ticket scope. All your thoughts are welcome.

[Meeting end]

#openverse-weekly-community-meeting

Zack Krida 2:45 pm on March 29, 2023

Openverse Monthly Priorities Meeting 2023-04-05

OpenverseOpenverse Openverse is a search engine for openly-licensed media, including photos, audio, and video. Openverse is also the name for the collection of related code repositories that make up the project. find Openverse at https://openverse.org. contributors will host a community meeting to discuss priorities for April at 1500 UTC on 2023-04-05.

A sync video chat link will be provided in the #openverse channel of the Making WordPress Chat. We hope to see you there.

You can read the ongoing notes document for these meetings here.

Dhruv Bhanushali 12:17 am on March 27, 2023
Tags: openverse ( 55 ), week-in-openverse ( 46 )

A week in Openverse: 2023-03-20 – 2023-03-27

openverse

Merged PRs

#1014: Pass ISSUE_ID and PROJECT_ID to the new_issue workflow
#1011: Add release-drafter APIAPI An API or Application Programming Interface is a software intermediary that allows programs to interact with each other and share data in limited, clearly defined ways. configuration to enable testing in #987
#1006: Add CNAME in other use of `actions-gh-pages`
#1005: Add docs CNAME to config
#1000: Fix diagrams with transparent background in README.md of "ingestion_server" for dark mode.
#994: Fix local build of the API and add its `recreate` just command
#991: Update URLs to point to docs.openverse.org
#987: Add tag app release action
#986: Remove XML from the API
#984: Add GH_TOKEN to the gh steps
#981: Switch to internal headerHeader The header of your site is typically the first thing people will experience. The masthead or header art located across the top of your page is part of the look and feel of your website. It can influence a visitor’s opinion about your content and you/ your organization’s brand. It may also look different on different screen sizes. on single results
#980: Change post status to 'publish'
#979: Add decoding of the strings that don't have backslashes
#977: Skip build and publish job if nothing to do
#975: Fix needs check on api staging deployment
#974: Improve documentation for partial stack setups
#973: Use formless default browsable API renderer
#969: make init script more system agnostic
#967: Update general setup guide for macOS
#965: Configure ingestion_server as a known first party for isort
#963: Swap from flake8 to ruff
#962: Update to new link for pipenv install instructions
#961: Update opensearch.xml to fix bad url
#959: Fix link to dev flow docs
#955: Make `searchTerm` for VAudioTrack and VImageCell optional
#952: Update Nuxt to v.2.16.3
#951: Update TS configuration to use composite projects and fix VSCode integration
#945: Log DB queries in production
#944: Absorb `build-nginxNGINX NGINX is open source software for web serving, reverse proxying, caching, load balancing, media streaming, and more. It started out as a web server designed for maximum performance and stability. In addition to its HTTP server capabilities, NGINX can also function as a proxy server for email (IMAP, POP3, and SMTP) and a reverse proxy and load balancer for HTTP, TCP, and UDP servers. https://www.nginx.com/.` job into `build-images` job
#943: Make Plausible setup idempotent

Closed issues

#1012: new_issue workflow is failing to add the issue to the new project
#993: Can't spin up the API locally
#983: The workflow for new project automation needs a GitHubGitHub GitHub is a website that offers online implementation of git repositories that can easily be shared, copied and modified by other developers. Public repositories are free to host, private repositories require a paid subscription. GitHub introduced the concept of the ‘pull request’ where code changes done in branches by contributors can be reviewed and discussed before being merged be the repository owner. https://github.com/ token
#982: Remove XML support from Django API
#976: Cutting a release does not successfully run CI/CD workflow
#972: Dump Django URLURL A specific web address of a website or web page on the Internet, such as a website’s URL www.wordpress.org resolver configuration and confirm all routes are expected API routes
#971: Staging API does not automatically deployDeploy Launching code from a local development environment to the production web server, so that it's available to visitors. after merge to main
#970: Reporting HTMLHTML HTML is an acronym for Hyper Text Markup Language. It is a markup language that is used in the development of web pages and websites. view `SELECT`s all media records
#968: [Improvement] Diagrams with transparent background are not great in dark mode
#966: General setup guide requires Homebrew, but has no info on installation
#960: timeout is required to successfully create the elastic search indexes using the just file
#958: Tags incorrectly escaped utf-8 characters to `uxxxx`
#953: Make `searchTerm` non-required for Audio track and Image cell
#942: Plausible DB setup is not idempotent
#923: Search on single result page does not work intuitively
#900: Dead thingiverse images are not filtered out because they return 403
#899: Include `collectstatic` step inside API Dockerfile
#892: Document how to setup just parts of the stack
#868: Add issues to the new Project and set Priority field value
#866: Use profiles in Docker Compose
#859: Consider JSON5 for `package.jsonJSON JSON, or JavaScript Object Notation, is a minimal, readable format for structuring data. It is used primarily to transmit data between a server and web application, as an alternative to XML.` files
#388: API ECS MigrationMigration Moving the code, database and media files for a website site from one server to another. Most typically done when changing hosting companies.
#463: Single result page should use header with navigation links
#478: Optimize CI pipeline avoiding running jobs for unrelated changes
#482: FilterFilter Filters are one of the two types of Hooks https://codex.wordpress.org/Plugin_API/Hooks. They provide a way for functions to modify data of other functions. They are the counterpart to Actions. Unlike Actions, filters are meant to work in an isolated manner, and should never have side effects such as affecting global variables and output. counter in button and tab
#675: Use `thumbnail_url` for thumbnail generation when present
#755: Build UIUI UI is an acronym for User Interface - the layout of the page the user interacts with. Think ‘how are they doing that’ and less about what they are doing. for API consumers to get their key and check their usage (original #335)

openverse-catalog

Merged PRs

#1065: Bump pre-commit from 3.1.1 to 3.2.0
#1063: Add required stack label to dependabot PRs
#1057: 🔄 synced file(s) with WordPress/openverseOpenverse Openverse is a search engine for openly-licensed media, including photos, audio, and video. Openverse is also the name for the collection of related code repositories that make up the project. find Openverse at https://openverse.org.
#1052: Update README.md with documentation reference
#1049: Handle the upper case licenses in the add_license_dag
#1048: Remove watermarked setting for SMK
#1040: Add SuggestedSubProvider type
#1011: Add option to skip specific ingestion errors

Closed issues

#1020: Revert wartermark value change for SMK after files are deleted
#702: Add configuration to skip specific ingestion errors
#394: Add old repo to documentation

openverse-infrastructure

Merged PRs

#428: 🔄 synced file(s) with WordPress/openverse
#426: Add monorepo required checks
#424: 🔄 synced file(s) with WordPress/openverse
#422: Reduce log levels in the API and disable DB query logging
#419: Send alerts to alerts channel
#405: Upgrade cloudflare ssl mode to strict for all managed zones
#403: Update ES node metadata & Cloudwatch dashboard for newly provisioned node

Closed issues

#425: Update required checks for the monorepo

#openverse, #week-in-openverse

Dhruv Bhanushali 12:18 am on March 20, 2023
Tags: openverse ( 55 ), week-in-openverse ( 46 )

A week in Openverse: 2023-03-13 – 2023-03-20

openverse

Merged PRs

#939: Add console_prod handler to query logging to allow in production
#936: Always build both api & ingestion server images for either service
#935: Deregister media model admins and dependents
#934: Add Django DB logging option
#933: Add application name to DB
#931: Remove Docker image loading from docs steps
#930: Fix links on the main Storybook page
#927: Fix global audio player's close button
#925: Build `api` when ingestion server changes
#922: Add `.githubGitHub GitHub is a website that offers online implementation of git repositories that can easily be shared, copied and modified by other developers. Public repositories are free to host, private repositories require a paid subscription. GitHub introduced the concept of the ‘pull request’ where code changes done in branches by contributors can be reviewed and discussed before being merged be the repository owner. https://github.com/` to CODEOWNERS
#918: Fix global audio player layout
#917: Update pinia and pinia/testing
#916: Update VueVue Vue (pronounced /vjuː/, like view) is a progressive framework for building user interfaces. https://vuejs.org/. from 2.7.10 to 2.7.14
#915: Fix background color on report pages
#910: Add user validation, concurrency, manual runs to deployment workflow
#909: Add get-image-tag as dependency for nginxNGINX NGINX is open source software for web serving, reverse proxying, caching, load balancing, media streaming, and more. It started out as a web server designed for maximum performance and stability. In addition to its HTTP server capabilities, NGINX can also function as a proxy server for email (IMAP, POP3, and SMTP) and a reverse proxy and load balancer for HTTP, TCP, and UDP servers. https://www.nginx.com/. build step
#895: Skip more jobs based on changed files
#894: Simplify and fix bundle size workflow
#893: Only generate POT file if `en.json5` has changed
#891: Add ability to boost search results by authority
#889: Prepare Docker setup for monorepo
#888: Adding brand assets
#886: Split deployment workflow into 4 separate workflows
#882: Only run stack label addition step on pull requests
#873: Project Proposal: Detecting, filtering, and blurring results that include sensitive terms
#844: Implement analytics in Nuxt
#828: Move peerDependencyRules to root package.jsonJSON JSON, or JavaScript Object Notation, is a minimal, readable format for structuring data. It is used primarily to transmit data between a server and web application, as an alternative to XML.
#397: Provider tally extraction script

Closed issues

#929: The links in Storybook have not been updated to monorepo
#928: Frontend PRs fail CI
#926: Global audio player cannot be closed when the audio is playing
#921: Action Required: Fix Renovate Configuration
#920: Django check in CI is flakey because of plausible check
#913: Global audio player is broken
#908: `SEMANTIC_VERSION` is not supplied to nginx image
#906: Port conflict with SlackSlack Slack is a Collaborative Group Chat Platform https://slack.com/. The WordPress community has its own Slack Channel at https://make.wordpress.org/chat/.
#879: Yellow background when reporting an image from GutenbergGutenberg The Gutenberg project is the new Editor Interface for WordPress. The editor improves the process and experience of creating new content, making writing rich content much simpler. It uses ‘blocks’ to add richness rather than shortcodes, custom HTML etc. https://wordpress.org/gutenberg/
#878: Update reverse proxy to allow for path prefix rewriting on the APIAPI An API or Application Programming Interface is a software intermediary that allows programs to interact with each other and share data in limited, clearly defined ways.
#877: Refactor deployment workflow into separate workflows per app and environment
#871: Jamendo thumbnails are failing
#865: Move Docker-only directories from root to `docker/`
#849: Skip frontend docker image build and its tests on non-frontend code changes
#827: Move pnpm peerDependencyRules.allowedVersions to the root package.json
#825: Set up wrangling for events
#380: Initial analysis of Redis provider tallies pre & post iNaturalist ingestion
#689: Add additional logging around search_controller's ES query building

openverse-catalog

Merged PRs

#1051: Adjust schedule for long running queries termination
#1050: Add DAG for terminating long-running queries
#1045: Use Python to group items by license to speed up the query
#1003: Remove alternate image extraction from SMK, fix foreign landing URLURL A specific web address of a website or web page on the Internet, such as a website’s URL www.wordpress.org

Closed issues

#1044: `add_license_url` DAG is inefficient and fails due to timeout
#1043: The Noun Project
#1039: Allow Flickr backfill to complete, turn notifications back on
#875: Duplicates identified in SMK data
#826: Provider: The Noun Project

openverse-infrastructure

Merged PRs

#420: secure staging api admin
#418: Add db logging and debug log level to production api
#417: Add api-production subdomain to access
#415: Add user validation, concurrency, manual runs to deployment workflow
#414: Add existing API aliases to ECS deployment
#413: Restore frontend capacity
#412: Add separate deployment workflows per environment/service
#411: Add photon auth key to ECS deployment
#401: Make desired count configurable, set to 5 in production

Closed issues

#399: Increase API ECS service count to match current EC2 production
#392: Point `api.openverse.engineering` to `api-production.openverse.engineering`
#366: Move staging ECS API to staging.openverse.org/api path route instead of openverse.engineering subdomain.

#openverse, #week-in-openverse

stacimc 2:11 am on March 15, 2023
Tags: openverse-weekly-community-meeting ( 49 )

Community Meeting Recap (14 March 2023)

[Meeting start]

📒 Agenda

This week’s main agenda item was a discussion of this PR proposal to automatically release to staging on version cuts in the monorepo. This prompted a larger discussion [ref] about whether manual releases are necessary moving forward, and raised questions about our versioning. We considered tagging published images with the version from package.json, in order to deployDeploy Launching code from a local development environment to the production web server, so that it's available to visitors. without cutting a release [ref]. Additional thoughts are welcome in the thread.

We also discussed a few issues which need to be assigned. This Jamendo thumbnail issue has since been closed, but we are still looking to assign the following:

ReadTimeout raised from general exception catching (high priority)
Translation strings partials should be linked with the whole sentence (medium priority)

We posted a reminder about the following RFCs which are open for feedback:

Project Proposal: Document all media properties (overdue by 10 March)
Project Proposal: Detecting, filtering, and blurring results that include sensitive terms (by 24 March)
Implementation plan: Managing a sensitive terms list (by 27 March 2023)
Project Proposal: Core UI improvement (by 28 March)

Finally, we announced that PyCascades Sprint Support is on Monday, 20 March with virtual sprints from 9am PDT – 5pm PDT [ref].

[Meeting end]

#openverse-weekly-community-meeting

Dhruv Bhanushali 12:13 am on March 14, 2023
Tags: openverse ( 55 ), week-in-openverse ( 46 )

A week in Openverse: 2023-03-06 – 2023-03-13

openverse

Merged PRs

#890: Add a stemming override for the word “universe”
#885: Add stack to label sync, allow emoji to be defined for whole group
#872: Make deployment action “uses” explicit
#870: Update sentry; fix config
#863: Fix weekly update workflow
#862: Add feature flag for fake marking results as sensitive
#858: Remove `prepare` script to prevent i18n overwrites inside Docker
#851: Make codeowners more specific
#848: Identify and fix cause of cURL error 23 when setting up pre-commit
#846: Bump boto3 from 1.26.81 to 1.26.84 in /ingestion_server
#843: Add preferences for analytics
#842: Update homepage copy to “700 million”
#841: Bump boto3 from 1.26.81 to 1.26.84 in /api
#840: Add production APIAPI An API or Application Programming Interface is a software intermediary that allows programs to interact with each other and share data in limited, clearly defined ways. deployment action
#838: Bump elasticsearch-dsl from 7.4.0 to 7.4.1 in /api
#836: Bump python-decouple from 3.7 to 3.8 in /api
#835: Bump python-decouple from 3.7 to 3.8 in /ingestion_server
#832: Bump elasticsearch-dsl from 7.4.0 to 7.4.1 in /ingestion_server
#831: Bump pytest from 7.2.1 to 7.2.2 in /ingestion_server
#830: Bump renovatebot/githubGitHub GitHub is a website that offers online implementation of git repositories that can easily be shared, copied and modified by other developers. Public repositories are free to host, private repositories require a paid subscription. GitHub introduced the concept of the ‘pull request’ where code changes done in branches by contributors can be reviewed and discussed before being merged be the repository owner. https://github.com/-action from 34.152.5 to 34.154.4
#806: Fix crash when more than one `q` parameter is provided in URLURL A specific web address of a website or web page on the Internet, such as a website’s URL www.wordpress.org
#804: RFC + POC: Add Plausible for analytics
#798: Handle incorrect types in cookie value
#788: Update home link screen reader text
#786: Add stack label if available, make get-changes composite action
#785: Add actions to search forms

Closed issues

#871: Jamendo thumbnails are failing
#857: Locales missing in Docker images
#854: Add non-production feature flag for marking half of results as sensitive
#852: `TypeError` term.trim is not a function
#850: PR review requests are not following the CODEOWNERS assignements
#845: Handle `precommit` recipe exiting with code 23
#839: Update homepage copy to “700 million”
#821: Add feature flag for analytics
#782: Invalid cookie value causes an error
#781: `setSearchTerm` fails when `query.q` is an array
#760: Update “Week in OpenverseOpenverse Openverse is a search engine for openly-licensed media, including photos, audio, and video. Openverse is also the name for the collection of related code repositories that make up the project. find Openverse at https://openverse.org.” script to support monorepo
#521: `DataCloneError` raised on search in Safari
#522: Switch to `ENTRYPOINT` instead of `CMD` in our Dockerfile
#545: Dependency Dashboard

openverse-catalog

Merged PRs

#1042: Update `LICENSE` to match main repo
#1041: Tweak Flickr time division settings, add logs
#1038: Add trailing slash to Jamendo thumbnail URLs
#1037: 🔄 synced file(s) with WordPress/openverse
#1036: Temporarily turn off scheduled image data refreshes, increase matview refresh timeout
#1035: Add logging to iNaturalist date check
#1034: Add flickr sub provider auditing dag
#1031: Adjust Flickr max records to account for incorrect reporting
#1028: Improve license URL validation
#1005: Add a DAG for backfilling license_url when meta_data is null
#976: Add Airflow variable used to configure overrides for task timeouts

Closed issues

#1027: `_get_valid_cc_url` makes a network request even for known valid license urls
#1024: Improve iNaturalist date check logging
#724: Allow execution timeouts to be overridden by Variables
#676: Identify new Flickr sub-providers
#511: Ensure that all media have `license_url` in `meta_data` field

openverse-infrastructure

Merged PRs

#410: 🔄 synced file(s) with WordPress/openverse
#408: Increase frontend memory and CPU back up
#406: Bump API to v4.0.0, point bump script to monorepo
#402: Add API service to ECS Cloudwatch dashboards
#398: Stand up production API on ECS
#397: Construct API URLs dynamically, change staging domain
#386: Add stack check as required for monorepo
#381: Set frontend memory and cpu to match staging

Closed issues

#407: Update deployment action to generate a blockBlock Block is the abstract term used to describe units of markup that, composed together, form the content or layout of a webpage using the WordPress editor. The idea combines concepts of what in the past may have achieved with shortcodes, custom HTML, and embed discovery into a single consistent API and user experience. per service
#400: Set up API in Cloudwatch ECS dashboard
#391: Set up and deployDeploy Launching code from a local development environment to the production web server, so that it's available to visitors. production API on ECS

#openverse, #week-in-openverse

Zack Krida 5:10 pm on March 8, 2023

Priorities Meeting Recap 2023-03-08

OpenverseOpenverse Openverse is a search engine for openly-licensed media, including photos, audio, and video. Openverse is also the name for the collection of related code repositories that make up the project. find Openverse at https://openverse.org. contributors discussed priorities for March and April in this meeting. View the full notes here and a summary below.

Open Bug Fixes

We discussed a number of open bugs. We’ve been balancing a lot of project work with metaMeta Meta is a term that refers to the inside workings of a group. For us, this is the team that works on internal WordPress sites like WordCamp Central and Make WordPress. work (monorepo cleanup in particular) and some bugs have languished a bit.

Solution: We added assignment of these bugs to the community meeting agenda for next week. We also deprioritized some high-priority issues that went unaddressed since fall of 2022:

#684 Tests make outbound network requests to providers for link validation and thumbnail generation
#504 FilterFilter Filters are one of the two types of Hooks https://codex.wordpress.org/Plugin_API/Hooks. They provide a way for functions to modify data of other functions. They are the counterpart to Actions. Unlike Actions, filters are meant to work in an isolated manner, and should never have side effects such as affecting global variables and output. sidebarSidebar A sidebar in WordPress is referred to a widget-ready area used by WordPress themes to display information that is not a part of the main content. It is not always a vertical column on the side. It can be a horizontal rectangle below or above the content area, footer, header, or any where in the theme. is not synced correctly with page navigation causing page jittering when navigating between search and a single result
#831 Add configuration to aggregate ingestion errors over reingestion

Current Infrastructure Issues

We discussed some issues that have recently appeared, namely:

The bad data node on our production ES cluster in need of replacement
The issues with the data refresh after adding 155+ million images from iNaturalist
Ongoing work to migrate the APIAPI An API or Application Programming Interface is a software intermediary that allows programs to interact with each other and share data in limited, clearly defined ways. to ECS

Solution:

We will immediately pause all data refreshes.
We will mitigate the ES data node situation this week.
We will use this ‘pause’ in the Catalog as an opportunity to run some outstanding cleanup work, DAG enables, and to plan for how to re-enable the data refreshes.
We will re-prioritize and begin the Popularity calculation optimizations (Matview refresh) #433 project this month, with Staci as lead.

Search Relevancy Sandbox

Originally slated to begin this month, we’ve decided to defer this project to April given the infra issues previously mentioned. Madison will lead the project, and we’d like to expand the scope from the single PR of making the staging API contain a proportionate subset of images from each provider to also include investigating and provisioning a managed ElasticSearch cluster to use in staging, but ultimately migrate to in production.

Solution:

Defer to April
Madison will lead
Expand scope to include a manged ES cluster for staging

Other

We remain confident in our ability to close out the Provider DAG stability milestone this month
We will pause design work on the Filter and blur sensitive results by term matching #377 project after getting feedback on the latest design iteration. This is to prevent additional assumptions from creeping into the work before the project is sufficiently planned.
Regarding the consent-based desicionmaking proposal shared earlier in February, we are going to wait for a final document to be shared for final iteration and feedback before implementing. Until then, our project process contains some guidelines on communication timelines.

Krystle Salazar 9:05 pm on March 7, 2023
Tags: openverse-weekly-community-meeting ( 49 )

Community Meeting Recap (7 March 2023)

[Meeting start]

📒 Agenda

After a long discussion of Done, In Process and To Do issues, one last-minute agenda item is raised: When should we update the community meeting documentation to use the new project and issue boards? @olgabulat suggested completing the automations for it first, which are compiled in the metaMeta Meta is a term that refers to the inside workings of a group. For us, this is the team that works on internal WordPress sites like WordCamp Central and Make WordPress. issue #818 Improve Openverse Project board. We added it to the To Do items so is up to grab for anyone during the week.

[Meeting end]

#openverse-weekly-community-meeting

Zack Krida 7:06 pm on March 6, 2023
Tags: make.wordpress.org/polyglots, p2-xpost ( 8 )

X-post: Updates to language support in Openverse

X-post from +make.wordpress.org/polyglots: Updates to language support in Openverse

Welcome!

Merged PRs

Closed issues

Merged PRs

Closed issues

Merged PRs

Closed issues

📒 Agenda

Merged PRs

Closed issues

Merged PRs

Closed issues

Merged PRs

Closed issues

Merged PRs

Closed issues

Merged PRs

Closed issues

Merged PRs

Closed issues

📒 Agenda

Merged PRs

Closed issues

Merged PRs

Closed issues

Merged PRs

Closed issues

Open Bug Fixes

Current Infrastructure Issues

Search Relevancy Sandbox

Other

📒 Agenda