A week in Openverse: 2024-02-05 – 2024-02-12

openverse

Merged PRs

  • #3779: Revert Add SEARCH_RESPONSE_TIME analytics event to searches (#3632)
  • #3773: Prevent running certain workflows on forks
  • #3767: Fixed Typo in Frontend Icons.md file
  • #3765: Publish changelog for ingestion_server-2024.02.07.18.41.16
  • #3762: Remove codeowners validator from pre-commit
  • #3759: Fix flaky Playwright tests
  • #3756: Fix incorrect key `pass_filesnames`
  • #3754: Bump cryptography from 41.0.6 to 42.0.0 in /automations/python
  • #3752: Publish changelog for api-2024.02.05.20.34.01
  • #3751: Publish changelog for frontend-2024.02.05.18.10.41
  • #3738: Update dependency @playwright/test to v1.41.2
  • #3737: Update @openverseOpenverse Openverse is a search engine for openly-licensed media, including images and audio. Find Openverse on GitHub and at https://openverse.org./eslint-pluginPlugin A plugin is a piece of software containing a group of functions that can be added to a WordPress website. They can extend functionality or add new features to your WordPress websites. WordPress plugins are written in the PHP programming language and integrate seamlessly with WordPress. These can be free in the WordPress.org Plugin Directory https://wordpress.org/plugins/ or can be cost-based plugin from a third-party
  • #3734: Update dependency @types/node to v18.19.14
  • #3733: Replace dependency npm-run-all with npm-run-all2 ^6.1.2 (edited by @obulat)
  • #3726: Raise an exception if any indexer workers fail their health check
  • #3632: Add SEARCH_RESPONSE_TIME analytics event to searches
  • #3407: Add source and provider definition tooltips

Closed issues

  • #3775: `SEARCH_TIME_EVENT` is unusable due to plausible and CORS limitations
  • #3766: Small Typo Correction Documentation File
  • #3744: TypeError: 'NoneType' object is not subscriptable
  • #3714: `visual-regression/v-headerHeader The header of your site is typically the first thing people will experience. The masthead or header art located across the top of your page is part of the look and feel of your website. It can influence a visitor’s opinion about your content and you/ your organization’s brand. It may also look different on different screen sizes.-internal.spec.ts:36:11 › VHeaderInternal › screen at breakpoint xs with width 340 › mobile-header-internal-modal-ltr` is flaky
  • #3713: `visual-regression/v-filterFilter Filters are one of the two types of Hooks https://codex.wordpress.org/Plugin_API/Hooks. They provide a way for functions to modify data of other functions. They are the counterpart to Actions. Unlike Actions, filters are meant to work in an isolated manner, and should never have side effects such as affecting global variables and output.-button.spec.ts:82:11 › VFilterButton › screen at breakpoint xl with width 1280 › pressed, focused, 12 filters` is flaky
  • #3712: APIAPI An API or Application Programming Interface is a software intermediary that allows programs to interact with each other and share data in limited, clearly defined ways. displays fewer (if any at all) results on the source collection page for Europeana than it does when filtering by `source=europeana`
  • #3693: Prevent workflows specific to this repo and its management from running on forks
  • #3554: Add linting for `CODEOWNERS`
  • #2840: Source and provider definitions in single result page
  • #2775: Update the single result page
  • #2708: Ingestion server does not appropriately report when indexer workers cannot be reached
  • #523: `window.webkit.messageHandlers` access errors prior to Sentry initialization
  • #1778: Identify fields from which the license can be obtained for certain Smithsonian museums (original #472)

openverse-infrastructure

Merged PRs

  • #798: 🔄 synced file(s) with WordPress/openverse
  • #797: Bump ingestion-server to rel-2024.02.07.18.41.16
  • #795: 🔄 synced file(s) with WordPress/openverse
  • #791: 🔄 synced file(s) with WordPress/openverse
  • #790: Update the Nuxt's `request_count_anomaly` alarm
  • #788: Allow GitHubGitHub GitHub is a website that offers online implementation of git repositories that can easily be shared, copied and modified by other developers. Public repositories are free to host, private repositories require a paid subscription. GitHub introduced the concept of the ‘pull request’ where code changes done in branches by contributors can be reviewed and discussed before being merged be the repository owner. https://github.com/ groups in CF Access, add content moderators to API access
  • #589: Initial setup of the Sentry provider

Closed issues

  • #748: Add `WordPress/openverse-content-moderators` to Django admin Cloudflare Access
  • #567: Better task count metric
  • #284: Silence ES CPU usage > 20% alert during data refreshes and add a separate, temporary 70% > CPU usage alert
  • #215: Manage Airflow Variables using Terraform (or some other programmatically handled and auditable way)
  • #153: Create a "deployment-permissions" singleton module to manage policy attachments for github AWS user
  • #96: Add cloudflare page cache rules for API and frontend

#openverse, #week-in-openverse

Openverse Monthly Priorities Meeting 2024-02-07

OpenverseOpenverse Openverse is a search engine for openly-licensed media, including images and audio. Find Openverse on GitHub and at https://openverse.org. contributors will host a community meeting to discuss priorities for February at 1500 UTC on February 7th, 2024.

A sync video chat link will be provided in the #openverse channel of the Making WordPress Chat. We hope to see you there!

You can read the ongoing notes document for these meetings here.

#openverse-priorities, #priorities

A week in Openverse: 2024-01-29 – 2024-02-05

openverse

Merged PRs

  • #3740: Update dependency pook to v1.4.0
  • #3739: Update dependency elasticsearch to v8.12.0
  • #3736: Update workflows
  • #3735: Update docker.io/redis Docker tag to v7.2.4
  • #3732: Add a link to the GitHubGitHub GitHub is a website that offers online implementation of git repositories that can easily be shared, copied and modified by other developers. Public repositories are free to host, private repositories require a paid subscription. GitHub introduced the concept of the ‘pull request’ where code changes done in branches by contributors can be reviewed and discussed before being merged be the repository owner. https://github.com/ file history on the Terms of Service
  • #3730: Remove preceding `$` in documentation code blocks, use bash for code blocks, add pre-commit check for console usage
  • #3728: Add storybook snapshot update command to testing documentation
  • #3727: Add the ToS to the navigation bar
  • #3724: Use new favicon
  • #3720: Bump aiohttp from 3.9.1 to 3.9.2 in /api
  • #3718: Publish changelog for frontend-2024.01.29.19.41.17
  • #3717: Publish changelog for api-2024.01.29.19.36.26
  • #3715: Prevent real network requests in TLS cleanup test
  • #3709: Remove sentry exception capture for thumbnail errors
  • #3688: Fix tag casing match in new tag view
  • #3650: Implementation Plan: Move Airflow and APIAPI An API or Application Programming Interface is a software intermediary that allows programs to interact with each other and share data in limited, clearly defined ways. to openverse.org
  • #3567: Add initial vale configuration and apply editorial changes to documentation

Closed issues

  • #3708: Remove Sentry exception handling for thumbnails
  • #3694: [Nuxt 3] Playwright Docker build fails on MacOS
  • #3666: [Nuxt 3] Re-add transition animation to the homepage images
  • #3665: [Nuxt 3] Re-add Prometheus module
  • #3664: [Nuxt 3] Re-add Storybook and Storybook tests
  • #3564: Add a terms of service page to the frontend
  • #3495: TagTag Tag is one of the pre-defined taxonomies in WordPress. Users can add tags to their WordPress posts along with categories. However, while a category may cover a broad range of topics, tags are smaller in scope and focused to specific topics. Think of them as keywords used for topics discussed in a particular post. casing is not matched in new tag view
  • #3566: Integrate Vale for linting docs according to editorial style guide
  • #2630: Remove the mature models from the API
  • #2629: Remove all code references to the mature-related models without removing the models themselves
  • #2628: Point the ingestion server to the new sensitive tables
  • #2627: Create a data management command to copy data from the old mature tables to the new sensitive ones
  • #2626: Add new models `Sensitive<Media>` and `<Media>Report`
  • #2038: Implementation Plan: Sufficiently change usage of `openverse.engineering` Cloudflare zone so that it can be switched to a free account
  • #903: Replace frontend favicon with yellow background version

openverse-infrastructure

Merged PRs

  • #767: 🔄 synced file(s) with WordPress/openverseOpenverse Openverse is a search engine for openly-licensed media, including images and audio. Find Openverse on GitHub and at https://openverse.org.
  • #766: Remove deprecated keys

Closed issues

  • #760: Add Nuxt 3 variables to frontend ECS template
  • #169: Add OATH-TOTP PAM module to all SSHSSH Secure SHell - a protocol for securely connecting to a remote system in addition to or in place of a password.-able infrastructure

#openverse, #week-in-openverse

X-post: Incident Response Team: Call for Nominations

X-comment from +make.wordpress.org/project: Comment on Incident Response Team: Call for Nominations

Community Meeting Recap (2024-01-29)

[Meeting start]

Agenda

  • Frontend code freeze lifted
    • The migrationMigration Moving the code, database and media files for a website site from one server to another. Most typically done when changing hosting companies. to Nuxt 3 has been put on hold due to memory leaks in upstream packages. We will wait for the ecosystem to mature a little before reattempting. The code freeze is lifted and as a result, 9 issues are moved into Todo and are now considered unblocked.
  • Does frontend terms of service need design work in order to be implemented?
    • We will add the updated Terms of Service using the current navbar design and Francisco will prioritise working on the updated navbar and footer designs that accommodate a link to the ToS more suitably.

[Meeting end]

#openverse-weekly-community-meeting

A week in Openverse: 2024-01-22 – 2024-01-29

openverse

Merged PRs

  • #3706: Publish changelog for catalog-2024.01.25.17.42.59
  • #3705: Use format over toJSON for escaping quotes in GHA
  • #3702: Do not use unsupported folded chomping multiline YAML blockBlock Block is the abstract term used to describe units of markup that, composed together, form the content or layout of a webpage using the WordPress editor. The idea combines concepts of what in the past may have achieved with shortcodes, custom HTML, and embed discovery into a single consistent API and user experience. syntax
  • #3701: Update dependency apacheApache Apache is the most widely used web server software. Developed and maintained by Apache Software Foundation. Apache is an Open Source software available for free.-airflow to v2.8.1 [SECURITY]
  • #3699: Wrap PR and discussion titles in `toJSON` to escape them in SlackSlack Slack is a Collaborative Group Chat Platform https://slack.com/. The WordPress community has its own Slack Channel at https://make.wordpress.org/chat/.
  • #3698: Enable the isort rules for ruff and fix linting issues
  • #3697: Use correct path when checking if DAGs.md needs to be regenerated
  • #3696: Update WordPress Photo Directory DAG to ingest weekly
  • #3692: Bump pillow from 10.1.0 to 10.2.0 in /utilities/project_planning
  • #3691: Update dependency Pillow to ~=10.2.0 [SECURITY]
  • #3690: Bump pillow from 10.0.1 to 10.2.0 in /utilities/provider_tallies
  • #3689: Revert "Toggle CloudWatch alarms actions during Data Refresh (#3652)"
  • #3687: Remove needs db and add patch
  • #3685: Bump jupyterlab from 4.0.8 to 4.0.11 in /utilities/project_planning
  • #3684: Bump notebook from 7.0.6 to 7.0.7 in /utilities/project_planning
  • #3680: Remove outdated Docker image deployment workflow and documentation
  • #3676: Combine write to file and STDOUT in one command using `tee`
  • #3669: Update Rawpixel image URLs for ingestion
  • #3596: Don't trigger PR limit reminder on PRs that are ignored
  • #3586: Make the terms of service apply to all services
  • #3568: Update copy changes implementation plan to avoid table rename
  • #3537: Add create_new_es_index DAGs

Closed issues

  • #3710: <Replace this with actual title>
  • #3695: Ruff failing to correctly sort imports
  • #3436: Skip `needs_db` and always map ES hits to DB results
  • #2492: Large SVGs can cause gunicorn workers to abort
  • #2372: Build ES index creation DAG
  • #2324: Implementation Plan – Nuxt 3 MigrationMigration Moving the code, database and media files for a website site from one server to another. Most typically done when changing hosting companies.

openverse-infrastructure

Merged PRs

  • #765: Add ssh-ed25519 key for Olga
  • #764: Add ssh-ed25519 key for Madison
  • #763: Bump catalog version
  • #762: 🔄 synced file(s) with WordPress/openverseOpenverse Openverse is a search engine for openly-licensed media, including images and audio. Find Openverse on GitHub and at https://openverse.org.
  • #751: Update stacimc key to ed25519
  • #750: Add playbook to sync ssh keys to all ec2 instances
  • #743: Configure ansible to route through the bastion

Closed issues

  • #707: Route Ansible SSHSSH Secure SHell - a protocol for securely connecting to a remote system in addition to or in place of a password. connections through the jumphost and disable public IPs on all infrastructure

#openverse, #week-in-openverse

Community Meeting Recap (2024-01-23)

[Meeting start]

Agenda

📢 Reminder: the frontend is still in a code freeze, but as soon as we get the Nuxt 3 PR merged, we can unfreeze it! :ice_cube:

[Meeting end]

#openverse-weekly-community-meeting

A week in Openverse: 2024-01-15 – 2024-01-22

openverse

Merged PRs

  • #3681: Add missing spaces in PR pingPing The act of sending a very small amount of data to an end point. Ping is used in computer science to illicit a response from a target server to test it’s connection. Ping is also a term used by Slack users to @ someone or send them a direct message (DM). Users might say something along the lines of “Ping me when the meeting starts.”
  • #3679: Add user-agent headerHeader The header of your site is typically the first thing people will experience. The masthead or header art located across the top of your page is part of the look and feel of your website. It can influence a visitor’s opinion about your content and you/ your organization’s brand. It may also look different on different screen sizes. for rawpixel
  • #3678: Publish changelog for ingestion_server-2024.01.18.18.31.08
  • #3677: Bump jupyter-lsp from 2.2.0 to 2.2.2 in /utilities/project_planning
  • #3674: Allow CI + CD workflow to deployDeploy Launching code from a local development environment to the production web server, so that it's available to visitors. from a different branch
  • #3672: Update alarms runbooks
  • #3671: Publish changelog for api-2024.01.16.17.28.14
  • #3668: Add Rawpixel to authority data as CURATED
  • #3667: Update tests for Nuxt 3 migrationMigration Moving the code, database and media files for a website site from one server to another. Most typically done when changing hosting companies.
  • #3662: fix: broken pipenv install link in general quickstart guide
  • #3656: Simplify load sample data
  • #3655: Bump jinja2 from 3.1.2 to 3.1.3 in /utilities/project_planning
  • #3653: Fix formatting of PR ping SlackSlack Slack is a Collaborative Group Chat Platform https://slack.com/. The WordPress community has its own Slack Channel at https://make.wordpress.org/chat/. messages
  • #3652: Toggle CloudWatch alarms actions during Data Refresh
  • #3649: Draft PRs fix for PR Board automation
  • #3629: Add documentation for yearly planning
  • #3618: Do not ingest Jamendo records with downloads disabled
  • #3590: Additional service checks for ingestion server health endpoint
  • #3570: Add codeowners pre-commit check
  • #3025: Prevent iNaturalist from running alongside any other DAGs

Closed issues

  • #3647: Disable alarm notifications during ES index creation
  • #3630: Fix link to `pipenv` installation instructions on general setup guide
  • #3530: Do not ingest Jamendo tracks that do not allow downloads
  • #3467: PRs that both have changes requested and are drafted should be in the drafted column of the PR board
  • #2505: Frontend response count alarms
  • #2504: Frontend response time alarms
  • #2501: General APIAPI An API or Application Programming Interface is a software intermediary that allows programs to interact with each other and share data in limited, clearly defined ways. response time alarms
  • #2344: ECS Alarms for anomalous behavior
  • #2019: Add additional checks to ingestion server healthcheck endpoint
  • #1276: Prevent iNaturalist from running alongside any other DAGs

openverse-infrastructure

Merged PRs

  • #759: 🔄 synced file(s) with WordPress/openverseOpenverse Openverse is a search engine for openly-licensed media, including images and audio. Find Openverse on GitHub and at https://openverse.org.
  • #757: Bump ingestion server to rel-2024.01.18.18.31.08
  • #754: 🔄 synced file(s) with WordPress/openverse
  • #753: Stabilize API Production alarms
  • #752: Stabilize and update Nuxt alarms settings
  • #749: 🔄 synced file(s) with WordPress/openverse
  • #736: Set up DNSDNS DNS is an acronym for Domain Name System - how you assign a human readable address to a website’s exact numeric coded location (ie. wordpress.org uses the actual IP address 198.143.164.252). records for ingestion server in staging

Closed issues

  • #14: Distinguish between dev & prod ingestion server DNS records

#openverse, #week-in-openverse

A week in Openverse: 2024-01-08 – 2024-01-15

openverse

Merged PRs

  • #3657: Fix PR limit reminders
  • #3654: Bump jinja2 from 3.1.2 to 3.1.3 in /documentation
  • #3651: Fix slackSlack Slack is a Collaborative Group Chat Platform https://slack.com/. The WordPress community has its own Slack Channel at https://make.wordpress.org/chat/. pingPing The act of sending a very small amount of data to an end point. Ping is used in computer science to illicit a response from a target server to test it’s connection. Ping is also a term used by Slack users to @ someone or send them a direct message (DM). Users might say something along the lines of “Ping me when the meeting starts.” action version reference
  • #3648: Add additional_query_params to provider DAG configuration
  • #3645: Publish changelog for api-2024.01.08.20.06.53
  • #3644: Increase ES throttling rate in Ingestion Server
  • #3643: Publish changelog for frontend-2024.01.08.19.12.18
  • #3628: pgcli version is extracted into a docker argument
  • #3627: Remove unused AWS variables
  • #3623: Add secret-key check
  • #3622: Convert unittest to pytest
  • #3594: Documentation for becoming a committer, reorganize contributing pages
  • #3577: Replaced `cURL` slack workflows with `slackapi`
  • #3258: Retrieve Auckland Museum Image Data

Closed issues

  • #3533: Add optional `additional_query_params` config to provider DAGs
  • #3461: Replace cURL to Slack with `slackapi/slack-githubGitHub GitHub is a website that offers online implementation of git repositories that can easily be shared, copied and modified by other developers. Public repositories are free to host, private repositories require a paid subscription. GitHub introduced the concept of the ‘pull request’ where code changes done in branches by contributors can be reviewed and discussed before being merged be the repository owner. https://github.com/-action`
  • #3456: Add a check on startup to ensure application is not using default key in live environments
  • #3425: Convert unittest.TestCase tests to pytest
  • #2959: Extract the pgcli version for the Dockerfile from the api Pipfile
  • #1771: Auckland Museum

openverse-infrastructure

Merged PRs

  • #745: Increase settings of Response Time alarms for the APIAPI An API or Application Programming Interface is a software intermediary that allows programs to interact with each other and share data in limited, clearly defined ways. and Nuxt

Closed issues

  • #729: Delete thumbnails log group

#openverse, #week-in-openverse

X-post: Call for Mentees & Mentors: Contributor Mentorship Program Cohort #2 (2024 Q1)

X-comment from +make.wordpress.org/community: Comment on Call for Mentees & Mentors: Contributor Mentorship Program Cohort #2 (2024 Q1)