Community meetings for 2023 cancelled – see you next year!

Our next two community meetings happen to land on two common holidays: Christmas (Dec 25th) and New Year’s Day (Jan 1st). For that reason we’ll be cancelling both meetings, and our next meeting time will be Monday, January 8th. See you all then, have a lovely end of 2023!

Holiday Food” by Travel Photographer/ CC0 1.0

#openverse-weekly-community-meeting

A week in Openverse: 2023-12-11 – 2023-12-18

openverse

Merged PRs

  • #3536: Revert "Remove mysqlMySQL MySQL is a relational database management system. A database is a structured collection of data where content, configuration and other options are stored. https://www.mysql.com/. from apt repository sources for catalog due to expired key (#3527)"
  • #3529: Remove outdated pre-monorepo info from `README.md`
  • #3527: Remove mysql from apt repository sources for catalog due to expired key
  • #3526: Allow more than one aspect label
  • #3524: Add Jamendo's audiodownload_allowed field to metadata
  • #3522: Update catalog pgcli version to 4.0.1
  • #3521: Rename catalog requirements files to match Renovate pattern
  • #3519: Pin `@actions/artifact` to version 1
  • #3518: Publish changelog for ingestion_server-2023.12.12.17.23.14
  • #3516: Fix source query caching
  • #3515: Fix incorrect default production data refresh limit of 0
  • #3512: Publish changelog for frontend-2023.12.11.19.44.28
  • #3511: Publish changelog for api-2023.12.11.19.20.45
  • #3509: Remove staging thumbnails deployment
  • #3506: Use `.rest` in Octokit calls
  • #3503: Point Airflow deployment information to updated deployment runbook
  • #3502: Update links to archived repositories in documentation
  • #3479: Add DAG for ccMixter
  • #3465: Add catalog to dependencies to update by Renovate
  • #3427: Update dependency vue-tsc to v1.8.25
  • #3361: Unify DAG creation/database cleaning fixtures for testing
  • #3069: Move Oembed endpoint validation onto the serializer
  • #2679: Incident Response Plan

Closed issues

  • #3491: Add Internet Archive Book Images source in Django admin
  • #3490: Backfill Internet Archive Book Provider source
  • #3489: Backfill Flickr invalid license fix
  • #3212: Upgrade Ruff (and other pre-commit hooksHooks In WordPress theme and development, hooks are functions that can be applied to an action or a Filter in WordPress. Actions are functions performed when a certain event occurs in WordPress. Filters allow you to modify certain functions. Arguments used to hook both filters and actions look the same.)
  • #3202: Remove line regarding moving the catalog into the repository from the README
  • #2961: Increase timeout while waiting for RDS database rename
  • #2622: Unify DAG creation/database cleaning fixtures for testing
  • #2063: Create an "outage protocol" runbook / "Incident Response Plan"
  • #2060: Postmortem improvements from "OpenverseOpenverse Openverse is a search engine for openly-licensed media, including images and audio. Find Openverse on GitHub and at https://openverse.org. degraded performance 2023-05-03"
  • #897: Fuse `label_new_pr` and `label_pr` workflows
  • #581: Dependency Dashboard
  • #1770: ccMixter

openverse-infrastructure

Merged PRs

  • #723: Remove the production thumbnails monitoring service
  • #722: Bump ingestion server to rel-2023.12.12.17.23.14
  • #721: 🔄 synced file(s) with WordPress/openverse
  • #720: Route all thumbnails traffic to general APIAPI An API or Application Programming Interface is a software intermediary that allows programs to interact with each other and share data in limited, clearly defined ways. service

#openverse, #week-in-openverse

X-post: Update on Matrix Migration: Pausing the Transition

X-comment from +make.wordpress.org/project: Comment on Update on Matrix Migration: Pausing the Transition

Introducing Enhanced Content Safety Features on Openverse

We’re excited to announce a significant update to OpenverseOpenverse Openverse is a search engine for openly-licensed media, including images and audio. Find Openverse on GitHub and at https://openverse.org. – the introduction of new features to filterFilter Filters are one of the two types of Hooks https://codex.wordpress.org/Plugin_API/Hooks. They provide a way for functions to modify data of other functions. They are the counterpart to Actions. Unlike Actions, filters are meant to work in an isolated manner, and should never have side effects such as affecting global variables and output. sensitive content. In addition to excluding known sensitive content, Openverse will now exclude media which contains sensitive textual content from search results by default. This helps ensure an accessible browsing experience for everyone, particularly young users and those in educational settings. This new text-based filtering feature aligns with our dedication to upholding WordPress’s Community Code of Conduct and creating a safe, inclusive, and accessible platform for all users.

Users can opt-in to view sensitive content through the “Sensitive results” and “Blur content” switches, easily accessible with the rest of Openverse’s search filters on both desktop and mobile devices. Sensitive content is now identified through automated detection of sensitive textual content, as well as through user reports. More information about the techniques used can be found on our website. These methods are only a starting point and we plan to implement continuous improvements and more robust filtering methods.

3 combined screenshots of Openverse on mobile. The first shows the filter sidebar open scrolled down to the safe browsing section to show the "sensitive results" and "blurred content" buttons. The second screenshot shows the results view for the search "bird" with some results blurred. The third screenshot shows the "sensitive content" warning screen when navigating to content.
New sensitive content controls

This update marks just the beginning of our journey towards enhancing content safety tools on Openverse. Your insights are invaluable in guiding our efforts to refine and improve our platform. Please share your thoughts by commenting on this announcement post or submitting an issue on our GitHub repository. Together, we will continue making Openverse a more inclusive and accessible search platform for everyone.

#sensitive-content

A week in Openverse: 2023-12-04 – 2023-12-11

openverse

Merged PRs

  • #3483: Use the GitHubGitHub GitHub is a website that offers online implementation of git repositories that can easily be shared, copied and modified by other developers. Public repositories are free to host, private repositories require a paid subscription. GitHub introduced the concept of the ‘pull request’ where code changes done in branches by contributors can be reviewed and discussed before being merged be the repository owner. https://github.com/ APIAPI An API or Application Programming Interface is a software intermediary that allows programs to interact with each other and share data in limited, clearly defined ways. to get the label's node ID
  • #3481: Add IDs for the maintainer attention labels
  • #3478: Update sample data to remove invalid licenses
  • #3477: Publish changelog for ingestion_server-2023.12.06.19.13.35
  • #3476: Fix path to test resources
  • #3474: Enforce braces for `if`-`else` and `switch`-`case`
  • #3466: Add project selection processing script
  • #3464: Bump jupyter-server from 2.10.1 to 2.11.2 in /utilities/project_planning
  • #3463: Pass `coreCore Core is the set of software required to run WordPress. The Core Development Team builds WordPress.` to the labelling job as well
  • #3462: Make punctuation marks consistent for singular & plural strings
  • #3455: Fix image proxy get error handling
  • #3453: Extend Europeana timeout to 1 week
  • #3452: Publish changelog for api-2023.12.04.19.51.23
  • #3451: Publish changelog for frontend-2023.12.04.19.35.53
  • #3443: Make assorted improvements to automations and workflows
  • #3442: Fix incorrect variable mapping for PR util
  • #3441: Add Internet Archive Book Images subprovider
  • #3440: Fix incorrectly parsed license url by-nc-nd for Flickr
  • #3439: Encode spaces in urls for SMK
  • #3438: Print diffs for new documentation directories
  • #3428: Update workflows
  • #3424: Update dependency @types/node to v17.0.45
  • #3422: Change Django secret key to make it more obvious it's a shim value
  • #3420: Add or update runbooks for alarms of anomalous behavior
  • #3406: Replace popper.js with floating-ui
  • #3232: Add DAG for creating staging indices
  • #3140: Add additional search view pages to the Nuxt app

Closed issues

  • #3498: Build the proportional-by-provider ES index creation DAG
  • #3454: Async `image_proxy.get` function uses the wrong error types
  • #3450: Extend Europeana timeout
  • #3386: Add ESLint rules to mandate braces around `if`-`else` and `switch`-`case`
  • #3377: Translation: Differences in strings between singular / plural: ending / not ending on period
  • #3353: Production thumbnails memory leak
  • #3326: PR documentation preview doesn't handle entirely new folders
  • #3276: Add `z` permissions option to compose mounts to accommodate SELinux hosts
  • #3246: Build the ES full index recreation DAG
  • #3197: API response time assessment and reduction
  • #3121: Add new Flickr subprovider: Internet Archive Book Images
  • #3115: SSLSSL Secure Socket Layer - Encryption from the server to the browser and back. Prevents prying eyes from seeing what you are sending between your browser and the server. error with uvloop and already-closed transport
  • #2858: Update the Nuxt store to allow for searches by source/creator/tag
  • #2774: Create a page for tag/creator/source views
  • #2701: Timeout related errors in `/v1/images/` endpoint
  • #1090: "Add stack label" CI/CD step fails on PRs from forks
  • #655: Thumbnail request raises "OSError: [Errno 107] Transport endpoint is not connected"
  • #506: Update from popper.js to floating-ui

openverse-infrastructure

Merged PRs

  • #718: 🔄 synced file(s) with WordPress/openverseOpenverse Openverse is a search engine for openly-licensed media, including images and audio. Find Openverse on GitHub and at https://openverse.org.
  • #717: Bump Ingestion Server
  • #716: Require the latest version of terraform in docs
  • #715: Temporarily disable num proxies setting
  • #714: Create new production service
  • #713: Add missing commit for staging frontend domain retention
  • #712: Delete redundant files that were deleted from the monorepo
  • #711: 🔄 synced file(s) with WordPress/openverse
  • #709: Remove unnecessary temporary variables used for staged deployments
  • #708: Add anomaly alarms for Nuxt response times and general Request Count
  • #702: Increase thumbnail p99 response threshold and anomaly points to alarm
  • #700: Set less restrictive throttling settings for anonymous
  • #690: Reduce Elasticsearch heap size
  • #685: Cleanup frontend module from temporary nginxNGINX NGINX is open source software for web serving, reverse proxying, caching, load balancing, media streaming, and more. It started out as a web server designed for maximum performance and stability. In addition to its HTTP server capabilities, NGINX can also function as a proxy server for email (IMAP, POP3, and SMTP) and a reverse proxy and load balancer for HTTP, TCP, and UDP servers. https://www.nginx.com/. conditional

#openverse, #week-in-openverse

Community Meeting Recap (2023-12-05)

[Meeting start]

Agenda

📢 Reminder 1: We have our Monthly Priorities Meeting tomorrow, December 6th, which will be the last MPM of 2023. Everyone is invited to join, more details are in the respective announcement.

📢 Reminder 2: These meetings are changing time, starting next week, we will gather to chat on Mondays at 1500 UTC.

[Meeting end]

#openverse-weekly-community-meeting

Weekly community meeting time is changing

Starting on December 11th, 2023, the OpenverseOpenverse Openverse is a search engine for openly-licensed media, including images and audio. Find Openverse on GitHub and at https://openverse.org. weekly community meeting will be changing times from 1500 UTC on Tuesdays to Mondays at 1500 UTC. Said another way, we’ll be moving the meetings 24 hours earlier than they currently are. This will happen after the next weekly meeting, so today/tomorrow’s meeting is still at it’s originally scheduled time!

#openverse-weekly-community-meeting

A week in Openverse: 2023-11-27 – 2023-12-04

openverse

Merged PRs

  • #3431: Update dependency @pinia/testing to ^0.1.0
  • #3429: Update Node.js to v18.19.0
  • #3426: Update dependency prettier to v2.8.8
  • #3423: Update dependency @octokit/rest to v19.0.13
  • #3419: Publish changelog for api-2023.11.29.22.20.31
  • #3418: Update the deployment runbook for the catalog
  • #3417: Publish changelog for catalog-2023.11.29.15.35.59
  • #3416: Use authenticated GitHubGitHub GitHub is a website that offers online implementation of git repositories that can easily be shared, copied and modified by other developers. Public repositories are free to host, private repositories require a paid subscription. GitHub introduced the concept of the ‘pull request’ where code changes done in branches by contributors can be reviewed and discussed before being merged be the repository owner. https://github.com/ requests to set up pre-commit
  • #3414: Fix loading skeletons
  • #3413: Handle existing stack label in CI labelling job
  • #3412: Add project planning weeks of work averaging script
  • #3411: Bump cryptography from 41.0.5 to 41.0.6 in /automations/python
  • #3410: Bump cryptography from 41.0.5 to 41.0.6 in /api
  • #3409: Allow None value for `NUM_PROXIES`
  • #3408: Use DB to augment ES hits for related media with required info
  • #3405: Publish changelog for frontend-2023.11.27.19.20.44
  • #3402: Sync project automation code to infra repo
  • #3401: Fix dynamically excluded providers
  • #3400: Publish changelog for api-2023.11.27.05.10.48
  • #3399: Revert "Fix dynamically excluded providers caching"
  • #3398: Put `NUM_PROXIES` setting in correct location for DRF
  • #3395: Drop separate download fallback
  • #3389: Add SENTRY_RELEASE environment variable to relevant dockerfiles
  • #3388: Convert entire image proxy route async
  • #3383: Add script for calculating available development weeks
  • #3360: Remove unused simple query string features
  • #3338: Add source, creator and tag links to the single result page media info

Closed issues

  • #3403: Related endpoint does not return all of the necessary propertes
  • #3327: Disable unused simple query string features
  • #2789: Convert the image proxy route to async
  • #2730: Component: `VByLine`
  • #2436: `get-translations.js` logs raw error to console with explanation despite the case being handled just fine
  • #490: Add some generic debug logs
  • #664: Thumbnails connection reset
  • #716: Objects returned by search result endpoint are missing some fields that are available on single result

openverse-infrastructure

Merged PRs

  • #706: Add Madison's gpg key from new laptop to git-crypt
  • #705: Bump catalog to latest release `rel-2023.11.29.15.35.59`
  • #704: 🔄 synced file(s) with WordPress/openverseOpenverse Openverse is a search engine for openly-licensed media, including images and audio. Find Openverse on GitHub and at https://openverse.org.
  • #703: Remove NUM_PROXIES until anon throttling is fixed
  • #701: 🔄 synced file(s) with WordPress/openverse
  • #698: Remove unused workflow
  • #697: 🔄 synced file(s) with WordPress/openverse
  • #691: Reduce legacy Elasticsearch module init to minimum for Ansible
  • #682: Set `NUM_PROXIES` into APIAPI An API or Application Programming Interface is a software intermediary that allows programs to interact with each other and share data in limited, clearly defined ways. environment

Closed issues

  • #482: Proposal request: route new `airflow` jumphost subdomain to Airflow, by-passing Cf Access and enabling future integrations to rely on Airflow directly
  • #382: Add `SENTRY_RELEASE` to the Docker image environment when built

#openverse, #week-in-openverse

X-post: Embracing Matrix for Enhanced Communication

X-comment from +make.wordpress.org/project: Comment on Embracing Matrix for Enhanced Communication

X-post: Create Tours for Make P2s

X-comment from +make.wordpress.org/meta: Comment on Create Tours for Make P2s