Title: Preferred Languages: Research Author: Pascal Birchler Published: November 25, 2016 --- [ ](https://profiles.wordpress.org/swissspidy/) [Pascal Birchler](https://profiles.wordpress.org/swissspidy/) 6:55 pm _on_ November 25, 2016 Tags: [feature projects ( 52 )](https://make.wordpress.org/core/tag/feature-projects/), [i18n ( 37 )](https://make.wordpress.org/core/tag/i18n/), [preferred-languages ( 7 )](https://make.wordpress.org/core/tag/preferred-languages/) # Preferred Languages: Research Following [the first meeting](https://make.wordpress.org/core/2016/10/29/preferred-languages-chat-summary-october-26/) of the [Preferred Languages project](https://make.wordpress.org/core/2016/10/18/preferred-languages/), some time was spent on researching popular platforms and other systems to see how they handle the issue of setting multiple preferred languages. To recap, this is needed to show things in the most suitable language for users in case their requested language is not installed. ### Browsers & Operating Systems All major browsers and operating systems have some UIUI User interface where the user can set their preferred languages. Mostly used for the `Accept-Language` HTTPHTTP HTTP is an acronym for Hyper Text Transfer Protocol. HTTP is the underlying protocol used by the World Wide Web and this protocol defines how messages are formatted and transmitted, and what actions Web servers and browsers should take in response to various commands. headerHeader The header of your site is typically the first thing people will experience. The masthead or header art located across the top of your page is part of the look and feel of your website. It can influence a visitor’s opinion about your content and you/ your organization’s brand. It may also look different on different screen sizes., one can set multiple languages and order them by preference. These systems usually know multiple variants like German and German( Switzerland), but not formal / informal variants. [](https://make.wordpress.org/core/2016/10/18/preferred-languages/preferred-languages-firefox-content-only/#main) Firefox [](https://make.wordpress.org/core/2016/10/18/preferred-languages/preferred-languages-chrome/#main) Chrome [](https://make.wordpress.org/core/2016/10/18/preferred-languages/preferred-languages-mac-choose/#main) macOS (overview) [](https://make.wordpress.org/core/2016/10/18/preferred-languages/preferred-languages-mac/#main) macOS (selection) Worth noting that on these systems you can often choose more settings that are influenced by the language, like temperature units and date formats. Related: [#18146](https://core.trac.wordpress.org/ticket/18146). #### The `Accept-Language` HTTP Header [RFC 7231](https://tools.ietf.org/html/rfc7231#section-5.3.5) about the HTTP/1.1 standard says the following about this header: > The “Accept-Language” header field can be used by user agents to indicate the > set of natural languages that are preferred in the response. Each language-range > can be given an associated quality value representing an estimate of the user’s > preference for the languages specified by that range. This header is quite powerful. For example, `Accept-Language: da, en-gb;q=0.8, en; q=0.7` would mean: “I prefer Danish, but will accept British English and other types of English”. I can only recommend reading more about it in the RFC, as it helps getting a better understanding of the problems it tries to solve. This header is usually used by websites to redirect users to the correct version or display a hint like “This content is also available in XY”. While each language in the header has a specific numeric priority, there is usually only a drag & drop interface to determine the order. ### Unicode Common LocaleLocale A locale is a combination of language and regional dialect. Usually locales correspond to countries, as is the case with Portuguese (Portugal) and Portuguese (Brazil). Other examples of locales include Canadian English and U.S. English. Data Repository The Unicode CLDR provides key building blocks for software to support the world’s languages, with the largest and most extensive standard repository of locale data available. It contains an [interesting chart about Language Matching](http://www.unicode.org/cldr/charts/dev/supplemental/language_matching.html), data that is used to match the user’s desired language/locales against an application’s supported languages/locales. There’s also a technical standard about [Unicode Locale Data Markup Language](http://unicode.org/reports/tr35/)( LDML), an XML format for the exchange of structured locale data. The section about [Locale Inheritance and Language Matching](http://unicode.org/reports/tr35/#Locale_Inheritance) is particularly interesting. For instance, it describes finding the most well suited language using a weighted graph and gives a better picture of dealing with more complex language fallbacks. It also takes geographic “closeness” into account, arguing that English (Slovakia) should fall back to something within Europe (e.g. British English) in preference to something far away and unrelated like English (Singapore). It’s clearly stated that these fallbacks aren’t as simple as just saying Spanish (Mexico) -> Spanish( Spain) -> English (US). Note that this technical standard is about finding the best supported locale based on the requested list of languages. The requested list could come from different sources, such as such as the user’s list of preferred languages in the OS Settings, or from a browser’s `Accept-Language` list. ### Wikipedia Wikipedia is built on the MediaWiki software, which has a hardcoded list of fallback chains for some locales. This is where MediaWiki will fall back on a different language if it cannot find what it needs. For example, French (Cajun) automatically falls back to French (France) when it doesn’t have all messages defined in it. Unfortunately, there’s only little [documentation](https://www.mediawiki.org/wiki/Manual:Language#Fallback_languages) about this. Apart from that, MediaWiki distinguishes between [various kinds of languages](https://www.mediawiki.org/wiki/Manual:Language): the _site content language_, the _user interface language_ and the _page content language_. The latter can differ from the first two and it influences the language the user views the page in, which depends on the user’s preferences, the available languages, and the defined fallbacks. It’s worth noting that on Wikipedia, you can not only select your preferred language, but also your preferred gender. In addition to that, you can select multiple different locales for both display _and_ input. [⌊Wikipedia Internationalization Settings⌉](https://make.wordpress.org/core/files/2016/11/wikipedia-i18n-settings.png) Wikipedia Internationalization Settings [⌊Wikipedia Language Chooser⌉](https://make.wordpress.org/core/files/2016/11/wikipedia-user-language.png) Wikipedia Language Chooser [⌊MediaWiki Fallback Chain Graphic⌉](https://make.wordpress.org/core/files/2016/11/mediawiki-fallback-chain.png) MediaWiki Fallback Chain Graphic ### Joomla I tried the official [Joomla Demo](https://demo.joomla.org) to test its [language management](https://docs.joomla.org/Help36:Extensions_Language_Manager_Content) which is a bit overwhelming at first. After installing all the available languages the user is eventually able to select their preferred language for the front end and the back end. There are a few other settings for language switching on multilingual sites, but that’s pretty much it. So basically the same settings as currently in WordPress 4.7. ![Joomla User Settings](https://make.wordpress.org/core/files/2016/11/joomla-language- setting-1024x720.png) Joomla User Settings   ### Drupal Drupal 8 has a rather powerful [user interface text language detection mechanism](http://hojtsy.hu/blog/2013-jun-19/drupal-8-multilingual-tidbits-4-highly-flexible-detection-options). There is a per session, per user and per browser option in the detection settings. However, users can only choose one language, so they cannot say (in coreCore Core is the set of software required to run WordPress. The Core Development Team builds WordPress. at least) that they want German primarily and Spanish if German is not available. But the language selected by the user is part of the larger fallback system, so it may fall back further down to other options. The [Language fallback module](https://www.drupal.org/project/language_fallback) allows defining one fallback for a language, while the [Language Hierarchy module](https://www.drupal.org/project/language_hierarchy) provides a GUI to change the language fallback system. It allows setting up language hierarchies where translations of a site’s content, settings and interface can fall back to parent language translations, without ever falling back to English. This module might be the most interesting one for our research. ![Drupal Language Hierarchy](https://make.wordpress.org/core/files/2016/11/drupal- language-hierarchy-module.png) Drupal Language Hierarchy ### TYPO3 TYPO3 has had a [locale hierarchy](https://typo3.org/download/release-notes/typo3-46-release-notes/) since 2011. Quote: > TYPO3 4.6 comes with a clever fallback mechanism when a label is not found in > the requested language; instead of returning the default (English) version, it > allows you to define your own hierarchy of locales. > By default, French (Canada) will first use French before falling back to English. > Similarly, missing labels in Brazilian Portuguese will first try to return Portuguese > labels before the English ones. This feature also accommodates completely custom > fallbacks. The same goes for the underlying Flow Framework and the Neos CMS. This is only a front end thing though. On the back end, a user can only configure one language. If there’s no translationtranslation The process (or result) of changing text, words, and display formatting to support another language. Also see [localization](https://make.wordpress.org/core/2016/11/25/preferred-languages-research/?output_format=md#l10n), [internationalization](https://make.wordpress.org/core/2016/11/25/preferred-languages-research/?output_format=md#i18n). in that language, it will fall back to the site’s default and eventually to English. ![TYPO3 Back end Language](https://make.wordpress.org/core/files/2016/11/typo3-backend- language.png) TYPO3 Back end Language Setting ### WordPress.comWordPress.com An online implementation of WordPress code that lets you immediately access a new WordPress environment to publish your content. WordPress.com is a private company owned by Automattic that hosts the largest multisite in the world. This is arguably the best place to start blogging if you have never touched WordPress before. 󠀁[https://wordpress.com/](https://wordpress.com/)󠁿 On WordPress.com there’s a user interface language selection in the account settings. One cannot select multiple preferred languages though, but only one. ![WordPress.com User Interface Language](https://make.wordpress.org/core/files/2016/ 11/wordpress-com-user-language-1024x353.png) WordPress.com User Interface Language Setting ### Facebook Facebook only has a really simple language settings. Although a user can select multiple languages they understand for use in the News Feed, they can only choose one specific locale for the overall UI.The locales that Facebook supports are referenced in the [Facebook Locales XML file](http://www.facebook.com/translations/FacebookLocales.xml). That file includes multiple variants for various languages, but only one variant for others. For example, there’s only de_DE, but no de_CH. Plus, you can’t choose between formal and informal variants. ![Facebook User Language Setting](https://make.wordpress.org/core/files/2016/11/ facebook-language-settings-1024x698.png) Facebook User Language Setting ### Summary We’ve now covered a bunch of well-known web platforms and content management systems to see how they are handling this problem. Various techniques exist to assist with finding the right translation. Sometimes they are automated, but most of the time the choice is left up to the individual user. This research should help with the next steps of the Preferred Languages project as these observations need to be adapted for WordPress so that we can learn from them. Feel free to leave any comments about this research in the comments. After that, I try to schedule a new meeting in December where the next steps can be discussed. [#feature-projects](https://make.wordpress.org/core/tag/feature-projects/), [#i18n](https://make.wordpress.org/core/tag/i18n/), [#preferred-languages](https://make.wordpress.org/core/tag/preferred-languages/) ### Share this: * [ Threads ](https://make.wordpress.org/core/2016/11/25/preferred-languages-research/?share=threads) * [ Mastodon ](https://make.wordpress.org/core/2016/11/25/preferred-languages-research/?share=mastodon) * [ Bluesky ](https://make.wordpress.org/core/2016/11/25/preferred-languages-research/?share=bluesky) * [ X ](https://make.wordpress.org/core/2016/11/25/preferred-languages-research/?share=x) * [ Facebook ](https://make.wordpress.org/core/2016/11/25/preferred-languages-research/?share=facebook) * [ LinkedIn ](https://make.wordpress.org/core/2016/11/25/preferred-languages-research/?share=linkedin) 1. ![](https://secure.gravatar.com/avatar/52d5621220a9afcf2b1cfe60038100e195af1b90105d71f6b4382e38c9e27334? s=32&d=mm&r=g) 2. [pcarvalho](https://profiles.wordpress.org/pcarvalho/) 7:04 pm _on_ November 25, 2016 3. Hi Pascal, great round up! 4. in the examples you studied, did you find any that allowed for example: 5. pt_PT fallback to pt_BR , then en 6. and at the same time: 7. pt_BR fallback to pt_PT, then en 8. if its a chain, would this lead to a circular dependency? 9. ![](https://secure.gravatar.com/avatar/d084f9861c652762fd89823c74896279cecb3577bbcf92245a6ab52dd7e38f22? s=32&d=mm&r=g) 10. [Abolfazl Ahani](https://profiles.wordpress.org/ironi2002/) 9:47 pm _on_ November 25, 2016 11. Dear Pascal 12. Thanks you for considering this feature in coreCore Core is the set of software required to run WordPress. The Core Development Team builds WordPress.. 13. 1. I thinks it is better to consider text direction (LTR vs RTL: Right To Left), like Drupal. This is important for most of MENA countries with Persian, Arabic, and Hebrew languages. 14. For example this English language profile in a RTL dashboard: [https://s16.postimg.org/klacuec9h/WP_4_7_User_Profile_Preferred_Language.jpg](https://s16.postimg.org/klacuec9h/WP_4_7_User_Profile_Preferred_Language.jpg) 15. 2. And it will be make it possible to select different languages for site backend and frontend, Like joomla. 16. At the moment I use this pluginPlugin A plugin is a piece of software containing a group of functions that can be added to a WordPress website. They can extend functionality or add new features to your WordPress websites. WordPress plugins are written in the PHP programming language and integrate seamlessly with WordPress. These can be free in the WordPress.org Plugin Directory [https://wordpress.org/plugins/](https://wordpress.org/plugins/) or can be cost-based plugin from a third-party. ([https://wordpress.org/plugins/wp-parsidate/](https://wordpress.org/plugins/wp-parsidate/)), which covers those points, as well as jajli date conversion utilities. 17. Thank you guys again 🙂 18. * ![](https://secure.gravatar.com/avatar/1718b7c2f2afc31b4e94805b6f1f6ae2b802cc4c696f99b11fde2186006c82c9? s=32&d=mm&r=g) * [Pascal Birchler](https://profiles.wordpress.org/swissspidy/) 7:11 am _on_ November 26, 2016 * > I thinks it is better to consider text direction (LTR vs RTL: Right To Left), like Drupal. * So you want to use English, but from right to left? That’s the first time I’m hearing this suggestion personally. Right now, the text direction is defined by the localeLocale A locale is a combination of language and regional dialect. Usually locales correspond to countries, as is the case with Portuguese (Portugal) and Portuguese (Brazil). Other examples of locales include Canadian English and U.S. English. you are using, e.g. English is LTR, Arabic is RTL. * I’m not sure a separate text direction option would be useful for the majority of users. Right now, you’d need a pluginPlugin A plugin is a piece of software containing a group of functions that can be added to a WordPress website. They can extend functionality or add new features to your WordPress websites. WordPress plugins are written in the PHP programming language and integrate seamlessly with WordPress. These can be free in the WordPress.org Plugin Directory [https://wordpress.org/plugins/](https://wordpress.org/plugins/) or can be cost-based plugin from a third-party. like [https://wordpress.org/plugins/rtl-tester/](https://wordpress.org/plugins/rtl-tester/) to achieve this. * > And it will be make it possible to select different languages for site backend > and frontend, Like joomla. * Good news: This is possible with WordPress 4.7, see [https://make.wordpress.org/core/2016/11/07/user-admin-languages-and-locale-switching-in-4-7/](https://make.wordpress.org/core/2016/11/07/user-admin-languages-and-locale-switching-in-4-7/). * > At the moment I use this plugin ([https://wordpress.org/plugins/wp-parsidate/](https://wordpress.org/plugins/wp-parsidate/)), > which covers those points, as well as jajli date conversion utilities. * Interesting. Is the date format not correct when using the Persian translationtranslation The process (or result) of changing text, words, and display formatting to support another language. Also see [localization](https://make.wordpress.org/core/2016/11/25/preferred-languages-research/?output_format=md#l10n), [internationalization](https://make.wordpress.org/core/2016/11/25/preferred-languages-research/?output_format=md#i18n). for WordPress? If so, we should try to make this better in WordPress itself. - ![](https://secure.gravatar.com/avatar/d084f9861c652762fd89823c74896279cecb3577bbcf92245a6ab52dd7e38f22? s=32&d=mm&r=g) - [Abolfazl Ahani](https://profiles.wordpress.org/ironi2002/) 3:26 pm _on_ November 26, 2016 - Hi - I use WP 4.7-RC1 and enjoy its new features, and lovely theme 2017. -   a. ** RTL/LTR Switch option in user Profile** - Currently site’s direction is only controlled by site’s language (1) not user’s language (2): -   - | 1. Site frontend & backend language & direction | Domain/site-name/ wp-adminadmin (and super admin)/options-general.php | | 2. User backend language to override site language | domain/user-name/wp-admin/profile. php | -   - It means that admin can set the first one in Persian (RTL site), but a user may set his profile language to English: in this case his site backend direction is still RTL. I tested it in [this site](http://wldevhost.ir/2017/) (a site in Persian) and a [user profile](https://s16.postimg.org/klacuec9h/WP_4_7_User_Profile_Preferred_Language.jpg) in English. So, I think it is good to have a backend direction option in user profile, too. -   - Also, in some cases, we need to switch temporarily backend of RTL sites from RTL to LTR. Because some pluginPlugin A plugin is a piece of software containing a group of functions that can be added to a WordPress website. They can extend functionality or add new features to your WordPress websites. WordPress plugins are written in the PHP programming language and integrate seamlessly with WordPress. These can be free in the WordPress.org Plugin Directory [https://wordpress.org/plugins/](https://wordpress.org/plugins/) or can be cost-based plugin from a third-party. / themes backend .css files do not support RTL and show awkwardly in RTL backends. As I said, currently I use [Parsi Date](https://wordpress.org/plugins/wp-parsidate) plugin to solve this problem. -   b. ** Jalali calendar in coreCore Core is the set of software required to run WordPress. The Core Development Team builds WordPress.** - [Jalali calendar](https://en.wikipedia.org/wiki/Jalali_calendar) (Official calendar in Iran) is not supported in core, and I use the above mentioned plugin. Yes, it is good to consider it in core. -   c. ** Lunar calendar** - Sometimes, I prepare Arabic language websites. But, interestingly I did not find any plugin to change the site dates to [lunar calendar](https://en.wikipedia.org/wiki/Lunar_calendar)( official calendar of many Arab countries). I searched the for it in [repository](https://wordpress.org/plugins/search.php?q=lunar+calendar). 19. ![](https://secure.gravatar.com/avatar/c0113b43c8dd56a18f20323ef51486b72927d0785a23d6d9f87d06d1c4e85bcd? s=32&d=mm&r=g) 20. [Greg Ichneumon Brown](https://profiles.wordpress.org/gibrown/) 10:20 pm _on_ November 28, 2016 21. I was recently looking at how to do better language guessing on WP.com so let me highlight a few other non-obvious pieces: 22. * On signup WP.com tries to guess the user’s language based on the following lookups: HTTPHTTP HTTP is an acronym for Hyper Text Transfer Protocol. HTTP is the underlying protocol used by the World Wide Web and this protocol defines how messages are formatted and transmitted, and what actions Web servers and browsers should take in response to various commands. accept-lang headers are used if they are set, otherwise the user’s IP address is used to do a country lookup. * When selecting a language with this lookup the language is chosen based on how well translated it is. There is a hard coded list of well translated languages that we try to steer the user towards. * There are a lot of cases where the localeLocale A locale is a combination of language and regional dialect. Usually locales correspond to countries, as is the case with Portuguese (Portugal) and Portuguese (Brazil). Other examples of locales include Canadian English and U.S. English. is ignored for various features. Really its a pretty simplistic way to do the fallback, the language matching chart you found is really interesting. For instance, when searching/ indexing content we mostly use just the language portion of the 23. We’ve discussed supporting multiple languages for things like the Reader, but don’t love the idea of adding more options to the adminadmin (and super admin) to support it. Comments are closed. # Post navigation [← Week in Core, November 15 – 22, 2016](https://make.wordpress.org/core/2016/11/25/week-in-core-november-15-22-2016/) [Extending the Custom CSS editor →](https://make.wordpress.org/core/2016/11/26/extending-the-custom-css-editor/)