This post was originally published at @media, MathML, and Django 1.11: MDN Changelog for May 2018
Editor’s note: A changelog is “a log or record of all notable changes made to a project. [It] usually includes records of changes such as bug fixes, new features, etc.” Publishing a changelog is kind of a tradition in open source, and a long-time practice on the web. We thought readers of Hacks and folks who use and contribute to MDN Web Docs would be interested in learning more about the work of the MDN engineering team, and the impact they have in a given month. We’ll also introduce code contribution opportunities, interesting projects, and new ways to participate.
Done in May
- Migrated CSS @media and MathML compat data
- Prepared for Django 1.11
- Started tracking work in ZenHub
- Continued HTML Interactive Examples
- Shipped tweaks and fixes by merging 397 pull requests, including 60 pull requests from 43 new contributors.
We’ll continue this work in June.
The browser compatibility migration continued, jumping from 72% to 80% complete. Daniel D. Beck completed the CSS @media rule features, by converting data (PR 2087 and a half-dozen others) and by reviewing PRs like 1977 from Nathan Cook. Mark Boas finished up MathML, submitting 26 pull requests.
There are over 8500 features in the BCD dataset, and half of the remaining work is already submitted as pull requests. We’re making steady progress toward completing the migration effort.
MDN runs on Django 1.8, the second long-term support (LTS) release. In May, we updated the code so that MDN’s test suite and core processes work on Django 1.8, 1.9, 1.10, and 1.11. We didn’t make it by the end of May, but by the time this report is published, we’ll be on Django 1.11 in production.
Many Mozilla web projects use Django, and most stick to the LTS releases. In 2015, the MDN team upgraded from Django 1.4 LTS to 1.7 in PR 3073, which weighed in at 223 commits, 12,000 files, 1.2 million lines of code, and at least six months of effort. Django 1.8 followed six months later in PR 3525, a lighter change of 30 commits, 2,600 files, and 250,000 lines of code. After this effort, the team was happy to stick with 1.8 for a while.
Most of the upgrade pain was specific to Kuma. The massive file counts were due to importing libraries as git submodules, a common practice at the time for security, reliability, and customization. It was also harder to update libraries, which meant updates were often delayed until necessary. The 1.7 upgrade included an update from Python 2.6 to 2.7, and it is challenging to support dual-Python installs on a single web server.
The MDN team worked within some of these constraints, and tackled others. They re-implemented the development VM setup scripts in Ansible, but kept Puppet for production. They updated submodules for the 1.7 effort, and switched to hashed requirements files for 1.8. A lot of codebase and process improvements were gained during the update effort. Other improvements, such as Dockerized environments, were added later to avoid issues in the next big update.
Django 1.4 wasn’t originally planned as an LTS release, but instead was scheduled for retirement after the 1.6 release. The end of support was repeatedly pushed further into the future, finally extending to six months after 1.8 was published.
On the other hand, the Django team knew that 1.8 was an LTS release when they shipped it, and would get security updates for three years. Many of the Django team’s decisions in 2015 made MDN’s update easier. Django 1.11 retained Python 2.7 support, avoiding the pain of a simultaneous Python upgrade. Django now has a predictable release process, which is great for website and library maintainers.
Django also maintains a guide on updating Django, and the suggested process worked well for us.
The first step was to upgrade third-party libraries, which we’ve been working on for the past two years. Our goal was to get a library update into most production pushes, and we updated dozens of libraries while shipping unrelated features. Some libraries, such as django-pipeline, didn’t support Django 1.11, so we had to update them ourselves. In other cases, like django-tidings, we also had to take over project maintenance for a while.
The next step was to turn on Django’s deprecation warnings, to find the known update issues. This highlighted a few more libraries to update, and also the big changes needed in our code. One problem area was our language-prefixed URLs. MDN uses Mozilla-standard language codes, such as
/en-US/, where the region is in capital letters. Django ships a locale-prefixed URL framework, but uses lowercase language codes, such as
/en-us/. Mozilla’s custom framework broke in 1.9, and we tried a few approaches before copying Django’s framework and making adjustments for Mozilla-style language codes (PR 4790).
Next we ran our automated tests, and started fixing the many failures. Our goal was to support multiple Django versions in a single codebase, and we added optional builds to TravisCI to run tests on 1.9, 1.10, and 1.11 (PR 4806). We split the fixes into over 50 pull requests, and tested small batches of changes by deploying to production. In some cases, the same code works across all four versions. In other cases, we switched between code paths based on the Django version. Updates for Django 1.9 were about 90% of the upgrade effort.
This incremental approach is safer than a massive PR, and avoids issues with keeping a long-running branch up to date. It does make it harder to estimate the scope of a change. Looking at the commits that mention the update bugs, we changed 2500 to 3000 lines, which represents 10% of the project code.
For the past two years, the MDN team had been using Taiga to plan and track our work in 3-week sprints. The team was unhappy with performance issues, and had been experimenting with GitHub’s project management features. Janet Swisher and Eric Shepherd led an effort to explore alternatives. In May, we started using ZenHub, which provides a sprint planning layer on top of GitHub issues and milestones.
If you have the ZenHub plugin installed, you can view the sprint board in the new mdn/sprints repository, which collects tasks across our 10 active repositories. It adds additional data to GitHub issues and pull requests, linking them into project planning. If you don’t have the plugin, you can view the sprint board on the ZenHub website. If you don’t want to sign in to ZenHub with your GitHub account, you can view the milestones in the individual projects, like the Sprint 4 milestone in the Interactive Examples project.
We’re still working through the technical challenges of shipping HTML interactive examples. One of the blocking issues was restricting style changes to the demo output on the right, but not the example code on the left. This is a use case for Shadow DOM, which can restrict styles to a subset of elements. Schalk Neethling shipped a solution based on Shadow DOM in PR 873 and PR 927, and it works natively in Chrome. Not all current browsers support Shadow DOM, so Schalk added a shim in PR 894. There are some other issues that we’ll continue to fix before including tabbed HTML examples on MDN.
Meanwhile, contributors from the global MDN community have written some interesting demos for <track> (Leonard Lee, PR 940), <map> (Adilson Sandoval, PR 931), and <blockquote> (Florian Scholz, PR 906). We’re excited to get these and the other examples on MDN.
There were 397 PRs merged in May:
- 176 mdn/browser-compat-data PRs
- 95 mdn/interactive-examples PRs
- 70 mozilla/kuma PRs
- 25 mdn/bob PRs
- 16 mdn/data PRs
- 8 mdn/kumascript PRs
- 2 mdn/dom-examples PRs
- 1 mdn/voice-change-o-matic PR
- 1 mdn/infra PR
- 1 mdn/mdn PR
- 1 mdn/learning-area PR
- 1 mdn/pwa-examples PR
60 of these were from first-time contributors:
- Chrome no longer supports
caretPositionFromPoint(BCD PR 1909), from Micah Cowsik-Herstand.
- Update MSE for
EventHandlersattributes support (BCD PR 1949), from Nicolas Le Gall.
- Record that
timeZoneworks (BCD PR 1959), from Chris Angelico.
- Mark the
scopeattribute as not deprecated (PR 1960), and Mark the
abbrattribute as not deprecated (PR 1961), to BCD from Roland Warmerdam.
- Fix Chrome/Opera versions
removeTrack(#1288) (PR 1963), from Josiah Ulfers (first contribution to BCD).
sourceelement and its attributes for Safari desktop/iOS (BCD PR 1964), from Josh Schneider.
- nodejs supports
Error.prototype.stack(BCD PR 1970), from Chaim-Leib Halbert.
append()for edge browser (BCD PR 1973), from midzer.
- Add compat data for css at-rules media
any-hover(BCD PR 1977), from Nathan Cook.
- EdgeHTML 17 updates (part 2) (BCD PR 1981), from Libby McCormick.
- Change MDN page for chrome support of
AnimationEvent.pseudoElement(BCD PR 1986), from xidachen.
- Update Edge desktop & mobile support of
text-decoration-styleCSS property (BCD PR 2004), from Lucas Volle.
HTMLIFrameelement (PR 2017), from SphinxKnight (first contribution to BCD).
WorkerGlobalScope.importScripts(BCD PR 2046), from mantou.
toLocaleStringcompat in Firefox Android (BCD PR 2055), from Zacqary Adam Xeper.
SameSitesupported in Firefox 60. (BCD PR 2088), from Dominik Picheta.
- Fix typo in
<form>(BCD PR 2142), from acshef.
- Add compatibility data for
AudioNodeOptions(BCD PR 2160), and Fix capitalization of Firefox’s
DocumentOrShadowRoot.fullscreenElement(BCD PR 2161), from haykam821.
ChildNode.removeworks in iOS Safari (BCD PR 2171), from Scott Koenig.
- Add compat data for Edge (BCD PR 2174), from Zouhir .
- Correct Chrome version for
issupport (PR 2175), from Jonathan Pool (first contribution to BCD).
- Inline text semantics for
br(Interactive Examples PR 828), from Kadir Topal.
pictureexample. (PR 842), Add
embedexample. (PR 863), and 7 more PRs to Interactive Examples from Leonard Lee.
- Improve js function
bindexample (Interactive Examples PR 886), from Théis Bazin.
- Add Input
<input type="button">(PR 891), and Add Input
<input>example (PR 892), to Interactive Examples from Neha Nupoor.
Array.prototype.values()(Interactive Examples PR 917), from Sid Vishnoi.
background-originexample. (Interactive Examples PR 918), from Paul Ferlito.
- Add HTML example for
<details>(PR 924), and Add HTML example for
<summary>(PR 926), to Interactive Examples from Christian Stuff.
(max-)widthto container (PR 929), Add HTML example for
<audio>(PR 930), and Change
tabbedin Publishing section (PR 942), to Interactive Examples from Stephan Max.
- Add object-keys.html (Interactive Examples PR 937), from Yahya Elharony.
- Implemented example for
<figcaption>tag (Interactive Examples PR 941), from daGo.
- Directly use
Math.roundin the demo code. (PR 956), from arai-a (first contribution to Interactive Examples).
- Remove depreciated
CELERYD_LOG_LEVEL(Kuma PR 4748), from mashrikt.
<title>in code samples (Kuma PR 4767), from Antonio Fernando Santos Ladeia.
- Add build status badge to README.md (PR 51), from ExE Boss (first contribution to bob).
list-itemlast (PR 220), from Oriol Brufau (first contribution to Data).
- Fix data for
line-height-step(PR 224), from Fuqiao Xue (first contribution to Data).
font-weightsyntax for CSS Fonts 4 (PR 234), Update
font-stretchfor CSS Fonts Level 4 (PR 235), and Update
font-stylefor CSS Fonts Level 4 (PR 236), from wbamberg (first contributions to Data).
- Add Ukrainian translation for
js_property_attributesmacro (KumaScript PR 700), from Nikola Myslovskij.
- Added server-sent-events/README.md (PR 20), and server-sent-events: Break the
connection_aborted()(PR 21), to dom-examples from Dumitru Uzun.
- Update app.js (voice-change-o-matic PR 14), from Alexander Leon.
- Focus the repository on MDN Web Docs (PR 1), from me (first contribution to infra).
- Fixed spelling error in comments (learning-area PR 71), from Stephen Borutta.
Other significant PRs:
- Add a name property to the browser schema (BCD PR 2025), from Connor Shea.
>>>(shadow-piercing descendant combinator) alias in Chrome (BCD PR 2158), from wbamberg.
- Add headless tests for CDN (Kuma PR 4765), from Ryan Johnson.
aria-hiddenfrom social SVG (Kuma PR 4776), from Schalk Neethling.
select_relatedto select only foreign key fields (Kuma PR 4804), from Safwan Rahman.
::slottedpseudo-element and necessary syntaxes (Data PR 169), from Chris Mills.
:host-context()(Data PR 171), from Chris Mills.
- Improve JSON schema errors output (Data PR 231), from Roman Dvornov.
We plan to ship Django 1.11 to production in early June, even before this post is published. We’ll spend some additional time in June fixing any issues that MDN’s millions of visitors discover, and upgrading the code to take advantage of some of the new features.
We plan to keep working on the compatibility data migration, HTML interactive examples, and other in-progress projects. June is the end of the second quarter, and is a deadline for many quarterly goals.
We are gathering in San Francisco for Mozilla’s All-Hands. It will be a chance for the remote team to be together in the same room, to celebrate the year’s accomplishments to date, and to make plans for the rest of the year.
This post was originally published at @media, MathML, and Django 1.11: MDN Changelog for May 2018