Mile
Zero

The problem with developing election data expertise is that they make you cover elections.

Even as they go, this was a rough one, and I'm just now starting to feel like I've recovered. After getting through a grueling primary season, we ended up rewriting our entire general election rig — we've got a post on the NPR News Apps blog about it, which is a pretty good overview.

This wasn't my first election night. But it was one of the biggest development efforts and teams I've led in journalism, and obviously the stakes were high. In the end, I'm mostly satisfied with the work we did — there's always room for improvement. I think 2020 is a year I'll look back on in terms of the simple (our technical choices) and the complex (weighing our responsibility for the results).

Act, React

For the primaries, I wrote our results displays as a set of custom elements. That worked well: it was fast, expressive, and with a little work to smooth over the API (primarily adding support for automatic templating and attribute/property mirroring), it was a pretty pleasant framework. However, for the general election, I wanted to build out a single-page application that shared state across multiple views. I didn't have a ton of experience with React, and it seemed like a worthwhile experiment (technically, we used Preact, but it's the same thing in practice).

Turns out I didn't love the experience, but it did have advantages. HTML custom elements do not come with a templating solution built-in, and they don't have a good way to pass non-string data across element boundaries. I wrote a miniature "data to element mapping" function to make that process easier, but it was never as straightforward as JSX templates were. In general, React's render cycle is reasonably pleasant to use, and easy to train people on. It's clear that this is what the library was primarily designed to do.

Unfortunately, it often feels like React doesn't really know how to organize the non-rendering parts of a web application, and when you're building live election results those parts matter. Unlike Vue or custom elements, which separate updates to display and data, changes to a React component properties and state all get piped through the render cycle. This is a poor match for operations that don't easily map to an idempotent template value, like triggering a server fetch or handling focus for accessibility purposes.

As best I can tell, the React community's answer to that problem has been to double down on its fixation on stateless programming, switch to pure functions and "hooks" for rendering, and recommend the use of global stores like Redux to manage data for the application. Class-based components are clearly considered gauche, and JSX templating encourages only using syntax that can fit into a return value, instead of natural structures like loops and conditionals. At times, working with React feels like being trapped with that one brogrammer at a party who keeps telling you about the lambda calculus. We spent decades waiting for better JavaScript syntax features, and now React wants to pretend they don't exist? In this economy?

The best explanation I've seen of this philosphy is Rich Harris's talk "Metaphysics and JavaScript" (be sure to bring up the slide notes). I'm not sure Harris has the right solution either — the new syntax in his own Svelte framework gives me the willies — but when he talks about React in terms of an ideology, I think he's right. And I'm ideological as hell, but I'm also a practical person. If your framework keeps choosing an elegant abstraction over how the actual world works, it's not doing what I need it to do.

So, long story short, it works, and I think it's probably good enough to re-use for another four to eight years. But I went into the experience hoping to see if there was some hidden virtues of React that I just hadn't found, and largely I was disappointed.

Tough Calls

It was obvious to anyone paying the slightest bit of attention that Trump was not going to accept the results of this election if he didn't win, and that his supporters would absolutely be flooding the country with misinformation about it. He'd done it before: in 2016, among other lies, he handed out misleading maps to try to persuade visitors that he had actually won the popular vote and the majority of the country.

But five days before that report surfaced, as usual having utterly failed to read the room, the New York Times had published a largely identical map, effectively pre-validating Trump's choice. Much like Trump's 2020 strategy, this shouldn't have been hard to predict: a map of vote margin by precinct is considered wildly misleading by data visualization experts, and right-wing operatives have consistently overemphasized the mostly-empty land mass that makes up "real America" when they're working the refs. If newsrooms themselves don't understand (or care) how their visual reporting contributes to misinformation, how can we expect voters to differentiate between journalism and lies?

My worst nightmare, going into 2020, was that Trump or someone in his orbit would use our reporting to try to illegally retain power. So we planned for uncertainty. We had a big note at the top of the page to warn readers that mail-in voting could delay results. We didn't mark races as "leading" on any displays until they hit 50% of votes in, to reduce the well-known red-to-blue shift from vote tabulation. We built new displays to emphasize electoral weight over geographic size. And behind the scenes, I built additional overrides into the pipeline just to be safe: although we never used them, there was a whole facility for flagging races with arbitrary metadata that could have been used for legal challenges or voting irregularities.

(In fact, if I can offer one bit of advice to anyone who's new to election results, it's to be extremely pessimistic about the things you might need to override. Since this wasn't my first rodeo, I had control sheets already set up that would let us reset candidate metadata, ballot rosters, and race calls. Sure enough, when Maine's public radio stations wanted long-shot independent candidates included to highlight their impact on ranked-choice voting for Susan Collins' seat, despite not clearing our vote threshold for display, it was just a matter of updating a few cells. Same for when data irregularities muddled the district results coming from those states that proportionally allocate their electoral votes.)

I honestly wanted to go further with our precautions, but I'm satisfied with what we got approved in a fairly traditional newsroom. And in the end, we got lucky: it wasn't actually that close. Biden won by the same electoral margin that Trump had won in 2016, and across enough states by a high enough margin that it couldn't be flipped with a single lawsuit (even assuming Trump's lawyers were competent which... was not the case).

So the good news was that we didn't have to worry about NPR results being posted to Trump's Twitter feed. The bad news, as you know, is that the losers of the election spent the next two months undermining those results, lying about illegal voting, and eventually inciting a riot by white supremacists and QAnon conspiracy theorists at the Capitol during final certification of the electoral college. Kind of a mixed bag.

No, this election wasn't like any other. However, I hope it serves as a chance for all of us who do this work to think carefully about what our role is, and how we use the power we've been given. I have long argued that real-time election results are a civic disaster — they're stressful, misleading, and largely pointless — but that ship has sailed, and like a hypocrite I cash the checks for building them regardless. In the era of The Needle, news organizations aren't going to give up the guaranteed traffic boost no matter how unhealthy it is for the country. At the very least we can choose to design responsibly, and prioritize more than "faster results" and "stickier pages." We're not big on long-term lessons in this industry, but we have to start somewhere.

Ah, budget day: the most annoying day of a data journalist's year. Even now that I'm no longer covering Congress, it still bugs me a little--except now, instead of being frustrated by the problem of finding stories, I'm just annoyed by the coverage itself. Few serious policy documents create so much noise from so little data.

For those who are unaware, on the night before budget day, each senator or representative places a constituent's tooth under their pillow before going to bed. While they're asleep, dreaming of filibusters and fundraising, the White House Chief of Staff creeps into their bedrooms and takes the tooth away, leaving a gift in return. Oh, the cries of joy when the little congresscritters wake to find a thick trio of paperback budget documents waiting for them!

Casting the budget as a fairy tale isn't as snarky as it might seem, because the president's budget is almost entirely wishful thinking. The executive branch, after all, does not control the purse strings of government--that power lies with the legislature. The budget is valuable in that it sets an agenda and expresses priorities, but any numbers in it are a total pipe dream until the appropriations process finishes. And if you want an example of how increasingly dysfunctional Congress has become, look no further than appropriations.

Although money for the next fiscal year is supposed to be allocated into appropriations bills by October 1st (the start of the federal fiscal year), they are increasingly late, often months late. In the meantime, Congress passes what are called "continuing resolutions"--stopgap measures that fund the government at (usually) reduced levels until real funding is passed. You can actually see the delays getting worse in a couple of graphics that my team put together at CQ: first, the number of "bill days" delayed since 1983, and then the number of "bill months" delayed by committee since 1990. Needless to say, this probably isn't helping the federal government run at its most efficient.

The connection between the president's budget and the resulting sausage is therefore tenuous at best (don't even get me started on tracking funds through appropriations itself). Even worse, from the perspective of a data-oriented reporter, is that the numbers in the budget are not static. They are revised multiple times by the White House in the months after release--and not only are they revised, they are often revised retroactively as new economic data comes in and the numbers must be adjusted to fit the actual policy environment. So even if we could talk about budget numbers as though they were "real money," the question remains: which budget numbers? And from when?

During my first couple of years at CQ, around January I would sit down with the Budget Tracker team and the economics editor, and propose a whole series of cool interactive features for budget season. And each time, they would politely and carefully explain all these caveats, which collectively added up to: we could talk about the budget in print, where numbers would not be charted against each other, and we could talk about the ways the budget/appropriations process is broken. But there simply isn't enough solid data to graph or visualize those numbers, since that lends them a visual credibility that they don't actually have.

The result is that I find budget day frustrating, even after leaving the newsroom, because it feels like a failure--something we should have been able to explain to our readers more fully, but couldn't quite grasp ourselves. Simultaneously, I often find coverage by other outlets annoying because they report on the budget as thought it's more meaningful than it actually will be, or they'll chart it across visualizations as though imaginary numbers could be compared to each other (there is an element of jealousy to this, no doubt: it must be nice to work in a place where you can get away with a little editorial sloppiness). It's a shame, because the budget itself is not broken. As an indication of what the White House thinks is important for the upcoming year, it's a great resource. But it is not a long-term financial plan, and shouldn't be reported as such.