I'm very proud to say that "Loaded with lead," a Seattle Times investigation into the ways that gun ranges poison their customers and workers, went live this weekend. I worked on all four interactives for this project, as well as doing the header design and various special effects. We'll have a post up soon on the developer blog about those headers, but what I'd like to talk about today is one particular graphic — specifically, the string-of-pearls chart from part 2.
The data underlying the pearl chart is a set of almost 300 blood tests. These are not all tests taken by range workers in Washington, just the ones that had to be reported after exceeding the safe threshold of 10 micrograms per deciliter. Although we know who some of the tested workers are, most of them are identified only by an anonymous patient ID and the name of their employer. My first impulse was to simply toss the data into a scatter chart, but as is often the case, that first impulse proved ill-advised:
Talking with reporters, what emerged was that the time dimension was not really important to this dataset. What was important was to show that there was a repeated pattern of negligence: that these ranges posted high numbers repeatedly, over long periods of time (in several cases, more than five years). Once we discard a strict time axis, a lot more interesting options open up to us for data visualization.
One way to handle this would be with a traditional box and whiskers plot, which shows the median and variation within a statistical set. Unfortunately, box plots are also wonky and weird-looking for most readers, who are not statisticians and would not know a quartile if it offered them a grilled cheese sandwich. So one prototype simplified the box plot down to its simplest form — probably too simple: I rendered a bar that began and ended within the total range of test results for each range, with individual test results marked with a line inside that bar.
This version of the plot was visually interesting, but it had flaws. It made it easy to see the general level of blood tests found at each range, and compare gun ranges against each other, but it didn't show concentration. Since a single tick mark was shown within the bar no matter how many test results at a given level, there was litttle visual difference between two employers with the same range of test results, even if one employer mainly showed results at the top of the range, and the other results were clustered at the bottom. We needed a way to show not only level, but also distribution, of results.
Given that the chart was already basically a number line, with a bar drawn from the lowest to the highest test result, I removed the bar and replaced the tick marks with circles that were sized to match the number of test results at each amount. Essentially, this is a histogram, but I liked the way that the circles overlapped to create "blobs" around areas of common test results. You can immediately see where most of the tests fall for each employer, but you don't lose sight of the overall picture (which in some cases, like the contractors working outside of a ventilation hood at Wade's, can be horrific — almost three times the amount considered dangerous by the CDC). I'm not aware of anyone else who's done this kind of chart before, but it seems too simple for me to be the first to think of it.
I'd like to take a moment here to observe that pretty much all data visualization comes down to translating information into a form that our visual systems are evolved to quickly understand. There's a great post on how that translation functions here, with illustrations that show where each arrangement sits on a spectrum of perceived accuracy and meaning. It's not rocket science, but I think it's a helpful perspective: I'm just trying to trick your visual cortex into absorbing a table's worth of data at a glance.
But what I've been trying to stress in the newsroom from this example is less technical, and more about how much effective digital journalism comes from the simple process of iteration and self-evaluation. We shouldn't expect to come up with a brilliant interactive on the first try every time, or even any of the time. I think the string-of-pearls is a great example of that, going from a visualization that I was confusing and overly-broad to a more focused graphic statement, thanks to a lot of evolution and brainstorming. It was exhausting work, but it's become my favorite of the four visualizations for this project, and I'm looking forward to tweaking it for future stories.
There's nothing I'm going to write this week that's as moving, as horrifying, or as important as Kathy Sierra's take on trolls and being trolled. Sierra, who was seriously harassed and abused by Andrew "weev" Auernheimer, writes about how awful this harassment was, how common it is, and devastating it was to see Auernheimer's history of sociopathic trolling dismissed after his (admittedly flawed) prosecution for "hacking" AT&T.
But you all know what happened next. Something something something horrifically unfair government case against him and just like that, he becomes tech's "hacktivist hero." He now had A Platform not just in the hacker/troll world but in the broader tech community I was part of. And we're not just talking stories and interviews in Tech Crunch and HuffPo (and everywhere else), but his own essays in those publications. A tech industry award. His status was elevated, his reach was broadened. And for reasons I will never understand, he suddenly had gained not just status and Important Friends, but also "credibility".
Did not see that coming.
But hard as I tried to find a ray of hope that the case against him was, somehow, justified and that he deserved, somehow, to be in prison for this, oh god I could not find it. I could not escape my own realization that the cast against him was wrong. So wrong. And not just wrong, but wrong in a way that puts us all at risk. I wasn't just angry about the injustice of his case, I had even begun to feel sorry for him. Him. The guy who hates me for lulz. Guy who nearly ruined my life. But somehow, even I had started to buy into his PR. That's just how good the spin was. Even I mistook the sociopath for a misunderstood outcast. Which, I mean, I actually knew better.
And of course I said nothing until his case was prosecuted and he’d been convicted, and there was no longer anything I could possibly do to hurt his case. A small group of people — including several of his other personal victims (who I cannot name, obviously) asked me to write to the judge before his sentencing, to throw my weight/story into the "more reasons why weev should be sent to prison". I did not. Last time, for the record, I did NOTHING but support weev’s case, and did not speak out until after he’d been convicted.
But the side-effect of so many good people supporting his case was that more and more people in tech came to also... like him. And they all seemed to think that it was All Good as long as they punctuated each article with the obligatory "sure, he’s an ass" or "and yes, he's a troll" or "he's known for offending people" (which are, for most men, compliments). In other words, they took the Worst Possible Person, as one headline read, and still managed to reposition him as merely a prankster, a trickster, a rascal. And who doesn’t like a "lovable scoundrel"?
The whole post is well worth reading before Sierra possibly takes it offline. And who could blame her? Even as she was writing this, the #gamergate movement on Twitter has literally harassed women out of their homes, and yet (in an echo of weev's "hacktivism") still insists that they're all about "ethics in journalism."
For most, that's an obvious smokescreen — a way to cloak their behavior in respectability while planning their next attack. For those who claim to actually believe it, that's a declaration that writing about videogames is more important than hurting women in the worst possible ways. And there's the problem: you don't get to claim the moral high ground when you got there through the pain and suffering of other people.
We came very close to using WebGL for a Seattle Times special report that will come out next week. Now that iOS 8 has shipped with support for WebGL, albeit in an unstable and slightly buggy form, it's common enough that I felt comfortable using it (with a scaled-down 2D fallback) for our audience. In the end, we went with a different design language and shelved the WebGL experiments, but the experience has left me very excited about the potential for mainstream usage.
There's an obvious parallel here, which is the first two major versions of Android. Because it was designed to run on low-end hardware, Android drew all its UI via software until 3.0 (and hardware acceleration didn't become widespread until 4.0). The resulting lag was never as bad as critics claimed, but it did mean that a lot of Android looked and felt a bit utilitarian. You wouldn't see something like Material Design emerge until the system supported using the GPU for rendering ordinary UI.
It's not a coincidence that Google's moving to Material Design on both Android and the web. Its design language — a smoothly-animated world of flat, geometric shapes — is attractive, but more importantly it's well-matched to the kinds of flat, geometric shapes that can be animated fluidly in a browser, using the 3D acceleration that's already built into the composition layer. Web Components will give developers a way to package those elements up, and make them reusable. Flexbox makes their layouts scalable and responsive.
But for the web platform to move forward, we need more than just a decent look and feel. We need the ability to write the kinds of applications that people insist that it can't run. WebGL is a step in that direction: graphics with near-native speed and capability, instantly deployed and paired with a surprisingly powerful UI toolkit. The kinds of apps and experiences we can write othe web, for a mainstream and mobile audience, just got a lot bigger. And I for one am looking forward to pushing those boundaries as much as I can.
I have started, and then failed to finish, three posts on the #GamerGate nonsense, in which a gang of misogynists led by 4Chan have attempted to hound Anita Sarkeesian and Zoe Quinn off the Internet for daring to be women with opinions about video games. There's very little insightful you can say about this, because they don't have any real arguments for someone to engage, and also because they're dumb as toast. At some point, however, someone decided that they'd use "ethics in journalism" as a catchphrase for their trolling.
What they mean by that is anyone's guess. This Vox explainer does its best to extract an explanation, but other than some noise about "objectivity," there aren't any concrete demands, and the links to various arguments are hilariously silly. One claims that there's a difference between "journalist" and "blogger" based on some vague measure of competence (read: the degree to which you agree with it), which veterans of the "blogger ethics panel" meme circa 2005 will enjoy. Frankly, the #GameGate movement's concept of journalism is itself pretty fuzzy, and tough to debate. As a journalist with actual newsroom experience, I think there are a few things that we should clear up.
Real journalists make phone calls. They dig into stories, find other viewpoints, and perform fact-checking. It's not glamorous work, which may be why reporters are so prone to self-mythologizing (a tendency I'm not immune to), but it is hard and often tedious. It's also, in its best moments, confrontational. There's an old saying: journalism is publishing what someone doesn't want to be published — everything else is just public relations. Some of that is just more myth-making, but it's also true.
When's the last time you read "gaming news" that had multiple sources? That actively investigated wrongdoing in the industry? That had something critical to say about more than how a single game played? You can probably think of exceptions, but that's what they are: exceptions. The vast majority of what gamers call "journalism" isn't anything like real reporting.
This isn't unusual, or even wrong. It's pretty typical for trade press, particularly in the entertainment industry. After all, there's only so combatative you can be when you're dependent on cooperation with game studios and publishers in order to have anything to write about. I don't expect hard-hitting investigations from Bass Player or The A.V. Club either. It's not journalism, but it still has value.
Unfortunately, as far as anyone can tell, the cries of "ethics in journalism" actually translate to a desire for more press releases and PR, like in the halcyon days of Nintendo Power. That's probably comforting for a lot of people, because PR is inherently more comforting than critical thought, but it means what they want and what they claim they want are very different things.
There's a kind of irony of calling for "objectivity" in gaming press, where the dominant mode of writing is through previews and reviews. People making this call aren't asking for actual objectivity, because that wouldn't make sense — what's an "objective" review? One that can definitively state that yes, the game exists? It's a code word for "tell me about the graphics, and the genre, and leave any pesky context out of it."
That isn't much of a review, frankly. It's the kind of thinking that gives four stars to Triumph of the Will because the cinematography is groundbreaking, no matter what the content might have been. Incidentally, the author of the definitive — if satirical — Objective Game Reviews site has a really nice post about this.
It's exactly what they're asking for! And they hate it! It's almost as though, protests to the contrary, this isn't about journalism at all. As if there's actually an agenda being pushed that's more about forcing women and alternative viewpoints out. Imagine that.
Over the weekend we soft-launched our Seahawks Fan Map project. It's a follow-up on last year's model, which was built on a Google Fusion Table map. The new one is better in almost every way: it lets you locate yourself based on GPS, provides autocomplete for favorite players, and clusters markers instead of throwing 3,000 people directly onto the map. It's also built using a couple of interesting technical choices: Google Apps Script and "web components" in jQuery.
The answer, as with many newsrooms, is heavy use of Google Sheets as an ad-hoc CMS. I recently added the ability to for our news app scaffolding to pull from Sheets and cache the data locally as JSON, which is then available to the templating and build tasks. Once every few minutes, a cron job runs on a machine in our newsroom, which grabs the latest data and uploads a fresh copy to the cloud. Anyone can use a spreadsheet, so it's easy for editors and writers to update the data or mark a row as "approved," "featured," or "blocked."
Getting data from the form into the sheet is a more interesting answer. Last year's map embedded a Google Forms page, which is the source of many of its UI sins: they can't be styled, they don't offer any advanced form elements, and they can't be made responsive. Nobody really likes Google Forms, but they're convenient, so people use them all the time. We started from that point on this project, but kept running into features that we really wanted (particularly browser geolocation) that they wouldn't support, so I went looking for alternatives.
Combining the web endpoint for Apps Script with our own custom form gave us the best of both worlds. I could write a form that had pretty styling, geolocation, autocomplete, and validation, but it could still go through the same Google Docs workflow that our newsroom likes. Through the API, I could even handle geocoding during form submission, instead of writing a separate build step. The speed isn't great, but it's not bad either: most of the request time is spent getting a lock on the spreadsheet to keep simultaneous users from overwriting each other's rows. Compared to setting up (and securing) a server backend, I've been very happy with it, and we'll definitely be using this for other news apps in the future.
I'm a huge fan of Web Components and the polyfills built on top of them, such as Polymer and Angular. But for this project, which does not involve putting data directly into the DOM (it's all filtered through Leaflet), Angular seemed like overkill. I decided that I'd try to use old-school technology, with jQuery and I Can Haz templates, but packaged in a component-like way. I used AMD to wrap each component up into a module, and dropped them into the markup as classes attached to placeholder elements.
The result, I think, is mixed. You can definitely build components using jQuery — indeed, I'm very happy with how readable and clean these modules are compared to the average jQuery library — but it's not particularly well-suited for the task. The resulting elements aren't very well encapsulated, don't respond to attribute values or changes, and must manually handle data binding and events in a way that Polymer and Angular safely abstract away. Building those capabilities myself, instead of just using a library that provides them, doesn't make much sense. If I were starting over (or as I consider the additional work we'll do on this map), it's very tempting to switch out my jQuery components for Angular directives or Mozilla's X-Tags.
That said, I'm glad I gave it a shot. And if you can't (or are reluctant to) switch away from jQuery, I'd recommend the following strategies:
The map you see today is only the first version — this is one of the few news projects I plan to maintain over an extended period. As we get more information, we'll add shaded layers for some of the extra questions asked on the form, so that you can see the average fan "lifespan" per state, or find out which players are favorites in countries around the world. We'll also feature people with great Seahawks stories, giving them starred icons on the map that are always displayed. And we'll use the optional contact info to reach out to a "fan of the week," making this both a fun interactive and a great reporting tool. I hope you enjoy the map, and if you're a Seahawks fan, I'll see you there!
This fall, I'll be teaching ITC 210 at Seattle Central College, which is the capstone class of the web development program there. It's taught as a combined class with WEB 210 (the designer's capstone). The last time I taught this course, it didn't go particularly well: although the goal is for students to implement a WordPress site for a real-world client, many of them weren't actually that experienced with the technology.
More importantly, they had never been taught any of the development methods that let teams work together efficiently. I suggested some of the basics — using source control, setting tasks, and using a "waterfall" structure — but I didn't require them, which was a mistake. Under pressure, students fell back on improvised strategies, and many of them ended up in a crunch as a result.
For the upcoming quarter, I plan to remedy those mistakes. But to do so, it's helpful to look at the web development program from a macro level. What is that we're trying to do here, and what should this capstone class actually mean to students?
Although the name has changed, Seattle Central is still very much a community college, and this is very much a trade program. We need to focus on practical job skills, not on CS theory. And so while the faculty are still working on many of the details, one of our goals for curriculum redesign was to create a simple progression between the three web applications classes: first teach basic programming in ITC 240, followed by an MVC framework in ITC 250, and finish the process with a look at development processes (agile, waterfall, last-minute panic, etc.) in ITC 260. By the end, students should feel like they can take a project from start to finish as part of a team in an organized fashion.
Of course, just because that's what our intentions were doesn't mean that it's working out that way. These changes are large shifts in the SCC curriculum, and like steering an Oldsmobile, those take time. So while it would be nice to assume that students have been through the basics of project management by the time that they reach the capstone, I can't count on it — and even then, they probably won't have put it to practice in teams, since the prior classes are individually-graded.
To bring this back to ITC 210, then, we have two problems. First, students don't know how to manage development, because they've spent most of their time just learning how to code. Second, the structure of the class hasn't historically encouraged them to develop those skills. Assignments on the development side tend to be based around the design milestones, which makes their workload "lumpy:" a lot of waiting for design resources, followed by an intense, panicky burst at the end. This may sometimes be an accurate picture of the job, but it's a terrible class experience. Ideally, we want the developers to be working constantly throughout the quarter.
So here's my new plan: this year, ITC 210 will be organized for students around a series of five agile sprints, just like any real-world coding project. At the start of each sprint, they'll assign time and staff to tasks, and at the end of each sprint they'll do a retrospective to help determine their velocity. Grades will be largely organized around documentation of this process. During the last sprint, they'll pick up another team's site and file bugs against it as QA, while fixing the bugs that are filed against them.
This won't entirely smooth out the development process — devs will still be bottlenecked on design work from time to time — but it will make it clear that I expect them to be working the entire time on laying groundwork. It'll also familiarize them with the ways that real teams coordinate their efforts, and it will force them to fit into a common workflow instead of fragmenting into a million angry swarms of random methodology.
I tend to make fun of programmers for thinking that they're the only ones who can invent a workflow, but it's easy to forget that coordinating a team is hard, and nobody comes by it naturally. I made that mistake last time around, and although we scraped by, there were times when it was rough. This quarter, I'm not giving students a choice: they'll work like a regular software team, or they'll fail the course. It may seem harsh, but I think it'll pay off for them when it comes time to do this for a living.
I would love to have been in the meeting where someone pitched Guardians of the Galaxy. "We're going to take all the good will you've built up through the Marvel comic movie franchise, and then spend it on a space movie with characters that nobody really knows, one of whom is a heavily-armed raccoon." And then even weirder, it worked: Guardians is pretty good. Maybe it tells when it should show a little too often, but it never stopped me from enjoying myself. It's got a great soundtrack, good writing, well-done special effects, and most importantly, a really watchable cast.
Of course, this has been the case with most of the Marvel movies. I mean, let's be honest about, say, the Thor franchise, which have been a fun pair of movies considering that they're composed almost entirely of gibberish: Norse gods with British accents (who are actually aliens) fighting against elves and ice trolls (who are also aliens)! The whole thing is completely incoherent, but nobody cares because of the casting: everybody onscreen is good-looking, compulsively charming, and clearly having fun with a very silly premise.
But there's one thing that's been bugging me about the Marvel flicks, including Guardians, which is their endings. Namely, that they've all got the same one: the bad guys summon/control/take over a huge flying object, which immediately crashes headlong into a city.
Explosions follow, while the heroes rush to tackle the portal/controller/big bad at the wheel. Lots of buildings fall over in the process, and people run through the streets while looking up and behind them (oddly enough, hardly anyone ever trips). Lance Mannion refers to it as the "obligatory ad for the video game," and while that's harsh it's not inaccurate, because it does feel a little bit (between the overused CGI and the framing) like watching someone else play God of War. A lot of money went into it, and someone's clearly having a good time, but it's not necessarily you.
And to be clear, Marvel's not the only company writing screenplays this way. Star Trek: Into Darkness, for example, was a movie that committed every sin in the screenwriting book (and then added a few) but arguably the worst part was the meaningless and cruel spaceship crash at its climax. Over at Fox, X-Men: Days of Future Past has Magneto tossing an airborne baseball stadium at the White House. Huge flying objects are the new glass jail cell.
The problem with these pyrotechnics isn't just that they're repetitive and tasteless (although they are both), it's that they're ineffective: by destroying a huge chunk of a city, the writers are aiming for huge cinematic stakes, but by portraying it from a wide-angle lens (and, in PG-rated films, refusing to show any of the resulting bodies and carnage) there's no sense of drama. It's just computer-generated buildings falling over in the distance: who cares?
For a movie like Guardians, which has no real point other than to be a fun space adventure, it's bad enough that there's a good fifteen minutes of watching architecture instead of the characters that are the real draw. But it's more acutely frustrating for something like The Winter Soldier, which spends its first 90 minutes referencing the modern surveillance state, which is a tricky, subtle political problem. And nothing says "tricky" and "subtle" like sending three flying aircraft carriers through a building in Washington, DC.
Maybe that's expecting a bit much from a huge media property with a multi-year cinema domination plan. Marvel wants to put people into seats twice a year, and if that means making the same movie over and over again, that's fine. It's certainly never stopped anyone else (see also: Transformers and Harry Potter). If they're not even sure they can make a movie starring a woman, the chances they'll mess with the formular are pretty slim.
But look at it this way: now you know when you can take a bathroom break without missing anything. I figure you can stay away until the end of the credits, at which point you'll learn which comic-book movie will drop a giant metal object on Paris next year. If we're all very lucky, it'll be Squirrel Girl's turn eventually.
This week, my interactive work for the Seattle Times examines the bidding wars that are part and parcel of being one of the fastest growing cities in the country. It's got everything you need to be horrified by your local real estate market: high prices, short days-on-market, and a search function to see how dire it is next door. It is also the third or fourth interactive that I've built with Angular this year (source code here). There aren't a lot of people building news apps with Angular, which I find amazing: if your goal is to surface data on a deadline, I'd argue it's the best option out there.
Let's review what Angular brings to the table. At the most basic level, it's a library for doing two things:
By contrast, when I look at code written in D3 (seemingly the most popular library for doing news visualizations), I see an entirely different set of priorities:
After years of debugging spaghetti code in jQuery, this design seems both familiar and ominous, particularly the lack of templating and the long call chains. I've written my fair share of apps this way, and they tend to sprawl out into an unstructured, unmaintainable mess. That may not be a problem for the New York Times, which has more budgetary and development resources than I'll ever have. But as the (for now) only developer in the Seattle Times newsroom, I need to be able to respond instantly to feedback from designers, editors, and reporters. One of my favorite things to hear is "we didn't expect a change so fast!" Angular gives me the agility I need to iterate rapidly, try things out, and discard what doesn't work in favor of what does.
Speed and structure are good reasons to use Angular in a newsroom, but there's another, less obvious incentive. Angular is basically training wheels for Web Components: although it lacks the Shadow DOM, it includes equivalents for custom elements and HTML imports. It's a short hop from Angular to libraries like Polymer, and from there to a whole world of deadline-friendly tooling and reuse. Make no mistake, this is the future of web development, and it can't get here soon enough: I'd love to be able to simply send off an <interactive-feature> tag to the web producers, and I imagine they'd appreciate it too. The Google Web Components tags would be a similar godsend.
For me, this makes using Angular a no-brainer. It's fast, it's effective, it's great for visualizations, and it's forward-thinking. It shocks me that more people haven't seen its advantages — but then, given the way that most newsroom hackers seem to think of the browser as "that embarrassing thing that loads my server code," it probably shouldn't be surprising.
We've just released a new interactive I've been working on for a couple of weeks, this time exploring the Oso landslide earlier this year. Our timeline (source) shows... well, I'll let the intro text explain it:
The decades preceding the deadly landslide near Oso reflect a shifting landscape with one human constant: Even as warnings mounted, people kept moving in. This interactive graphic tells that story, starting in 1887. Thirteen aerial photographs from the 1930s on capture the geographical changes; the hill is scarred by a succession of major slides while the river at its base gets pushed away, only to fight its way back. This graphic lets you go back in time and track the warnings from scientists; the failed attempts to stabilize the hill; the logging on or near the unstable slope; and the 37 homes that were built below the hill only to be destroyed.
The design of this news app is one of those cases where inspiration struck after letting its idea percolate for a while. We really wanted to showcase the aerial photos, originally intending to sync them up with a horizontal timeline. I don't particularly care for timelines — they're basically listicles that you can't scan easily — so I wasn't thrilled with this solution. It also didn't work well on mobile, and that's a no-go for my Seattle Times projects.
One day, while reading through the patterns at Bocoup's Mobile Vis site, it occurred to me that a vertical timeline would answer many of these problems. On mobile, a vertical scroll is a natural, inviting motion. On desktop, it was easier to arrange the elements side-by-side than stacked vertically. Swapping the axes turned out to be a huge breakthrough for the "feel" of the interactive — on phones and tablets that support inertial scrolling for overflow (Chrome and IE), users can even "throw" the timeline up the page to rapidly jump through the images, almost like a flipbook. On desktop, the mouse wheel serves much the same purpose.
On a technical level, this project made heavy use of the app template's ability to read and process CSV files. The reporters could work in Excel, mostly, and their changes would be seamlessly integrated into the presentation, which made copy editing a cinch. I also added live reload to the scaffolding on this project — it's a small tweak, but in group design sessions it's much easier to keep the focus on my editor for tweaks, but let the browser refresh on another monitor for feedback. I used Ractive to build the timeline itself, but that was mostly just for ease of templating and to get a feel for it — my next projects will probably return to Angular.
All in all, I'm extremely happy with the way this feature turned out. The reporting is deep (in a traditional story, it would probably be at least 5,000 words), but we've managed to tell this story visually in an intuitive, at-a-glance format, across multiple device formats. Casual readers can flip through the photos and see the movement of the river (as well as the 2014 devastation), while the curious can dig into individual construction events and warning signs. It's a pretty serious chunk of interactive storytelling, but we're just getting started. If you or someone you know would like to work on projects like this, feel free to apply to our open news app designer and developer positions.