Up: Colorless Green Ideas

[Related]   «^»   «T»

Real Editors Ship

By Paul Ford

tl;dr: needs editing.

There's a useful dialogue making the rounds. A man named Tom Taylor wrote about shipping product (I picked that up off Waxy Links). He's shipped good work himself, and makes the point that getting stuff out the door is a noble thing. I agree with that. Here's the closing graf:

And the next time someone produces an antenna with a weak spot, or a sticky accelerator, you're more likely to feel their pain, listen to their words and trust their actions than the braying media who have never shipped anything in their lives.

Now as for this—well, as Taylor later pointed out (and I love blogs that point out when someone disagrees with them—very classy), this bothered his friend Bobbie Johnson. Johnson wrote:

Tom's assessment is that these people (let's call them critics) have never shipped anything and therefore can't understand what they're talking about. I'd suggest the opposite is in fact the case: the trouble is that media ships constantly, and therefore becomes inured to the difficulties and delicacies of launching a product of any size or scale.

I agree with that even more. (Well, mostly; they may be inured to the difficulties, or maybe just not impressed by them.) And I think the discussion between Taylor and Johnson brings up a good point. I remember when I used to write for All Things Considered, my editor there sent me a few pictures from the whiteboard they used to put together the show. It changed constantly throughout the day; they kept a webcam trained on it (this was a few years ago; maybe they use websockets and node.js now). There were an insane number of variables that went into creating that big hunk of nightly audio: Recordings created months ago or two hours ago; people working together in a dozen time zones; contracts, permissions, fact-checking. It had to fit together technically; it had to be transmitted efficiently at a high bitrate to maintain quality (but may be sped up or slowed down to the limits of Fourier transforms); it had to be edited to match certain durations; it had to have a certain consistency and flow; and so on. It requires the human equivalent of map-reduce to manage it. And they—meaning editors and producers—managed a release every night, with 12 million users.

People often think that editors are there to read things and tell people "no." Saying "no" is a tiny part of the job. Editors are first and foremost there to ship the product without getting sued. They order the raw materials—words, sounds, images—mill them to approved tolerances, and ship. No one wrote a book called Editors: Get Real and Ship or suggested that publishers use agile; they don't live in a "culture" of shipping, any more than we live in a culture of breathing. It's just that not shipping would kill the organism. This is not to imply that you hit every sub-deadline, that certain projects don't fail, that things don't suck. I failed plenty, myself. It just means that you ship. If it's too hard to ship or you don't want to deal with it, you quit or get fired.

I recently left zineland and did a bunch of freelance work and hooboy do people not know how to ship. A three-year project that yielded only 90-second page load; or $1.5 million down the drain with only a few microsites to show. And I've started to find myself going, God, these projects need editors. Editors are really valuable, and, the way things are going, undervalued. These are people who are good at process. They think about calendars, schedules, checklists, and get freaked out when schedules slip. Their jobs are to aggregate information, parse it, restructure it, and make sure it meets standards. They are basically QA for language and meaning.

But can they deal with character encoding issues when the parser breaks? Not really. They're often luddites of the kind that calls the mouse a clicker, even the young ones. That said, I think there're weird content times afoot. Google just acquired MetaWeb, which is not user-generated as much as user-edited content. (C.f. the Shakespeare page). Wolfram Alpha is purely about curating data sources and then calculating atop the restructured data. Wikipedia growth is slowing, but editing and tagging continue; the infoboxes are a wealth of semantic data. Meanwhile F——b—— and Tw—— (I can't bear to write those words again) continue to dump forth information by the gallon, now tagging their core objects with all manner of extra metadata. Everything is being knit together in all sorts of ways. User-generated content is still king, because it generates page views and inculcates membership (the concept of the subscription being dead, the concept of the membership being ascendant) but user-edited content is of increasing importance because of what I call, having just made it up, "the Barnes & Noble problem."

Until I was about 26 almost everything I wanted to read was in Barnes & Noble. Eventually they had less and less of what I wanted. Now B&N's a place I go before a movie, and I get my books anywhere else. I'm increasingly having B&N moments with full text search ala Google. It's just not doing the job; you have to search, then search, then search again, often within the sites themselves. The web is just too big, and Google really only can handle a small part of it. It's not anybody's fault. It's a hard, hard problem.

Remember when everyone was into the idea that Google is a media company, back in 2008 when YouTube was two? Google is not really a media company as much as a medium company. Google creates forms—i.e. structured ways of representing data—and then populates them with search results. They're the best at that. Google doesn't do the best job making it easy to edit the nodes in every case (they can when they want, though—it's easy to edit in Gmail, or upload a video), or even particularly want you to edit much of their data. Knol being the exception that proves that Knol is kind of eh. And I haven't checked in on Orkut (the 65th largest website in the world) in quite a while.

Now, though, they've bought ITA (a very interesting company that has had tons of weird database stuff going on for a while) and Metaweb. So clearly structured—meaning edited, meaning user-edited—data is now going to be a big part of the web. There are going to be all kinds of new slots and tabs and links and nodes. And whether the users want this or not, it looks like they're going to get it, and the state of NLP being what it is, not to mention NPC, humans will need to be involved. Unfortunate but true. (Then again I've been off in the high wilderness for five years; I have no clue what people think in Mountain View. I could just be blowing more smoke.)

The Semantic Web is basically the edited web, for some very nerdy take on editing. Which implies editors. Facebook has gone turtles all the way down. Django, Rails, and other frameworks make it possible to build custom-structured-and-semantic data acquisition tools with very little pain; Django's admin, in particular, is optimized for exactly that sort of thing. Solr and related technologies make it possible to search through that structured information. And nearest to my heart there's an insane glut of historical data, texts, and so forth, billions of human, historical, textual objects to come online from the millennia before the web. Plus a gaggle of history bloggers trying to contextualize it (the history bloggers are the best bloggers out there—but that's for a different day). Dealing with the glut—and we must deal with this glut, because what is more important than sorting all human endeavor into folders?—will require all manner of editing, writing, commissioning, contextualizing, and searching. (Take a look at Lapham's Quarterly to see one very successful approach, using paper and ink.) Fortunes will be made! Not mine, of course, because I lack the qualities that money likes, but someone's. History is big business.

I see three problems with my idea. First, editors and journalists are mostly luddites, as already noted, and they don't really hang out in places where you might think to hire them. (I think the Awl should have a jobs board; that would be perfect.) But I think this one can be solved: even my most technically mystified editor pals could be trained to use Freebase Gridworks. Add to that the willingness to schedule the living shit out of everything, the ability to see patterns, a total dedication to shipping, and willingness to say "no," and you start to have this very interesting source of power inside your organization, especially given the changes coming in web content, where you need structure and connections in order to play with others. Editors can help you play nice. And they actually do understand standards, at least conceptually. If you tell them the line needs to end with a semicolon they will end it with a semicolon. Words into Type and ISO 8879 are of similar complexity.

Second problem: most editors want to be editing for print or broadcast, not for the web, which is still seen as slumming it. But that said more and more of the big-deal journalism is about aggregating data. Which means that more and more journalists are getting exposed to thinking in grids and bulk-editing and so forth. Or at least getting interns to do it for them. Which is interesting. Also, getting fired or taking a buyout helps people gain perspective on what they like doing; there's that.

Third problem: I've worked on various big content engagements, and I've talked to a number of people with more big-content experience than me. And people agree that big orgs, even if they now have content problems, won't hire editors, or enough editors, to manage their content. Think: museums, non-profits, giant corporations, government. I get very sadpanda when I see someone spend $500K plus deployment, development, and licensing costs on a Java EE-based multilingual platform incorporating a JSR-238 repository with a custom workflow/process approval engine. Because they could build out something for about 20 percent of that (or sometimes 1/2 a percent of that), and hire a few editors to wrangle the content. The content, were it approached strategically, could be of far higher quality—better SEO, more durable, consistent voice, vetted for legal compliance, primed for re-use. And you can make an end-run around workflow if you add versioning and reversion capability to your text fields (like Wikipedia), give most users the ability to edit, and give the editor full revert and publish privileges. Most CMSes are parasitic technologies dedicated to preserving the cultural and hierarchical status quo of their hosts no matter the cost, literally. People hear me whine about this and they say: Our case is different; we need to have a system that sends out seven thousand "todo" emails per day. And I grieve for the spirit of Work, killed by her evil child, Workflow.

That's it. This of course is already too long because I don't have an editor. Sadly. But to summarize: Good conversation between Taylor and Johnson. Editors ship. There's no place to hire the nerdier ones because the Awl won't set up a job board. That's sad. The web is changing and it needs more editors. Do not dispute me. I love you. Goodbye.