Up: Colorless Green Ideas

[Related]   «^»   «T»

Time's Inverted Index

By Paul Ford

(1) When robots write history we can get in trouble with our past selves. (2) Search-generated, "false" chrestomathies and the historical fallacy.

I use a number of tools to sort things through. For example, I've made my email searchable back to 1995. I can type mailsearch f:ford procrastination and in response two commands run (mairix and mutt), and up pops a terminal window filled with 15 years of emails, from me to other people, about putting things off. When I set this up I hoped it would be an aid to memory, a way to be smarter.

It's overwhelming at times to reach back across a decade-and-change and find so much, so fast (the mairix program, being local, is much faster than the same Google mail search). It brings those years back, quicker than memory. I can't imagine what it will be like in a decade or two. While right now it's unusual in general population for a person to have all this history so close, so quickly searchable, obviously the world will go this way. There will be many new forms of art and commerce over time, I think, that allow us to interact with, and share from, our private archives. There is going to be an urgent market need for tiny mechanical historians who can live in our pockets and point out our flaws.

Slicing open my own corpus, I've been surprised—and upset—to find that ideas I had then were very similar to the ones I have now. I found that my thinking on things like time, technology, or human behavior hasn't changed much since I was a 22-year-old studying document markup languages in the dark while writing advertising copy in the daytime.

At first I was pleased, not upset, noticing this. “Look at that,” I thought, reading through thirty emails resulting from a search for “e-readers.” “I've really been on this beat for a long time.” But it kept happening: I'd have an idea, search through my archive, and find that I'd already had that idea, some variation on it, six years ago. I was, without a doubt, repeating myself. Spinning the wheels of my hobbyhorses. It was depressing, because I have a major need for progress right now. These search results showed me that it will come slowly if at all.

And I don't recall myself fondly. I see Paul-1996 as an inveterate and unrepentant wearer of loose-fitting vests, untucked shirts, and a knit cap—a lost dog trotting around New York City with his tongue out, hoping for a pat on the head. But here, in my searches, was a somewhat vital creature, tongue unlolled, concerned about deadlines and doing hard work. This younger I seemed far more engaged than the venal goon I remembered. Why did I want to judge him as lacking? Jealousy? Fear that what was lacking was my own progress? I walked around for months bothered by this. Finally I realized the problem: The issue was not with me but with my software.

I was biasing the results by using full-text search to explore my email. I would look for the things I found interesting that day—searching on terms like “Google” or “literature” or “e-reader”—and see a chronological list of exactly what I said about those very terms. The pattern-seeking engine in my brain would fire on all cylinders and make a story of the searches, creating an unintentional email-chrestomathy, a greatest-hits collection of ideas I'd had around a single word or phrase. The results seemed weirdly definitive. I thought I was doing history in a mirror, but because the emails were pure matches for key terms, devoid of all but a little context, I fell for the historical fallacy, which is when, as John Dewey described it, somewhat impenetrably:

A set of considerations which hold good only because of a completed process, is read into the content of the process which conditions this completed result. A state of things characterizing an outcome is regarded as a true description of the events which led up to this outcome; when, as a matter of fact, if this outcome had already been in existence, there would have been no necessity for the process.

That is, I had lost sight of time. Search—especially search across such a limited corpus as one fellow's email—made it seem that I had been on a path when I hadn't been, and made me, the outcome, seem inevitable. And if I was inevitable, well then, why bother?

It is not the fault of the software, which is by definition unwitting. No one writing code said, “Let's totally mess with his perception of self and understanding of free will.” The code sees an email that is filled with words as something like a small bucket that is filled with coins. It takes the coins out one by one and stacks them in squares marked out for each denomination, adding a slip of paper between each coin that indicates the coin's bucket of origin. Then it does the same with the next bucket. There are many denominations of coins; a truly big table, called an inverted index, is needed to hold them all. The coin-sorting program cannot think or advise, but it does allow you to go to a pile of coins on the table, pick them up, and know their buckets of origin. It's my unrigid brain that turns a sorted stack of coins into a story—so there I was, querying against my life, tapping the touchscreens of fate and clicking the mouse of destiny, all of it suffused with a sort of sweet nostalgia. (Of my emails, 893 mention libraries; 311 mention libraries and love.)

To prove that I was prey to this fallacy I picked a few random days and looked at all the emails I generated. And yes. There was, without the acute knife-edge of a search query slicing my life, a wealth of goofiness, a catalog of wasted flirtations and dumb thoughts and mistakes made, all displayed without consciousness of the future. A conversation organizing a cup of coffee that lasted thirty emails; a boring description of a just-purchased shirt. It was exhausting to read what this banal nerd had to say, except for one thing: The old me was so damned hopeful. I ended up liking him even if he did borrow grieving, even if his ego was a great swollen balloon that he dared the world to pop. (Often the world did. Sometimes he put the balloon in his mouth.) Unlike the portrait of self that emerged from my tightly constrained searching, this fellow was hard to classify. He was alive in his own moment, not mine. What I had assumed to be artifacts of destiny were simply precedents.

I had, as you would expect, the fantasy of sitting down with my 1996, 22-year-old self, for just a minute or two. Before today I would have wanted to tell him things, give him nostrums for his ailments—give a lecture, with the 14-years-later me at the podium. But reading the younger I's emails I can't imagine him being such a good listener as all that. I imagine we'd both start talking at once, each angling for the other's respect and admiration. Reading his old messages, even though they were intended for someone else, I know that he wanted more life, more love, and more space to think, and I've given him, imperfectly, not always willingly, but best I could, all three. Still, I hope he would forgive me that I've gone exactly this far, but no further, with my life. And he would. He more than anyone would understand how uncertain things can be, how slow progress can come. He would peek in with tremendous, understandable curiosity at my wife as she slept in the other room, at her dark silhouette. “Huh,” he would say, oblivious but always helpful. “That's good to know.” And soon after he would drift back into the realm of chaos or serendipity where our past selves mill, on his face that sad smile, so familiar from movies with ghosts, of those who fade.