Algorithms, AI, and Emergence

The story about why Google overtook AltaVista is a fascinating one. Up until Google, there were two ways of accessing web pages. The first was Yahoo!'s portal approach, in which a single authority determines the web pages worth viewing, and you trust the experts to have done their work. You want to read sports news? Go to Yahoo! and then the Sports category.  Simple.

Lightning fast!

The obvious problem is that the experts at Yahoo! might have missed something. For this reason, I always preferred AltaVista (or WebCrawler, before that), which used an indexing approach, similar to an index in the back of a book. AltaVista used simple programs to search for key terms in a page's text and metadata and then aggregate that data in a way that later could be easily matched and ranked according to search terms you fed it.

AltaVista was better than the alternatives for a few years, but it had trouble determining relevance. If you're searching for a recipe with certain ingredients, you don't want just any recipe; you want the best recipe. If you want to read about last night's game, you don't want just anyone's commentary; you want the best commentary.

Enter Google's PageRank algorithm. PageRank takes authority into account, like Yahoo!, but it doesn't require a single source of information, like AltaVista. Pages that have more links to them are ranked higher than ones that don't. And the pages that link to these pages give an even higher ranking if many pages link to them. PageRank lets you find what the most important people are talking about.

.304 (at the center) is the highest page rank in this contrived example

It's a bit more complicated than this, but what is revolutionary here (and I use the term revolutionary sparingly, unlike, ahem, Apple) is that Google's algorithm did not try to reproduce the intelligence of the web that was already out there. Instead, it tried to capitalize on the accumulated intelligence of millions of individuals who had made a simple choice: to link to or not link.

Linking is a binary action that, when accumulated into a large network of linked pages, allows for an emergent kind of intelligence. Emergence happens when a simple set of rules add up to something greater than their parts. Ants, for example, are very stupid individually, but when they signal to each other using pheromonic rules that have evolved, some very complex behavior emerges. Similarly, neurons in themselves have no intelligence. They can either fire or not. But when you have a few hundred billion of them, you're on your way to self-consciousness. Markets too display emergent phenomena when the buying and selling of millions of people shape the direction of societies.

PageRank is just one of a few algorithms that have begun to capitalize on emergence. For decades, it was assumed that language translation software would have to know the rules of grammar of the source and destination language in order to decode and encode meaning appropriately. This was a miserable failure. The success of modern translation algorithms is that they use statistics to compare millions of existing translated documents. The algorithm knows that, for example, it has seen 'Je suis' much more frequently than 'Je es'.  It doesn't need to know anything about grammatical rules.

An ant fugue

The problem with algorithms like PageRank is that they are parasitic upon an existing emergent system, whether it is a network of linked pages or a network of translations. The algorithms themselves are not emergent. They require human maintenance and tweaking, and they lead to people gaming the system. PageRank spawned web spam, in which people post links to sites in order to increase their ranking. This is why it's now only one of 200 or so indicators of rank for Google. The whole point of Gödel, Escher, Bach is that intelligence cannot be decomposable into a set of rules. Algorithms like PageRank are still just that.

While we've made some tremendous advances in machine learning by fitting algorithms to emergent systems, the future of AI can't be algorithmic. It can't be only a set of rules. But it might be a set of rules that, when used in aggregate, lead to emergent and artificial intelligence. What set of rules could be this flexible? Perhaps only the binary links of neural networks.

Music for Computers

ENIAC boards or early keytars?

I've been thinking about human - computer interaction, and it strikes me that the philosophical questions central to early Artificial Intelligence research do not seem to have much resonance any longer. In the 50's and 60's, research into AI led to many innovations in computing. But it also caused people to worry about the future of humanity. Joseph Weizenbaum, designer of the ELIZA program, which seemed so close to passing the Turing Test half a century ago, afterwards turned against AI research due to its imagined consequences. The most striking image of the computer that the 60's left us with is HAL, an embodiment of the singularity in which human life is no longer rational.

But now, the singularity is mostly the stuff of Hollywood, filling in for Nazi's or Communists. The philosophical question that arises from the the Matrix is, 'Dude, what if we're in the Matrix?' The doomsday scenario today is a product of human hubris (man-made climate change, bio-medically-engineered zombies, or radiation-induced mutation), not thinking machines. The singularity, according to futurologists like Ray Kurzweil, is something to be embraced, not feared

I think the main reason the mood has changed is that computers are no longer as alien as they once were. They used to be huge machines that had to be talked to with 0's and 1's. They were very smart in certain respects, but unfathomably stupid in others. They existed in places like MIT, not the living room or minivan. In the 70's, however, computers started to become a part of our everyday lives. One way this happened was through music.

I used to have one of these

The 80's might have been the decade in which electronic music became mainstream, but it was the 70's when new technologies like synthesizers, drum machines, and tapes were really explored and developed. Here's a broad sample of some early electronic music. It's not always made by computers, but I often listen to it in front of one. Check out:

Sly and the Family Stone - It's a Family Affair - First use of a drum machine.

Brian Eno - Big Ship - Pioneering warm, synthesized sound.

Kraftwerk - Robots - Hilarious 'live' show.

Donna Summer and Giorgio Moroder - I Feel Love - Post-disco is a lot better than disco.

Einojuhani Rautavaara - Cantus Articus - Tape machines and orchestras unite.

In sum, the philosophical questions of early AI have faded away with the growing ubiquity of computers. Computers have become products and extensions of our thought, not alternatives to it. We already know what computers are. They send email and show us YouTube videos. We stare at their spreadsheets all day long at work. I wonder if computers have become too near for us to ask questions about their objective value. But perhaps this is just a story of a very abstract question ('Will computers one day think?') becoming more concrete ('How can we make a search engine that knows what you want even if you don't?' or 'How can we make a better-sounding guitar?').

Gödel, Kuhn, and Human/Computer Co-evolution

Kurt Gödel showed that all formal systems are incomplete, meaning that there are truths that can be stated within a system that cannot be demonstrated within it. This means that truth cannot be reducible to axiomatic principles and, therefore, that there is no single system to which all truths could be reduced, whether it be physics, mathematics, logic, or any other system of abstraction.

These principles have become widely accepted within mathematical and scientific communities. After Thomas Kuhn's Structure of Scientific Revolutions, it has become common to think of domains of knowledge as paradigms which are never proven or disproven but which are eventually abandoned in favor of new, more helpful, or more interesting paradigms. What makes a paradigm valuable cannot be shown within the paradigm itself but by the problems it helps solve or the vistas it allows.

I've been trying to think through the implications of these insights for technology ever since reading Douglas Hofstadter's Gödel, Escher, Bach. As I understand it, the book's main point is that any attempts to create artificially intelligent systems that are grounded solely in deductive logic are doomed to failure. Human thought involves deduction, but it involves much more. As Hofstadter writes in his introduction to Ernest Nagel and James Newman's Gödel's Proof, the goal of AI research should be to devise systems "guided by visual imagery, the associative patterns linking concepts, and the intuitive processes of guesswork, analogy, and esthetic choice that every [person] uses."

The kind of AI we are most familiar with is Google. It is probably a stretch to say that the Google engine thinks, but it is certain that the algorithms it uses to filter and aggregate trillions of bits of information are guided by analogies and associations if not esthetic choice. Google gets feedback from users in the form of click-throughs, so it can better predict what sites to show in the future. Feedback loops like this are at the center of thought and learning. The more the Google engine can modify itself instead of relying on engineers to tweak algorithms, the more it can be said to think.

But the computer side of this loop is only one half of the picture. Just as computer systems learn from us, we learn from the systems. It's a dynamic, co-evolutionary process, and we need to think about the kinds of choices that computers make available to us. Eli Parser has coined the term 'filter bubbles' to describe the ways that applications like Google and Facebook filter information for us and thus structure the choices we make. This is not necessarily a bad thing, since we desperately need ways of filtering out information that is not relevant to our purposes. However, it will be a co-evolution that requires careful attention, for, unlike the bee and the flower, we can control our collective destiny.

This brings me back to Kuhn, since he suggests that the terms we currently use to think about our co-evolution are not necessarily the best for the job. One of the interesting things about computing is the way its concepts and language have been used to understand other things. Metaphors from other walks of life permeate computing, like 'the cloud', 'friend', or 'stream', but computing has also shaped the ways we understand each other and the world. It's not simply a matter of words like 'Google', 'text', and 'filter' becoming commonplace, because everything is either a computer or something to be computed today. For example, it's hard not to think of human minds (or 'wetware') along computational lines now. Not only do we 'process data' and 'filter out noise', but we act on information that has been computed so that we can better operate computers. Commercials tell us that our very personhood is threatened if our personal data are lost.

I wonder if we're seeing the waning of a paradigm, or the waxing of a new one. Will the next years of computing see more of the same, only faster? Or will we come to understand thought, action, art, ethics, and even humanity in an entirely different, techno-saturated vein? Are computers just machines (with which have have been long familiar--we are eating machines, sex machines, and poop machines). Or are they something different?

Links:
-Jaron Lanier's You Are Not A Gadget