Bigger Faster Stronger

Scalability is one of those words that everyone uses but few understand. It's a measure of how adding resources (typically hardware) affects performance. You can scale vertically by increasing the power of a server. You can scale horizontally by adding servers. The scalability of a system depends on how performance is defined. Martin Fowler suggests a few categories:

Expect more posts on this one

• Response time, or the amount of time it takes to process a request
• Responsiveness, or the amount of time it takes to acknowledge a request
• Latency, or the amount of time it takes to get a response (this is especially important when there is no data to return)
• Throughput, such as transactions / second
• Load, or the amount of stress a system is under
• Load Sensitivity, or response time / load
• Efficiency, or performance / resources
• Capacity, as in maximum throughput or load

Systems must be designed to scale, but what scaling means will depend upon the purposes for which a system is built. It might be tempting for database professionals to think about scalability in terms of transactions / second or the number of active accounts. But what really matters is whether or not the system is usable given an increase in transactions or accounts, and this depends upon the use for which the system was created. If we're talking about an e-Commerce system, throughput is probably more important than response time, as long as responsiveness is high. If we're dealing with a manufacturing system, we'll probably be most interested in throughput.

It's important to design systems to be scalable. The Internet has increased adoption rates to unprecedented rates. Consider Instagram, which has 30 million users after 2 years. Draw Something had 36 million users in three weeks. Scalability is a prerequisite for virality.

In the case of N-tier applications which have a Service-Oriented Architecture, it's usually easy to add hardware to the web and application servers. Load balancers and web farms can take care of extra load by distributing it evenly across a number of servers. The real problem is, as always, the database layer.

You can't just add servers to the database layer, because databases must be architected across multiple database servers. Concurrency adds to the difficulty, as database transactions must be ACID (atomic, consistent, isolated, and durable). In other words, you have to manage updates to multiple servers, making sure that an update to Server 2 does not depend on Server 1.

Lighting bolts make it faster

I thought the Cloud might be the solution to database scalability, but Microsoft Azure currently supports databases of only 150 GB in size. In talking with Microsoft consultants, they recommend 'sharding' databases. This means having a master database that directs transactions to the appropriate database server. For instance, all transactions dealing with North American accounts should go to Server 1, South America to Server 2. Sharding adds a layer of abstraction and a layer of complexity, and it requires duplication of database schema, but it's an increasingly popular approach.

Another option is Oracle's RAC system or Microsoft's MatrixDB, which has basically been ported to Azure. I'm skeptical that MatrixDB will make it in to the next edition of SQL Server (2012 has AlwaysOn, which is close, but the mirrors are read-only). In RAC or MatrixDB, databases are replicated across multiple servers and a load balancer directs reads and writes to the server with the least load. Changes are replicated asynchronously between database servers. Still, there are limitations to the size of databases for which this would be feasible.

Relational databases are great up to a certain size (though this is growing, thanks to SSD's and improved caching). It's hard to say exactly what this size is. In the end, scalable databases adhere to principles of normalization and partitioning. After a certain amount of data, RDMS's will be of no use, and NoSQL solutions are the answer to a different problem. Are you ready to scale?

Technology and Collective Problem-Solving

"Technology" signifies all the intelligent techniques by which the energies of nature and man are directed and used in satisfaction of human needs; it cannot be limited to a few outer and comparatively mechanical forms.
--John Dewey

In a previous post, I explained how many philosophers, including Heidegger and Marcuse, see a rift between ethical reflection and technology. They worry that the means-ends thinking at the heart of technology can cause us to ignore other kinds of reflection--especially about who we want to be, what we hold to be just, and how we can lead more meaningful lives.

There is obviously a difference between painting a picture and developing a manufacturing plant to make paints and brushes, but what's wrong with solving problems? Is it really so dangerous as philosophers--who aren't typically known for being technologists--seem to think? John Dewey says no, arguing that all inquiry has a technological component insofar as it is meant to solve problems. If moral inquiry helps us solve problems, it's as technological as lasers and airplanes are. Theories are just tools for solving problems.

Understanding technology as problem-solving may seem impossibly vague, but it's actually very powerful. Whenever considering a new gadget, theory, or way of doing things, Dewey suggests we ask: what is the problem this is meant to solve? Remarkably, many new products don't seem aimed at solving any problems, or at least not any serious ones.

What about the problem of collective decision-making? Humans have created two lasting technologies for this purpose: representative governments and markets. Governments are good at ensuring certain behaviors that its people think should be ensured. They define and enforce justice, including the means of determining what justice is. This wasn't always the case and took many years of trial and error. Life used to be filled with a lot more anxiety, because the world was so much more unpredictable, and the means of determining fairness were uncertain.

Althingi, where Icelanders have solved problems since 930 CE
Governments, however, can only solve certain problems. They're bad at picking market winners, for example, and they're slow to react to change. They are good at prohibiting certain behaviors, but it's hard for them to make citizens moral, healthy, intelligent, or cultured. As Cass Sunstein argues in his book Nudge, the best governments may be able to do is incentivize certain behaviors so that people will make the right choices on their own.

Markets, on the other hand, provide a highly responsive way of determining what people value and what should be produced. As Friedrich Hayek recognized, markets aggregate people's individual choices and values and thus collectivize intelligence in a very efficient manner. Markets will always have the input of more people than governments as well as higher levels of participation. And, since people often know what they want better than 'experts,' markets can be more rational than governments.

Unfortunately, many things cannot be quantified in dollar values, such as the environment, health, or justice. We can adjust markets so that they take hidden costs into account, as cap-and-trade systems do, but these work best when you have a metric that can be easily tied to cost. Another criticism of markets is that people do not always act rationally, as Daniel Kahneman and other behavioral economists have shown. Even if we know what we want, we can't be sure to act accordingly.

Given the limitations of governments and markets, Deweyans turn to small groups for salvation. There are many interesting examples of small-scale collective problems solving, such as the rebirth of Pittsburgh or river management in Mexico, but it's hard to see how such solutions will scale. As our interactions become ever more global, we need globalized methods of collective decision making.


For these reasons, Clay Shirky and other technologists point to the internet as a possible third way of making intelligent choices collectively. It's not enough to say that the internet connects people. The idea of the internet as a 'Global Village' has become a joke, as new technologies help us filter each other out like never before. What Shirky points to is the way the internet lowers barriers to participation. Shirky's poster child is Wikipedia, which, like most internet phenomena, displays a long tail of participation. Many people work together, though the vast majority only contribute a little.

Lowering barriers is great, but it is probably not enough if we are to find a third way to compete with governments and markets. Can new technologies help us better solve collective problems? The question becomes ever more pressing as big players like Google, Microsoft, and Facebook become ever bigger and structure the ways we interact more and more. Not being evil is not the same thing as providing venues for increasing collective intelligence. What other problems should we be trying to solve?

Gödel, Kuhn, and Human/Computer Co-evolution

Kurt Gödel showed that all formal systems are incomplete, meaning that there are truths that can be stated within a system that cannot be demonstrated within it. This means that truth cannot be reducible to axiomatic principles and, therefore, that there is no single system to which all truths could be reduced, whether it be physics, mathematics, logic, or any other system of abstraction.

These principles have become widely accepted within mathematical and scientific communities. After Thomas Kuhn's Structure of Scientific Revolutions, it has become common to think of domains of knowledge as paradigms which are never proven or disproven but which are eventually abandoned in favor of new, more helpful, or more interesting paradigms. What makes a paradigm valuable cannot be shown within the paradigm itself but by the problems it helps solve or the vistas it allows.

I've been trying to think through the implications of these insights for technology ever since reading Douglas Hofstadter's Gödel, Escher, Bach. As I understand it, the book's main point is that any attempts to create artificially intelligent systems that are grounded solely in deductive logic are doomed to failure. Human thought involves deduction, but it involves much more. As Hofstadter writes in his introduction to Ernest Nagel and James Newman's Gödel's Proof, the goal of AI research should be to devise systems "guided by visual imagery, the associative patterns linking concepts, and the intuitive processes of guesswork, analogy, and esthetic choice that every [person] uses."

The kind of AI we are most familiar with is Google. It is probably a stretch to say that the Google engine thinks, but it is certain that the algorithms it uses to filter and aggregate trillions of bits of information are guided by analogies and associations if not esthetic choice. Google gets feedback from users in the form of click-throughs, so it can better predict what sites to show in the future. Feedback loops like this are at the center of thought and learning. The more the Google engine can modify itself instead of relying on engineers to tweak algorithms, the more it can be said to think.

But the computer side of this loop is only one half of the picture. Just as computer systems learn from us, we learn from the systems. It's a dynamic, co-evolutionary process, and we need to think about the kinds of choices that computers make available to us. Eli Parser has coined the term 'filter bubbles' to describe the ways that applications like Google and Facebook filter information for us and thus structure the choices we make. This is not necessarily a bad thing, since we desperately need ways of filtering out information that is not relevant to our purposes. However, it will be a co-evolution that requires careful attention, for, unlike the bee and the flower, we can control our collective destiny.

This brings me back to Kuhn, since he suggests that the terms we currently use to think about our co-evolution are not necessarily the best for the job. One of the interesting things about computing is the way its concepts and language have been used to understand other things. Metaphors from other walks of life permeate computing, like 'the cloud', 'friend', or 'stream', but computing has also shaped the ways we understand each other and the world. It's not simply a matter of words like 'Google', 'text', and 'filter' becoming commonplace, because everything is either a computer or something to be computed today. For example, it's hard not to think of human minds (or 'wetware') along computational lines now. Not only do we 'process data' and 'filter out noise', but we act on information that has been computed so that we can better operate computers. Commercials tell us that our very personhood is threatened if our personal data are lost.

I wonder if we're seeing the waning of a paradigm, or the waxing of a new one. Will the next years of computing see more of the same, only faster? Or will we come to understand thought, action, art, ethics, and even humanity in an entirely different, techno-saturated vein? Are computers just machines (with which have have been long familiar--we are eating machines, sex machines, and poop machines). Or are they something different?

Links:
-Jaron Lanier's You Are Not A Gadget

Erlang and Concurrency

I was excited to learn me some Erlang, since the mysterious language has recently gotten a lot of press for being the tool of choice for Facebook Chat. It's often cited as a way to break the performance barriers of multi-processor architecture and mainstream programming languages that use multi-threading. A program is as fast as its slowest part, and this is typically the resources shared by threads. These resources also require some fancy coding techniques to ensure their integrity. And, of course, the more complicated the code, the more likely it is to break and be difficult to maintain.

Erlang's simple model is based around processes that pass messages to each other, crash, and respawn very quickly. After compiling a module, you spawn a process using:

handle = spawn(module, function, parameters).

The process is defined as a function that shreds out the parameters with a series of case statements:

function -> receive
	{parameter1} ->
		%do something
	{parameter2} ->
		%do something else
	Unexpected ->
		%handle exceptions
end.

You can pass messages to a spawned process using:

handle ! parameters.

This architecture allows you to quickly create, monitor, message, and respawn processes whenever they fail. Because Erlang is a functional language, there are no variables or any other shared resources that can form a bottleneck between processes liked global variables. Processes communicate using messages, and these can be processed asynchronously or independent of any other processes.

Erlang's concurrency model is similar to Service Oriented Architecture, except that concurrency is built into the system at the lowest level, not added on top of an Object Oriented framework and requiring several other technologies. Note also that concurrency is very different from parallelism. If you need to crunch a lot of numbers, you'd need to use a parallel processing system, not a concurrency model.

Erlang is best for soft real-time, distributed, and highly-available applications that could be composed of message-handling systems, like Facebook Chat or low-level telecommunications software. (These are very specific applications that I tend not to think about very often, so I don't have a mock-up like I have tried to have in other posts.)

For these reasons, Erlang is not a cure-all for performance-related problems, though it is important to note that there are limits to any concurrent or parallel approach. Amdahl's law states that even with 95% parallelism, performance benefits will quickly plateau as the number of processors increase.  E.g., if your program takes 20 seconds to run, and 1 second of it cannot be parallelized, the least amount of time it could run in is 1 second.  No matter how many processors you add, you cannot break that 1 second barrier without further parallelizations.


As a side note, I had not heard of Amdahl's law before researching Erlang. Interestingly, I also just learned of IBM's SyNAPSE project, which has the grand aim of functionally replicating the human brain. One argument for a new hardware architecture based on neural networks is that, though the firing of individual neurons is slow, the massively parallel processing power of a brain more than makes up for the performance of any individual part. I'm not sure how this argument stacks up against Amdahl's.

I'll be looking for opportunities to write me some Erlang, but--given my own limits to processing in parallel--this may not be any time soon.

Links:
-Erlang home page
-Eugene Letuchy's notes on designing Facebook Chat
-Learn You Some Erlang, a free online book