Erlang FTW!

If you write code and you haven't learned Erlang, you should do so, right now. It will stretch your thinking, for sure, but it may do much more than that.

Erlang represents a fundamental shift in the way we think about solving problems. The fact that it is a functional language is part of the necessary reality that makes Erlang successful, but it is not the primary reason. What makes Erlang very special is its concurrency. Erlang is a language for "concurrent" programming and a virtual machine (like Java) for running compiled Erlang code.

The Java language is almost always considered in light of its runtime environment, the JVM. Erlang even more so, it would not be what it is without the runtime that implements it. So when I say Erlang, I mean the whole package, not just the language.

Erlang is oriented around concurrency on a massive scale. The fundamental unit of thinking in Erlang code is the process. One Erlang interpreter/virtual machine can handle millions of concurrent processes. The processes interact by passing immutable messages (everything is immutable BTW, Erlang uses single-assignment variables).

Guess what? The Internet IS concurrency. Practically any piece of software connected to a network these days must handle concurrency. Most languages do a pathetic job of this (or no job at all). They don't give us tools to build concurrent systems with any kind of ease. Most languages make it quite difficult, in fact.

Erlang makes it so easy and so natural that you begin to think about solving problems differently. The ability to run millions of concurrent processes can radically change your approach to a problem. What are social networking and large user-driven sites always running into? Scalability. How do you scale with all these users?

There are two problems here, the first being how do you scale to handle the load, and the second being what do you have to sacrifice in terms of code elegance, simplicity, etc. to get there? Will the complexity of your system grow with the scale? More little optimizations that disguise the real logic of the application?

What if you could run a unique process for every single user on your system? How would that change your approach?

That is what Erlang makes possible. Instead of a pool of workers crunching away in serial, individualized processes working in parallel, on a massive scale.

Plus, it is built to cluster, and all of its message-passing works identically across a cluster as it does locally. So if one machine can handle a million or millions of processes, how many processes could you handle on a large cluster?

One of the exercises in the Erlang book is to write a process ring benchmark. The idea is that you create a ring of N processes and then pass M messages through the ring and see how long it takes. On my Mac Pro (couple years old, quad-core 2.4GHz, 6G RAM), I ran the benchmark with 1000 messages and 1 million processes (!!). That means Erlang passed 1 billion messages. The benchmark ran in 8 minutes, which boils down to 2 million messages every second, or 2 messages every 1 microsecond. That's amazing. And powerful.

Another interesting tidbit: the statistics report "run" time and user time. User time was half of the run time. Why? Single-assignment + message passing = no locking, no contention = SMP-friendly. Erlang is built to run on multi-core computers. So it spent twice as much time "on processor" as it took in "real life" because it was using the multiple CPUs/cores offered by the Mac Pro.

Here's the code:

-module(ring).
-compile(export_all).

run(M,N) ->
   Pid = start(N,M,self()),
   statistics(runtime),
   statistics(wall_clock),
   send_messages(Pid, M),
   receive
      finished ->
         void
   end,
   {_, Time1} = statistics(runtime),
   {_, Time2} = statistics(wall_clock),
   io:format("~p ~p ~n", [Time1, Time2]).

start(N,M,RunPid) ->
   spawn(fun() -> start_master(N,M,RunPid) end).

send_messages(Pid, M) ->
   for(1,M,fun() -> Pid ! {init, message} end).

for(M,M,F) ->
   F();
for(N,M,F) ->
   F(),
   for(N+1,M,F).

start_master(N,M,RunPid) ->
   LastPid = start_ring(self(), N),
   master_loop(LastPid, RunPid, M).

start_ring(Pid, 1) ->
   spawn(fun() -> loop(Pid) end);

start_ring(Pid, N) ->
   NextPid = spawn(fun() -> loop(Pid) end),
   start_ring(NextPid, N-1).

loop(NextPid) ->
   receive
      stop ->
         NextPid ! stop;
      Any ->
         NextPid ! Any,
         loop(NextPid)
   end.

master_loop(Pid, RunPid, 0) ->
   RunPid ! finished,
   io:format("Finished~n"),
   receive
   after 100 ->
      Pid ! stop
   end;

master_loop(NextPid, RunPid, M) ->
   receive
      stop ->
         NextPid ! stop,
         io:format("Master stopping ~n");
      {init, Msg} ->
         NextPid ! Msg,
         master_loop(NextPid, RunPid, M);
      _Any ->
         master_loop(NextPid, RunPid, M-1)
   end.