Wednesday, December 16, 2009

Comet is dead long live websockets

I've just had a chance to play with the implementation of websockets
in Googles Chrome browser. This post started me off.

After a small amount of experimentation I was able to make Erlang talk to a web page using pure asynchronous message passing.

I think this means the death of the following technologies:
  • comet
  • long-poll
  • AJAX
  • keep-alive sockets
All the above are merely hacks, inadequate ways of programming round the central problem that web-browsers could not simply open a socket and do asynchronous I/O like any other regular application.

Well now things have changed, web sockets are here

It's not a question of if long-poll etc. will die, it's just a question of when.

Once the handshake is over, web sockets work more or less like regular sockets. The important point to note is that the overhead involved in managing a web socket is minimal. In interactive applications, for example, when providing auto-complete of characters that are typed into a buffer, the current AJAX technology imposes an enormous overhead since virtually all the time in the server is spend parsing HTTP headers.
Pushing data to a client has for years been a total mess. Attempts to circumvent this have involved things like long-poll and comet, or even simply just polling the server on a regular basis. All this is now irrelevant.
So let's see what this means with an Erlang example

Sending messages to a region in a browser from Erlang is extremely easy using websockets.

Add this to your web page:
<div id="tag"></div>
Then do this in Erlang:
Browser ! {send, "tag ! XXX"}
XXX will appear in the div

The server code

start() ->
F = fun interact/2,
spawn(fun() -> start(F, 0) end).

interact(Browser, State) ->
{browser, Browser, Str} ->
Str1 = lists:reverse(Str),
Browser ! {send, "out ! " ++ Str1},
interact(Browser, State);
after 100 ->
Browser ! {send, "clock ! tick " ++
interact(Browser, State+1)
To send a message to a tagged region tag in the browser I just say Browser ! {send "tag ! message"} this could hardly be easier.

The details

If you want to try this yourself you'll need a version of chrome that supports web sockets. Then two files local_sever.erl and interact.html

They are listed at the end of this article:

To run things you'll need a local web server and Erlang.

Compile local_server.erl and start by evaluating local_server:start(). Then serve the web page interact.html from your local server.




%% start()
%% This should be in another module for clarity
%% but is included here to make the example self-contained

start() ->
F = fun interact/2,
spawn(fun() -> start(F, 0) end).

interact(Browser, State) ->
{browser, Browser, Str} ->
Str1 = lists:reverse(Str),
Browser ! {send, "out ! " ++ Str1},
interact(Browser, State);
after 100 ->
Browser ! {send, "clock ! tick " ++ integer_to_list(State)},
interact(Browser, State+1)

start(F, State0) ->
{ok, Listen} = gen_tcp:listen(1234, [{packet,0},
{active, true}]),
par_connect(Listen, F, State0).

par_connect(Listen, F, State0) ->
{ok, Socket} = gen_tcp:accept(Listen),
spawn(fun() -> par_connect(Listen, F, State0) end),
wait(Socket, F, State0).

wait(Socket, F, State0) ->
{tcp, Socket, Data} ->
Handshake =
"HTTP/1.1 101 Web Socket Protocol Handshake\r\n",
"Upgrade: WebSocket\r\n",
"Connection: Upgrade\r\n",
"WebSocket-Origin: http://localhost:2246\r\n",
"WebSocket-Location: ",
" ws://localhost:1234/websession\r\n\r\n"
gen_tcp:send(Socket, Handshake),
S = self(),
Pid = spawn_link(fun() -> F(S, State0) end),
loop(zero, Socket, Pid);
Any ->
wait(Socket, F, State0)

loop(Buff, Socket, Pid) ->
{tcp, Socket, Data} ->
handle_data(Buff, Data, Socket, Pid);
{tcp_closed, Socket} ->
Pid ! {browser_closed, self()};
{send, Data} ->
gen_tcp:send(Socket, [0,Data,255]),
loop(Buff, Socket, Pid);
Any ->
loop(Buff, Socket, Pid)

handle_data(zero, [0|T], Socket, Pid) ->
handle_data([], T, Socket, Pid);
handle_data(zero, [], Socket, Pid) ->
loop(zero, Socket, Pid);
handle_data(L, [255|T], Socket, Pid) ->
Line = lists:reverse(L),
Pid ! {browser, self(), Line},
handle_data(zero,T, Socket, Pid);
handle_data(L, [H|T], Socket, Pid) ->
handle_data([H|L], T, Socket, Pid);
handle_data([], L, Socket, Pid) ->
loop(L, Socket, Pid).

And the web page interact.html
<script src='/include/jquery-1.3.2.min.js'></script>

var ws;

if ("WebSocket" in window) {
debug("Horray you have web sockets
Trying to connect...");
ws = new WebSocket("ws://localhost:1234/websession");

ws.onopen = function() {
// Web Socket is connected. You can send data by send() method.
ws.send("hello from the browser");
ws.send("more from browser");

run = function() {
var val=$("#i1").val(); // read the entry
$("#i1").val(""); // and clear it
ws.send(val); // tell erlang
return true; // must do this

ws.onmessage = function (evt)
var data =;
var i = data.indexOf("!");
var tag = data.slice(0,i);
var val = data.slice(i+1);
$("#" + tag).html(val);

ws.onclose = function()
debug(" socket closed");
} else {
alert("You have no web sockets");

function debug(str){
$("#debug").append("<p>" + str);


<h1>Interaction experiment</h1>

<div id="debug"></div>

<div id="clock">I am a clock</div>

<div id="out">Output should appear here</div>

<p>Enter something in the entry below,
the server will reverse the string and send it to the
out region above</p>

<P>Enter: <input id="i1" onchange="run()" size ="42"></p>



lispmeister said...

The pain - it's fading! Wow, this is really beautiful. I've been waiting for this ever since the second web conference in '94.

Anonymous said...

Yes this is a lot better that the comet and long-polling hack where we try to fake synchronous communications. AJAX is not going anywhere however. AJAX is used where one wants to update parts of the page without reload in a request-response manner. For example a comment form where a user posts a comment and it appears in the comment list without reload. AJAX solves that problem wonderfully and without any major problems. It's easy to make it degrade gracefully(non-js browsers for example) and it's a good and solid solution that requires nothing from the web server. It's a use case where HTTP solves the problem in a good way. Using Web Sockets there seems to me like adding complexity and code bloat. HTTP is a good standard and it should be embrased where it's strengths are.

Websockets is however great news. I look really forward to testing it out. The idea of trying out what I can do by combining it with the HTML5 Web Workers is giving me a warm fuzzy feeling already. I am however painfully aware of the fact that this is not a standard yet and only available in V8. TraceMonkey may support it soon'ish. Not when SquirrelFish will move to it but it may be soon as well. Opera's Carakan and Microsoft's JScript engines I am afraid will not support this in the forseeabe future. This will mean that again we have to rely on some ugly hacks. And the hack for this will be ugly beast indeed. We are talking about javascript that will failover to using comet and things are worse than that as the failover will have to exist on the server as well.