F# and Silverlight

I finally took some time to learn enough about Silverlight to try to create a little F# demo to attempt to host in my blog.  It turns out that Windows Live Spaces has a bunch of content restrictions that appear to prevent embedding Silverlight directly, so I hosted it elsewhere instead.  Click here to go have a look at the demo, and then come back here and read the rest of this blog :)

You’ll recognize this as the second client-side demo I mentioned in a recent blog entry.  I only had to make minor changes to get it running in Silverlight.  Though the standard/recommended way to use F# with Silverlight is to create e.g. a C# Silverlight app (and use designer tools or what have you) and then reference an F# Silverlight library, I did a little hacking to make this completely F# – no C#, no XAML, just pure F# code.  The source code is below (I also had to do a little hacking on the project file to get it to compile into .xap with the correct entry point); it is very similar to the WPF app.

namespace Module1

open System.Windows 
open System.Windows.Controls 
open System.Windows.Media 

type MyApp() as this =
    inherit Application()
    let cc = new ContentControl()
    let CYAN = new SolidColorBrush(Colors.Cyan)
    let BLACK = new SolidColorBrush(Colors.Black)
    let canvas = new Canvas(Width=600.0, Height=400.0, Background = CYAN)
    let WIDTH = 20.0
    do
        this.Startup.Add(fun _ ->
            let chars = 
                " F# reacts to events!"
                |> Seq.map (fun c -> 
                    new TextBlock(Width=WIDTH, Height=30.0, FontSize=20.0, Text=string c, 
                                  Foreground=BLACK))
                |> Seq.toArray 
            for tb in chars do                                     
                canvas.Children.Add(tb) |> ignore 

            canvas.MouseMove 
            |> Observable.map (fun ea -> ea.GetPosition(canvas))
            |> Observable.add (fun p -> 
                async {
                    for i in 0..chars.Length-1 do
                        do! Async.Sleep(90)
                        Canvas.SetTop(chars.[i], p.Y)
                        Canvas.SetLeft(chars.[i], p.X + float i*WIDTH)
                } |> Async.StartImmediate
            )
            cc.Content <- canvas
            this.RootVisual <- cc
        )

That’s all for today – I just wanted to take my first steps into the world of Silverlight!

F# async on the server side

Last time I talked about cool ways to use F# asyncs to help write client GUIs.  Today I’ll talk about the killer app for F# asyncs, namely, non-blocking I/O on the server side.  Once again, this is material I lifted from a recent talk that Matthew Podwysocki and I gave at TechReady.  (So if today’s blog entry sounds like I’m presenting a powerpoint deck and a demo to an audience in a room, that’s why!)

A server example: stock-quote server

Most typical server applications have a number of things in common:

• Many clients
• Streams of data to/from clients
• I/O to carry out the guts/machinery of the service

There are various kinds of file & application servers: web servers, file servers, chat servers… As an example for today, I’ll show a mocked-up stock-quote server:

• Clients connect with info about stocker ticker they’re interested in (MSFT, GOOG, whatever), and the server will send back a continual stream of stock price updates, maybe about 1 per second. 
• We want to be able to handle thousands of concurrent clients.

This small sample application demonstrates some essential scaling features found in many server scenarios.

Server commonalities

At a very high level, servers often have this basic pseudocode structure:

• Server code often has the form

while (true) {
    client = socket.AcceptClient()
    BeginServicingClient(client)
}

• Where BeginServicingClient has the form

try {
while (true) {
     SendOrReceiveData()  // local & remote I/O
}
} catch (Exception e) { Handle(e) }

There’s a main loop that listens for new client connections, and each connection spawns a loop that is dedicated to servicing that client, and typically involves I/O (talking to the file system, a database, a web server, whatever).

• In the case of our stock-quote server, we’ll have a main loop that accepts new clients, and then for each client, we will send updated price data every second until the connection ends.
• We need to be able to handle multiple clients concurrently; the simplest (but naïve) approach is to use threads – e.g. BeginServicing() creates a new thread, one thread per client.  This is very easy to code up and lets us continue to use easy ‘sync’ APIs (e.g. “Stream.Write”) rather than difficult ‘async’ APIs (BeginWrite/EndWrite).
• I’ve coded an example of this strategy in C#.  Rather than using any real stock quote data, the server just sends dummy packets back to its clients, but otherwise this is similar to what you might find in practice.  Let’s have a look at the C# code.

(The entirety of the code can be found at the end of this blog entry.)

A synchronous solution in C#, using threads

I’ve coded the server as a short C# program, only about 100 lines of code.  I’ll just call out a couple noteworthy bits:

static void SyncMain(string[] args)
{
...
        while (true)
        {
            var client = socket.AcceptTcpClient();
...
            var t1 = new Thread(new ThreadStart(() =>
...
                        ServiceClient(client);

In the main loop we accept new clients (AcceptTcpClient), and for each one we start a new thread (ThreadStart) that calls ServiceClient(), which is this little function that is a lot like the pseudocode I mentioned earlier:

static void ServiceClient(TcpClient client)
{
    using (var stream = client.GetStream())
    {
        stream.Write(quote, 0, 1);  // write header
        while (true)
        {
            WriteStockQuote(stream);
        }
    }
}

We write an initial header byte, and then just sit in a loop sending stock quotes back to the client.  To keep the example simple, this just goes forever, until e.g. the client disconnects (which would cause an exception and get out of the loop).

Ok, let’s try running it.  I’ll start the server, and I’ll fire up Task Manager so we can look at CPU usage.  To simulate clients, I’ve got a short F# script I can send to the F# Interactive to add clients on the fly.  So I’ll highlight this F# code and “Send to Interactive”, and now I have 1000 clients hitting the server.  (You can try this yourself with the code attached at the end of this blog entry.  Or just follow along with my narration and imagine you’re seeing on the screen what I’m talking about.)

(Incidentally, this shows a nice use of F#’s Interactive REPL: you can easily write little snippets of test code to test some library or app, whether it be C#, or F#, or whatever.)

Ok, so there’s no flashy UI, but you see the server is handling 1000 quotes per second and we’re hardly using any CPU, maybe just 3-5%.  So everything looks great.  But let’s try adding another 1000 clients.  (I highlight some F# script code and “Send to Interactive” again.)  BOOM, on the screen we get “CSSyncServer has stopped working… Unhandled exception: System.OutOfMemoryException”.  Our server crashed! 

Threads don’t scale well!

Using all those threads ran us out of memory.  Threads are expensive.  We created one thread per client, but our threads were mostly wasted waiting on blocking operations – despite all these threads, we saw the CPU utilization was next to nothing.  After a little more than a thousand clients (and thus a thousand threads, since we had one thread per client), the app falls down, running out of memory since each thread typically needs a megabyte of memory for its call stack.

• One thread per client is not the right way to write a scalable .NET server.
• The real solution is to use non-blocking I/O.  This requires switching to Begin/End APIs.  Then we can use the .NET ThreadPool to use only a few threads to service many clients.

So let’s do it right.  It turns out I’ve already written this solution with C#, so let’s have a look at how it performs.

An asynchronous solution in C#, using Begin/End methods

I can switch the “startup project” in the solution to the “CSAsyncServer” project, where I’ve written the same server code in a non-blocking fashion.  That is, I’m using the asynchronous APIs like BeginWrite/EndWrite, so as not to block and to utilize the .NET ThreadPool for callbacks.  If I run that code, I can start the server, and hit it with 1000 clients. Ok, everything looks fine like before, now let’s add another 1000 clients.  (Server keeps running.)  Another 1000.  (Keeps running, starting to show a little CPU usage, in the 10-20% range…) Another 1000 clients.  You get the idea, it scales now.  Hurrah – problem solved?

• Ok, I’ve convinced you that one-thread-per-client is bad, and that async non-blocking I/O is good…
• But I did it all with C#.  This is an F# talk, right?  So what’s the story?

The story is that in C#, the async programming model is awful.  Let’s have a look at the C# code written using Begin/End APIs to do async I/O.  Recall back in the synchronous version, our ServiceClient() was this little method we saw before:

static void ServiceClient(TcpClient client)
{
    using (var stream = client.GetStream())
    {
        stream.Write(quote, 0, 1);  // write header
        while (true)
        {
            WriteStockQuote(stream);
        }
    }
}

When I change to using the Begin/End pattern to make it non-blocking, it becomes this monstrosity:

static IAsyncResult BeginServiceClient(TcpClient client, AsyncCallback cb, object state)
{
    var stream = client.GetStream();
    var ar = new ServiceClientAsyncResult(stream, cb, state);

    try
    {
        stream.Write(quote, 0, 1);  // write header
    }
    catch
    {
        stream.Dispose();
        throw;
    }
    var everWentAsync = false;
    Action<Action> wrap = (a) =>
    {
        try { a(); }
        catch (Exception e)
        {
            ar.Complete(!everWentAsync, e);
        }
    };
    Action loop = null;
    Action<IAsyncResult> end = (iar) =>
    {
        wrap(() => EndWriteStockQuote(iar));
    };
    AsyncCallback callback = (iar) =>
    {
        if (iar.CompletedSynchronously)
            return;
        end(iar);
        loop();
    };
    loop = () =>
    {
        wrap(() =>
        {
            while (true)
            {
                var iar = BeginWriteStockQuote(stream, callback, state);
                if (!iar.CompletedSynchronously)
                {
                    everWentAsync = true;
                    break;
                }
                end(iar);
            }
        });
    };
    loop();
    return ar;
}

static void EndServiceClient(IAsyncResult iar)
{
    try
    {
        AsyncResult.End<ServiceClientAsyncResult>(iar);
    }
    finally
    {
        (iar as ServiceClientAsyncResult).Dispose();
    }
}

Gaaaah! You end up duplicating the exception handling for Begin/End calls, putting state in the IAsyncResult object, dealing with CompletedSynchronously (see e.g. the “stack dive” section of this blog by Michael Marucheck for an explanation), … I’m not going to go into all the little details.  You get the point – writing async code in C# is a difficult mess.

So it’s possible to ‘do it right’ in C# to utilize your server hardware and the .NET ThreadPool, but it takes a lot of crazy difficult code.  So now you see why I was showing the C# code in an F# talk: now we can have F# come to the rescue!  Let’s look at the F# code!

F# solutions – sync and async

Here’s the F# synchronous code for serviceClient, it looks very much like the C#:

let serviceClient (client: TcpClient) =
    use stream = client.GetStream()
    stream.Write(quote, 0, 1)  // write header
    while true do
        writeStockQuote(stream)

Now here’s the F# async code:

let asyncServiceClient (client: TcpClient) = async {
    use stream = client.GetStream()
    do! stream.AsyncWrite(quote, 0, 1)  // write header
    while true do
        do! asyncWriteStockQuote(stream) }

See how similar they are?  We just had to wrap the code in an ‘async{…}’ block, and then change “Write” to “AsyncWrite” with a “do!”, and “WriteStockQuote” to “AsyncWriteStockQuote” with a “do!”.  That’s it!  Using F# async, we can write non-blocking I/O, but our code structure stays exactly the same as if we’d written things synchronously.  We can evolve our naïve code or proof of concept ideas smoothly and easily into production quality code.

(Once again I do the demo, this time with the F# async server scaling to thousands of clients.  Cue applause.)

Using F# async has other benefits too.  For example, we get cancellation for free; all F# asyncs use the new .NET 4.0 APIs for cancelation (CancelationTokenSource & CancelationToken) and check for cancelation at every “let!” or “do!”  If I were to add the same checks to the C# code, it would take even more effort there.

The final score

Let’s try to sum this all up.  This slide summarizes what I just demonstrated:

 

First off, look at column 1: regardless of what language you use, using asynchronous non-blocking I/O is a big win for scaling; you can handle far more concurrent clients this way than by using dedicated threads and synchronous APIs.

Second, looking at the lines of code (LoC) columns objectively, a solution with the traditional .NET Asynchronous Programming Model (Begin/End methods) is much longer and more complex than the sync version (by about three times in this example).  Whereas with F# async, the solutions are about the same length.

Finally, looking at the ‘coding’ columns, just anecdotally, the APM is much harder to reason about and debug.  These numbers describe how long it took me to code and debug these solutions: though it was easy to write the “sync” versions in both C# and F#, it took me about 3 hours to get the C# async example working correctly, compared to about just 10 more minutes for the F# version.  I admit I’m a little out of practice with C# async, but I don’t lack experience: before working on F#, I was a developer on WCF (Windows Communication Foundation), and the entire runtime there is built out of tons of Begin/End calls.  This stuff is just intrinsically difficult to write and debug in C#.

The two take-home messages here are clear:

• If you are writing .NET server code, you need to use async non-blocking I/O calls to scale.  Otherwise you’ll be throwing money away on extra servers since your software doesn’t scale well to utilize them.
• F# makes it much easier to write, debug, and reason about async code with non-blocking I/O.

So there you go.  Who would have guessed that a programming language could help save customers money on server hardware?  :)

Source code

You can find a Visual Studio 2010 solution with all the source code here.

F# async on the client side

Though the killer app for F# async is non-blocking I/O on the server side, F# asyncs are also useful and fun to use for client-side programming on the UI thread.  I’ve mentioned this before in the context of a large example, but I wanted to call out this detail in a separate blog post today.  I’ll show two small examples of F# async lets you do non-blocking loops on the UI thread.

Clock example

The first example is a clock.  Say I want to write a simple WinForms app that displays a digital clock that updates every second:

There are a variety of ways you can “get off the UI thread” to do background work (e.g. BackgroundWorker) and get back to the UI thread (e.g. Control.Invoke, SynchronizationContext.Post).  When you don’t need to do any work, and just want to sleep or get called back in the future (as we’d need to for this ‘clock’ app), you could just use a Timer.  But who wants to fiddle with the details of all those cumbersome APIs?  In my head, I just want to do a simple loop:

    while true do
        sleep for 1 second
        display the new time on the clock

Simple, right?  The problem is, if you author a synchronous infinite loop on the UI thread like that, you end up with a blank UI that just has the ‘wait cursor’ (e.g. the hourglass or the spinny blue circle) forever.

Fortunately, F# offers “async”, so we can write the code just as I conceptualized it.  Here’s the entire program (add a reference to “System.Windows.Forms” to run it as an application):

    open System.Windows.Forms 
    let form = new Form(Visible = true)
    let textBox = new TextBox(ReadOnly = true, Text = "Hello world")
    form.Controls.Add(textBox)
    async {
        while true do
            do! Async.Sleep 1000
            textBox.Text <- System.DateTime.Now.ToString()
    } |> Async.StartImmediate         
    [<System.STAThread>]
    do Application.Run(form)

Right in the middle of the code you see my “while true” loop, just like I imagined it.  They key is that it’s inside an “async{…}” block, and when I sleep, I do an Async.Sleep with a “do!”.  Recall that F# asyncs enable you to “let go of the thread” for long-running async operations at “let!” and “do!” points; when the async operation finally completes, your code will pick back where it left off (via an implicit callback).  So this code “just works”.  The call to Async.StartImmediate starts running the async block on the current thread (the UI thread, in this example).

So there you have it – an infinite loop running on the UI thread without jamming up the UI – thanks to asynchronous non-blocking calls.

Text chasing the mouse

This is a cute little demo that is fun to play with.  The screenshot doesn’t quite do it justice:

so you should grab the code below, put it in an F# application (add the standard WPF references: PresentationCore, PresentationFramework, System.Xaml, UIAutomationTypes (if .NET 4.0), and WindowsBase) and run it right now.  The sentence “F# reacts to events!” will chase your mouse around the window in a manner that is surprisingly amusing.  (One of these days, I really need to learn enough Silverlight to host these cute UI apps directly in my blog; I’m such a slacker, I bet it’s easy to do.)

In F#, events like "MouseMove” are first class entities – I can pass them as arguments to functions, use them in pipelines, etc.  Furthermore, in F#, events are IObservables (which I’ve discussed previously).  So it’s trivial to write code to respond to these events using Observable functions (there’s no need for elaborate Visual Studio tooling to write “event handler callback method skeletons” for you, or whatnot).

If I imagine in my head how to write this app, the idea I have is: every time the mouse moves, I want to have the first letter of the sentence move to follow it, and then after a short delay have the second letter move to follow it, and then after a short delay the next letter, and so on.

Once again, I can encode this logic directly in F#.  The code is below, the first 20 lines of code just create a WPF Window containing a Canvas, and an array of TextBlocks (called “chars”) for each character in the sentence.

open System.Windows 
open System.Windows.Controls 
open System.Windows.Media 

type MyWindow() as this = 
    inherit Window()   

    let WIDTH = 20.0
    let canvas = new Canvas(Width=800.0, Height=400.0, Background = Brushes.White) 
    let chars = 
        " F# reacts to events!"
        |> Seq.map (fun c -> 
            new TextBlock(Width=WIDTH, Height=30.0, FontSize=20.0, Text=string c, 
                          Foreground=Brushes.Black, Background=Brushes.White))
        |> Seq.toArray 
    do
        this.Content <- canvas 
        this.Title <- "MyWPFWindow" 
        this.SizeToContent <- SizeToContent.WidthAndHeight  
        for tb in chars do                                     
            canvas.Children.Add(tb) |> ignore 

        this.MouseMove 
        |> Observable.map (fun ea -> ea.GetPosition(this))
        //|> Observable.filter (fun p -> p.X < 300.0)
        |> Observable.add (fun p -> 
            async {
                for i in 0..chars.Length-1 do
                    do! Async.Sleep(90)
                    Canvas.SetTop(chars.[i], p.Y)
                    Canvas.SetLeft(chars.[i], p.X + float i*WIDTH)
            } |> Async.StartImmediate
        )

[<System.STAThread()>] 
do  
    let app =  new Application() 
    app.Run(new MyWindow()) |> ignore 

Look at the bit starting with “this.MouseMove”.  I pipe the MouseMove event into a call to Observable.map to project out just the position (x-y coordinate) of the MouseMove event relative to this window.  Now I have an IObservable<Point>, so I pipe that result into Observable.add to add a handler.  In this handler, I just run the async loop I imagined in my head: for each character in the sentence, I have a short delay, and then move that character to chase the mouse (by updating its top & left positions relative to the Canvas, to match the mouse-move point “p”).

Since this little loop-over-each-character runs asynchronously, we don’t jam up the UI; while the latter portion of the sentence is still chasing the old mouse position, we can still react to newer mouse moves and have the earlier characters chasing the updated position.  You end up with lots of interleaved loops, all running asynchronously on the UI thread, and it “just works” and you get a cute, fluid animation.

There’s a line commented out; if you uncomment it, you can probably guess what happens.  The line filters only those locations whose X coordinate is less than 300, so if you run the app with that line uncommented, then the letters only chase the mouse while it’s on the left portion of the Canvas, but quit chasing when the mouse goes too far to the right. 

By the way, this demo is one I stole from a recent presentation I did with Matthew Podwysocki at TechReady.  We, in turn, stole the idea from the “time flies like an arrow” sample that Erik Meijer and the Rx team use to show off Reactive Extensions. :) Such a cute little demo!

Summary

These examples demonstrate how easy it is to use F# asyncs to create some simple UI effects.  Normally you can’t write a long-running loop on the UI thread, as it will make the UI non-responsive (no repainting, no responding to the mouse, etc.).  But with F# async, we can write loops with non-blocking calls (like “Async.Sleep”), so that we can write for-loops or while-loops that run for many seconds (or even forever) without blocking the UI.

(The real killer app for F# asyncs is server-side, and I hope to blog about that sometime soon.)