In my previous post, I detailed a common software pattern that anyone writing media-rich iOS apps probably runs into, so I was surprised Apple hadn't provided a generic implementation in the SDK.  Another pattern I'm running into is a prioritized and throttled HTTP connection pool.

As you navigate through a media-rich application that loads resources, the resources need to be streamed in to support the user's working set.  However, it's possible the user transitions among working sets very quickly, invaliding the previous working set before all of its resources are even loaded.  For example, you can imagine a photo gallery app that has one continuous view of a large photo album.  The user may scroll through lots of photos before they are even loaded in an effort to get to the end of the album.

While the cache explained in my previous post would help keep everything stable, the cache still needs to go out to storage to retrieve the photos.  In order to make things simple, developers typically write code that serially retrieves required resources, disregarding whether they are in the working set anymore.  If the cache is still interested, it will store the retrieved data.  Otherwise, it will drop it on the floor.  However, this can result in a poor user experience in the scenario above, where the user must wait for all the previously requested images to load before the ones at the bottom do.  The challenge is to make the code better to not retrieve resources once they are outside the working set.

In more complex scenarios, the application may want to lazily load resources in the background while the user works on the current working set.  However, this can once again create the same poor experience if the user must wait for these background resources to complete loading before resources in the user's working set are loaded.  Even worse, these background resources may not even be visible, so the user has no idea why she is waiting!

Hence the need for prioritizing HTTP connections.  To do this, one needs to assign a relative priority to each connection and place it in a pool of requests.  Once in the pool, some logic needs to decide the right connection to execute.  You can easily add more complexity by allowing multiple connections in flight.  The generic pattern here is that of a scheduler: "objects" become "ready" and are then "scheduled" for "execution" once "compute resources" are available.  The "scheduling" of an object can be simple or complex: operating systems are known for having algorithms that attempt to fairly give compute resources to lots of objects with differing priorities.  Validating the fairness of a scheduling algorithm can get pretty hairy.  For our problem, simpler solutions are typically good enough.

An orthogonal, but also related, problem one runs into when developing apps that have a lot of HTTP traffic is that of throttling.  Lots of web services want clients to throttle their connections in an effort to curtail abusive usage.  However, the HTTP library on iOS doesn't have a way to throttle a bunch of connections to the same HTTP server.  Most web services only allow a certain number of connections every period of time.  However, in an age where multiple connections to each server are the norm, that could mean several things.  Conservatively, an HTTP library could provide a static period between the closure of one connection and the opening of the next to the same server.

I've solved these problems within the context of the app that I'm currently writing.  Currently, it's not the most general solution as it is tailored to the MPOAuth connection library.  Hopefully I will get some time to refactor it and throw it up on github.  Let me know if this would be useful to you!

~s