Read our latest: The year so far & 1.0 update.
At Caplin Systems we help financial institutions build web trading platforms. We license a number of backend technologies to these companies, along with front-end SDKs to enable the development of web trading applications.
On top of this technology we built something called WebSheet; an SDK framework that enabled the development of realtime displays of financial prices running in a web browser.
WebSheet has now long been replaced with something called Caplin Trader, a web SDK for developing HTML5 web trading applications, now at version 3.
Caplin Trader is an SDK on top of which you build your web trading applications. The SDK is made up of:
In addition to this, Caplin offers a set of reference implementations, each designed to serve as the starting point for a trading GUI. The Foreign Exchange version consists of:
When we're talking about scaling, the assumption that this equates to "lots of code" is indeed true. Since it's a large application it's also highly likely that some of the code will have been - or is going to be - around for a long time. In the case of Caplin Trader we have some libraries that have been around for years.
Before we look at Caplin Trader, let's take a look at an app that most people are more familiar with: Gmail.
When it comes to complex web applications Gmail is the de facto reference. It offers lots of functionality, all within the browser, without requiring a page refresh:
That's a hell of a lot of functionality.
Next let's have a look at a Caplin Trader application:
In order to provide this level of functionality these applications have to interact with numerous other pieces of technology. Frequently, an interaction with the UI will result in either other parts of the UI updating or a call being made to a back-end service. Back-end services will also invoke client functionality resulting in UI changes.
It's easy to see how the code behind an application with this kind of complex functionality and interactive nature could become very tangled.
How do you ensure that the code stays clean and the way that application components interact doesn't get overly complex or tightly coupled, making it very hard make changes without unexpected knock-on effects?
A factor that is often overlooked when it comes to large applications is the human factor. A large scale complex web app simply can't be developed and maintained by a single developer. It's unlikely that it can be developed by a single team. So we know an application has achieved the status of "large scale" when it requires the contribution of many individuals and numerous teams.
I don't know about GMail, but at Caplin we have more than 80 developers. We consistently have one core Caplin Trader team and from time-to-time we may have more teams working on other Caplin Trader components. We then have one or more teams developing and maintaining the reference implementation. In addition, we have numerous teams - both internal professional service teams and customer developer teams - who are either starting out using the reference implementation or writing their own functionality using the Caplin trader SDK.
Lots of teams and lots of developers. But it's not just developers. Applications can have contributions from:
Our teams are generally reasonably small and either located in our office or on-site at a customer. Customer teams may be large or multiple teams can spread across an organisation. Members of any of these teams may change over time.
With input from so many people it can be difficult to ensure things are done in a consistent way. Different people can do the same things in different ways. Variety is great, but when it comes to building a codebase that everybody has to be able to understand, extend, improve and maintain you need people to do things in a consistent ways.
How do you make sure that all these individuals can work in harmony? We do have source control, but the best type of merge conflict is where there isn't a conflict at all. How do you ensure people aren't treading on each others toes by editing the same code or changing the same assets?
In the next post I'll cover what those pain points were, and finally - by-example - explain what the specific solutions that we've incorporated into BladeRunnerJS are.
If you want to find out more before the next post then I'll be speaking at FOSDEM on Sunday 2nd February and Andy Berry will be talking about the BladeRunnerJS workflow at GlasgowJS. Or for more thought-provoking background reading you should take a look at this excellent answer to Why do dynamic languages make it more difficult to maintain large codebases?