Peter Holditch's Blog

Firstly, a full quarter of a year seems to have passed since I last wrote anything.  Not a great start for the series of use cases I had planned to write about!

That said, I have been scalded into action by a surprising set of results that a customer achieved by testing one of their systems on Azul over the last few weeks.

The system in question was a "SOA style" data movement application which extracts data from various systems, transforms it into a canonical XML format and from there passes it on to other downstream systems.  A lot, but not all, of the data transfer is done in overnight batches.

At first blush, it was not clear how much moving to Azul might help this system.  there was some parallelism,  but not an overwhelming amount,  and quite a lot of file i/o.  This coupled with the fact that garbage collection during a batch process is not often a big problem (since there is no user present to complain if the system stops to garbage collect for a few minutes) made me wonder how much benefit Azul would show.  Undaunted, we started to test.

In the event I am delighted that we did, since the results were spectacular!  All of the test showed an increase of around 5x in single-threaded speed!  Given Azul's highly multicore nature, single threaded speed (in the absence of a large volume) is not often an interesting metric.

It then transpired that another vendor had historically promised a 30% speedup by moving to their JVM and modern hardware.  Tests showed no speedup at all.

Clearly, it is not CPU speed that is the problem for this system.  Rather, it is memory bandwidth, a growing problem that has been written about elsewhere.  When serially crunching through XML documents, the raw speed of the CPU just governs how fast it will wait while the memory gets back to it...  You are likely never to see a CPU with a data cache large enough to hold a multi-megabyte XML document...

What helps Azul in this scenario is our extremely large memory bandwidth.  We have publicly demonstrated this to place us in the league of the top 20 supercomputers which is all very nice in theory, but here is how it helps a real application in practice!

This coupled with our ability to handle large memory footprints makes Azul an excellent XML processing engine - in a separate customer benchmark which I carried out a while ago, I demonstrated the ability to process required data volumes in 7x less conventional hardware footprint, which was respectable.  As icing on that particular cake, I then demonstrated the same result after removing some specialised XML processing appliances from the mix and performing the whole XSLT processing workload in java.  An excellent result both in terms of the resulting deployment simplicity as well as the deployment cost.

Taking all this coupled with our ability to directly connect communicating VMs to bypass the need for streaming network bandwidth, I think the Azul story around XML-centric applications (OK, call them SOA style if you must) is compelling.

I'm off now to find some more XML centric applications to test...

Permalink » | Comments (0) » | TrackBack (0) »

As usual, I need to start with the disclaimer that it’s been many moons since I last posted.  I always start off with such good intentions, but somehow that never translates into more frequent posting…  Hey ho.

I have recently been putting together a whose set of talks about Azul, one for last week’s London Finexpo event, and another for the forthcoming TradeTech Architecture conference, also in London.

The good news is that with the increasing maturity of Azul’s technology and customer base, the presentations can shift more and more from the “what could be” mould to the “what is” – many people now have significant production footprints of Azul, and patterns are emerging of where the sweetest spots are for the technology.

And reading Kirk Pepperdine’s blog of the other day, one of the major underlying themes for these use cases is clearly front and centre in his mind too, namely the need for ultra-high throughput data based applications.

Particularly in financial services, there are many trends that are all pointing toward the need for very high throughput data based applications…  Among these are such things as algorithmic trading – as this becomes more and more mainstream, the winning institutions will be those that drive decisions based on real time risk assessments (meaning real-time processing of lots of data to feed decisions) and those that make the best decisions as to which trading venue offers the best return for a given trade (again, something requiring real-time processing of lots of data)  This level of throughput forces the data in question to not be held in databases, but in memory.

This all leads to some of Azul’s customers’ most important applications being built around large (in the order of hundreds of GB per node) replicated caches.

It is interesting to note following on from this that although the feature mentioned by Kirk - Azul's transactional-memory type abilities to increase throughput - are important, at least as important is the ability to handle a multi-hundred gigabyte heap without the VM suddenly having to take a 2-hour sabbatical (known as an outage to all the application owners I have met) to compact the heap.

Thus, even when other platforms start to support notions of transactional memory, Azul’s hardware-based lead will still be very much alive-and-kicking in the form of Pauseless Garbage Collection.

I suspect this will continue to generate much interest in the value of our technology among the owners of many of the most critical, memory intensive applications for some time to some.

Permalink » | Comments (1) » | TrackBack (0) »

Well, I somewhat shamefacedly realized that it has been… Well…  rather a long time since I last posted.  I guess I must have been distracted, or busy…  There is certainly plenty going on here at Azul to keep me off the street.  My hanglider is ready, and I feel a chasm coming on.

I spent the majority of last week at The Server Side Java Symposium in Barcelona, along with fellow Azul blogger Cliff Click and Gil our esteemed CTO.  It was interesting to meet developers at the coal-faces of java, and delighting to see how many of them experience the day-to-day pain that the Azul products eliminate.  Needless to say a large amount of this pain is around garbage collection pauses, and the consequent downward pressure on heap-sizes, eliminating useful caches and other wished-for things from peoples’ architectures.

Cliff gave 2 very interesting talks, one on what lurks under the covers of the JVM, and a Birds of a feather session about building and debugging code for highly parallel systems.  Gil participated in a most illuminating panel discussion about high performance computing.  When the videos etc become available, I’ll update this with some links.

For me, looking at the sponsoring vendors and listening in to a number of the sessions where people described their real-life architectures (VOCA, the Guardian) and also some more “futurologist” type presentations with the goal of predicting where next for java, one overriding theme clearly emerged.  This won’t exactly come as a massive revelation to anyone with open ears, but still…

From the ‘future’ perspective (at least, the near future) it is clear that J2EE is widely regarded as “too heavyweight” (read, dead) - especially things like EJB.  This comes as no surprise really, a remotable persistent object always seemed rather 2-tier to me.  It is quite clear that most people are using frameworks like spring (in fact, probably most people are using spring) to wire together their logic simply because this approach offers more flexibility.  It will be interesting to see if EJB3.0 will change this to any extent – it certainly seems to be a pretty strong tide.

From the long-term future perspective, there is (as ever) a variety of opinion on whether java as a language in the form we perceive it today will continue its dominance (personally, I can’t see it going away any time soon – there’s simply too much of it out there) What most did agree on, however, was that the java virtual machine is likely to underpin whatever becomes the next big language thing from a runtime perspective, which is great news for people wishing to future-proof an Azul investment!  It will be interesting for dedicated followers of fashion to see what comes off the pegs next season.  I firmly believe that the whole “which language” debate is more about personal taste and aesthetics than about any technical differentiation – a point of view expressed on the panel by Ola Bini

From the vendors perspective, several of those present seem to reflect a general trend to try and cluster multiple VM instances together to achieve scale in applications.  Gigaspaces are touting their “Space Based Architecture” as the route to scale, whilst Appistry were present with their application fabric.  Both technologies are complementary to our own (Gigaspaces are actually a signed up Azul partner) and would thus allow you to cluster many large, pauselessly garbage collected virtual machines together, but I can’t help reflecting as to why this VM-level clustering is such a hot topic.  At least in some cases, it is surely because a single VM instance won’t cut the mustard due to limits on CPU or memory scale.  For those cases, I guess you have a choice… use a software framework to try and manage the extra complexity that distributing your application implies in production, or just augment your existing server hardware with Azul and get single instance scale for your VMs, and enjoy the resultant simplicity (not to mention the absence of those 3am “my app has gone down!” calls)  I can’t help recalling one of Gil’s comments on the high performance panel…  the first rule of distributed computing…  Don’t distribute!

Permalink » | Comments (3) » | TrackBack (0) »

Roaming the blogosphere, as I all too seldom do, I often feel like a party-goer the morning after a major rave-up...  Stumbling between different points of view and commentary, finding interesting tidbits here and there as I pass by.

On one such mission recently, I found Phil Dawes' post from the back-end of February, detailing some very interesting pieces he had discovered on a subject near to my heart, that of multi-threading. In particular, a post from Ridiculous Fish here and a commentary on that by Michael Suess here.

The conclusion that Phil comes to from all this is:

think carefully before employing shared-state multi-threading as a central theme in your next architecture

I would, perhaps unsurprisingly, come to a different conclusion.  I would say, use java to implement your next architecture that requires shared-state multi-threading.  With its standardised memory model, java avoids all the portability pitfalls that Messrs Fish and Suess write about, and with the new java 5 concurrency features, scalable concurrent programming in java got even easier.  Of course, there's no such thing as a free lunch...  Your code still needs to be fundamentally correct with respect to threaded behaviour, but this is easier to achieve with the higher level perspective that programming to the JVM brings.

Finally, if you still need convincing, take a look at Cliff Click's non-blocking hashmap for the ultimate in free lunches.  For that matter, take a look at his blog, posted alongside mine...  Hours of threading and concurrency fun for all the family, man can that guy write... where does he get the time ;-) ?

And finally finally (honest), don't forget that once you have written your scaalable, threaded, java code you can run it corectly on whatever small box you have to hand but when the going gets tough and you really need throughput, you can just offload it to Azul and watch it scream :)

Permalink » | Comments (0) » | TrackBack (0) »

I read an interesting posting from Kirk Pepperdine the other day on the server side, entitled “Can JIT’ed code be faster than Hardware acceleration?” which picked up on the first of a fascinating series of blogs (the rest are here, and here) from Sun’s Mark Lam.

Unluckily for me, John Davies of C24 posted pretty much exactly what I thought about the subject, which, in a nutshell, is that the solution to “performance” in the large (I rambled before on what that might be) is necessarily going to be a mixture of software and hardware.  It is hard to conceive of the two ever really competing, mostly due to the dramatically different development cycle length of the two.

A lot of the noise in the press about Azul has been focused on our 48 core chip design, so it is not impossible for outsiders to go away with the notion that Azul makes something that amounts to java on a chip. Since this is very far from the truth, I thought a few minutes might be well spent here in a quick tour of exactly what Azul have made…

Firstly, it is true that we design our own CPU.  This is a necessary evil, given that there is no commercial general purpose CPU on the market that packs the density that we need at the required power footprint.  The good news about being a java appliance is that the CPU (whatever it may be) is abstracted below the JVM, and never sees the light of day – totally bypassing the conventional instruction-set compatibility and portability problems of a new CPU.  So in some bright tomorrow, there is nothing stopping Azul from powering our nth generation appliances with an off-the-shelf CPU such as those currently in development, at the likes of Intel.

The Azul Virtual Machine, which enables unchanged java applications to run on our hardware is – in fact – simply a port of Sun’s Hotspot, so it benefits from all the JITing and other Hotspot features that Mark talks about.

The Azul features that deliver the throughput gains and single VM instance scalability that customers enjoy are, however, hardware features at base.  It all goes to prove that performance is a way more complex thing than simply how fast the instructions that execute a program are.

Pauseless Garbage Collection is a major performance boost to applications running on Azul.  The memory management hardware provides support for objects being relocated in parallel on the fly.  This can be done in software at the expense of an “is it still there?” check before each write OR read of any object.  The performance hit from that is disastrous to a software only implementation, but manageable with the assistance of hardware.  It is not an accident that at least some well-known java benchmarks measure performance for a length of time guaranteed to EXCLUDE the possibility of a full GC occurring.  GC pauses make dents in your average response time, and massive dents in your worst-case response times, ultimately hurting throughput.

Of course, the direct input that our CPU has on performance is the actual level of parallelism you can achieve.  On our largest 2nd generation system, you can get over 700 threads of activity in a single JVM instance truly concurrently.  Who needs queuing theory?  In support of this, the other Azul hardware feature is Optimistic Thread Concurrency, so multiple readers through a lock can still achieve correct results, and execute in parallel.

In summary, Azul is building a hardware system, not a java CPU. With that system, plus hotspot with all its JIT magic, we achieve the dramatic levels of throughput that save customers money and datacentre space and meet their SLAs even under load.  It’s a good mix for a successful recipe.

Permalink » | Comments (0) » | TrackBack (0) »

I spent some time a short while ago speaking to Peter Williams of Bloor Research about Azul. I was interested to read his thoughts after our conversation as it's always good to hear other perspectives on what you're doing...

The initial impetus for our conversation came from a series of pieces that Peter had written about the issue of power consumption in datacentres.  As he reflects in the conclusion of his article, I have not seen power consumption as a primary driver for hardware acquisitions on my travels and reading around, it seems I am not alone.

Of course, the good news is that because of the high-density, low-power design of Azul's machines, you can buy them to solve your application problems today, and then wait for your conscience to benefit tomorrow, when you realise the power savings.  What is certain is that if you do what you have always done in terms of hardware selection and provisioning, you will get what you always got in terms of the power footprint!

Permalink » | Comments (0) » | TrackBack (0) »

James McGovern professed himself impressed by Azul's new Vega 2 product family in a blog posting a while ago.  He also raised some questions, and wondered if I could take up the gauntlet and answer some of them.  How could I refuse...

Rather than write war and peace in return (although that was tempting) I have taken some high level themes from the questions and posted some high level thoughts on the subjects.  May conversation ensue! These themes are:

·       Grid

·       Memory scale

·       CPU scale

·       Security

Grid

Clearly I cannot speak for the 451 group’s view of Azul’s relationship to grid; what I have seen personally, however, is that there is confusion between Azul’s technology and grid that arises since, at the highest level, both are technologies one of whose aims in life is to optimise hardware asset utilisation.

In fact, the reasons that hardware is more efficiently utilised in the 2 technologies differ considerably.  In Grid, it’s the ability to use otherwise idle machines for calculations.  With Azul, it derives from the fact that several applications share one coherent machine so that in extremely short order (10ms) applications that experience a spike in load can have CPU resources allocated to them.  The short order part is the key – with such predictability, the appliance is truly shareable since applications cannot starve one another of resources.  This is very different from what is offered by a conventional grid and positions Azul firmly in the space of on-line transactional applications where responses must generally be sub-second, whilst the grid will be best used for calculations whose results are needed in minutes or more.

There is another point about grid that segues neatly into the next bullet, memory, which does highlight some overlap between the 2 approaches (there is always more than one way to skin a cat).  Conventional grid requires that data and processing is shipped to a node in the grid, and then results are assembled afterwards.  Clearly, that restricts the amount of data that can be sent per calculation and forces calculations to be explicitly decomposed.  I do know of customers who use a single multithreaded process with a very large memory footprint to do big calculations on Azul – the time to results is accelerated by this approach since the entire result set is already in one place.

Memory scale

I can’t speak for why servers aren’t all bristling with terabytes of RAM (although I suspect the high cost of dense DIMMS is a lot to do with it)  Another limiting factor in the java world in particular is the time it takes to garbage collect a large heap.  The parallel calculation application I was just writing about boasts a 200GB heap, made possible by Azul’s hardware assisted Pauseless Garbage Collector (coupled, of course, with a bucket load of physical memory!)

CPU Scale

A key metric of applications running on Azul is the single instance scalability.  As I have posted before, in real world terms, this equates to how much headroom you have in terms of load before your application tops out.  This is a good ceiling to set high, since it could be the difference between hitting and missing your SLA just when it matters most.  Who wants their applications crashing when a successful marketing campaign hits?

This does raise the question of to what extent individual applications can scale to some arbitrary number of CPUs.  This varies as you would expect.  The good news is that Azul provides the tooling to determine the degree to which they can scale, in the form of RTPM, and facilities such as Optimistic Thread Concurrency (OTC) among others to maximise the extent to which singles instances can scale.

Security

The question posed was whether Azul appliances support SAML or XACML, or whether they require their own identity store.  In fact, Azul appliances have less need for an idea of identity than one might think.  Because Azul represents a virtualisation layer for the execution of java applications, the only identified individuals that a compute pool needs to be aware of is the administrators – usually a small and well defined set.  Of course, their credentials do not have to be stored locally within a compute pool; they can be accessed from a corporate LDAP, where such credentials commonly exist in large organisations.

There is an interesting sidebar, however, in terms of the possibilities for security using the Azul architecture.  It is common for applications that make intensive use of crypto type services to be supported by cryptographic hardware accelerators.  This fact in turn also becomes a limiting factor on security.  If the risk from not fully securing some application is present, but not large enough to cost justify hardware acceleration (or increased hardware footprint) then it will not get secured.  Now think yourself into the world where all your java processing is running on Azul; all the processing now happens centrally on the pool of compute appliances.  Imagine the appliances in the pool get hardware crypto accelerators…  Now there is no per-application incremental cost for using the hardware crypto facilities, since they are provisioned at the pool level.  Suddenly, more facilities can be more secured; why not use SSL (for example) everywhere if it’s effectively free?

I hope that goes at least some of the way to providing answers, or at least thoughts, James?

Permalink » | Comments (0) » | TrackBack (0) »

I couldn't resist revisiting the theme of my last blog entry, in view of the recent announcement of Sun's successful tapeout of their 2nd generation T1 processor, named "Rock"

A couple of things sprang to mind.  Firstly, it is reassuring to see developments like multicore CPUs continuing to gather momentum in the industry.  Being in a minority of 1 is a good thing when it can be termed innovation, but unless at least someone starts heading in your direction, there comes a point where you must realise that you are not an innovator, but just some fool.  The validation of others following Azul's suit is good;  From a selfish point of view, what is also good is that we already have systems in the marketplace based on our second generation CPUs, whilst the taping out of Rock is just another step along the road to a marketable product by the second half of 2008 - the other responsibility of innovation is that you need to stay ahead of the curve, and things in that respect are looking good from where I stand.

There is another hint as to Azul's lead in the Rock press reports that I read... 

"...which means an eight-socket box would have 256 thread--about the upper limit that an operating system can handle these days. "

Herein lies part of the explanation for why Azul took the step of building network attached compute appliances, rather than conventional large servers.  We have boxes in production today that can quite happily schedule threads across 384 processor cores (note, cores, not hyperthreaded smoke and mirrors) and the yet to be released 768 way box is working fine too.  Clearly, evolving a general purpose OS to cope with boxes this size, and then evolving software support matrices to match is a long term prospect.  Since Azul's hardware is all hidden beneath the well defined and compatibility tested JVM layer, we can short-circuit a lot of that effort, and bring the technology to bear on real-world problems today.

I am always keenly aware that the technology business is a big glasshouse, so one needs to be very careful about throwing stones - or rocks - but right now I am very encouraged by the growing production footprint of our technology, and even more importantly a demonstrable ability to provide solutions to customers' real and burning issues with their java applications.  As someone once said, the best way to predict the future is to invent it...

Permalink » | Comments (0) » | TrackBack (0) »

It is interesting to see Mark Reinhold of Sun talking about Azul’s technology…

Sun said that broad server platforms will be the best solutions for Java performance, despite impressive benchmarks for Azul Systems, a company that makes Java-dedicated appliances based on its own processor designs. Sun is currently involved in a legal dispute with Azul.

“In general, custom hardware tends not to win in the long term,” said Mark Reinhold, Sun’s chief engineer for Java SE. “There are so many resources [at firms such as Intel, AMD and Sun] that it’s hard to bring a competitive solution for one type of language.”

This is an interesting comment, but I can’t help wondering if he hasn’t just come back from a holiday sitting on a beach telling the tide to desist. Of course, I work for Azul so I would say that wouldn’t I?  Well, maybe but I think I’m in good company for a few different reasons:

Firstly, the abstract notion of “java performance” as I and others have discussed is foolish; last time I looked, java was a general purpose programming language. I’m not sure how one of them ‘performs’ in a vacuum; performance is a highly context sensitive issue.  For server side, high throughput, high concurrency applications, especially those with tight QoS constraints, there is a broad consensus that a fresh approach to providing machine resources is necessary, or did I just dream Sun's own T2000 range and BEA's project bare metal?

Secondly, the move to custom hardware is gathering pace – in approximate reverse chronological order; XML processing, cryptography, storage, network routing.  Much of what was done by traditional servers yesterday is moving to specialist appliances; network attached processing is just another logical step along the road.  There is more to this than just performance – Who fills their servers with “commodity “disks any more?  Everyone is too busy putting special purpose ones into their NAS arrays, because they’re so much easier to manage there. On a smaller scale, the same is happening, with established vendors being driven by the laws of Moore and Physics to upend the notion of a general purpose computer – the task in hand and today’s constraints are simply starting to mean that the power to do more processing each year can no longer keep page with demand for processing, and if you don’t want to take my word for it, how about Intel’s?

To me, the irony of all this is that increasingly, the server is no longer the computer; the network is. Did I hear that notion somewhere before?  Once Sun was trying to crack into the stranglehold that was glasshouse computing with a progressive vision; now it has become the establishment, burdened with all the dilemmas that implies.

In conclusion, I have always believed that actions speak louder than words, so maybe damning Azul with faint praise is perhaps not the whole story with respect to Sun’s feelings toward Azul’s technology.

The future’s bright, it’s network attached!

Permalink » | Comments (0) » | TrackBack (1) »

Not surprisingly, a good deal of coverage has appeared in relation to Azul's announcement of the first of our second-generation appliances.  Many of the pieces (e.g. ZDNet, Computing, Techworld)talk about applications' ability to scale better on Azul (importantly, whilst preserving reliability and meeting Service Level Agreements)  This is hardly surprising, since Azul's technology assists in achieving all of these goals.  A considerable amount of exposure has also been given to the Openreach deployment of the technology, also unsurprising since that has been driven entirely by a need for massive scale.

I have always been pretty close to things that help scale - in an earlier incarnation I was in R&D and then consultancy on the Tuxedo product, which is famous for scale - almost without question, all of the biggest open systems based TP systems use Tuxedo as the substrate to provide scale to applications, so I feel right at home here...  I am also mindful of lessons learned from those days about scale, so  I thought I'd take the opportunity to put down some thoughts on the topic.

Scale is a verb...

I guess it's just natural human enthusiasm to hear the word scale and think big!  I remember in the Tuxedo days endlessly talking about the Employment Services with 32,000 clients attached to the system.  People often got the impression that their system was "too small" to need Tuxedo.  This is a mistake; things change.  How many times have you seen that little system that just got knocked up for some one-off purpose become the lynch-pin of something big that you never foresaw?  The trick to scale is to remember that it's a verb...  An ability to scale is key, especially as application monoliths break down into distributed services, what only needs to be small today may need to be bigger tomorrow - that's the price of success.  Who out there is planning for failure?

Don't be piqued by peaks!

Another factor to consider is peaky load.  What is a quiet system for most of the time may suddenly become busy with no warning.  In the case of the Openreach deployment, the gateways running on Azul service the whole UK telecoms industry.  If one vendor runs a special offer on, say, broadband then this could prompt hundreds of consumers to look into the viability of switching broadband provider.  It is also likely that they will shop around to see if they can find a better deal.  Suddenly, many individual consumers are on-line doing several enquiries about broadband provision with serveral vendors.  Guess what... the Openreach gateway gets to see each of these as a separate enquiry.  When it sees a spike, it could well see a big one!  For this (and many) deployments it is important to remember that the 'peak hour' load could exceed the steady state load by a large factor.

What are your building blocks?

So, armed with the thought that your system will be small sometimes, and big at others and (hopefully) armed with some kind of projection as to how small and how big it is likely to be you can go off and plan a deployment.  It's easy just to think "scale it out, throw more boxes at it and it will get bigger" but the reality is seldom so simple.  It might be boxes you need to throw, but not necessarily.  And if you do want to throw boxes, you'd better be pretty comfortable with your ability to power them, cool them and manage them.  This was another feature with the Openreach system.  datacentres don't just grow on trees you know...

The smallest unit of scale, in a java system, is the JVM.  How much load can you pass through a single JVM instance?  This will be limited by contention on resources;  these could be external resources such as i/o bandwidth, cpu, memory, etc. or internal resources such as lock structures within your code (or libraries your code uses).  The process of finding out how much load a single JVM instance can handle involves lightly loading it and gradually increasing the number of transactions per second you are driving at it whilst monitoring response times.  When the response times start to depart, you know you have found a bottleneck.   External bottlenecks can be identified with system tools such as sar, vmstat and the like, whist the Azul Real Time Performance Monitor (RTPM) allows internal bottlenecks to be identified, as I wrote about in my last blog posting.  Once the bottleneck has been identified, you can see if it can be removed (faster disk, more CPUs, change a simple lock into a read/write lock, turn on Optimistic Thread Concurrency, etc) and continue ramping up the load until you find a bottleneck that you cannot remove.  You now have the fundamental unit of capacity of your system.  In the case of the Openreach gateway, these units are the individual nodes that combine together to form a b2b gateway instance.

Now, how do these fundamental units compose together?  If they are something as simple as web servers serving static content, then you can simply sit lots of them next to each other behind a TCP/IP load balancer.  Life is rarely so simple, however.  In the case of the Openreach gateway, the traffic is stateful, reliable ebXML conversations so a simple load balancer won't cut it.

The Openreach gateway is architected to support clusters of nodes to allow the capacity of a single VM to be aggregated with others, conversational state is held centrally within the cluster so that any node in a cluster can handle any incoming message, irrespective of which conversation it is part of.  Of course, there is overhead to clustering so a JVM in a cluster can't handle as much load as a JVM on its own, so there is more testing to be done, along the previous lines to determine the optimum cluster size to maximise throughput per unit CPU resources.

So how does Azul help?

There are several things that Azul brings to the table which help in the Openreach scenario (and indeed in the general case!)

• Raise the ceiling of throughput per JVM
• Provide capacity for peaks that can be safely shared with other systems when the gateway is not peaking
• Remove the need for delicate, and load profile dependant, GC tuning
• Provide dense computing - maximising the throughput per unit {rackspace|cooling|insert your favourite datacentre metric in here}

I could write an entire article on each one of these, but for brevity the one I will zoom in on here is the throughput per JVM.  if a peak exceeds the amount of throughout that can be handled by the JVMs you have deployed, then you just blew your SLA.  This is generally regarded as bad news.  To work an example, if one JVM can handle 4tps and you deployed 2 of them you can handle a maximum of 8tps.  if Azul increases the capacity of one JVM to 8tps, then you can either reduce the deployment down to 1 VM to simplify maintenance etc., or you just doubled the ceiling of the maximum spike your system can handle.  This flexibility is key.  If you see a spike that stretches beyond your capacity to service, you need to deploy more VMs into production - with all the attendant change control etc. to worry about.  The better the dynamic range of each VM, the fewer you need and the more able you are to meet your SLAs by receiving additional Azul compute resources in milliseconds (vs. deploying more kit and software in months (or 10s of minutes, if you have some kind of automated provisioning of traditional hardware))

And all this adds up to more sleepful nights for operations managers.

So, for scale don't think big, think stretchy!

images courtesy of bigfoto

Permalink » | Comments (0) » | TrackBack (0) »