Big Wish List for Windows Azure - PDC10 update

At Lokad, we have been working with Windows Azure for more than 2 years, received the 1st Windows Azure Award and serving large and small companies through a 100% powered by Windows Azure technology since the commercial availability in Q1 2010.

In my previous Big Wish List for Windows Azure, I was stating that Microsoft was a late entrant in the cloud computing arena. Considering the tremendous efforts that Microsoft has pushed around cloud technologies in 2010, I believe this aspect is no more relevant.

With all the PDC10 announcements and all the improvements delivered in 2010 in Windows Azure, it’s time to revisit this list.

Windows Azure

Top priority:

• (Nice improvement) Faster CPU burst: Compared to 20min VM burst observed at the very beginning of 2010, it now takes about 8min to get the first extra requested VMs. That’s a major speed-up already, and I am really looking forward an equivalent improvement in 2011. Then, near real-time VM instanciation would open tons of new possibilities, but it’s beyond the strict requirement of Lokad.
• (Done!) Smaller VMs: Quarter VMs have been announced which is going to very handy for tactical apps.
• (No update) Per minute CPU billing (but no cloud provider delivers this feature either).
• (No update) Per-VM termination control.

 Nice to have:

• (Downgraded) Bandwidth and storage quota: we have now about ~20 cloud apps running at Lokad, and the cloud consumption proves to be very predicable. Hence the need for quota is nearly not as bad as I was expecting almost 1 year ago.
• (No update) Instance count management through RoleEnvironment.
• (Downgraded) Geo-relocation of services: After extensive use of Windows Azure, you just get used to chose the right service location from the start.

Overall feedback: even if occasional glitches have been observed after 1 year of services in production of Windows Azure, it clearly proved to be the most stable hosting environment we ever experienced.

SQL Azure

Top priority:

• (Nice improvement) DB snapshot & restore toward the Blob Storage: the copy feature of SQL Azure is a big step forward. It costs more than a Blob Storage dump, but in practice, VM costs are dwarfing SQL Azure costs anyway.
• (Interested & surprised) Smaller DB (starting at 100MB for $1 / month): No update on that one, but the announcement of federations for SQL Azure might bring a solution from a fresh angle for multi-tenant apps.
• (Nice Improvement) Size auto-migration: Still no auto-scaling, but changing the database size can now be done with a tiny SQL command which is really nice.

Nice to have:

• (Downgraded) Geo-relocation of service: Idem.

Overall feedback: a really distinct feature of the Azure platform. Its near seamless integration with SQL Server proved to be very handy for a couple clients of Lokad to expose their data to Salescast in order to avoid overloading of their on-premise databases with intensive read operations.

Table Storage

Top priority:

• (NEW) Upsert operation: group entity transactions are provided for insert or update, but neither of those operations are idempotent an important aspect of large scale computations. Hence, it would be really nice if Table Storage was supporting upsert (update or insert) entity transactions, as it facilitates the design of large scale data crunching apps.
• (NEW) Indexed get-many entity retrieval: if it is possible to update up to 100 entities in a single request, it is not possible to efficiently retrieve 100 entities at once from the same partition while explicitly specifying entity identifiers. Indeed, the get-many request triggers a linear scan of the table partition.
• (No update) REST level .NET client library.

Nice to have:

• (No update) Secondary indexes.

Overall feedback: very powerful storage service, still lacking the .NET client library it deserves. Work-around exists for the lack of upsert and get-many operations, but they complicates the client code.

Queue Storage

Nice to have:

• (NEW) Increase scalability beyond 500 messages / sec: Queues have been announced to be caped at 500 messages / sec which is definitively not large scale. Yet, I am impressed by the attitude of Microsoft in this area: Azurescope being an excellent initiative. In comparison, the SLAs offered by the other cloud providers are rather fuzzy.
• (No update) Push multiple messages at once.

Overall feedback: Just exactly what you would expect from a FIPFO. The throughput cap at 500 msg / sec is annoying, but not a show stopper. The work-around consists of sharding a single logical queue over multiple Azure queues. It’s not too complicated to implement, but it adds extra layers of code.

Blob Storage

Nice to have:

• (No update) Reverse Blob enumeration.

Overall feedback: Although the most common usage pattern consists of using this storage as a substitute for a classical file system, Lokad mostly uses the Blob Storage for pre-aggregated data chunks - in order to make our data accesses more coarse (handy to improve overall process latencies). It works just fine.

Windows Azure Console

(Big updates coming) At PDC10, Microsoft unveiled an entirely redesigned web console for Windows Azure. I did not get the chance to have a try yet, but I believe big changes (for the better) are coming soon in this area.

New services

Although it was not part of my initial wish list, but the Windows Azure AppFabric Caching (distributed cache) and Windows Azure Virtual Network (IP management) are impressive additions that I am very eager to see in production.

Concerning other services:

• (Interested & surprised) .NET Role Profiler: Not there yet, but it could come as a latter extension of Intellitrace an impressive addition to .NET that I wasn’t even close to expect from Microsoft. I don’t think any cloud offer a similar feature at present day.
• (No update) Map Reduce.

I am eager to see how Windows Azure unfold in 2011. This upcoming year is likely be turning point in terms of widespread adoption of the cloud among traditional software companies (not just Californian startups :-).