The bane of every successful company, athlete or artist harkens back to the vaudevillian phrase, "What do you do for an encore?" A company like AWS that pioneered cloud infrastructure services and still dominates the market, despite many big-name competitors, forces itself to answer that question every year at its widely-anticipated re:Invent conference.

Having defined the categories of cloud compute, storage, networking and database services, recent years have seen AWS expand into every corner of IT services including AI/ML applications, development tools, infrastructure management, security automation and data analytics. After a virtual-only interregnum last year, AWS resumed a live-plus-virtual format this year, with much anticipation over what new CEO Adam Selipsky's would deliver in his first keynote.

AWS spent much of the last two pivoting from its long-held belief in the primacy of centralized hyperscale shared cloud services to embrace hybrid and remote (edge) infrastructure. Now spanning the cloud landscape like the Colossus of Rhodes, AWS seemingly had nowhere else to go but up, as in abstraction layers, by packaging and automating infrastructure products into managed platforms and adding business and developer applications.

Hence, a major theme at re:Invent is "serverless," today's buzzword for managed software platform  — many of which are still called PaaS and not to be confused with the initial category of serverless functions — that insulate users from many details of infrastructure provisioning and management. In the nouveau version of serverless, cloud operators like AWS handle details like resource sizing, configuration, placement and deployment while customers focus on their usage.

Whereas serverless versions of databases and analytics products simplify cloud operations, graphical and no-code UIs — another theme at re:Invent —  streamline and enrich cloud usage by reducing complexity, visually exposing options and guiding non-experts through unfamiliar jargon and technology.

If you posit that serverless is fundamentally a packaging of a cloud product with management automation, my re:Invent predictions from early November were spot on.

Thus, predicting what AWS might introduce at re:Invent has become less interesting as it’s grown more predictable by annually providing more of the same.

• More instance types further tailored to different applications with a recent focus on AI development, model training and inference.
• Additional management features to automate cloud operations, security configuration and resource optimizations.
• New database configurations to accommodate those few enterprise apps not already supported.
• Better price-performance as AWS exploits improved processor, storage and network technology and, recently, by employing custom chip designs using alternative processor (Arm) and machine learning architectures.

Simplifying data organization

The centerpiece of Selipsky's keynote was a discussion of the business problem of our age: collecting, organizing and analyzing the vast and expanding sources of data produced by devices, customers, transactions and business processes. AWS organizes the problem into four categories:

• Databases for collection and organization
• Data lakes for aggregation and consumption
• Analytics and machine learning for summarization, extraction, analysis and visualization.

Of course, AWS has services in all of these categories and has stratified its offerings over the years with database and analytics products optimized for particular needs. Many of these, like RDS and EMR were cloud-ified versions of existing products, but over time, AWS has introduced cloud-native products like Athena, Aurora and Kinesis that improve performance and scalability. This year saw AWS focus on usability and manageability via several new 'serverless' products, including:

• Redshift Serverless deploys and configures the resources necessary for a particular data warehouse workload and automatically scales capacity to accommodate more demand. The serverless controller also shuts down compute resources when there's no activity and fires them when new requests arrive.
• DevOps Guru for RDS identifies and analyzes database performance problems such as resource starvation, I/O bottlenecks and inefficient SQL queries using ML algorithms that also recommend the best fixes. The system displays detected issues in its management console and can also send notifications to EventBridge or SNS (Simple Notification Service).
• Kinesis Data Streams On-Demand is a managed stream processing service that aggregates data and automatically provisions and scales capacity up and down as needed.
• AWS Lake Formation is a managed data lake with three new features:
  • Governed Tables, a form of S3 tables that support ACID transactions for reliable multi-user changes.
  • Storage Optimization automatically compacts small S3 objects in governed tables into larger objects to improve performance when accessed by analytics software like Athena.
  • Row and Cell-Level Security provides granular query and ETL access control over particular rows and columns.
• SageMaker Canvas is a visual, no-code interface for SageMaker that allows novice business analysts to create and run ML models with a graphical interface for browsing and importing datasets for model training.
• SageMaker Inference Recommender assists with ML cost and performance optimization and testing by identifying the best instance type for a particular model and automating deployment.

Combined these features simplify the development and operation of complex, multi-faceted business analytics systems and broaden the scope of data analytics developers and users.

My take

re:Invent was unusual this year because AWS used it to roll out both new products and a new CEO. On the former, the company did an acceptable, although given past performance, exceptional job. Unfortunately, the latter was lackluster, akin to opening a bottle of flat champagne after enjoying some Perrier-Jouët.

While watching Selipsky's keynote, my reaction expressed the frustration of someone expecting the rapid-fire product barrage delivered by his predecessor, Andy Jassy, or the GOAT of keynote content and analysis, NVIDIA's Jensen Huang. Instead, we got lots of table-setting with child-sized entrees:

"So far, #reinvent2021 keynote is long stretches of narrative building winding up with spurts of brief, detail-free announcements. 🥱"

As the event wore on, I wondered if I missed the memo announcing that re:Invent had been re-invented from a forum for cloud experts into a C-level tutorial and schmooze fest:

"Did I miss something? Is this the cloud-for-dummies session before the actual #reinvent2021 keynote?"

Fortunately, secondary keynotes from old-timers like Peter DeSantis and Werner Vogels did a better job catering to cloud hounds. Furthermore, the technical intermediate and advanced (200- 300-level) sessions I attended were quite useful by minimizing the fluff and right to the technical content. I plan on aggregating some of these into some thematic columns in future columns here and at other venues.

The dichotomy between executive-focused keynotes and "leadership sessions" and the conference presentations illustrates the advantages of hybrid or virtual-only events. With more than 300 sessions (not counting the many non-English sessions which I assume are repeats), it's far easier for an attendee to find and attend the most exciting and relevant talks using the online interface than by schlepping from room-to-room (or hotel-to-hotel in the case of re:Invent). While the keynotes serve to highlight the topics, products and case studies the company deems most significant, they are far less efficient at summarizing new announcements than an organized web page like the synopsis of top re:Invent announcements produced by the outstanding AWS blog team.

In sum, while re:Invent attendance might have been one-third of its 2019 peak, don't expect a return to those levels as people and event sponsors internalize the advantages of a hybrid format and most choose to forgo the hassles of travel.