Caylent Goes to re:Invent
Each year cloud christmas happens in Las Vegas and AWS Santa Clause gifts all the good little architects new features and new ideas for what to build. This magical event is called re:Invent. To celebrate the 10th annual re:Invent the Caylent team was there in force! Luckily the fun doesn’t end when we all go home from Vegas. AWS publishes many of the best sessions on YouTube so we can learn at our leisure. In 2021 AWS released, by our count, 479 recorded sessions from re:Invent, or put another way, about 400 cumulative hours of video. That’s almost 17 days of continuous viewing (we may or may not have watched a few of these on 1.5x speed).
The Caylent team popped some popcorn, sat back, and watched every single talk. Why? First of all, it’s our job and our mission to keep pace with everything AWS releases. Second, it’s how our team likes to unwind after a busy year. I know, kind of weird, but hey it’s how we make sure we’re providing the latest and greatest for our customers. Below we’ve written about some of our favorite sessions from AWS re:Invent 2021.
You may view the entire playlist on YouTube by following the link here.
Keynotes
Several of my colleagues have already written about the keynotes so I’ll briefly summarize everything here.
Ross wrote about Adam Selipsky’s and Peter DeSantis’ Keynotes late last year.
Adam’s Keynote is required viewing for all aspiring cloud folks, but Peter’s 2021 keynote was something special. Peter’s keynote talked about the years of research and innovation that allow AWS to operate at truly dazzling scales. You must not miss this talk.
Another colleague Mark wrote about Werner Vogel’s keynote.
Finally, our fearless leader JP wrote about Doug Yeum’s partner summit keynote.
That only leaves the Databases, Analytics, and Machine Learning keynote by Dr. Swami Sivasubramian which is definitely worth a watch.
Leadership Sessions
While all of the re:Invent leadership sessions are worth watching we’ve picked out the ones below as particularly interesting and insightful.
Driving innovation and insight with cloud data on AWS Storage – Mai-Lan Tomsen Bukovec
In this session, Mai-Lan walks through several customer journeys through AWS storage, mainly S3, but also talks about the new releases. Pinterest is a featured customer in this session.
Reinvent your business for the future with AWS Analytics – Rahul Pathak
Building on 15 years of compute innovation – Dave Brown
AWS Networking: Making all workloads possible – Dave Brown
I shared some notes on LinkedIn already but I’ll quickly summarize them here.
- In the first five years of operation, AWS had four regions. In the next five years, they had 11. In the last five years, they had 25 regions. In the next n+1 years AWS expects to open 9 more regions. All of these regions are connected by AWS’s global backbone of 100GbE fiber comprising terrestrial and undersea transoceanic cables.
- Nitro is a fundamental difference between AWS and other cloud providers, it lets AWS offload virtualization to a hardware layer and provide the instance’s entire capacity to customers.
- Covid-19 caused a 400% increase in AWS networking throughput over a matter of days.
- In 2021 AWS processes 11 trillion metrics per day for AWS Networking.
- In 2021 50gbps baseline EC2 Outbound Networking and up to 800gbps for HPC instances. Direct Connect now supports 100gbps ports!
- Coming in 2022, SRD on ENAs will provide drastic bandwidth improvements.
- All traffic is encrypted over the wire. IAM is an application layer security envelope. Traffic sent between regions with TGW or VPC peering is encrypted by default. Encryption methods are quantum-safe, AES-256, and post-quantum key exchange algorithms. Even if someone were to record your traffic today they wouldn’t be able to crack it in 10+ years when quantum computing tools are more readily available.
- AWS is making strides in IPv6 across all of its infrastructure.
- AWS Local Zones are exploding in usage and popularity. NASDAQ built its own “local zone” with outposts. 5G is a promising growth area for edge computing and networking.
Containers on AWS: Everywhere and anywhere – Deepak Singh
It’s always incredible to hear from Deepak and even though he couldn’t be in front of a live audience this year it was still a fantastic session.
Abstractions: Creating the best developer experience – Ken Exner, Emily Freeman
If you’re a hands-on-keyboard coder you can’t miss the insights from this talk.
Continuous security improvement: Strategies and tactics – Steve Schmidt
Migrations
Evolve existing apps to microservices using Migration Hub Refactor Spaces
Belinda Quick (AWS), Michael Sciscenti (AWS), Jonah Jones (AWS)
In this talk, Belinda Quick walks us through some of the stories and learnings from large-scale migration and modernizations that AWS customers have done. Belinda quickly covers leave and layer approaches as well as strangler-fig approaches to migrations as well as multi-account strategies. Next Michael Sciscenti walks through how Refactor Spaces can deploy an application proxy to combine API Gateway, NLB, Transit Gateway, and Resource Access Manager to incrementally route traffic between legacy and migrated applications. Finally, Jonah Jones closes with a real-world demo of Refactor Spaces. If you take a look at the screenshot from this talk you might see a familiar name as one of the launch partners!
Learn how MAP can help you achieve your business outcomes
Faraz Shafiq (AWS), Alon Radzi (Amdocs)
In this session, Faraz Shafiq tells us how to navigate the tension between business and technology goals by showing examples from companies like Expedia, Dow Jones, and Coca-Cola. Faraz talks about the motivations for migrations and how customers tend to evolve as they modernize. He points out that, on average, customers realize 31% cost savings over comparable infrastructure, 62% more efficient IT infrastructure teams, and a whopping 75% increase in agility in delivering new features. Next Alon Radzi walks through a real-world example of using MAP to help telecom providers migrate to AWS.
DevOps, Developers, and Architecture
Beyond five 9s: Lessons from our highest available data planes
Colm MacCárthaigh (AWS), Yasemin Avcular (AWS)
Building modern cloud applications? Think integration
Gregor Hohpe (AWS)
Building next-gen applications with event-driven architectures
Sam Dengler (AWS), Robbie Kohler (Taco Bell)
Application integration patterns for microservices
Dirk Fröhner (AWS)
Amazon Builders’ Library: Operational Excellence at Amazon
David Yanacek (AWS)
Don’t miss this one
Best practices for securing your software delivery lifecycle
Curtis Rissi (AWS) and Sundeep Goswami (Dish Network)
Building with the new AWS SDKs for Rust, Kotlin, and Swift
Zelda Hessler (AWS – Rust), Ian Botsford (AWS – Kotlin), and Nicki Stone (AWS – Swift)
This talk walks through how the new AWS SDKs are built on top of the AWS Common Runtime (CRT) libraries. It’s a masterclass in how to build SDKs at scale in 2022. It walks through Smithy an open-source SDK specification language and even shows the SDKs being used in a demo.
DevOps Revolution
Emily Freeman (AWS)
A new model for thinking about DevOps as concentric circles of responsibility. Just a heads-up that the audio in this video is somewhat choppy. Towards the end of this talk, Emily walks through two concrete examples that navigate the new model. Finally, Emily closes with a “hexagons are the bestagon” skill assessment visualization.
Incorporating continuous resilience in your development ecosystem
Curtis Rissi (AWS)
In this talk, Curtis starts by describing the cost of downtime and the way downtime grows in applications. He then walks through tools, services, and best practices for reducing the probability of failure. Among those recommendations are setting timeouts, retry backoffs, jitter, queue size limitations, rate limiting, load shedding, cellular architectures, shuffle sharding, immutable infrastructure, monitoring, and alerting. Curtis closes with a brief discussion of Chaos Engineering. Whew! A great talk with a whirlwind of advice.
Networking
Networking foundations
Alan Halchmi (AWS), Madhura Kale (AWS)
Start here if you need a refresher on how networking behaves in AWS.
Advanced Amazon VPC design and new capabilities
Matt Lehwess (AWS), Alexandra Huides (AWS)
This talk is a fantastic primer on what VPC design looks like for 2022. The complexity of that first slide definitely triggers PowerPoint PTSD though.
{New Launch} Introducing AWS Cloud WAN and AWS Direct Connect SiteLink
Nick Matthews (AWS), Santiago Riesco (AWS)
In this talk, a couple of AWS Product Managers walk us through the reasoning and use cases for SiteLink and AWS Cloud WAN.
How Netflix is using IPv6 to enable hyperscale networking
Donavan Fritz (Netflix)
Netflix Engineer Donnavin Fritz talks about how they use Titus. They think 5-10 years ahead when preparing their network. There was a great laugh on “old code is not fun, especially Java”.
Amazon Route53: A year in review
Jeffrey Damick (AWS), Gavin McCullagh (AWS)
AWS Well-Architected Framework for hybrid networks
Aarthi Raju (AWS), Rohit Aswani (AWS)
How to choose the right load balancer for your AWS workloads
James Wenzel (AWS)
Integrate Amazon EKS with your networking pattern
Theo Salvo (AWS), Tom Adamski (AWS)
Databases and Analytics
DynamoDB deep dive: Advanced design patterns
Rick Houlihan (AWS)
Rick does this talk every year and you can’t miss it. Each year Rick dispenses NoSQL wisdom at 100mph. You might have to run it at .75 normal speed to keep up. Unique takeaways from this year include: how DynamoDB request routers actually improve in performance as you start to reach peak load because they’ve all cached your table config, how Snapchat dealt with the 2019 Superbowl spike using burst bucket capacity, global tables, and — I mean come on you should just watch the talk!
Data modeling with Amazon DynamoDB
Alex DeBrie (Stedi)
Alex literally wrote the book on DynamoDB. Do not miss this talk!
Drawing the New York City skyline with Amazon Aurora Serverless v2
Renato Losio (Funambol)
This is a completely absurd, but effective, way to demonstrate the difference between Aurora Serverless V1 and V2.
Design patterns to consider when moving to Amazon Aurora
Jatin Singh (AWS), Kiran Singh (AWS)
Jatin and Kiran provide concrete and useful advice for teams moving to Amazon Aurora. Things like setting apg_ccm_enabled=on
to make sure caches are warm on failovers, setting TCP keepalives to good values for longlived connections, setting DNS cache timeouts lower so failovers get detected faster, and using the right cluster endpoints for the right needs.
Accelerating data-led migrations
Eric Heikkila (AWS)
What’s new with Amazon DynamoDB
Chad Tindel (AWS)
Is relational the new COBOL?
Mark Porter (MongoDB)
Announcing AWS DMS Fleet Advisor in the new AWS DMS studio console experience
John Winford (AWS)
We use a lot of DMS at Caylent so it’s exciting to see all these new features coming soon.
Deep dive on Amazon Aurora
Richard Waymire (AWS)
Deep dive on Amazon ElastiCache for Redis
Joe Travaglini (AWS), Lindsey Berg (Groupon)
Deep dive on Amazon MemoryDB for Redis
Jon Fritz (AWS), Mahesh Lagishetty (TSYS)
Enabling workloads for high availability and durability with Amazon RDS
Andy Katz (AWS), Jason Burns (Robinhood Markets)
Introduction to AWS Data Exchange for Amazon Redshift
Ryan Waldorf (AWS), Neeraja Rentachintala (AWS), Alex Bohl (Mathematica)
Introducing Amazon Kinesis Data Streams On-Demand Mode
Sai S. Maddali (AWS)
Security
{New Launch} The new Amazon Inspector for vulnerability management
Rick Anthony (AWS), Kashis Wadhwa (AWS), Anit Kumar (Uber)
A least privilege journey: AWS IAM policies and Access Analyzer
Brigid Johnson (AWS)
This is a can’t miss talk for any security-conscious AWS guru.
Locks without keys: AWS and confidentiality
Colm MacCárthaigh (AWS)
For anyone interested in privacy and encryption this talk goes deep on some of the most interesting current topics in that space.
Securing your data perimeter with VPC endpoints
Becky Weiss (AWS)
Deep dive on Amazon S3 security and access management
Becky Weiss (AWS), Meg Rose (AWS)
Be prepared: Security is shifting left
Gert Leenders (DPG Media IT)
Serverless
Best practices of advanced serverless developers
Julian Wood (AWS)
Building a serverless banking as a service platform on AWS
Dennis Winter (Solarisbank AG)
{New Launch} Introducing Amazon EMR Serverless
Radhika Ravirala (AWS)
{New Launch} Introducing Amazon MSK Serverless
Usama Naseem (AWS)
{New Launch} Introducing Amazon Redshift Serverless
Yan Leshinsky (AWS), Neeraja Rentachintala (AWS)
Evolutionary AWS Lambda functions with hexagonal architecture
Luca Mezzalira (AWS)
I’m a sucker for hexagons so this talk appealed to me.
Inside a working serverless SaaS reference solution
Tod Golding (AWS)
Storage
{New Launch} Introducing Amazon FSx for OpenZFS
Delwin Olivan (AWS), Darryl Osborne (AWS)
1M IOPS and 12.5 GB/s throughput? You have our attention.
{New Launch} Introducing the Amazon S3 Glacier Instant Retrieval storage class
Andrew Kutsy (AWS), Jessie Felix (AWS), Srinivas Bobba (Splunk)
$0.004 per GB per month, millisecond access, too many 9s of durability to count.
{New Launch} Protect data and lower costs with EBS Snapshots
Venky Nagapudi (AWS), Ashish Palekar (AWS)
Recycle Bin, EBS Snapshots Archive Tier (90 day minimum retention period)
Amazon EBS under the hood: A tech deep dive
Marc Olson (AWS), Prathana Karmakar (AWS)
I really enjoyed this deep dive into EBS. Especially the ideas behind Physalia and the SRD transport protocol. If you’re building storage-intensive applications this is required viewing.
AWS Transfer Family: The future of managed file transfer
Smitha Siriam (AWS), Russ Boyer (AWS)
Definitely worth a watch if you want an overview of the AWS Transfer Family of services and when it makes sense to use them. There’s even a walkthrough and demo at the end.
Building a data lake on Amazon S3
Oleg Lvovitch (AWS)
Deep dive on Amazon EFS
Abrar Hussain (AWS)
Deep dive on Amazon FSx for Lustre
Jordan Dolman (AWS), Jeffrey Lee (Foundation Medicine)
Compute / Containers
{New Launch} Deep dive into AWS Graviton3 and Amazon EC2 C7g instances
Ali Saidi (AWS), Jonathan Tronson (DIRECTV)
{New Launch} Introducing AWS Trainium-based Amazon EC2 Trn1 instances
Chetan Kapoor (AWS), Ron Diamant (AWS)
{New Launch) Introducing AWS Outposts servers
Tatiana Cooke (AWS), Josh Coen (AWS), Atif Kureishy (visitry.ai)
{New Launch} New EC2 storage optimized instances and parallel file system (PFS) on AWS
Amit Shah (AWS), David Desroches (AWS), Ameer Hakme (AWS)
Powering next-gen Amazon EC2: Deep dive on the Nitro System
Ben Serebrin (AWS)
The journey of silicon innovation at AWS
Ron Diamant (AWS)
You have a container image: Now what?
Massimo Re Ferrè (AWS)
Massimo walks us through the 34 ways of running a container on AWS!
Deep dive on Amazon EKS
Mike Stefaniak (AWS)
Delivering code and architectures through AWS Proton and Git
David Killmon (AWS), Rafa Alvarez (AWS)
Getting to large Amazon EKS clusters
Alex Kestner (AWS), Shyam Jeedigunta (AWS)
Kubernetes at AWS: Strategy, road map, and vision
Allan Naim (AWS), Anubhav Dhoot (Salesforce)
The powerful simplicity of Amazon ECS
Inbal Shani (AWS), Deepak Kallakuri (AWS), Dustin Pearce (Instacart)
Amazon EKS SaaS deep dive: A multi-tenant EKS SaaS solution
Tod Golding (AWS)
AI/ML
{New Launch} Introducing Amazon SageMaker Ground Truth Plus
Kumar Chellapilla (AWS), Rick Carragher (Samsara)
{New Launch} Amazon SageMaker serverless inference
Rishabh Ray Chaudhury (AWS), Ram Vegiraju (AWS)
{New Launch} Introducing Amazon SageMaker Canvas
Kosti Vasilakakis (AWS), Marc Neumann (BMW Group)
{New Launch} Introducing Amazon SageMaker Studio Lab
Michele Monclova (AWS), Emily Webber (AWS)
{New Launch} Introducing Amazon SageMaker Training Compiler
Pinary Yilmaz (AWS), Jeff Boudier (Hugging Face)
{New Launch} Reduce model deployment times with Amazon SageMaker Inference Recommender
Santosh Bhavani (AWS)
{New Launch} ML-powered call summarization in Contact Lens for Amazon Connect
Kaustubh Khanke (AWS), Naomi Hall (AWS)
Other Cool Things!
{New Launch} Remove guesswork when launching new features with CloudWatch Evidently
Varsha Velagapudi (AWS), Balamurugan Muthiah (Amazon Alexa), Adel Lahlou (AWS)
{New Launch} Optimize applications through end user insights with Amazon CloudWatch RUM
Varsha Velagapudi (AWS), Matt Crouch (Booking.com), Dan Nations (AWS)
Under the hood at Amazon Ads
Rachit Chawla, Anuj Joshi, Kun Liu, Ish Singh (all AWS)
Winging It – Jeff Barr and Corey Quinn
Jeff Barr (AWS), Corey Quinn (Duckbill Group)
This is just a fun talk between Jeff Barr and Corey Quinn. It’s nice to see an AWS Legend and an AWS pundit share some stories and chat about the history of re:Invent and AWS.
The architect elevator: Connecting IT and the boardroom
Gregor Hohpe (AWS)
I absolutely loved this talk from Gregor Hohpe and it even convinced me to buy his book. I don’t want to spoil too much but if you’re interested in being a cloud architect this should be required viewing.
Optimizing robotics and autonomous systems with AWS and Amazon
Christine Osik (Amazon), Eric Anderson (AWS)
This is a very interesting talk that walks through how AWS works with some of Amazon’s own robots.
Riot Games at the edge: Launching Valorant with AWS Outposts
Ashwin Raghuraman (AWS), David Press (Riot Games)
For the gamers out there this is a great conversation on hybrid cloud networking infrastructure combining AWS Global Accelerator, Outposts, and Riot’s own edge devices.
Building the re:Invent app with Amazon Location Service
Fernando Rocah Silva (AWS)
In this talk Fernando introduces Amazon Location Service, which became generally available in June of 2021, and how the re:Invent app team used Location Service in the re:invent app! We like this talk because it walks through a less often talked about service and how it provides real-world value to customers.
15 years of AWS with Jeff Barr
Felipe LeMaitre (AWS), Jeff Barr (AWS)
Warning, volume is very loud at the start
This is just a fun dive through the history of AWS.
Emotional intelligence to supercharge your success
Richard Hua (AWS)
This was the first time I’d seen a talk like this at re:Invent and I really enjoyed it. Richard walks through some best practices for maintaining emotional intelligence in your professional life.
Just one more video…
Just kidding! If you managed to stick with us through the end of this monster post then thank you so much for your time. We started out just trying to pick our 50 favorite sessions but we ended up with 93.
We hope to see you at re:Invent 2022!