Breaking Speed Limits: Rust and the Next-Gen S3 Transfer Manager
by Waqar Ahmed Khan
Discover how we are rewriting our C-based S3 Transfer Manager in Rust, achieving over 100Gbps transfer speeds. Join us to explore how this open-source library empowers customers to run massive AI/ML workloads at scale, revolutionizing data-intensive operations in the cloud.
Speaker
Waqar Ahmed Khan
I’m a Software Development Engineer at Amazon, where I’ve been a key member of the S3 Client team for over two years. My journey in high-performance computing began with optimizing our C-based S3 Transfer Manager, pushing it to become one of the fastest in the market. Now, I’m diving deep into Rust, working on rewriting this critical component to unlock even greater performance and safety.
With a passion for systems programming and a knack for squeezing out every bit of performance, I thrive on the challenges of large-scale data transfer in cloud environments. When I’m not optimizing code, you might find me playing video games or reading books - always eager to explore new worlds, whether digital or on paper.
I’m excited to share my experiences and insights from the frontlines of high-performance cloud computing, and to contribute to the growing Rust community. And while I can’t choose a favorite member of One Direction (they’re all equally talented, right?), I can definitely pick a favorite between C and Rust - but you’ll have to attend my talk to find out which!