MinIO, the leader in high-performance object storage for the exabyte AI era, is debuting new AIStor support for Amazon's S3 Express API, becoming the first and only AI data storage platform to adopt Amazon's S3 Express API, according to MinIO.
Amazon's S3 Express API is a more streamlined iteration of the general-purpose S3 API, centered more on maximizing performance for AI and data-intensive analytics workloads. While capable of delivering 10x faster performance compared to the S3 Standard storage class, it comes with a hefty price tag and a lack of data protection and replication capabilities.
With AIStor support, MinIO makes S3 Express API more accessible to its customers, combining the API’s performance advantages with full active/passive, disaster recovery (DR), and batch replication support—at no extra cost. This update affirms MinIO’s ongoing commitment to provide full and complete S3 API compatibility.
"Until now, use of S3 Express has typically been reserved for data that requires high speed access at the expense of enterprise features like replication and lifecycle management, due to its cost premium over S3 Standard," said Garima Kapoor, co-founder and co-CEO, MinIO. "We're changing that because we believe it can and should become the standard API for all analytical data and AI workloads. With the AIStor S3 Express API, pricing remains the same so enterprises can now put all of their analytical and AI data, not just a subset, in 'express mode.' This is drawing tremendous excitement from customers."
AIStor support allows enterprises to continue using the general purpose S3 API for raw source data requiring versioning and immutability before any processing, while also implementing S3 Express API for all other downstream workloads and derived data sets, including high intensity data processing for data lakehouse analytics with Apache Spark and Iceberg, AI data pre-processing, and AI model training and inference.
"In most lakehouse environments, versioning is already handled by table formats like Apache Iceberg, which provide atomic operations at the file or table level," said Sanjeev Mohan, principal analyst at SanjMo. "But when versioning is also enabled at the storage layer, it can create unnecessary duplication and complexity. Removing it from the storage layer simplifies the backend architecture, lowers CPU usage on storage servers, and helps prevent accidental storage growth and rising costs."
Coming at no extra cost, AIStor’s S3 Express API support offers the following benefits:
- Accelerated PUT and LIST operations, delivering up to 20% faster PUT operations, lowering CPU utilization, and up to 447% faster time-to-first-byte (TTFB) LIST operations compared to AIStor general purpose S3 API
- New atomic, exclusive append operations that empower direct and safe object modification, eliminating multi-step update workflows
- Full active/passive, disaster recovery (DR), and batch replication support, bringing the full performance boons of the S3 Express API without incurring a loss in business continuity
- Streamlined and simplified API behavior through eliminated redundancies and new simplifications and guardrails that improve the developer experience as well as application resiliency, predictability, and security
To learn more about AIStor’s support for the S3 Express API, please visit https://min.io/.