AWS is announcing the general availability of AWS Glue 6.0, providing 30% lower pricing than previous AWS Glue versions and introducing full support for Apache Iceberg v3 features.
According to AWS, AWS Glue 6.0 is built on a fully modernized runtime, Apache Spark 4.1, Python 3.13, and Scala 2.13, delivering faster performance.
With this release, “AWS Glue provides the most complete Iceberg v3 implementation on any fully serverless managed Spark service, along with new capabilities that simplify ETL authoring, improve PySpark performance, and enable real-time streaming with single-digit millisecond latency,” the company said.
AWS Glue 6.0 delivers the complete Apache Iceberg v3 specification, built on Iceberg 1.11.0. The headline feature is the VARIANT data type with shredding support, which achieves faster query read performance compared to traditional string data type columns for semi-structured data.
With VARIANT shredding, users can store and query JSON, logs, and event data without flattening schemas, eliminating duplicate data copies, custom parsing code, and pipeline breakage when schemas change. This capability transforms how teams handle semi-structured data at scale, the company said.
Additional Iceberg v3 capabilities include:
- Geometry and Geography data types: Enable native spatial processing for GIS analytics, location intelligence, and geospatial data pipelines directly on managed Spark.
- Nanosecond-precision timestamps: Support IoT sensor data, scientific computing, and high-frequency financial workloads that require precision beyond standard milliseconds.
- Unknown type handling: Process data with unexpected or evolving schemas without pipeline failures, providing resilience against upstream schema changes.
AWS Glue 6.0 also includes a significant upgrade in Spark 4.1, the modern runtime engine, including features such as
Spark declarative pipelines, Arrow-native Python UDFs and UDTFs, and a real-time streaming mode.
AWS Glue 6.0 is generally available today in all AWS Regions where AWS Glue operates.
For more information about this news, visit https://aws.amazon.com.