Preview release of Spark 4.0

To enable wide-scale community testing of the upcoming Spark 4.0 release, the Apache Spark community has posted a Spark 4.0.0 preview2 release. This preview is not a stable release in terms of either API or functionality, but it is meant to give the community early access to try the code that will become Spark 4.0. If you would like to test the release, please download it, and send feedback using either mailing lists or JIRA. The documentation is available at the link.

There are a lot of exciting new features added to Spark 4.0, including ANSI mode by default, Python data source, polymorphic Python UDTF, string collation support, new VARIANT data type, streaming state store data source, structured logging, Java 21 support, Java 17 by default and many more.

We’d like to thank our contributors and users for their contributions and early feedback to this release. This release would not have been possible without you.

Spark News Archive