Gluten’s target user is anyone who wants to accelerate SparkSQL fundamentally. As a plugin to Spark, Gluten doesn’t require any change for dataframe API or SQL query, but only requires user to make correct configuration. Gluten is a middle layer responsible for offloading JVM-based SQL engines' execution to native engines. - gluten/gluten-core/src/main/java/org/apache/gluten/memory/memtarget/DynamicOffHeapSizingMemoryTarget.java at main · apache/gluten Here's the architecture, how to wire it up on a Scala workload, and where the rough edges are. Spark's JVM is great at distributed scheduling and not particularly great at single-node compute. The basic rule of Gluten’s design is that we would reuse spark’s whole control flow and as many JVM code as possible but offload the compute-intensive data processing part to native code. Gluten use off-heap memory for certain operations. The absolute amount of memory which can be used for off-heap allocation, in bytes unless otherwise specified. Note: Gluten Plugin will leverage this setting to allocate memory space for native usage even offHeap is disabled. This document provides an overview of Apache Gluten's architecture and design principles. It explains the three-tier architecture that enables Apache Spark to offload query execution to native engines This document covers the memory management system within the Velox backend of Apache Gluten. It details how native memory allocation is coordinated between the JVM (Spark) and native execution engines 以下配置可加入 spark-defaults.conf 文件以优化Gluten性能： 参数列表. 设置Java OffHeap内存大小（Gluten会利用此配置为Native操作分配内存，即使OffHeap未启用）。 若Gluten出现OOM问题建议增大此值. 启用Gluten（默认 true，实验性参数。 建议通过 spark.plugins 控制启用） 实验性：启用隔离内存模式。 若Gluten需处理同一会话中的并发查询，建议启用以避免OOM。 启用后建议降低 spark.memory.storageFraction. 强制使用ShuffledHashJoin而非SortMergeJoin。 若需利用存储分区连接优化则设为 false. DynamicOffHeapSizingMemoryTarget relies on JVM heap shrinking (returning committed heap pages to the OS) to reclaim memory for off-heap usage. The static initializer already checks for -XX:+ExplicitGCInvokesConcurrent and -XX:+DisableExplicitGC, which would prevent shrinking from working. Gluten requires setting both on-heap and off-heap memory sizes, which initializes different memory layouts. Improper configuration of these settings can lead to lower performance. To fix this issue, dynamic off-heap sizing is an experimental feature designed to simplify this process.