Monte Carlo Energy Aggregation for Mobile 3D Gaussian Splatting

Rendering high-fidelity 3D scenes on a smartphone used to require hundreds of megabytes of memory. The authors of Flux-GS report bringing that down to just 2.1 MB for certain indoor scenes. This reduction makes complex 3D environments practical for mobile devices and web browsers.

The field of 3D reconstruction was recently revolutionized by 3D Gaussian Splatting (3DGS). This technique models scenes as millions of tiny, translucent, anisotropic ellipsoids. You can think of these as 3D "blobs" that can be stretched in specific directions to represent surfaces. While this approach allows for high-fidelity novel view synthesis (generating new perspectives of a scene) on desktop GPUs, it faces a massive hurdle on mobile devices.

The bottleneck is twofold: storage and compute. To capture complex lighting, such as the glint of sun on water, standard 3DGS uses high-order Spherical Harmonics (SH). These are mathematical functions used to represent how color changes depending on your viewing angle. These coefficients require significant memory. When scaled to millions of Gaussians, the volume of data overwhelms mobile memory bandwidth. Furthermore, the standard way of adding or removing these "blobs" during training often creates too many redundant points. This slows down rendering on mobile hardware.

The high cost of view-dependent light

Current mobile-optimized Gaussian Splatting methods try to solve this by using simpler, lower-order SH representations. However, these approaches often come with a price. The authors note that prior distillation-based methods (transferring knowledge from a large model to a small one) impose an extremely heavy training burden. Other methods that simply use low-order SH lose high-frequency specular details. These are the sharp, shiny highlights that make a scene look realistic.

Additionally, many existing lightweight variants rely on single-view gradient-based densification. This is a process where the model looks at a single camera angle and adds more Gaussians to areas with high error. The authors argue this is prone to overfitting to a specific view. It also produces an excessive number of primitives (individual 3D points). As shown in, these issues cause a massive storage footprint and low frame rates (FPS).

This makes fluid interaction impossible on a handheld device.

Compressing light through Monte Carlo sampling

To resolve this, the authors propose Flux-GS. It is built around a Monte Carlo Specular Energy Aggregator. Instead of storing every nuance of high-order SH coefficients, the method samples radiance residuals across a uniform sphere. Think of this like taking a quick, representative snapshot of how light hits a surface from all sides. This is much more efficient than recording every single photon's path.

The process follows a specific sequence: 1. Spherical Sampling: The system samples $K$ points on a unit sphere. This captures the directional energy of specularities. 2. Energy Aggregation: It calculates the total energy magnitude ($E_{mag}$) and the dominant direction ($E_{dir}$). This collapses bulky high-order parameters into a compact latent space. 3. SH Enhancement: Simple compression loses fine detail. The authors introduce an Attribute-Conditioned SH Enhancement module to fix this. It uses a lightweight Multi-Layer Perceptron (MLP) to predict small corrections (offsets) to the base color.

Crucially, the authors designed this enhancement to be "baked in." The offsets are calculated during training and added to the Gaussian parameters themselves. This means the device does not run a neural network during actual rendering. It simply reads the pre-calculated, enhanced values.

To clean up the scene structure, the authors also implement a Multi-view Alpha-based Densification and Pruning strategy. Instead of looking at one camera, the system uses stratified camera sampling to look at the scene from multiple angles. It uses the "alpha" (transparency or opacity) contribution of Gaussians to decide which ones are necessary. As illustrated in, this allows the model to prune redundant points in well-reconstructed areas.

It also adds new Gaussians only where multi-view errors are high.

Efficiency gains on mobile hardware

The results reported by the authors suggest a breakthrough in balancing visual quality and hardware constraints. Testing on a smartphone with a Snapdragon 8 Gen 3 GPU showed impressive results. Flux-GS achieved a rendering speed of 147 FPS for indoor scenes. This significantly outperforms the Mobile-GS baseline .

The efficiency in storage is even more striking. For the Mip-NeRF 360 indoor subset, Flux-GS required only 2.1 MB of storage. In contrast, the standard 3DGS approach would require much more. The paper also highlights a drastic reduction in the number of Gaussian primitives. As shown in, Flux-GS achieves 61% and 26% reductions in memory footprint.

This is done by decoupling SH into base and view-independent components.

The authors also emphasize the training speed. Because the multi-view pruning strategy prevents unnecessary Gaussians, training converges much faster. For example, on the "bonsai" scene, Flux-GS completed training in 11 minutes. This is much faster than the 114 minutes required by Mobile-GS [Table 9].

Limits of the compressed representation

The authors acknowledge several trade-offs. First, compressing high-order SH into a lower-order subspace causes some loss. Flux-GS inevitably loses the capacity to model extremely complex, mirror-like specular reflections. A full third-order SH model would capture these better. Users might notice a slight loss in sharpness on highly reflective surfaces.

Second, the training process itself is not entirely "light." For the first 3,000 iterations, the system still optimizes full third-order SH. This establishes a high-frequency foundation. This means peak memory consumption during the initial training phase remains high. It is comparable to standard 3DGS, even if the final deployed model is tiny.

Finally, the multi-view pruning strategy carries a risk of over-pruning. If a geometric detail is only visible from a very narrow angle, the algorithm might miss it. The stratified camera sampling might not include that specific angle. In this case, the algorithm might incorrectly label that Gaussian as redundant. This could create holes or artifacts in the reconstruction.

Verdict: A viable path for mobile AR

Flux-GS is a practical tool for building augmented reality (AR) or mobile web applications. It moves technology from "desktop-only" to "handheld-ready" by prioritizing inference-time efficiency. By baking neural enhancements into static parameters, the authors ensure intelligence does not hurt the frame rate.

The method is suitable for production deployment where storage and battery life are vital. This holds true as long as scenes do not rely on perfect, mirror-like reflections. Code is reportedly available at the project's official site: https://xiaobiaodu.github.io/flux-gs-project/.

Figures from the paper