αDepth: Learning Single-Pass Soft Boundary Decomposition for Stereo Conversion

αDepth: Solving Soft Boundary Ambiguity for High-Fidelity Stereo Conversion

When turning a 2D photo into 3D, things like hair or blurry backgrounds often look messy or "bleed" into each other. This happens because the software struggles to decide which depth belongs to the foreground object and which belongs to the background when they are blended into the same pixel. αDepth solves this by mathematically splitting these tricky areas into separate layers of foreground and background. This makes the 3D effect look much more realistic.

The Problem

The core issue in stereo conversion—the process of synthesizing two slightly different views from a single image to create depth—is the "soft boundary." At these boundaries, such as strands of hair or defocus blur (camera-induced blurring), the observed color is a mixture of foreground and background. Most monocular depth estimators, which predict the distance of objects from the camera, assign a single depth value per pixel .

This creates a fundamental depth ambiguity. If a pixel contains 50% hair and 50% sky, assigning it one depth value causes the resulting 3D warp to look broken. This often manifests as "flying pixels" or heavy background bleeding [Figure 3a].

Existing solutions usually fall into two camps, both of which fail in production pipelines. Alpha matting techniques can extract opacity to model layers. However, they typically require manual guidance like trimaps (user-drawn outlines) or segmentation masks to identify foreground and background . This is a non-starter for automated, scene-level processing. On the other hand, auxiliary-free matting models exist. They are often specialized for specific categories like humans. They fail to generalize to varied phenomena like camera defocus blur.

How It Works

The αDepth approach moves away from trying to extract specific objects. Instead, it focuses on decomposing local boundaries. The architecture, detailed in, uses a dual-path encoder to pull both high-level semantic context and fine-grained structural details.

The implementation relies on three core pillars:

1. Circular Alpha Representation (CAR): This is the mathematical heart of the paper. Standard alpha matting often suffers from the "alpha valley" issue. This occurs when neural networks struggle to predict sharp transitions between overlapping targets [Figure 3b]. To fix this, the authors project the alpha value $\alpha \in [0, 1]$ into a continuous trigonometric space. They use sine and cosine transformations: $\alpha_{sin} = \sin(2\pi\alpha)$ and $\alpha_{cos} = \cos(2\pi\alpha)$. This wraps the linear scale around a circle. Consequently, $\alpha=0$ (background) and $\alpha=1$ (foreground) map to the same coordinate. This turns discrete jumps into a continuous manifold. This makes it much easier for the model to optimize .

2. Layered Decomposition: Instead of a single depth/color prediction, the model jointly estimates layered foreground (FG) and background (BG) information. Specifically, it predicts layered color ($\bar{I}{FG}, \bar{I}}$) and layered depth ($\bar{D{FG}, \bar{D}$), along with blending weights. The authors note that a two-layer local decomposition is sufficient to resolve most soft boundaries .

3. Layered Warping: During inference, the predicted alpha, color, and depth are used to perform a modified version of softmax splatting (a technique for projecting pixels into 3D space). The model projects the foreground and background layers separately. It then composites them back together using the estimated alpha. This ensures the reconstructed 3D view respects the underlying layered structure.

Numbers

The authors report significant improvements over existing state-of-the-art (SOTA) methods. In stereo image conversion on the Mono2Stereo dataset, αDepth achieves an S-PSNR of 25.60. This metric measures accuracy specifically in soft regions. It outperforms the previous best, HairGuard, which scored 25.05 [Table 1].

For video conversion on the Marvel-10K dataset, the authors highlight superior temporal consistency. They report an FVD (Fréchet Video Distance, a metric measuring how "realistic" and stable a video looks) of 1.72. This is a notable improvement over HairGuard’s 2.14 [Table 1]. Regarding computational efficiency, the model is relatively lightweight. For an input size of $448 \times 640$, the model requires 607.87 MB of peak GPU memory. It achieves an inference speed of 0.0153 seconds per image on an NVIDIA GeForce RTX 4090 [Section B.2].

What's Missing

While the results are strong, there are three clear gaps that a practitioner should consider:

• Dependency on Base Geometry: αDepth is a "plug-and-play" module that refines boundaries. However, it does not fix the global scene structure. If the underlying monocular depth model produces severe geometric distortions in opaque regions, αDepth cannot correct them [Section D].
• Complexity Limits: The model assumes a two-layer local decomposition is sufficient. In highly complex scenes where three or more semi-transparent layers overlap at a single pixel, the model may not capture all layered information [Section D].
• Temporal Flickering: The core model is image-based rather than video-based. Therefore, it lacks explicit temporal constraints. The authors concede that the absence of spatio-temporal modules may cause flickering in dynamic scenes [Section D].

Should You Prototype This

Yes, if you are building an automated 3D/stereo conversion pipeline. The transition from manual, guidance-based matting to a single-pass, autonomous decomposition is the real win here. The fact that it runs in ~15ms on a 4090 suggests it can fit into a production path without becoming a massive bottleneck.

However, do not treat it as a replacement for a good depth estimator. Think of it as a specialized "boundary refiner" that sits on top of your existing geometry stack. If your current pipeline is plagued by halos and bleeding around hair or glass, this is worth a one-week prototype. Code is reportedly available; see the paper for the canonical link.

Figures from the paper