Function2Scene: 3D Indoor Scene Layout from Functional Specifications

Function2Scene: Designing 3D Indoor Layouts Based on Human Activity and Needs

Instead of just asking an AI to "put a bed in a room," this system lets you describe how you live. You might ask for "a quiet place for a senior to read." It then uses a smart loop of checking and fixing. This ensures the furniture actually works for those specific human needs.

In 3D scene synthesis (the process of generating digital indoor environments from text), the goal is to create realistic rooms. Currently, the state of the art relies on object-centric prompts. Users specify what should be in a room, such as "a bedroom with a queen bed." However, a major gap remains. Current models struggle to understand how a space should be used. They can place objects that look plausible. Yet, they often fail to account for ergonomic (human-body compatibility), social, or environmental requirements.

The Question

Researchers from Simon Fraser University and Brown University addressed a specific problem. They asked how to move from generating mere collections of objects to designing functional spaces. They investigated if a framework could translate high-level "design briefs" into 3D layouts. These briefs describe occupant personas (user profiles) and intended activities. The core challenge is that functional specifications are high-level and heterogeneous (diverse in type). They do not provide coordinates. Instead, they provide requirements like "ensure the cook stays part of the gathering."

Why The Old Answer Was Incomplete

Historically, the field has oscillated between two extremes. Early work relied on hard-coded design guidelines. These were mathematical cost functions (rules used to calculate how "good" a design is) for things like clearance and alignment. However, these required massive manual effort. They were also brittle when faced with new scenarios. Later, the field pivoted toward data-driven deep learning. This used transformers and diffusion models to learn statistical patterns of furniture placement.

While these modern models produce visually coherent scenes, they use an "implicit" approach. They learn where a chair usually goes. But they do not understand why it goes there. As noted in the paper, an LLM (Large Language Model) tasked with direct scene generation often fails. It lacks an explicit understanding of functional demands. As seen in the initialization phase of, a raw LLM-generated layout might be physically plausible but practically unusable.

It may feature overlapping objects or blocked pathways.

What They Did

The authors developed Function2Scene. This framework treats interior design as a multi-stage optimization problem. It uses a "check-and-repair" loop. The process begins with an initialization stage. An LLM parses a functional prompt into two outputs. First, it creates a structured scene description. Second, it generates a customized set of design constraints. These constraints come from a taxonomy (a classification system) of 17 criteria. They cover four domains: Spatial, Ergonomic, Activity, and Environmental .

The system does not ask an LLM to produce a finished JSON file in one shot. Instead, it employs a tool-augmented iterative refinement process. After generating an initial layout, the agent enters a sequential evaluation loop. It invokes specialized tools to validate the scene. These include geometric algorithms for measuring path widths or collisions. It also uses LLM-based reasoning for semantic judgments. Finally, it uses VLMs (Vision-Language Models) to assess visual qualities like balance or glare . If a constraint is violated, the LLM generates a targeted refinement action. This might involve repositioning or resizing an object. The agent then re-evaluates the scene.

What They Found

The researchers evaluated their method using 30 professional interior design cases from Architectural Digest. They conducted a two-alternative forced-choice (2AFC) perceptual study. In this study, 30 human participants chose between two layouts. The results show that Function2Scene is more effective at satisfying functional requirements than existing baselines.

The authors report an aggregate preference rate of 94.3% for their method. This means that in nearly all head-to-head comparisons, humans preferred Function2Scene. Compared specifically against Holodeck, the preference rate was 92.2%. Against iDesign, it reached 94.4%. Against LayoutVLM, it hit 96.7% [Table 2]. This high preference rate indicates the system produces much more usable designs.

Crucially, ablation studies (tests where components are removed to see their impact) revealed a key insight. The "intelligence" of the system depends on the interplay between the LLM and grounded tools. The paper finds that iterative updates without specialized evaluation tools actually perform worse. This suggests that unguided refinement is counterproductive. The qualitative results in and reinforce this.

The system successfully handles complex requirements like "blackout sleeping" or "workspace for dual monitors." It does this by adjusting the layout to meet specific ergonomic and environmental thresholds.

What This Changes

The success of Function2Scene suggests a shift in how we approach generative spatial AI. We should stop treating layout generation as a simple translation task. Instead, we should treat it as an agentic optimization task. This moves from Intent $\rightarrow$ Constraints $\rightarrow$ Iterative Refinement.

If this methodology generalizes, several implications emerge. First, it provides a blueprint for building "functional" agents. This could apply to robotic environment generation. In that field, the utility of a space is as important as its appearance. Second, it demonstrates the value of tiered constraint hierarchies. These start with rigid spatial rules (Tier 1) and move toward subjective environmental comforts (Tier 6). This hierarchy helps manage multi-objective optimization in LLM-driven pipelines.

However, the paper does not show how this system scales to large, multi-room environments. It also does not address how it handles extremely vague user inputs. Future work could integrate the framework with a physics engine. This would replace current numeric checks with true embodied simulation. Such a step would test if the agent can truly navigate the space it has designed.

Figures from the paper