Skip to main content Skip to secondary navigation

Neural Design Phase: Bridging Schematic and Development Phases

Main content start

Research Team

Our Motivation:

"The research was motivated by the need to better serve designers and owners at the early stages of the design process. In addition, we were inspired by recent advancements in deep generative models and generative design. The goal is to let designers and owners generate a BIM by simply sketching and describing their buildings.

Imagine for a second a method capable of retrieving similar buildings or sites worldwide and generating new sustainable buildings assimilating various lessons learned.


Icon Contribution

Research Contribution

The research aims to answer questions such as:

“How can a BIM be generated conditioned also on a single perspective sketch?”.

It will explore deep generative models, neural fields and their applications in design for the AEC. 

Icon Problem


Practical Problem

The high latency between the schematic and development phases causes a delayed validation of the initial design ideas, making it difficult and costly to reset the initial design directions.

Conceptual Problem

The high latency between is caused also by the lack of a real-time generation of BIM outcome from simple inputs like sketch, text, image or mock-ups.

Icon Solution


The solution consists in a method that combines schematic and development phases into a unique one:

A Neural Design Phase. From a text, speech, sketch, image, or mock-ups, our AI generates a building information model visible in immersive mediums.

Icon added value

Added Value For The Industry

It will facilitate the industry partners to generate BIM from initial design ideas reducing the latency between schematic and development phases. 

Icon Cooperation partner

Cooperation Partner

The referenced media source is missing and needs to be re-embedded.
Goldbeck GmbH, Germany
NVIDIA Corporation, USA
NVIDIA Corporation, USA
Stanford University Human-Centered Artificial Intelligence, USA


Icon Timeline





Year 2022

Research became awarded: Neural Design Phase: Bridging Schematic and Development Phases


Dec 2022

Paper Submitted to Automation in Construction (under review)


Mar. 2023

Update & Progress Report


Apr. 2023

Presenting Poster


Sep. 2023

Final Report + Summary Update


If you want to participate in the project please reach out to Alberto Tono.


Project Summary

The CIFE Seed project provides a step towards converting hand-drawn building sketches into Building Information Models using a deep learning framework. However, the current version needs more detailed information and editability. Recognizing the need for a user-friendly platform to test our method better, we started the development of eSketch. eSketch is a design interface that evaluates performance during a neural design phase (NDP). The forthcoming CIFE Seed 23/24 project, "eSketch: A Multimodal Generative AI Design Interface and Human-Centered Approach," promises to address these needs, finalizing the development of eSketch to test our method more accurately. Collaboration with CIFE partners pinpointed industrialized construction (IC) as an ideal testing ground for our discrete diffusion model. Our ongoing efforts aim to resolve the challenges presented by the model's discrete nature, enhancing its functionality and application in practical design scenarios.

Relevant Links for this Research

This research will be continued and the Final Report will be issued as part of the ongoing study:

eSketch: A Multimodal Generative AI Design Interface and Human-Centered Approach for a Neural Design Phase