Skip to main content Skip to secondary navigation

Developing Virtual Assistants for the AEC-FM Industry

Main content start

Research Team

Our Motivation:

Many BIM-based information systems have been developed for the AEC-FM industry. However, practitioners today still cannot realize the full benefits of rich, up-to-date information because of accessibility issues. This is because they must master not just the technical systems but also the digital information, language, and tools describing the systems.


Icon Contribution

Research Contribution

Theoretically, this study will contribute a natural language (NL) -driven ontology model that supports an end-to-end semantic parser to translate complex NL utterances into system queries or functional calls.

Practically, it is expected to develop VA prototypes integrated with existing systems for use cases.

Icon Problem


Practical Problem

The “three-hand problem” was formulated to illustrate how virtual assistants (VA) can help improve the efficiency of retrieving information about construction. 

Slow hand: Practitioners who don’t have the skills to use the systems need to wait for technical assistance to access information, resulting in decision latency. 

Difficult hand: It is difficult for practitioners to learn how to use new systems, because of lack of training resources, different backgrounds, and reluctance to change. 

Busy hand: Practitioners are often busy with their hands when performing tasks or wearing gloves. If they lack the needed information midway, it can result in reduction in productivity or poor quality of work.

Conceptual Problem

The AEC-FM industry lacked a NL-driven ontology that can map between NL query utterances and schemas in BIM-based systems.

Icon Solution


To address the problem, the proposed VA can provide a natural language (NL) interface so that practitioners can directly “ask for” needed information from the systems with little learning.

NL is powerful in composing various filter conditions in queries and retaining contexts in the history of dialogues. As a result, users can spend minimal effort searching for the data.

Lastly, equipped with a speech recognition module, VA can make use of the voice to enable on-site and on-task information delivery in the hands-free condition.

Icon added value

Added Value For The Industry

By completion, this project will facilitate the development of VA for the AEC-FM industry, reducing the learning cost for systems and improving the efficiency of information retrieval as well as the user experience. In the long term, the NL interface will help bridge the gap between the white- and blue-collar workers in the AEC-FM industry by providing better access to technology.

Icon Cooperation partner

Cooperation Partner

Autodesk, USA
Logo Stanford Healthcare
Stanford Healthcare, USA


Icon Timeline





Fall 2022

User study: Testing the VA prototype with Stanford students 


Winter 2023

Use case 1: Data collection, prototyping and field testing


Spring 2023

Use case 2: Data collection, prototyping, and field testing 


Summer 2023

Deliver a summary report and presentation to CIFE members 



If you want to participate in the project please reach out to Junwen Zheng (

Relevant Links for this Research