Skip to main content Skip to secondary navigation

Exploring human-robot collaboration in industrialized construction assembly production by using computer vision algorithms

Main content start

Research Team

Our Motivation:

"After visiting many factories practicing industrialized construction, we observed thousands of steps of manual work in the assembly processes. This reality pointed out that we need to have a systematic way of understanding the off-site production to assess whether using the manual work is the best fit, or should we use fully automatic processes, or deploying human-robot collaborate on the work."


Icon Contribution

Research Contribution

Develop a methodology using computer vision algorithms to identify work activities and opportunities for implementing an optimal level of human-robot collaboration. 

Define a set of metrics to evaluate the impact of human-robot collaboration in prefabricated part assembly processes. 

Icon Problem


Practical Problem

The construction industry is adopting the industrialized construction concept in its new projects through off-site production for superior quality, short cycle time, and minimized environmental impact. 

However, we observe many companies just moved the production process from on-site to off-site without rethinking the manufacturing strategies, leaving the production line to be highly manual and labor intensive. 

Therefore, further analysis of the correlation between factors such as the degree of automation, the performance of machinery and tools, the level of lean and standardization, as well as the size of the workforce and business will answer the question. 

Conceptual Problem

Identifying and implementing optimal levels of automated assistance in construction workplaces is still a significant challenge.

To find an optimal prefabricated component assembly solution, the factory owners need to understand the cost and benefits of implementing human, robotics, and human-robot collaboration to ensure the implemented production solution is productive, safe, and profitable.

Icon Solution


A management framework to evaluate the impact of different levels of automation on off-site production assembly processes. 

We propose

  • metrics to measure the quality, safety, schedule, and cost of the current process and use computer vision algorithms to capture data without interrupting the work.
  • to develop a computer vision-based production analysis method to analyze the production procedures without disrupting the current workflow or the workers.
Icon added value

Added Value For The Industry

·      This framework could serve as a basis for the AEC companies to evaluate the impact of implementing human, robotics, or human-robot collaboration in a systematic manner without interrupting current processes. Once the method is translated into practice, it could help the designers, builders, manufacturers, and owners identify opportunities to optimize and automate the production process. 

Reviewing and identifying places for improvement in off-site construction processes could trigger innovative practices such as human-robot collaboration in the AEC industry, ultimately improving the productivity and safety of the projects. 

Icon Cooperation partner

Cooperation Partner

Autodesk, USA
Icon Timeline





Year 2022

Research became awarded: Exploring human-robot collaboration in industrialized construction production by using computer vision algorithms


Milestone 1

(in progress)

Month 1-6

  • visit prefabrication factories and collect information (e.g. activities, time, utilization rate…)
  • collect data regarding their current component assembly processes.
  • Establish metrics for measuring process performance.
  • Map the current process
  • Analyze the process to identify opportunities for implementing robotics solutions.

Initial interviews and visits to factories are done, we are building a platform for deploying computer vision algorithms for real-time streaming analysis. 

Milestone 2

(in progress)

Month 3-8

  • identify human-robot collaboration practices in assembly production.
  • Extract information regarding the solutions’ impact on productivity, safety, and schedule of quality.

We are trying to obtain production images and videos for computer vision based analysis and production data extraction.  

Milestone 3

Month 1-8

  • train and tune computer vision algorithms to capture work activities and data of proposed metrics with video observations.
  • Validate the results to understand the accuracy of the model.

Milestone 4

Month 8-12

  • contrast human, robot, and human-robot collaboration with the same assembly actions.
  • Understand ergonomics, efficiency, and intrinsic human value through quantifiable measurements.

If you want to participate in the project please reach out to Bochen Zhang.

Relevant Links for this Research