Validating the Robotics Evaluation Framework
Research Team
Research Overview
Observed Problem:
Recent advances in sensing, computing, and mapping technologies are enabling the use of robots in unstructured environments like construction; however, innovators in construction companies do not have a comprehensive and consistent evaluation method to assess the best option for a given project.
Primary Research Objective:
This research aims to validate the Robotics Evaluation Framework (REF) formalized with previous CIFE funding for innovators in construction companies looking to deploy robots on site.
Potential Value to CIFE Members and Practice:
- Enable the successful adoption of construction robots with corresponding safety benefits for the workforce, and quality, schedule, and cost benefits for projects and companies.
- Participate in a course on deploying robots on construction sites leveraging the REF.
- Evaluate over 15 construction robots that are in use today.
Research provides relevant insights for:
Construction
Research and Theoretical Contributions
A REF for comprehensive and consistent comparison of robotic and manual construction work.
Opportunities to deploy construction robots and managerial challenges to do so.
Industry and Academic Partners
HD lab
Keywords:
Robotics, Automation
Original Research Proposal
Previous Research
Robotic Evaluation Framework - 2nd Year
Research Updates & Progress Reports
Progress Report June 2021
The course involved 11 General Contractors and 10 robots from 8 countries in total. Pairs of students from Stanford University and the University of Lima applied the REF to consistently analyze each of the cases.
# GC Robot
- DPR Hilti concrete drilling
- Obayashi Material Handling (Obayashi)
- Bechtel Kewazo scaffolding
- Megacentro Exyn autonomous drones
- Produktiva Safeai
- NCC Boston Dynamics Spot
- Swinerton Canvas drywall finishes
- DPR Canvas drywall finishes
- HDlab Exowear
- MT Højgaard Civ Robotics
- Implenia Tybot
- Traylor Brothers Tybot
The class evaluation found that all robot cases improved safety conditions for workers by reducing strenuous and repetitive manual work. In two cases, the general hazardous conditions of the work were not reduced because the robots augmented the human but did not remove the operator from the place of danger. Accuracy was improved on average by 55% with a range from 25-90%. In two cases we could not quantitatively assess accuracy improvements due to lack of experimental data. Traditional work was on average 5% and could be reduced by 62% according to the robot rework reported. Finally, the schedule was reduced in 8 of the robots under study by 1.5x on average; in one case remained the same, and in one increased. Finally, the costs were reduced by 15% on average, with 6 cases reducing the cost and 4 increasing the traditional cost to perform the task.
We surveyed the class participants (industry partners and students) to gain insights on the applicability of the REF to support decision making and whether the variables in the REF are comprehensive, i.e., enable all the comparisons the industry partners want to do. We were also able to determine how consistently the students applied the framework, the effort required, and the main bottlenecks for such comparisons.
97% of the participants answered that the framework included all the variables needed for decision-making. Suggestions for REF modifications included adding environmental criteria, increasing the details of the cost analysis with categories of indirect costs, including optional hypothetical scenarios to run sensitivities on the REF, a feedback loop to improve the robot design based on the REF results, and adding relevant institutional standards.
All survey participants but one answered the evaluation sequence was logical and agreed the REF was useful or mostly useful. 75% of the participants did not find any of the variables included unnecessary.
The overall student effort to perform the comparative analysis with the REF was ~20 hours. The median delta of hours working in the REF between pair of students studying the same robots was 3 hours. The students’ effort was focused on the process and schedule analysis (12% and 13% of the total REF time respectively). Nonetheless, the coordination and meetings with the key stakeholders to gather relevant project information took on average 37% of the time spent working on the evaluation.
The bottlenecks of the evaluation were Schedule and Quality analysis for students and Cost and Safety according to the Industry Partners. The schedule was difficult due to the lack of data, the uncertainty of estimations, and the need to trace back to the schedule critical path to decide the impact to deploy the robot. Quality was difficult to evaluate due to the lack of experimental data and subjectivity involved in the evaluation. From a cost perspective, the industry partners found reluctance to share cost information and the complexity of the analysis. Finally, safety was difficult to evaluate because the long-term effects of repetitive and ergonomically stressful tasks were not well documented or broken down in a way to compare with the robot impact in the short term.
All pairs reached the same conclusions and recommendations for their industry partners. One of the groups, although they reached similar final numbers, disagreed on their final recommendation. When we look at the final tab of their REF, the result is 47% preference toward the traditional method and 52% to the robot. This competitive result justified the split recommendation from the students.
All in all, it appears from the feedback that the process of getting answers following the framework allowed the students and practitioners to gain a consistent understanding of how the robot will impact their business. According to one industry partner, “it is good to have a baseline to compare". One of the practitioners from Lima stated: “I thought it [the robot] was more expensive. I think its adoption is closer than we think. However, it is important to analyze the alternatives early. Helps you see the closeness or distance of the benefits or disadvantages.” One of the robot manufacturers stated that they “used the conclusions from the evaluation to improve the process and design of the robot, to give it a better evaluation. Thus, we used it as a design tool and not just as an advanced business case tool.”
Even at this early stage, most robots showed potential benefits for adoption in construction with 5 of the 10 robots performing better than traditional methods in the four variables studied. According to one of the industry partners, their vision for construction robots is “that all construction projects will consider the use of robots and even aim for at least one robot to be used in the project, which they have not used before.” Another partner identified that robots could achieve “70% productivity improvement by 2025.”
Some remaining challenges are change-resistance, training effort, cost of deployment, unpredictable site conditions, lack of information about robots’ capabilities, technology not sufficiently mature to buy out of the box solutions, lack of robot adaptability to the construction and design conditions, contract changes, and lack of suppliers (e.g., in Latin America).
Progress Report November 2020
We are seeking construction companies and robot start-ups interested in participating in the class next Winter quarter.
Recording of kickoff event for CEE students: Kickoff recording
Event Slides
Detailed Research Overview & Progress Updates
Overview & Observed Problem
Over the past few years, progress in mobility, autonomous manipulation skills, Artificial Intelligence reasoning, and physical interaction through multimodal sensing and environment modeling present new opportunities for the application of robotics in unstructured environments (Khatib et al., 2016). The International Federation of Robotics forecasts 4,200 construction robotics units to be sold from 2019 to 2021 (IFR, 2018) and Bock and Linner (2016) outlined 24 categories of on-site task-specific construction robots. Tasks such as drilling, painting, brick-laying, and excavating are being automated and performed with the aid of robots (Ragaglia et al., 2017; Usmanov et al., 2017; Yamada et al., 2017).
As these robotic construction methods are increasingly being prototyped and adopted in the field, innovation leaders in construction must be able to consistently evaluate the impact of deploying robots on site compared to traditional construction methods. It is critical to answer if the robot will be successful for a given project, and how the robot impacts the schedule, cost, quality, and safety, particularly for the workforce.
Theoretical & Practical Points of Departure
To answer these questions, a previous CIFE seed project developed three case studies comparing on-site robots to traditional construction methods to formalize an evaluation framework for robots. This project analyzed three single-task robots being tested or deployed on construction-sites: a drilling robot in Norway, a drywall placing robot in Sweden, and a layout robot in the US.
For each case, we analyzed the practices that project teams must decide to do or not do following CIFE’s Product, Organization, and Process (POP) Matrix (Kunz & Fischer, 2012) and compared the Safety, Quality, Schedule, and Cost (SQSC) impacts to deploy construction robots. The cases pointed to the need to break down the SQSC to the organization-process units that are different between the robotic versus manual method. To set up these comparison units, construction companies should have access or start tracking the traditional project production data.
We also found that these variables had different priorities according to the construction task, and so, defining the goal to introduce the robot on a particular project was important for the evaluation (Brosque et al., 2020). For example, reducing long term injuries of lifting drywall boards had a higher priority for general contractors than reducing injuries while performing the walls’ layout.
The framework was prototyped in Microsoft Excel and entails two main steps. The first one analyzes the feasibility of the robot for a given project looking at the three independent variables. The second establishes the comparative evaluation between the robot and the traditional method based on four dependent variables and utilizes a multi-criteria decision-making method to aggregate the data of the four variables and provide a recommendation.
Research Methods & Work Plan
To validate the completeness of the REF and assess the speed and consistency of comparing robotic and manual construction work, we propose to develop a graduate course on construction robotics to enable case studies by the students in the class on the applications of robots CIFE members want to explore.
Through applying the REF to a number of case studies, each carried out independently by three students, we will be able to determine whether the variables in the REF are comprehensive, i.e., enable all the comparisons the industry partners want to do. We will also be able to determine how consistently the students applied the framework, the effort required for a comparison, and the main bottlenecks for such comparisons.
To aid the development of the class we will develop three case studies with CIFE members that want to evaluate potential robots for their projects with the REF. The first test case will evaluate an interior demolition robot for a refurbishment project. The Danish demolition robot, developed by HD lab in collaboration with two demolition contractors, aims to substitute the manual grinding and blasting work of a renovation project. Next, we plan to evaluate a walking robot for reality capture, and finally, test the REF with an interior drilling study.
Expected Contributions to Practice
The practical contribution to CIFE members and industry is the development of 15 case studies that consistently compared traditional and robotic tasks. Through deploying the REF in a construction robotics course, we will understand the readiness of promising construction robots and the managerial challenges to deploying them.
Expected Contributions to Theory
This research aims to validate a comprehensive and consistent Robotics Evaluation Framework (REF) formalized with previous CIFE funding for innovators in construction companies looking to deploy robots on site.
If innovators in construction companies had a consistent and comprehensive way to evaluate robotics according to different priorities, they could make more informed decisions about their feasibility. This could reduce the effort of deploying construction robots and maximize the best robot and project configuration for success. We expect that if applied to practice, the REF will contribute to the successful adoption of construction robots with corresponding safety benefits for the workforce, and quality, schedule, and cost benefits for projects and companies.
Publications
- Brosque, C., Skeie, G., Orn, J., Jacobson, J., Lau, T., & Fischer, M. (2020). Comparison of Construction Robots and Traditional Methods for Drilling, Drywall, and Layout Tasks. In 2020 International Congress on Human-Computer Interaction, Optimization and Robotic Applications (HORA) (pp. 1–14). IEEE. https://doi.org/10.1109/HORA49412.2020.9152871
References
- Bock, T., & Linner, T. (2016). Construction robots: Elementary Technologies and Single-Task Construction Robots. Cambridge: Cambridge University Press. https://doi.org/https://doi.org/10.1017/CBO9781139872041
- Brosque, C., Skeie, G., Orn, J., Jacobson, J., Lau, T., & Fischer, M. (2020). Comparison of Construction Robots and Traditional Methods for Drilling, Drywall, and Layout Tasks. In 2020 International Congress on Human-Computer Interaction, Optimization and Robotic Applications (HORA) (pp. 1–14). IEEE. https://doi.org/10.1109/HORA49412.2020.9152871
- IFR. (2018). Executive Summary World Robotics 2018 Service Robots. Executive Summary World Robotics 2018 Service Robots (Vol. 11). Retrieved from https://ifr.org/downloads/press2018/Executive_Summary_WR_Service_Robots…
- Khatib, O., Yeh, X., Brantner, G., Soe, B., Kim, B., Ganguly, S., … Creuze, V. (2016). Ocean one: A robotic avatar for oceanic discovery. IEEE Robotics and Automation Magazine, 23(4), 20–29. https://doi.org/10.1109/MRA.2016.2613281
- Ragaglia, M., Argiolas, A., & Niccolini, M. (2017). Cartesian-Space Motion Planning for Autonomous Construction Machines. In 34th International Symposium on Automation and Robotics in Construction (pp. 983–990). Taipei: International Association for Automation and Robotics in Construction. https://doi.org/https://doi.org/10.22260/ISARC2017/0136
- Usmanov, V., Bruzl, M., Svoboda, P., & Šulc, R. (2017). Modelling of industrial robotic brick system. In 34th International Symposium on Automation and Robotics in Construction (pp. 1013–1020). Taipei: International Association for Automation and Robotics in Construction. https://doi.org/https://doi.org/10.22260/ISARC2017/0140
- Yamada, M., Fujino, K., Kajita, H., & Hashimoto, T. (2017). Survey of the line of sight characteristics of construction machine operators to improve the efficiency of unmanned construction. In 34th International Symposium on Automation and Robotics in Construction (pp. 588–593). Taipei: International Association for Automation and Robotics in Construction. https://doi.org/https://doi.org/10.22260/ISARC2017/0082
- Zedin, T., Vitalis, L., Guéna, F., & Marchand, O. (2017). A method based on C-K Theory for fast STCR development: The case of a drilling robot design. In 34thInternational Symposium on Automation and Robotics in Construction (Vol. 34, pp. 464–471). Taipei: International Association for Automation and Robotics in Construction. https://doi.org/https://doi.org/10.22260/ISARC2017/0064
Funding Year:
2021