Skip to content Skip to navigation

A Proposed Trajectory of Validation Experiments for Computational Emulation Models of Organizations

TitleA Proposed Trajectory of Validation Experiments for Computational Emulation Models of Organizations
Publication TypeWorking Paper
Year of Publication1998
AuthorsThomsen, J, Levitt, RE, Kunz, J, Nass, CI
Date Published03/1998
Publication Languageeng
KeywordsCenter for Integrated Facility Engineering, CIFE, Computational Organizational Simulation Models, Contingency Theory, Information Processing, Intervention, Organizational Design, Stanford University, Validation
AbstractValidation of complex simulation models, with multiple inputs and feedback loops, is a challenging, multifaceted problem. It has received considerable attention in the Computational Organizational Science (COS) literature. The COS literature typically calls for extensive validation and discusses some guidelines for when certain techniques are appropriate. However, reports of rigorous external model validations are limited. In this paper, we use an organizational design and analysis tool, called the Virtual Team Alliance (VTA), to illustrate that one needs to perform a series of validation steps in a predefined sequence to accomplish a comprehensive, credible validation of a computational organizational simulation model. The purpose of the VTA model is to provide managers with a tool that they can use to test "virtual prototypes" of project organizations and predict outcomes prior to actual implementation-it can therefore be described as an "emulation" system. The ultimate external validation of VTA is whether or not it is useful to managers for this purpose. VT A is useful if the model forecasts problems that will occur without organizational change, managers redesign the organization based on the model's problem predictions and suggested remediations, and the organizational risks, which the model predicted are thereby reduced. To reach this kind of ultimate external validation of usefulness, VTA has to go through a "trajectory" of different validation methods. The primary contribution of this paper is the development of an innovative validation trajectory strategy for complex computational emulation models. We present a validation trajectory that includes (1) computational synthetic experiments, (2) retrospective validation and comparison with manager's "what-if' predictions, (3) contemporaneous validation, and (4) prospective validation with intervention. We discuss in some detail how we applied VTA to two portions of an ongoing aerospace project. The VTA model made predictions about severe bottlenecks and potential quality problems for one sub-team within the two project teams but no problems for the other sub-team. These predictions were subsequently confirmed. The results of our experiments agreed with those of extant, qualitative organizational theory.
PDF Link
Citation Key938