by LTC Chris Nannini and Assoc. Prof. Andy Hernandez
Researchers from the Naval Postgraduate School (NPS) participated in the 25th International Data Farming Workshop in Istanbul, Turkey from September 17 – 21, 2012. The workshop was sponsored by the Turkish War Colleges, Ataturk Wargaming and Convention Center and was held in conjunction with the North Atlantic Treaty Organization Modeling and Simulation Task Group (MSG-088) Meeting 9. Workshop objectives facilitate international collaboration for exploring diverse military and global security issues.
A team led by Associate Professor Andy Hernandez, Department of Systems Engineering, NPS, investigated the development of more effective scenarios for test and evaluations, and peace support operations through the use of large-scale design of experiments. This team was cross-disciplinary, consisting of military and national defense professionals, software developers, and research analysts: Mr. Alan Howard, United States Partnership for Peace Training and Education Center (USPTC) Program Deputy Director; Lt. Hamadi Bouaicha, Tunisian Navy (Department of Operations Research masters student); LTC Chris Nannini, Military Assistant Professor; Ms. Anna Gordon, PhD, Research Professor; Mr. Steve Upton, Research Associate; and Mr. Paul Roeder, Research Associate;. The latter four are members of the NPS Simulation Experiments and Efficient Designs (SEED) Center.
Simulations are valuable tools for preparing organizations to achieve their mission and to help understand the capabilities of new systems and technologies. However, defense, peacekeeping, and security organizations responsible for test and evaluation, training, and other analysis efforts have limited time and resources to perform numerous physical experiments or even a handful of computer-assisted events. They recognize that better operational scenarios are critical for efficient and timely improvement of systems, force capabilities, and training programs. The development of robust peace operations scenarios is critical to the preparation of peacekeepers, as well as to the defense initiatives for the United States, allied forces, and the international community. The workgroup focused on building stressor scenarios in the Peace Support Operations Model (PSOM), using the conflict in Sudan as the backdrop.
Peace Support Operations Model
The PSOM is a turn-based, agent-based, human-in-the-loop model developed to represent military and civilian organizations deployed in support of peacekeeping operations. The Defense Science and Technology Laboratory in the United Kingdom developed the PSOM to investigate non-traditional military operations. The USPTC uses the PSOM to support training and educational programs for Peace Support Operations. During the workshop, the team created a scenario representing the conflict region between Sudan and South Sudan. The resulting scenario can serve as the foundation for future wargames, training exercises, and education programs.
The modeled area of interest during the workshop was the disputed territory along the border between South Sudan and Sudan known as the Abyei Area. The country of South Sudan was formed in 2011 after a referendum passed in overwhelming support of independence from Sudan. The region had been enveloped in civil war for several decades and is still witnessing mass atrocities in the Darfur region. Over thirty countries currently have peacekeepers supporting UN missions in the region. Stability in the region is critical in order to end cross-border violence between Sudan and South Sudan; provide humanitarian aid to the affected population; and, resolve outstanding issues concerning borders, residence, and oil. This complex environment contains many possible scenarios for peacekeepers.
The team applied a structured, rigorous statistical approach to identify controlled and uncontrolled factors that significantly influence the performance of a system or outcomes of an operation. By understanding the sensitivity of different metrics to those factors a set of “stressor” scenarios could be developed. These select scenarios become the environment in which peacekeepers are placed to "stress" their abilities.
A general peace operations scenario was the basis of the workshop effort. Past wargaming efforts in the region and open-source reviews were leveraged to identify the regional boundaries, population distributions, force compositions, and critical socio-economic events. The team identified operational objectives, key measures of effectiveness, and initial factors of interest to construct an efficient design of experiment (DOE) recently developed in the SEED Center. The DOE allowed the team to explore 80 factors within 512 design points, replicating each point 15 times. The simulation runs were completed within 48 hours, as opposed to classic approaches which would have required millions of runs and eons to complete. In an iterative process the analysis team may refine the scenario based on intermediate findings in order to explore “what-if” questions. If warranted, the DOE may also be modified. This iterative process allows the team to link specific factor levels to particular scenario outcomes of interest.
A study of initial scenario outcomes revealed some interesting insights that point to conditions that would significantly stress players and systems. For instance the deployment of criminal elements in the region affects its intelligence gathering capability. Enhanced intelligence allows criminals to avoid police forces. The team also identified a threshold for rules of engagement for government forces that increased government casualties. By combining these elements into an event during an exercise or game, trainers can understand how players will react to a high stress situation. Continued analysis will show new events to develop. Planned inclusion of subject matter experts for the specific scenario will also help refine it.
Computer experimentation and design of experiments can develop better scenarios by identifying conditions that stress systems and players in their training and performance. The process developed at the workshop serves as a proof of concept for creating various stressor scenarios for peacekeeping environments. The methodology is simulation and application independent. In fact, it is not limited to computer-based events. This event development tool has the flexibility to support the development of better scenarios for system acquisition, improvement programs, and future peacekeeper training efforts.