Game Trains Soldiers in a Virtual Iraq or Afghanistan

A training tool being developed by a research team from the Arts and Technology (ATEC) program may soon make it easier for military service men and women to perform their missions in Iraq and Afghanistan.

The project offers virtual villages for soldiers in Iraq or Afghanistan to practice their training skills.

Dr. Marjorie Zielke, the principal investigator on the project, holds the award plaque with her co-investigators, Dr. Thomas Linehan (left), director of Arts and Technology at UT Dallas; and Dr. Frank DuFour, assistant professor of sound design.

“The work we’re doing has to do with the facilitation of cultural training,” said Dr. Marjorie Zielke, an assistant professor in the ATEC program and the principal investigator on the project.  “The way some of that training has been done in the past and may still be done in certain areas is to build actual villages and hire actors to replicate a particular culture,”  Zielke said. “That kind of approach has some limitations in the sense that it’s expensive, not everyone can attend, it’s not easily changed because it’s a physical structure, you have to work with actual actors, and so forth.”

The ATEC team set out to re-create  a realistic virtual environment instead.  The result is First Person Cultural Trainer (FPCT), a 3D interactive game that  teaches soldiers the values and norms of Iraqi and Afghan cultures.  FPCT is a serious game, which means that it is designed for purposes other than pure entertainment, in this case, cultural training.

Players enter the community from the first-person point of view and collect information based on verbal and non-verbal cues observed in characters encountered along the way.

FPCT recently won the Cross-Function Team Award at the 2010 Modeling & Simulation Leadership Summit, held in Virginia Beach. Presented annually by the National Training & Simulation Association (NTSA), the Modeling & Simulation (M&S) Awards recognize achievement in the M&S functional areas of training, analysis and acquisition, and in support of the overall M&S effort.

First Person Cultural Trainer was also a finalist at the Interservice/Industry Training, Simulation and Education Conference (I/ITSEC) in Florida in early December.  I/ITSEC promotes cooperation among the armed services, industry, academia and various government agencies in pursuit of improved training and education programs.

Zielke says ATEC has been working in the cultural training and simulation area for about three years, with about 15 students – at the undergraduate, graduate and doctoral levels – working on this project.  Her co-investigators are Dr. Thomas Linehan, Endowed Chair and director of Arts and Technology at UT Dallas; and Dr. Frank DuFour, assistant professor of sound design.

The project team conducted extensive research to get the characters to look, sound and act like the culture they’re representing.

“Much of the cultural data is being developed in real time by the military,” Zielke said.  “By having it in a systems-based approach that is composable — in other words, we can generate culture in certain aspects of the game on the fly — we can respond to the data as soon as it becomes available.  We could change it overnight if we needed to.”

The project is supported and sponsored by the U.S. Army Training and Doctrine Command (TRADOC) G-2 Intelligence Support Activity (TRISA).

“This prototype, now on its way to a full-fledged model, is a highly flexible tool for training battle staffs and individual soldiers at the tactical level,” said Mel Cape, senior knowledge engineer for TRADOC G2. “The recognition [the UT Dallas team] has received within the modeling and simulation community is well deserved, and we at TRISA are proud of their superlative efforts in the development of a culturally-based training device.”

Part of this cultural training is to familiarize soldiers with what they will face when arriving in their theaters of operation. Researchers worked to make the game’s characters look, sound and act as much as possible like people from the culture they represent.

The project offers virtual villages for soldiers in Iraq or Afghanistan to practice their training skills.

In the game, the player enters a community from the first-person point of view.  He doesn’t know much about the community, how the people feel about him, or who the key figures are in the village. The goal is to move through the community and try to understand the social structures and issues, then address those issues and work with the community to affect missions.

The people in the community form opinions about the player based on how the player treats them.  If the player doesn’t interact properly with them, the villagers discuss his behavior among themselves.  Some individuals in the village have more clout than others.

The player collects information based on verbal and non-verbal cues he observes in the characters he encounters and then rates those characters based on a scale of four emotions: anger, fear, gladness and neutrality.

One of the most rewarding aspects of this program for Zielke is the opportunity to help her students make connections within the industry. “Between this project and other similar projects in the ATEC program, we have at least 30 students employed at any given time,” she said. “The students develop great portfolios, gain work experience, go to conferences, write research papers based on an incredibly rich data set and then hopefully leverage all of those things to get industry jobs.”

ATEC Prof to Take Clinical Concepts to Virtual World

Assistant Professor Marjorie Zielke has been awarded a three-year, $350,000 grant to create online training in neonatal nursing through an ongoing collaboration with the UT Arlington School of Nursing (UTASON).

“Student learning will be enriched by faculty perspectives from across the country,” Marjorie Zielke said of her project.

Zielke, with Arts and Technology (ATEC) faculty members Monica Evans, Frank Dufour and Todd Fechter, will build a Web site that allows student nurses to be taught concurrently by faculty from Dartmouth University, the University of Texas Health Science Center at Houston, the University of Texas at Arlington and Stony Brook University School of Nursing in New York.  The new research enhances subject matter review and creates a more portable learning experience.

“Student learning will be enriched by faculty perspectives from across the country,” said Zielke. “Students also will benefit from the social community of other graduate-level nursing students through this virtual learning environment.”

The course subject matter covers conditions in fetuses and babies under 2 years old.  Students can download a lecture as a podcast or video, and then follow up with a virtual examination of an infant in a no-risk environment. The virtual environment allows for endless practice and limitless scenarios in a risk-free environment. For instance, the instructor can program symptoms such as respiratory distress into a 3D model, rather than teach based on the conditions of patients visiting a clinic or hospital on a given day.

ATEC students at both the undergraduate and graduate levels also will participate in this project, performing research, modeling, animation, story development and computer programming.

Research project sponsors are the UTASON, with principal investigator Dr. Judy Leflore, and the Health Resources and Services Administration.

Professor Zielke is thinking beyond the virtual classroom. In addition to her work on grants from The University of Texas System and other healthcare organizations, she plans to create a virtual baby, which is difficult because of the current technology used to capture motion. A researcher cannot, for instance, direct an infant to raise its right arm so that the cameras and computers can capture every nuance of the movement as a basis for animation.

“We like to take on complex projects that no one really knows how to do,” she said. “I’ve always been interested in being able to represent humans virtually, both physically and cognitively. Representing non-verbal communication is especially challenging.  This line of research has the potential to make a major difference in the way online medical education and medical simulations are done today.”

Listening to Paintings: ATEC to Present Soundscapes

In a creative blending of sense experiences, students in UT Dallas’ Arts and Technology (ATEC) program will present their soundscapes to go with works at the Dallas Museum of Art on Friday, May 15, at 8 p.m.

Students in Dr. Frank Dufour’s Sound Design for Interactive Applications course were commissioned by Nicole Stutzman, director of learning partnerships with schools and the community at the museum, to compose soundscapes or auditory translations for a selection of paintings from the DMA’s permanent collection.

This collaboration marks the second time ATEC students have been invited to create sound-design projects inspired by paintings, sculptures, and other art objects at the DMA and to present them at the museum with the actual art that inspired them.

This upcoming presentation will be held in conjunction with the DMA’s Late Nights program, held on the third Friday of each month, when the museum is open until midnight. General admission is $10 or less, and DMA members and children under 12 are admitted free.

The Audio Tour will start at 8 p.m. in the Center of Creative Connections. The visitors will be lead by UT Dallas’ students through the galleries of the museum to the selected artworks, in front of which the soundscapes will be then played.

Students and Frank Dufour will explain the various options chosen to “translate” art into sounds and music.