ATEC Team Receives Healthy Dose of Grants for Virtual Medical Work

Dr. Zielke and research team
Researchers from the Center for Modeling and Simulation and the Virtual Humans and Synthetic Societies Lab include, (from left), Stephen Rodriguez, Erik DeFries, Sean Lenox, Jacob Keul, Dr. Marjorie Zielke, Nick Orr, Gautham Mathialagan, Dylan Fino, research manager Gary Hardee, Leonard Evans, Djakhangir Zakhidov and Joel Rizzo.

A research team from the School of Arts, Technology, and Emerging Communication at The University of Texas at Dallas has received two grants — one each from Southwestern Medical Foundation and the National Institutes for Health — to fuel ongoing research into virtual reality-based medical experiences.

The Center for Modeling and Simulation and the Virtual Humans and Synthetic Societies Lab, both led by ATEC professor Dr. Marjorie Zielke, are developing an emotive “Virtual Reality Patient,” or VRP, in conjunction with Southwestern Medical Foundation, that medical students will be able to use to improve their patient communication skills.

The center also has received a clinical trial planning grant from the National Institutes of Health (NIH) to explore virtual reality-graded exposure therapy for those with chronic back pain.

“Both of these new projects continue to establish the center’s growing presence in the medical simulation space,” Zielke said. “Serious games for health and medicine along with our virtual humans program are both critical research areas that we want to continue to grow and nurture.”

Revolutionizing the Medical Interview with Virtual Reality Patients

virtual reality patients
A visualization of an emotive “virtual reality patient” experience is shown. The project, under development by Dr. Marge Zielke’s research team in the Center for Modeling and Simulation and the Virtual Humans and Synthetic Societies Lab, has received funding from Southwestern Medical Foundation.

Working alongside subject-matter experts at the UT Southwestern Medical Center, Zielke’s team hopes to create a platform that will replicate medical interviews with the help of virtual patients and caregivers.

Zielke said the platform will offer high-quality simulations, known as emotive Virtual Reality Patients, which can exhibit medical symptoms to help medical students improve their verbal and nonverbal communication skills.

The virtual humans will complement other training methods, and ideally possess a lifelike ability to have both a conversation and convey emotion — something Zielke said is particularly important in the interview process, given that patients express some things nonverbally.

“Virtual humans have always been a major focus for the center,” Zielke said. “We’ve been working on this project for quite a while, and we would really like this to be a stake in the ground for developing world-class research on virtual patients in Texas. We are very grateful for this grant from Southwestern Medical Foundation to continue our research track focused on virtual humans here at UT Dallas. We hope to develop one of the first augmented or virtual reality-based conversational digital patients right here in our lab.”

With the $200,000 grant from the foundation, Zielke’s team will first develop a state-of-the-art “natural language interface” capable of responsive and realistic communication, with the team compiling data on body language, facial cues and other physiological information.

Zielke said the center has long been interested in creating virtual robots that can either work in tandem, or in some cases, replace the need for medical mannequins often used in educational scenarios. The advantage of a training simulation is its potential to physically emulate what symptoms the patient is presenting.

Given the lab’s past work on game-based medical simulations featuring stroke patients, Zielke said her team has a rich backlog of data regarding stroke-specific dialogue and symptoms they can use as their first case in this new project.

“From its very beginning, Southwestern Medical Foundation has sought to advance medical knowledge to benefit our community,” said Kathleen Gibson, president and CEO of Southwestern Medical Foundation. “As new methods of advanced learning become available, we want to support those innovations that keep medical education at UT Southwestern at the forefront. This collaboration between UTD and UTSW is an exciting example of such innovation and progress.”

Serious Games for Serious Pain

The center — along with colleagues at the University of Alabama at Birmingham, Ohio University and others — also has received a $700,000 grant from the NIH’s National Institute on Drug Abuse to develop a serious game aimed at helping patients with chronic back pain.

Unlike most games, serious games are not designed to entertain but to teach, and they’re used in industries such as defense, education and health care. The game Zielke’s team is developing employs the use of graded exposure therapy, which is a method of reducing physical or psychological impairments through gradual exposure to the source of pain or fear.

Titled VRGE (Virtual Reality Graded Exposure), the game uses graded exposure to allay physical disabilities by promoting engagement in physical activities that might otherwise seem intimidating to patients with back pain.

Zielke said graded exposure therapy has traditionally been delivered in clinical settings, so its ability to help patients at home has been limited. VRGE will use motion-tracking technology, ongoing onboard assessments and motivational rewards within the game to reinforce traditional graded exposure therapy.

This ongoing project also received support from the American Pain Society and the North American Spine Society through Dr. Zina Trost at the University of Alabama at Birmingham in 2015 and 2016.

Note: The research was supported by the National Institute on Drug Abuse of the National Institutes of Health under Award Number R34DA040954. The content is solely the responsibility of the authors and does not necessarily represent the official views of the National Institutes of Health.

ATEC Team Developing Virtual Teachers to Help Dyslexic Students

Marjorie Zielke

 

A professor in the School of Arts, Technology, and Emerging Communication is researching the development of a virtual teacher that will facilitate the most demanding portions of learning curricula for children with dyslexia.

Dr. Marjorie Zielke, who is also director of UT Dallas’ Center for Modeling and Simulation, and her team are collaborating with the Luke Waites Center for Dyslexia and Learning Disorders at Texas Scottish Rite Hospital for Children (TSRHC) to create the simulation.

“For several compelling reasons, we are researching a virtual teacher,” said Zielke, director of ATEC’s Virtual Humans and Synthetic Societies Lab. “The challenge is that there aren’t enough trained therapists for some of the most challenging portions of the hospital’s dyslexia curriculum termed ‘new learning.’ We’re researching a variety of technology approaches to fill this gap.”

According to TSRHC, dyslexia is a learning disorder that affects approximately 10 percent of children. Those diagnosed with dyslexia have trouble connecting sounds to letter symbols. This affects the way children with dyslexia learn to read and spell.

Dr. Jeffrey Black, medical director of the Luke Waites Center, looks forward to implementing this new technology-based system.

“The center has had success using technology in the form of recorded teaching paired with teacher-facilitated learning in helping the child with dyslexia to read and spell,” Black said. “Advances in the understanding of dyslexia treatment and the development of interactive technology-driven instruction make this the right time for the next TSRHC curriculum innovation. Making effective intervention accessible to more students is the goal.”

Zielke said several components of the technology development will provide excellent research opportunities for the Sim Center team.

The interface under consideration for the project will include the live teacher, the students and the virtual teacher, among other technologies the team is considering.

“The interaction between this triad is a design challenge,” Zielke said. “The persona of the virtual teacher is another research design component, and we are studying several model therapists to create the right visualization, voice and personality for the potential virtual teacher.”

One important aspect of this virtual teacher research is that the curriculum and the content is critical, but we also must consider everything a therapist does. In this particular project, we are studying the back-and-forth interaction between the students and the virtual teacher. The children have to enjoy the experience and understand what they’re supposed to learn.

Dr. Marjorie Zielke,
assistant professor of arts and technology and director of UT Dallas’ Center for Modeling and Simulation

Researchers also must immerse themselves into the subject matter to understand the program fully. She said researchers must consider elements such as vocalization and human physiology.

“An important part of what we do is collaborating with subject matter experts,” Zielke said. “We need to be able to work really closely with them. A big part of the project is the collaborative design phase, so we’ve been studying videos of live lessons and analyzing them as well.”

One of the main challenges in the initial design phase is making the tradeoff between the live teacher and the simulation seamless.

“One important aspect of this virtual teacher research is that the curriculum and the content is critical, but we also must consider everything a therapist does,” she said. ‘In this particular project, we are studying the back-and-forth interaction between the students and the virtual teacher. The children have to enjoy the experience and understand what they’re supposed to learn.”

Zielke sees the potential for incorporating new technology like the Oculus Rift, a virtual reality head-mounted display, into future projects.

Although she’s not considering these technologies for the dyslexia project, Zielke said she has expanded her research focus from screen-based environments to virtual reality to better represent face-to-face interactions.

“We don’t really focus on one particular technology, but rather we try to consider the research objective,” she said. “In fact, an important area of research and a question we are frequently asked is, ‘How do various media work together to create the holistic curriculum?’”

The Sim Center team has enjoyed recent successes in health education-related simulations.

In 2015, the team won Best in Show: Academic Faculty or Staff Category in the peer-reviewed Serious Games and Virtual Environments Competition at the International Meeting for Simulation in Healthcare. It was honored for its work on the TIME (Transformation in Medical Education) Learning Portal — an internally funded project done in collaboration with UT Southwestern Medical Center.

The team also won the same award in 2014 for GLIMPSE — A Game to Learn Important Communications Methods for Patient Safety Enhancement — which simulates communication between physicians and nurses. The project was a collaboration with UT Arlington’s College of Nursing and Health Innovation, and Baylor Scott & White Health. It was funded by the Agency for Healthcare Research and Quality through UT Arlington.

Zielke recalled some of the research design issues in some of her team’s earlier projects, such as GLIMPSE.

She said the medical component of the dialogue has to be precise because researchers are portraying behavior in the context of the medical profession. Audiences lose interest if they encounter unrealistic situations in games, but her team must simultaneously try to persuade users to adopt these new communication strategies.

“In the case of the project with TSRHC, the design is very deliberate in terms of what has to happen,” she said. “The curriculum is complicated for therapists to learn. By creating a virtual therapist and considering other technology solutions, we can potentially have educational programs more widely available to children because there will be more resources to help teach.”

ATEC Game Wins First Place at Medical Simulation Conference

A collaborative game-based simulation project between UT Dallas, UT Arlington and Baylor Scott & White Health that seeks to improve physician-nurse communication received first place at a serious games competition at the 14th Annual International Meeting on Simulation in Healthcare (IMSH).

GLIMPSE offers situational learning through conversations with the game's characters. Its players role play as medical professionals and must select appropriate responses to other nurses or physicians.
GLIMPSE offers situational learning through conversations with the game’s characters. Its players role play as medical professionals and must select appropriate responses to other nurses or physicians.

The project also won fourth place overall out of approximately 60 entries in the Technology Innovations Abstract Category at the conference.

A panel of judges selected GLIMPSE (A Game to Learn Important Communications Methods for Patient Safety Enhancement) for top honors in the Faculty Category of the IMSH Serious Games and Virtual Environments Showcase and Arcade, the world’s largest conference on simulation in health care.

Dr. Marjorie Zielke
Dr. Marjorie Zielke

A UT Dallas team led by Dr. Marjorie Zielke, assistant professor of Arts and Technology and director of ATEC’s Virtual Humans and Synthetic Societies Lab, developed the game in collaboration with Dr. Mary E Mancini, associate dean and chair for undergraduate nursing programs at UT Arlington; Dr. Yan Xiao, Baylor’s director of patient safety research; and Dr. Susan Houston, Baylor’s director of nursing research.

“The overall track record we have with our game-based simulations and the international recognition we are receiving is very gratifying, particularly when the subject matter is as challenging as it is in GLIMPSE,” Zielke said. “As always, we owe our continuing success to the project team and the great faculty, staff and students involved in these important research projects. The encouragement and support we get from our administration is also critical.”

Mancini is the project’s principal investigator.

“Our hope is that this project will enhance patient safety and, ultimately, improve patient outcomes,” Mancini said. “Being honored by the judges at this year’s International Meeting on Simulation in Healthcare tells us that the virtual learning environment we’ve built is among the very best in terms of content and design.”

Dr. Katie White, secretary for the Serious Games and Virtual Environments Interest Group that organizes the IMSH competition, praised the quality of the entries.

“The entries for this year’s arcade were technologically sophisticated and innovative in the way that they combined gaming concepts and clinical teaching,” White said. “It’s great to see the game developers improve their products from year to year and to see the growth of the arcade into a fun place for IMSH attendees to be introduced to serious games as a teaching tool.”

The overall track record we have with our game-based simulations and the international recognition we are receiving is very gratifying, particularly when the subject matter is as challenging as it is in GLIMPSE.

Dr. Marjorie Zielke,
assistant professor of Arts and Technology and director of ATEC’s Virtual Humans and Synthetic Societies Lab

The project, funded by a grant from the Agency for Healthcare Research and Quality, started with the development of a communications curriculum based on a research study involving physicians and nurses at two Baylor Scott & White hospitals. The goal is to improve the patient experience by improving communication between physicians and nurses. Results of evaluation of the game will be made available later this year when insights and effects of the game are analyzed on a deeper level for education, behavioral changes and improved learning potential.

“A great deal of health care errors are due to miscommunication between physicians and nurses, which can present patient safety issues,” said Houston. “The nurses and physicians who played the game were extremely supportive. Overall, the collegiality and collaboration has been wonderful in an effort to pull off this three-year project. Ideally, we will see the long-term effects, not only just in the results of this study, but through a marked decrease in health care errors that occur due to miscommunication.”

GLIMPSE also was selected in December 2013 as a finalist in another serious games contest at the Interservice/Industry Training, Simulation and Education Conference (I/ITSEC), the world’s largest modeling and simulation conference. Game-based simulations developed in the VHSS Lab have been recognized with nine major awards since 2010.

ATEC Project is Finalist at International Serious Games Competition

A new game born out of a collaboration involving UT Dallas’ Arts and Technology (ATEC) program, The University of Texas at Arlington’s College of Nursing and Baylor Scott & White Health is a finalist in an international serious games competition.

The game-based simulation called GLIMPSE (Game to Learn Important Communications Methods for Patient Safety Enhancement) is being used in research on communication practices among physicians and nurses.

“GLIMPSE features a robust educational curriculum,” said Dr. Marjorie Zielke, director of ATEC’s Virtual Humans and Synthetic Societies Lab and ATEC’s principal investigator on the project. “We designed the game for busy health care professionals. The curriculum content is broken down into short episodes using audio, video and 3-D virtual game play to allow perspective sharing and situational learning.”

The project, which made its debut in October, is funded by a $969,604 grant from the federal Agency for Healthcare Research and Quality.

Dr. Mary E. Mancini, professor, associate dean and chair for undergraduate nursing programs at UT Arlington’s College of Nursing, is the project’s principal investigator at UT Arlington. A UT Dallas team led by Zielke constructed the game in collaboration with Mancini. Also on the project were Baylor Scott & White Health’s Dr. Yan Xiao, director of patient safety research, and Susan Houston, director of nursing research.

GLIMPSE is a finalist in the Business Category of the 2013 Serious Games Showcase & Challenge − one of 18 finalists out of 50 games submitted. Winners will be announced at the Interservice/Industry Training, Simulation and Education Conference (I/ITSEC) this month. I/ITSEC is the world’s largest modeling and simulation conference.

Mancini, past president of the Society for Simulation in Healthcare, said the project’s use of game technology is unique. She hopes its benefits will be far-reaching.

“Research has shown that communication is a contributing factor in the majority of cases resulting in near-misses and actual patient harm,” Mancini said. “By improving the ability of health care professionals to communicate with each other, GLIMPSE will enhance patient safety and, ultimately, improve patient outcomes.”

The project is also one of six finalists being evaluated for the Adaptive Force Award – this year’s Special Emphasis Award. According to the conference website, Adaptive Force games encourage the player to repeatedly try new or different strategies to solve problems while considering feedback with the purpose of improving overall success.

The selection of GLIMPSE as a finalist in the serious games competition marks the ninth time that Zielke’s projects have been nominated for national and international awards. Should GLIMPSE win, it will be the eighth major award in the last few years for Zielke’s lab.

“The ongoing lab recognition is a real tribute to our students, researchers, faculty and staff, and the overall support we get from our administration for our research,” said Zielke.

Zielke also serves as the vice president for education for the Society for Modeling & Simulation International and the deputy chair of the National Modeling and Simulation Coalition.

“The fact that we continue to win and be finalists in extremely competitive events shows the depth of strength across all of our research teams,” she said.

ATEC Course Explores Creating Autonomous Technology

Driverless prototype developed by Audi
Driverless prototype developed by Audi

ATEC 6375 “Topics in Emerging and Cognitive Design: Creating Autonomous Technology” with Dr. Marjorie Zielke.

Coined by Langdon Winner, autonomous technology (A.T.) refers to technologies that operate on their own.

In this NEW course, students will learn about:

  • Humanoid robots and other hardware and software designed to work hand and hand with humans but operate also on their own
  • Research literature in areas such as human-robotic interactions (HRI)
  • Design implications
  • Areas for students to research and develop projects
  • A.T.’s philosophical background
  • A.T.’s psychological underpinnings


The course will incorporate current and historical readings, papers, and projects designed to help students to understand and apply what they learn. The class will meet from 4:00 p.m. to 6:45 p.m. on Thursdays in Room 3.605 in ATC. If a student decides to take the class, he or she needs to enter course number 25591 when registering online.

ATEC Nurse Training Simulations Singled Out for Awards

Two nursing education research projects developed by the Institute for Interactive Arts and Engineering (IIAE) at UT Dallas in collaboration with the UT Arlington College of Nursing have received national and state recognition this spring.

One project — “Can Game Play Teach Student Nurses How to Save Lives?” — has been named a 2012 Computerworld Honors Laureate. This game-based simulation uses 3D infant patients in a synthetic environment to give undergraduate nursing students virtual clinical

One project — “Can Game Play Teach Student Nurses How to Save Lives?” — has been named a 2012 Computerworld Honors Laureate.  This game-based simulation uses 3D infant patients in a synthetic environment to give undergraduate nursing students virtual clinical practice opportunities.  The project was funded through a UT System Transforming Undergraduate Education grant. It will be recognized for its applications of information technology to promote positive social, economic and educational change at the Computerworld Laureate Medal Ceremony and Gala Evening on June 4 in Washington, D.C.

A second research project, NursingAP.com, tied for first place as Best Demonstration Project at the “Innovations in Health Science Education” conference sponsored by the University of Texas Academy of Health Science Education. The recognition is voted on by attendees at the conference, which is sponsored by the six health science campuses within the UT System.

A project called “Can Game Play Teach Student Nurses How to Save Lives?" uses 3D infant patients in a synthetic environment.

NursingAP.com is a blended-learning website that incorporates interactive technology and virtual environments to assist graduate students seeking nurse practitioner degrees and certifications. The project started with neonatal nurse practitioner (NNP) curriculum.   The project is funded by the federal Health Resources and Services Administration.

NursingAP.com affords students opportunities to practice the knowledge acquired through lecture material through the use of interactive modules and a 3D virtual Neonatal Intensive Care Unit (NICU). The components of the NNP curriculum are presented through lecture notes with embedded media, and a variety of other multimedia forms, including videos, interactive games, simulations and virtual equipment demonstrations.  Students can practice clinical skills in the virtual NICU, an immersive environment where 3D patients present medical conditions covered in lecture content.

The project gives undergraduate nursing students virtual clinical practice opportunities.

Both projects are research collaborations between Dr. Marjorie A. Zielke, in Arts and Technology assistant professor and the associate director of IIAE, and Dr. Judy LeFlore, professor at the UT Arlington College of Nursing.  “These continual awards reinforce the deep talent of our student developers,” Dr. Zielke said.  “I also think we need to give a great deal of credit to our strong collaboration with the UT Arlington College of Nursing.”

“I am particularly proud of the scope of the recognition we are receiving from international conferences to internal recognition by the UT System Health Science campuses,” Dr. Zielke continued.

At the Computerworld awards dinner, the UT System project will be presented with a medallion inscribed with the program’s mission statement, “A Search for New Heroes.”

“The Computerworld Honors program was especially competitive this year, as more than 500 IT initiatives were nominated for their innovation and benefit to society,” said Julia King, executive editor of events for Computerworld.

These new honors are just two of the several awards received by IIAE projects over the past two years.  The UT System project also received a first place award for Emerging and Innovative Technology and Methods at the 2011 International Meeting on Simulation in Healthcare (IMSH). The simulation was used in a randomized, controlled study designed to compare the clinical application of undergraduate nursing students using a virtual clinical experience compared to students receiving the same pediatric respiratory content in traditional lecture format. Results of the study were published this spring in Simulation in Healthcare, the journal of the Society of Simulation in Healthcare.  Another project, The First Person Cultural Trainer (FPCT) won first place in the government category of the Serious Games Showcase at the Interservice/Interindustry Training, Simulation and Education Conference (I/ITSEC) in Orlando in December 2011.  Earlier in 2011, FPCT earned first-place in the Innovations in DoD Gaming Competition at the 2011 Defense GameTech Users’ Conference also in Orlando.

Virtual Texans Celebrate Centenary of Birth of Alan Turing

The Arts and Technology program will participate in a high-tech, 24-hour international multimedia show honoring the father of computer science, Alan Turing.

On March 23 and 24, UT Dallas Arts and Technology faculty members Dr. Marjorie Zielke and Dr. Roger Malina, professor Judy LeFlore of University of Texas at Arlington and ATEC students Sanger Doane and Steven “Slade” Jansa, will participate as virtual Texans in a worldwide streaming extravaganza celebrating the 100th anniversary of the birth of Alan Turing.

Alan Turing’s accomplishments made a fundamental impact on the development of the computer and to our contemporary networked digital culture.

Alan Turing is sometimes called the father of computer science.  In 1935, at the age of 23 he invented the concept of abstract computing machines – now known simply as Turing machines – on which all subsequent stored-program digital computers are modeled.

Turing also pioneered the field of artificial intelligence, and he developed the idea what is now called the “Turing Test,” a test of a machine’s ability to exhibit intelligent behavior.

The UT Dallas ATEC program has a number of award winning research and development initiatives which seek to create virtual environments with virtual humans for applications in health care and education.

With colleague Dr. Judy LeFlore, associate professor at the University of Texas at Arlington College of Nursing, the ATEC team has developed a serious game to teach undergraduate nurses how to treat respiratory distress in infants, a health-care professional assessment program for a local hospital, and a full online nurse practitioner curriculum for neonates.

These projects have won a variety of awards, to include first place at the 11th International Meeting on Simulation in Healthcare (IMSH) in the category of Emerging and Innovative Technologies and Methods in January 2011 and most recently in February 2012 a tie for first-place demonstration at the Eighth Annual Innovations in Health Science Education Conference, sponsored by the University of Texas Academy of Health Science Education.

These Virtual Texans from ATEC projects will be participating in the worldwide celebration Decode/Recode.

Decode/Recode is globally networked interactive event celebrating the centenary of Alan Turing’s birth, as part of the official opening of the University of Salford building at MediaCity, England on March 23. For this event ATEC will be connecting for 24 hours with 24 partners in a worldwide live digital media performance.

Video Game to Help U.S. Troops Wins New Award

Honor is Arts and Technology Research Project’s Third National Honor in 2 Years

For the third time in two years, the First Person Cultural Trainer (FPCT), a research project from the UT Dallas Arts and Technology(ATEC) program, has won a national award for serious gaming.

The First Person Cultural Trainer game designed by ATEC simulates the challenges a soldier might encounter on patrol in a village.

FPCT received the Best Game award in the Government Category of the 2011 Serious Games Showcase and Challenge. FPCT is sponsored by U.S. Army Training and Doctrine Command G-2 Intelligence Support (TRADOC).  The Serious Games Showcase is part of the Interservice/Interindustry Training and Simulation Education Conference (I/ITSEC), and was held in Orlando, Fla., from November 28 through December 1.

Earlier in 2011, FPCT earned first place in the Innovations in DoD Gaming Competition at the GameTech Users’ Conference  in Orlando. In 2010, FPCT won the cross-function award from the National Training and Simulation Association (NTSA).

FPCT is a four-level immersive game that allows Army leaders and other appropriate personnel to practice culturally correct ways of interacting with different populations around the world.  The game features a variety of innovations, like a branching conversation system and methods for displaying nonverbal communication and environmental perception. The program can also be ported to different game engines with minimal redevelopment.

The video game simulates conversations with people that a soldier might meet, in this case a village elder.

More than 50 games were entered in the I/ITSEC Serious Games contest, which had five categories – government, business, student, mobile and a special category, adaptive stance.  The work was reviewed by a panel of military, academic and industry gaming experts.  About 20,000 government, business, military and academic total registrants attend I/ITSEC every year. The conference is widely considered to be the largest and most competitive worldwide in modeling and simulation.

“This honor and the overall visibility that FPCT, UT Dallas and ATEC received at I/ITSEC this year is a real tribute to our sponsors at TRADOC, students,  faculty, project staff and administrators who have nurtured this project for going on four years,” said Dr. Marjorie Zielke, ATEC assistant professor.

Zielke is the associate director of the Institute for Interactive Arts and Engineering (IIAE) and principal investigator of the FPCT project.  Other faculty co-investigators on the project include Dr. Frank Dufour, assistant professor and director of the ATEC PhD program; Dr. Gopal Gupta, professor and head of the UT Dallas computer science department; and Dr. Thomas Linehan, professor and director of ATEC and the IIAE.  More than 20 students,  staff and faculty worked on the project for this development phase. The project has employed many more undergraduate, masters and PhD student developers over its four-year life cycle.

The FPCT captures the sights and sounds of life in a specific deployment area.

As part of the award, the development team received a kiosk display area at the conference where live gameplay was demonstrated to the large conference delegation.  Key developers from ATEC serious games projects gave demonstrations throughout the entire conference.

The developers were able to show the game to key government and business entities involved in modeling and simulation, including a representative from the White House, who visited the Serious Games pavilion to learn about national and international research in serious games.

In addition to winning the award, Zielke, Dufour and ATEC  Research Manager Gary Hardee presented a paper titled  “Creating Micro-expressions and Nuanced Nonverbal Communication in Synthetic Cultural Characters and Environments,” which highlighted some of the new FPCT development recently completed in October.

Virtual Medical World Has Real-Life Value

Researchers from UT Dallas and the UT Arlington College of Nursing have created a virtual environment where graduate students can train online for the medical challenges that await them in the real world.

NursingAP.com is designed to help Advanced Practice Nursing students hone critical skills interactively, on their own schedules via distance learning.

Dr. Marjorie Zielke

“Medical simulation offers the potential to be critical technology for many reasons,” says Dr. Marjorie Zielke, assistant professor of Arts and Technology (ATEC) at UT Dallas and principal investigator for the site’s technology design.  “We can train nurses more quickly, leverage doctoral-level faculty, train on rare but potentially fatal conditions and ultimately save lives. It allows nursing students from remote locations to practice in a controlled, reduced-risk, cost-effective environment.”

Researchers believe this emerging field of virtual gaming medical simulation can improve the accuracy and speed of cognitive and behavioral skills, better preparing nursing students for real-life situations.

A medical care environment complete with virtual babies and nurses helps students prepare for critical situations.

“We have seen some promising results from a randomized study we conducted in the spring of this year,” said Dr. Judy LeFlore, associate professor of nursing at UT Arlington and principal investigator for the project. “Game students (in an experimental group) were more likely to select the right intervention and do it in a more timely manner compared to the lecture only group (control group). However, more research is needed to assure cognitive, behavioral, and/or psychomotor knowledge obtained in the virtual world can successfully be translated into the physical world.”

NursingAP.com is funded by a grant from the Health Resources and Services Administration, an agency of the U.S. Department of Health and Human Services, and serves as both a supplement to traditional classroom instruction and a potential resource for the ongoing education of working professionals in the future.

LeFlore and Zielke submitted a research abstract about their project, titled, “Can Game Play Teach Student Nurses How To Save Lives: An Undergraduate Training Proposal for Student Nurses in Pediatric Respiratory Diseases with a Living World Gaming Construct,” to the 11th International Meeting on Simulation in Healthcare (IMSH), and placed first in the topic category of Emerging and Innovative Technologies & Methods.

Information screens on the NursingAP.com site guide students with both pictures and text.

The initial phase of NursingAP.com is focused on Neonatal Nurse Practitioner training, with subject matter covering conditions in newborns and babies under 2 years old. The training consists of three components:

  • Lectures by subject matter experts like LeFlore, which lend a human intelligence component and provide guidance and content.
  • Complex interactives, such as the virtual ventilator, which allow for specific training exercises and familiarity with equipment.
  • The immersive game itself, “Virtual NICU,” in which the interactive equipment appears in context, allowing nursing students to feel they are actually practicing on a patient in a virtual hospital setting.

In the game, students make decisions within their scope of responsibility on a variety of neonatal conditions.   The virtual environment allows for endless practice and a variety of scenarios in a risk-free environment that can be altered to create dilemmas that can’t be found in a textbook. The student can “feel” tired or stressed, or might have to deal with the virtual patient’s family – all real-life situations that a nurse would have to handle. Such an emphasis on psychological, behavioral and social modeling is unique, providing a distinctive teaching modality that may be used in conjunction with physical simulation, says LeFlore.

But Zielke makes it clear that simulation and virtual patients should not substitute for  traditional lectures and practicum – they simply provide another avenue of learning. She says, “I received my PhD at age 52. So I came in with much more industry experience than 20-something students who got their degrees straight out of college. I might gain more knowledge from the interactive game than from the lectures, simply because of what I already knew coming in.”

That’s one of the unique characteristics of the site – students can enter the program from different navigations, experience levels and perspectives, but will still gain the pertinent knowledge and skills they need. A system of learning metrics and usage patterns are in place to help aggregately track students’ progress and show professors how they are using the materials. The instructors are able to gauge the effectiveness of a variety of different learning modules, to determine what works best for each student.

NursingAP.com provides more than 60 lectures accompanied by dynamic learning modules designed to appeal to various learning styles. All lecture content and interactive courseware meets the core requirements defined by the National Certification Corporation and the National Association of Neonatal Nurses. Researchers are hopeful that the neonatal platform will be applied to other medical issues in the future.

Game Trains Soldiers in a Virtual Iraq or Afghanistan

A training tool being developed by a research team from the Arts and Technology (ATEC) program may soon make it easier for military service men and women to perform their missions in Iraq and Afghanistan.

The project offers virtual villages for soldiers in Iraq or Afghanistan to practice their training skills.

Dr. Marjorie Zielke, the principal investigator on the project, holds the award plaque with her co-investigators, Dr. Thomas Linehan (left), director of Arts and Technology at UT Dallas; and Dr. Frank DuFour, assistant professor of sound design.

“The work we’re doing has to do with the facilitation of cultural training,” said Dr. Marjorie Zielke, an assistant professor in the ATEC program and the principal investigator on the project.  “The way some of that training has been done in the past and may still be done in certain areas is to build actual villages and hire actors to replicate a particular culture,”  Zielke said. “That kind of approach has some limitations in the sense that it’s expensive, not everyone can attend, it’s not easily changed because it’s a physical structure, you have to work with actual actors, and so forth.”

The ATEC team set out to re-create  a realistic virtual environment instead.  The result is First Person Cultural Trainer (FPCT), a 3D interactive game that  teaches soldiers the values and norms of Iraqi and Afghan cultures.  FPCT is a serious game, which means that it is designed for purposes other than pure entertainment, in this case, cultural training.

Players enter the community from the first-person point of view and collect information based on verbal and non-verbal cues observed in characters encountered along the way.

FPCT recently won the Cross-Function Team Award at the 2010 Modeling & Simulation Leadership Summit, held in Virginia Beach. Presented annually by the National Training & Simulation Association (NTSA), the Modeling & Simulation (M&S) Awards recognize achievement in the M&S functional areas of training, analysis and acquisition, and in support of the overall M&S effort.

First Person Cultural Trainer was also a finalist at the Interservice/Industry Training, Simulation and Education Conference (I/ITSEC) in Florida in early December.  I/ITSEC promotes cooperation among the armed services, industry, academia and various government agencies in pursuit of improved training and education programs.

Zielke says ATEC has been working in the cultural training and simulation area for about three years, with about 15 students – at the undergraduate, graduate and doctoral levels – working on this project.  Her co-investigators are Dr. Thomas Linehan, Endowed Chair and director of Arts and Technology at UT Dallas; and Dr. Frank DuFour, assistant professor of sound design.

The project team conducted extensive research to get the characters to look, sound and act like the culture they’re representing.

“Much of the cultural data is being developed in real time by the military,” Zielke said.  “By having it in a systems-based approach that is composable — in other words, we can generate culture in certain aspects of the game on the fly — we can respond to the data as soon as it becomes available.  We could change it overnight if we needed to.”

The project is supported and sponsored by the U.S. Army Training and Doctrine Command (TRADOC) G-2 Intelligence Support Activity (TRISA).

“This prototype, now on its way to a full-fledged model, is a highly flexible tool for training battle staffs and individual soldiers at the tactical level,” said Mel Cape, senior knowledge engineer for TRADOC G2. “The recognition [the UT Dallas team] has received within the modeling and simulation community is well deserved, and we at TRISA are proud of their superlative efforts in the development of a culturally-based training device.”

Part of this cultural training is to familiarize soldiers with what they will face when arriving in their theaters of operation. Researchers worked to make the game’s characters look, sound and act as much as possible like people from the culture they represent.

The project offers virtual villages for soldiers in Iraq or Afghanistan to practice their training skills.

In the game, the player enters a community from the first-person point of view.  He doesn’t know much about the community, how the people feel about him, or who the key figures are in the village. The goal is to move through the community and try to understand the social structures and issues, then address those issues and work with the community to affect missions.

The people in the community form opinions about the player based on how the player treats them.  If the player doesn’t interact properly with them, the villagers discuss his behavior among themselves.  Some individuals in the village have more clout than others.

The player collects information based on verbal and non-verbal cues he observes in the characters he encounters and then rates those characters based on a scale of four emotions: anger, fear, gladness and neutrality.

One of the most rewarding aspects of this program for Zielke is the opportunity to help her students make connections within the industry. “Between this project and other similar projects in the ATEC program, we have at least 30 students employed at any given time,” she said. “The students develop great portfolios, gain work experience, go to conferences, write research papers based on an incredibly rich data set and then hopefully leverage all of those things to get industry jobs.”