Researchers Study Hand Gestures For Children With Disabilities

Montana children with learning disabilities could get a boost thanks to new research at Montana State University that seeks to better understand how visual hand gestures can aid in communications in the classroom and home.

The project, which started in August and is funded with $67,000 from Montana INBRE, is a collaboration between researchers in MSU’s Norm Asbjornson College of Engineering, Mark and Robyn Jones College of Nursing and the College of Education, Health and Human Development. The goal is to develop training videos and other aids that could help teachers, families and others interacting with children, especially in rural areas where resources such as speech therapy are often hours away, according to project leader Bernadette McCrory, assistant professor in the Department of Mechanical and Industrial Engineering and faculty for MSU’s Biomedical Engineering program.

“Anything we can do to help our kids in Montana, that’s something we’re committed to,” McCrory said.

For the project, child volunteers will play a series of language games on a computer. The games include a variety of hand gestures, and state-of-the-art eye-tracking devices will help measure the volunteers’ comprehension. The results will advance understanding of which kinds of visual gestures most enhance language comprehension.

“The idea is that language isn’t just verbal — we do a lot of communication with our face and hands,” said Nadya Modyanova, project co-leader and a research scientist in MSU’s engineering college. Kids, especially those with developmental disabilities such as autism, can struggle with verbal comprehension and speech but are often more attuned to visual signals, she noted.

Helping teachers and caregivers develop visual hand gesture skills could go a long way toward improving learning outcomes and quality of life for those children, she said.

The new project expands on a phase of preliminary research that in 2022 received a pilot grant of $50,000 from Montana INBRE, which is a statewide network of Ph.D.-granting institutions, baccalaureate schools and community and tribal colleges that invests in Montana’s biomedical research capacity and workforce.

“Ultimately, what we want to do is to create a training video for children and adults of all abilities that shows how to use gestures effectively to support communication,” Modyanova said.

The eye-tracking equipment is housed in MSU’s Biomedical Innovation for Research and Development Hub, or BioReD Hub, which was established last spring with a $429,000 grant from the M. J. Murdock Charitable Trust to expand biomedical engineering research and education with an emphasis on developing new technologies to improve rural health care. McCrory is leading BioReD Hub, which emphasizes working across academic disciplines to solve pressing health challenges. McCrory has previously collaborated on the eye-tracking technology with MSU nursing professor Elizabeth Johnson, who is also contributing to the new project.

“We are working together as nurses, engineers, educators and computer scientists toward common goals,” McCrory said. “I’m excited about how we can take these tools we use in engineering and use them in public health, pediatrics — all these challenges such as in education.”

With the newly funded INBRE project, she said, “I’m excited to see how we can integrate gestures to have more positive outcomes for our children who learn in a different way than most.”

Montana INBRE is an IDeA Program funded by the NIGMS division of the National Institutes of Health under award P20GM103474.

For more information about the project, contact Bernadette McCrory, 406994-2801, bernadette.mccrory@ montana.edu or Nadya Modyanova, 406-848-1535, nadezhda.modyanova@montana. edu. If you are interested in participating as a research subject, contact Modyanova at 406-848-1535 or language- research@montana. edu.