Motion capture was being used in the healthcare space decades before Hollywood adopted the technology. Since then, a technological gap has emerged between the industries. Now the healthcare sector is catching up again.
Four decades before Andy Serkis’ motion capture-enabled depiction of Gollum stunned cinema-goers in the early 2000s, modern motion capture was taking its first baby steps.
Contrary to popular belief, the technology was not initially meant for the film industry.
Instead, it has its roots in healthcare, starting as a photogrammetric analysis tool in biomechanics research in the 1970s and 80s.
The early technology that would come to birth modern motion capture was used as early as the 19th century when photographer Eadweard Muybridge studied the motion of humans and animals through stop-motion photography.
Among the first institutions to develop the technology was The University of Strathclyde which continues to innovate today through a long-standing partnership with Vicon, a global leader in motion capture cameras, software and motion capture systems for the VFX, life science and engineering industries.
In addition to its work in the film and game industries, Vicon’s motion capture technology is used by numerous research institutions to improve patient outcomes in areas ranging from the treatment of neuromuscular conditions, such as cerebral palsy or Parkinson’s, to recovering from ACL reconstruction.
Health Tech World sat down over Zoom with the research director for biomedical engineering at the University of Strathclyde, Professor Phillip Rowe and Dr Kim Duffy, head of life sciences at Vicon to delve into the evolution of mo-cap technology, its applications in the healthcare industry and their exciting partnership that aims to explore its future potential.
Building the foundations of modern mo-cap
Professor Philip Rowe’s “academic grandfather” and early pioneer of motion capture, Professor John P. Paul, was approached in the 1960s by an orthopaedic surgeon who was attempting, unsuccessfully, to replace joints in the body.
Professor Paul set out to understand why the implants kept breaking. He used cine cameras and markers to study the loading on joints and improve orthopaedic implants.
“He used cine cameras at right angles to each other, recorded people with dots on them, projected them on the wall, measure the position of the dots and by triangulation, worked out where the markers were,” Professor Rowe explained.
“He did that for about a dozen people and basically showed the profile of loading on the hip.
“Then, when everybody started making those hip implants stronger, [patients] started to survive and that really led through in the late 60s to a breakthrough in orthopaedic surgery.”
In the 1970s, efforts were made to automate the process using early video cameras, leading to the creation of Oxford Metrics, which later became Vicon. Headed by Julian Morris, the small medical firm lay the groundwork for the eye-popping visual effects seen in film and TV today.
The emergence of a technological gap
While the animation and film industry quickly embraced the possibilities of mo-cap technology, the healthcare sector has been more conservative in its adoption.
Traditional clinical practices and the need for accuracy and risk mitigation have often limited the sector’s exploration of the full potential of mo-cap.
“When the animation industry got involved, the icon upscaled massively right, which led to better service for everybody but my sector, the medical sector was sort of stuck with methods that it was used to.
“We would put up with all sorts of things that the animation industry didn’t have the time to put up with. The animation industry wanted to push things further.
“They wanted to monitor multiple people and whole people and then show it in real-time.
“The technical capabilities of their equipment moved on, but the medical application industry didn’t from the 90s.”
One example of this technological gap is the use use of multiple markers.
Mo-cap systems in the medical space generally use single markers attached to key anatomical areas which are used to digitally reconstruct movement.
After the data has been recorded, each of these markers needs to be labelled manually; a time-consuming process.
Current mo-cap gait analysis conducted by a clinical provider costs in the region of £1500, mostly due to staff time.
Meanwhile, the animation sector developed the use of clustered markers.
These patterns can be recognised by software which then automatically labels the markers in real-time.
“There’s something good about stability in that all the medical records that these groups have over years are done with the same methods, so they can compare them,” Professor Rowe said.
“But that also means that you, in a way, don’t move on with the technology.”
However, there is now a gradual shift occurring.
While she “agreed to an extent” with Professor Rowe’s view, there are certain clinical groups, she said, that are trying to breach the boundaries of what traditional clinical gait analysis can be.
While some groups have been pushing the technology forward over the last few years with real-time monitoring and VR, Dr Duffy argues that the issue lies in translating research into clinical practice.
“There are only certain clinical groups that use [real-time monitoring],” Dr Duffy said.
“When you go into the research world, that’s when you start seeing this blending of people trying new innovations and new technologies, but not always does it translate back to clinical practice.
Dr Kim Duffy
“It does in some cases, but not always. I think it’s starting to shift with changes in technology.
“Things are becoming easier to do and you’re starting to see people more willing to try different things.”
The partnership between the University of Strathclyde and Vicon is at the forefront of this shift, aiming to make mo-cap technology more accessible and further explore its applications in musculoskeletal and neurological therapy.
Dr Duffy added: “Phil is right, especially in the UK, that there aren’t many centres that are doing what Phil and Strathclyde are doing and that’s what’s the most exciting thing about this project.”
The game-changing potential of mo-cap, VR and gamification
The partnership between the University of Strathclyde and Vicon aims to explore the potential of mo-cap in healthcare, particularly in musculoskeletal and neurorehabilitation, by developing new protocols to change the entry barrier and make the technology more accessible.
The collaboration was initiated when Professor Rowe approached Vicon with the idea of using mo-cap for movement analysis with clusters of markers, similar to the techniques employed by the animation industry.
“I had seen what the animation industry could do and I was frustrated about what the clinical sector did and I felt there was an opportunity there,” Professor Rowe said.
After initial proof of concept studies, the university opened its in-house treatment centre which works mostly with stroke patients.
The centre has incorporated real-time monitoring to offer patients feedback during their session rather than waiting several weeks for a written report.
“If we want people to rehabilitate, we need to be able to give them this feedback,” Professor Rowe said.
“Unlike with sports biomechanics, where they can spend a fortune to do this on a few elite athletes, we need to be able to do it at scale.
“It was clear from the Vicon technology that it was possible to give real-time feedback of how you were moving and the moments that you were generating in your joints, but the medical sector wasn’t using the technology.”
Gamifying neurological therapy
Traditional exercises for patients in these types of therapy often have low uptake due to their repetitive nature.
But now, innovative projects such as those led by the University of Strathclyde and Vicon, are using mo-cap and immersive gaming systems to transform the patient experience.
The gamified, virtual reality environments provided by these systems offer a new level of engagement and focus.
No longer limited to mundane exercises, patients can now embark on journeys where every movement translates into points and achievements.
This technology not only enhances patient care but also offers personalised and effective treatments.
Initially, the team at Strathclyde believed the gamification element was just helping boost motivation, but as the project has developed along with the emergence of new research, it is becoming increasingly clear that it also has an inherent therapeutic benefit.
Prof. Philip Rowe
While most people can walk without consciously thinking about it, people with brain injury cannot due to the pathways to their muscles being damaged.
New understanding of VR, however, suggests that the cognitive element of using a gamified system can help people rebuild these pathways.
“When they think about [walking] in their cortex, they then find a different way of achieving it,” Professor Rowe said.
“Adding a cognitive load improves the speed of reconnection and relearning. They are, If you like, going around the roadblock they have in their brain.
“It’s more than just making it interesting. Now. I think it’s actually part of the process of relearning.”
Now, the University of Strathclyde is working with NHS Lanarkshire to implement the technology as a service.
The two institutions are set to open a “technology-enhanced gym” in one of NHS provider’s acute stroke units and in a community technology hub.
Right now, mo-cap is chiefly employed in treating neurological conditions, but Dr Duffy believes the technology has potential that extends beyond this category of conditions.
“I’d like to see more clinical labs pop up with motion capture because I think it could help such a wide array of patients,” Dr Duffy said.
“Typically right now, it’s neurological conditions, mainly cerebral palsy, a stroke and a little bit of Parkinson’s, but it’d be great if it could be a wider application, for example, with podiatry [and] diabetes.
“There are so many different applications in measuring someone’s walking or rehabilitation in a walking environment.
“You can tell so much about a person’s health and how you can more appropriately rehabilitate.
“That’s where it’s exciting and as it becomes easier to use and more accessible, having facilities such as Strathclyde’s rehab centre, it opens up the possibilities for other clinical groups.”
