Solutions Developed by VU Neurobiologists Open New Opportunities for Music Performers

The neuroscientists from the Institute of Biosciences at Vilnius University Life Sciences Centre focuse on the development of brain-computer interface. In addition to using measured brain impulses for rehabilitation, the scientists use them to control assistive mechanisms for people with disabilities, and to use brain responses to amplify the emotions of audiences during concerts and performances. Dr. Inga Griškova-Bulanova, a researcher at the VU Institute of Biosciences, discussed the emotions evoked by music, her international project, and the possibilities of “mind reading.”

Music is language, emotion, and mathematics

Everyone knows that music evokes some kind of emotion, but is it possible to identify the genre of music that has the greatest impact on feelings? According to Dr. I. Griškova-Bulanova, emotions are very individual and difficult to study, depending on both the listener and the context.

“Different genres of music evoke different sensations. But, in this case, much more important is the personal experience of the listener. The most emotional works are the ones you recognize, the genre is not that important. And if you listen to a piece of music in the laboratory and are told what emotion you should feel, then that emotion is the most likely to emerge. So, the music is important, but even more important is the person listening to it,” she says.

Music, in its structure, involves a lot of mathematical features: there are certain patterns in the way it is created, not only in terms of sequential processing but also in terms of spatial processing, in terms of predicting consequences, or in terms of picking up on relationships that are not immediately apparent.

“Listening to music activates many areas of the brain, including areas responsible for decision-making, which are also important in mathematics. So, learning mathematics and learning music are both mutually reinforcing processes,” she says.

Music is also a form of language. According to the researcher, listening to music activates a large number of areas in the brain that overlap with language areas as well as emotional and memory centers.

“It seems that for people with Alzheimer’s disease listening to music can help filling in the gaps in memory and finding the secret tracks that lead to memories. People with this disease may not remember much, but they may know the lyrics of a song or the emotions associated with it,” says the researcher.

Using a brain-computer interface to amplify the senses evoked by music

One area of brain-computer interface implementation is music performance. During the project “Brain-Computer Interface for Music Embodiment Research”, Dr. I. Griškova-Bulanova, in cooperation with musicians from Latvia and engineers from Taiwan, delved deeper into the brain-computer musical interface and aimed to use brain signals to amplify the sensations evoked by music.

“This is an area of interest for musicians who don’t have enough “hands on deck” to make their performances unforgettable. They need a tool that uses the brain to help the performer convey their music in a way that has the greatest possible impact on people. We were first approached by musicians from Latvia. Their main representative is a PhD student in systematic musicology who is looking for more innovative ways to teach music performance,” says the researcher.

To develop the tool, researchers used electroencephalography to measure electrical activity in the brain. They wanted to create conditions that were as close as possible to a real performance in concert.

“At the very beginning of the project, we had an ambitious idea. We wanted to study performers while they perform under natural conditions, i.e., in concert with an audience. But the Covid-19 situation ruined our plans. It so happened that for most of the study, the musicians performed the pieces in the lab without an audience, so the emotion was not entirely accurate. We are very pleased to have been able to carry out brain activity recordings this spring with students playing for their classmates under observation and evaluation. This brings us at least a little closer to the real conditions,” says Dr. I. Griškova-Bulanova.

Brain-controlled light effects to enhance emotion

The researchers were able to record brain activity under controlled conditions and identify the parameters that need to be taken into account to reflect the emotion being conveyed. This would be of great help in the future development of an instrument that works under real conditions.

“With the results of the project at their disposal, the colleagues were able to enliven the musical performance. The performer was allowed to play as he or she wished, and our team used the brain activity data to create effects that enhance the listener’s emotions. The performance sound and visual effects were modulated depending on brain activity,” the researcher said, discussing the results.

The project team’s vision for the future is to carry out “real” research where musicians can perform under natural conditions: in a hall, with an audience. Researchers would also like to find out what makes the listener feel the emotion that the performer conveys.

“This would require brain activity research to record several people at the same time: the one who plays the music and the one who listens to it. But that’s a topic for future research,” she says.

“Mind reading” is a long way off

Brain research, including artificial intelligence techniques, is increasingly talking about possible “mind reading.” According to Dr. I. Griškova-Bulanova, the successful “mind reading” trials carried out so far have taken place in a strictly controlled environment and the brain signal received has been thoroughly processed using special techniques and professional equipment.

“Research in the laboratory takes place under strictly controlled conditions. So, the brain signals recorded here are as clean as possible from all environmental disturbances. But if we were to try the same experiment in a shoe shop, both the conditions and the results would not be equivalent. Equally important is the equipment used, which is designed for research in the laboratory with no ambient noise, and which would be totally unsuitable for other conditions,” the researcher says.

Dr. I. Griškova-Bulanova also points out that all people are individual and it is very difficult to devise methods and systems that can “read” the minds of all people.

“In our research, we have only tried to control light, and still faced many obstacles. Our success rate is around 80%, but it was all done in a laboratory under strictly controlled conditions and we knew in advance what we wanted to measure and test. In my opinion, the most that the brain can be trained to do is to “use the mind” to turn the pages because that is a motor action. But I can’t yet imagine global decision-making through “mind reading,” the researcher says.