Sooraj Shah ’24

The relationship between humans and technology is one which advanced the world to where it is today. By physically pressing a few buttons, we are able to express our thoughts and ideas onto a digital screen. However, this might not always be the case. Researchers at the University of Helsinki have developed neuroadaptive generative modeling, in which a computer creates a visual model via feedback from brain signals of individuals who were concentrating on a specific image. BCI systems, which rely on pre-determined user actions for output, have traditionally been used; yet a major limitation of such a system is the prevention of complex mental visualization. The study builds upon BCI systems in a way that can achieve more advanced mental visualization.

The experiment involved exposing participants to computer based, realistic images with a broad range of physical features. Participants were instructed to focus on images of a specific category, such as “young” or “old.” The resulting electroencephalogram (EEG) signals were then used to produce an image similar to the category that the participant was consciously viewing and thinking of. The models were updated by observing spikes in the event-related potential of the user, which is a brain response recorded when the individuals are observing the images. The use of Generative Neural Networks (GAN), which can generate previously non-existing images, was formally tested when participants were asked to select the computer produced images that best matched the ones they personally viewed. The results were broken down into three different model types. Positive (most relevant), negative (least relevant), and random (random feedback). The positive images were chosen 90% of the time, the negative 2.5%, and random 42.5%. The results exemplified that neuroadaptive modeling is a highly accurate, precise generation of the perceptual visualization provided by the participants. The relationship between the generated images and the human approval of the images was proportionally positive, indicating its accuracy.

The practical use of neuroadaptive generative modeling can lie in a number of fields. For scientists, it can provide a direct pathway for illustrating ideas and designs far more accurately than attempting to write it on paper. A major focus of future research is making this technology more dependable and error-free in order to fully capture the cognitive representations of humans. 

Works cited:

[1] https://www.nature.com/articles/s41598-020-71287-1

L. Kangassalo, M. Spapé, T. Ruatsalo. Neuroadaptive modelling for generating images matching perceptual categories. Scientific Reports 10, (2020).

[2] Image retrieved from: https://www.nature.com/articles/s41598-020-71287-1/figures/1