Weizmann AI Can Reconstruct Images From Brain Scans in Just One Hour

Researchers at Israel’s Weizmann Institute of Science have developed a new artificial intelligence system capable of reconstructing images a person is looking at by analyzing their brain activity.

Called Brain-IT, the system uses functional magnetic resonance imaging (fMRI) data to identify patterns of brain activity associated with visual information. More importantly, researchers say the system can adapt to a new person using approximately one hour of brain-scan data, a major improvement over earlier approaches that could require around 40 hours.

How Brain-IT Reconstructs Images

When a person looks at an image, different areas of the brain respond to its colors, shapes, locations and overall meaning. An fMRI scanner can measure changes in blood oxygen associated with this neural activity.

Brain-IT analyzes these patterns and attempts to reconstruct the image that produced them.

The technology is built around a Brain-Interaction Transformer (BIT). Rather than treating every person's brain as completely unique, the system searches for functional patterns that are shared across different individuals.

Researchers also developed a Universal Brain Encoder. This component works in the opposite direction by predicting brain activity that could be generated when a person views a particular image.

Together, these technologies allow the researchers to make better use of the limited amount of available brain-scan data.

Why One Hour of Training Matters

One of the biggest challenges in brain-decoding research is the amount of personalized data required.

Previous systems generally needed many hours of fMRI recordings from a new participant before they could produce useful reconstructions. Brain-IT reportedly achieves comparable results after approximately one hour of data from a new individual.

This could make future brain-decoding research considerably more practical.

The researchers trained the system using data from eight participants who viewed thousands of images while undergoing fMRI scans. Because collecting this type of data is expensive and time-consuming, the team developed a method for expanding its training information without having to scan people for every additional image.

AI Learns Shared Brain Patterns

The research team divided brain activity into thousands of small regions called voxels. By studying how these regions responded to different visual characteristics, the researchers were able to identify patterns that appeared across multiple participants.

The Universal Brain Encoder identified 128 functional brain regions that were shared among participants and appeared to play different roles in visual processing.

The researchers also reported an interesting finding involving the parahippocampal place area, a part of the brain associated with processing scenes and places. Their analysis suggested a previously unrecognized distinction between responses related to indoor and outdoor environments.

More Than Just Recognizing the Subject

Earlier brain-to-image systems could often determine the general subject of an image but struggled to reproduce its finer visual characteristics.

For example, an AI might understand that someone was looking at a dog, but the reconstructed image could differ significantly in terms of the dog's position, composition, colors or surrounding environment.

Brain-IT is designed to preserve both high-level semantic information and lower-level visual structure. Its architecture uses different visual features to guide the reconstruction process, helping the resulting image more closely resemble what the person actually saw.

Researchers demonstrated the technology using scenes such as a baseball game, a dog leaning out of a car window and people walking through a snowy environment.

Is This Really Mind Reading?

Despite the impressive results, Brain-IT should not be considered a general-purpose mind-reading system.

The research focuses specifically on reconstructing externally presented visual images under controlled experimental conditions. It does not demonstrate that the system can read a person's private thoughts, beliefs, intentions or memories.

Dream decoding has also not been demonstrated by this research.

The technology currently depends on fMRI scanners, which are large, expensive machines that require controlled laboratory environments. That makes the system very different from a practical consumer mind-reading device.

Researchers Are Exploring EEG and Other Applications

The Weizmann team is also investigating whether similar approaches could eventually work with electroencephalography (EEG), which records electrical activity using sensors placed on the scalp.

EEG equipment can potentially be much more portable than an MRI scanner, although current research has not established that an EEG-based version can match Brain-IT's fMRI performance.

The researchers are also interested in decoding other forms of information, including auditory signals. Video reconstruction is considerably more difficult because video changes rapidly while fMRI measurements are comparatively slow.

Potential Benefits for Assistive Technology

One of the most interesting potential applications of this research is communication assistance.

Brain-decoding technology could eventually contribute to systems designed to help people who are unable to speak or move communicate more effectively. Similar research has already explored using brain activity to decode intended speech and other forms of communication.

However, turning Brain-IT into a practical medical technology would require extensive additional research, testing and safeguards.

Privacy Concerns Could Become More Important

As AI becomes increasingly capable of interpreting neural activity, privacy will become an important issue.

Brain data is fundamentally different from many conventional forms of personal data because it can potentially reveal information about how a person responds to visual or other stimuli.

Any future system capable of decoding neural information outside a laboratory would therefore need strong safeguards covering consent, data security and individual control over neural information.

A Significant Step for Brain-AI Research

Brain-IT represents an important advance in the effort to connect artificial intelligence with human neuroscience. Its ability to reconstruct viewed images while requiring substantially less calibration data could help researchers develop more efficient brain-decoding systems.

However, the technology remains a research project rather than a consumer mind-reading device. The need for fMRI equipment, controlled experiments and additional validation means that widespread real-world applications are still some distance away.

For now, the most significant achievement may be the demonstration that AI can identify common patterns across different human brains and use those patterns to reconstruct visual information with far less personalized training.