Video pulled from people's thoughts

6th

Scientists have discovered how to “read” minds by scanning brain activity and reproducing images of what people are seeing — or even remembering.


Researchers have been able to convert into crude video footage the brain activity stimulated by what a person is watching or recalling.

The breakthrough raises the prospect of significant benefits, such as allowing people who are unable to move or speak to communicate via visualisation of their thoughts; recording people’s dreams; or allowing police to identify criminals by recalling the memories of a witness.

However, it could also herald a new Big Brother era, similar to that envisaged in the Hollywood film Minority Report, in which an individual’s private thoughts can be readily accessed by the authorities.

Earlier this year, Jack Gallant and Thomas Naselaris, two neurologists from the University of California, Berkeley, managed to ‘decode’ static images seen by the person from activity in the brain’s visual cortex. Last week Gallant and Shinji Nishimoto – another neurologist – went one step further by revealing that it is possible to decode signals generated in the brain by moving scenes.

In an experiment which has yet to be peer reviewed, Gallant and Nishimoto, using functional magnetic resonance imaging (fMRI) technology, scanned the brains of two patients as they watched videos.

A computer programme was used to search for links between the configuration of shapes, colours and movements in the videos, and patterns of activity in the patients’ visual cortex.

It was later fed more than 200 days’ worth of YouTube internet clips and asked to predict which areas of the brain the clips would stimulate if people were watching them.

Finally, the software was used to monitor the two patients’ brains as they watched a new film and to reproduce what they were seeing based on their neural activity alone.

Remarkably, the computer programme was able to display continuous footage of the films they were watching — albeit with blurred images.

In one scene which featured the actor Steve Martin wearing a white shirt, the software recreated his rough shape and white torso but missed other details, such as his facial features.

Another scene, showing a plane flying towards the camera against a city skyline, was less successfully reproduced. The computer recreated the image of the skyline but omitted the plane altogether.

“Some scenes decode better than others,” said Gallant. “We can decode talking heads really well. But a camera panning quickly across a scene confuses the algorithm.

“You can use a device like this to do some pretty cool things. At the moment when you see something and want to describe it to someone you have to use words or draw it and it doesn’t work very well.

“You could use this technology to transmit the image to someone. It might be useful for artists or to allow you to recover an eyewitness’s memory of a crime.”

Such technology may not be confined to the here and now. Scientists at University College London have conducted separate tests that detect, with an accuracy of about 50%, memories recalled by patients.

The discoveries come amid a flurry of developments in the field of brain science. Researchers have also used scanning technology to measure academic ability, detect early signs of Alzheimer’s and other degenerative conditions, and even predict the decision a person is about to make before they are conscious of making it.

Such developments may have controversial ramifications. In Britain, fMRI scanning technology has been sold to multinational companies, such as Unilever and McDonald’s, enabling them to see how we subconsciously react to brands.

In America, security agencies are researching the use of brain scanners for interrogating prisoners, and Lockheed Martin, the US defence contractor, is reported to have studied the possibility of scanning brains at a distance.

This would allow an individual’s thoughts and anxieties to be examined without their knowledge in sensitive locations such as airports.

Russell Foster, a neuroscientist at Oxford University, said rapid advances in the field were throwing up ethical dilemmas.

“It’s absolutely critical for scientists to inform the public about what we are doing so they can engage in the debate about how this knowledge should be used,” he said.

“It’s the age-old problem: knowledge is power and it can be used for both good and evil.”

Alot in here to take in and ponder. Sure at this stage it might just be science fiction but with the direction they are going its something to watch.

I'm just wondering how easy it would be to cheat something like this? Easier than a polygraph I image.

nyarlothothep

I expect the little hitlers of this world to use the tech irresponsibly and stupidly in 100-150 years time.

humanji

I was thinking the same thing. Decoding what someone is currently thinking about seems easy enough (in basic terms, the science behind it is giving me a headache), but considering the billions upon billions of images that a human will see in their life, I wonder how this would be able to pick out individual thoughts.

Then there's the reliability of the memory itself that's a problem. If I witnessed a robbery and thought the robber looked a bit like Jack Palance, then if my memory was read by the police, would their suspect come out as Jack Palance?

Even if it were being designed for nefarious reasons alone, I can't see any reliable prototype being made for a good few years, and even then I can't see how they'd over come the problem of soring someones memories.

humanji

nyarlothothep wrote:

I expect the little hitlers of this world to use the tech irresponsibly and stupidly in 100-150 years time.

Don't worry, they'll have killed us all long before that.

RoboClam

humanji wrote: »

I was thinking the same thing. Decoding what someone is currently thinking about seems easy enough (in basic terms, the science behind it is giving me a headache), but considering the billions upon billions of images that a human will see in their life, I wonder how this would be able to pick out individual thoughts.

Then there's the reliability of the memory itself that's a problem. If I witnessed a robbery and thought the robber looked a bit like Jack Palance, then if my memory was read by the police, would their suspect come out as Jack Palance?

Even if it were being designed for nefarious reasons alone, I can't see any reliable prototype being made for a good few years, and even then I can't see how they'd over come the problem of soring someones memories.

Good point. So far, all this this technology can do so far is pick out your reactions to different stimuli and interpreting them. So obviously your brain pattern looks a certain way when you see a man in a white coat and another way when you see a city skyline. Seems to have a problem picking up multiple inputs though (ie the city skyline AND the plane).

This particular method doesn't seem like it would be able to read memories, it also assumes that everyone brain activities are identical. So unless you were walking through the airport thinking "bomb bomb bomb bomb BOMB" then this technology won't work. Accessing memories is a lot more complex than what this addresses.

But of course, science advances every day, so there may be a way to do it in the future. I wonder if it would be possible to ever distinguish real memories from imagined memories though.

seamus

humanji wrote: »

I was thinking the same thing. Decoding what someone is currently thinking about seems easy enough (in basic terms, the science behind it is giving me a headache), but considering the billions upon billions of images that a human will see in their life, I wonder how this would be able to pick out individual thoughts.

I think the intention is to decode the actual patterns themselves as opposed to trying to "guess" what the person is looking at from a fixed set of possible guesses.

In other words what I mean is, the may be looking for a distinct coding that the brain uses when thinking about images. A computer doesn't need to know what a .jpg file contains to decode it, it simply needs to know how to turn the raw stream of data into an image. If we all think in the same "coding", then it would be theoretically possible to decode our thought stream and display exactly what we're thinking.

Some very big ifs in there though. "Missing" some key parts of the image may also be down to the way we are interpreting the data. If the subject, for example, was focussed on Steve Martin's white shirt, then his brain may simply not be processing the finer details of his face.

6th

Well they arent making any crazy claims about what they have achieved but rather are saying its very primative and talking about the "potential" of such tech.

seamus

6th wrote: »

Well they arent making any crazy claims about what they have achieved but rather are saying its very primative and talking about the "potential" of such tech.

From a CT point of view, the current path they're going down is a million miles away from plucking information from your brain to use against you. This application doesn't address the issue of taking non-active thoughts and memories from one's brain and to use the bomb example above, would be easily fooled by simply thinking of sunshine and rainbows more strongly than anything else. Although we can successfully have a number of different thought "streams" active simultaneously, it may not be possible for a compute to accurately separate them.