Can You Recover Sound From Images?

  visningar 2,483,026


År sedan

Is it possible to reconstruct sound from high-speed video images?
Part of this video was sponsored by LastPass:
Special thanks to Dr. Abe Davis for revisiting his research with me:
This video was based on research by Dr. Abe Davis and colleagues. I found out about this work years ago and was fascinated by the way he was able to capture vibration information in image-only video. I always imagined the motions of objects would be visible as when recording a tuning fork in slow motion - so deriving sound from high speed images seemed a feasible task. But the reality is much more difficult.
Sound vibrations only cause objects to wiggle by about a micrometer. This is much smaller than a pixel, so the algorithm must understand the characteristics of the image. A move in one direction should cause some pixels to lighten slightly, while others darken - and this behavior is correlated along the edges of the image. So noise can be reduced because it's random over the image and there are enough places to sample that you can get it to cancel out.
Something I'm wondering now is - would it be possible to capture sound in a single image? I'm thinking it would have to be an image of a large object or space because the wavelengths of typical sounds are quite long. Maybe a high frequency sound could be imaged in a suitable medium...
Animations by Alan Chamberlain
Music from "Seaweed"

Stefan Hrvatski
Stefan Hrvatski 2 dagar sedan
Haha, he's using Matlab :D
A.M. Goudarzi
A.M. Goudarzi 3 dagar sedan
think about all the real-world implications that this technology had for the surveillance technology, and would have in the years to come.
A. B.
A. B. 4 dagar sedan
If we now know about this the Gov definitely damn sure knows and has known! Prob has already used it to spy on others too.
Matt Stroker
Matt Stroker 4 dagar sedan
You didn't really think the piece of foil through, did ya? 🤣🤣🤣 Should have used an open chips bag, for real, from multiple angles.
David P.
David P. 6 dagar sedan
You should do a video over TEMPEST
StrangeVortex 6 dagar sedan
what a nerd
Bearwastaken 6 dagar sedan
and thats how a lazer record player works
Nikhil Agrawal
Nikhil Agrawal 6 dagar sedan
If the sound recovery is possible from the objects, can we recover what conversation happened in the history??????????????????????
shawn chartrand
shawn chartrand 7 dagar sedan
the rcmp already have a working device that does this over distance
Bob Coco
Bob Coco 7 dagar sedan
9:11 that's how proper Beats headphones have been designed to sound. (like an earthquake... got it? 9, 1, 1... Boom Ok that 1 was far streched from the ªss hairs)
Ruthless Cardigan
Ruthless Cardigan 7 dagar sedan
sounds like some old obsolete spook tech to me like microwave transmitters
Lisa Price
Lisa Price 7 dagar sedan
what stupid idea, why not try to use a washing machine to record sound.? Anyway, replacing the bright light with a LASER LIGHT WILL GIVE YOU RELEVANT CHANGES that are RECORDABLE ON a CAMERA. now replace the camera with a very low resolution light sensor chip, you can easy record at a speed for clear audio without having to pay lots of money.
Siddique Khalid
Siddique Khalid 7 dagar sedan
so in future students will have assignments like this.
Bass Boosted
Bass Boosted 8 dagar sedan
me tries to extract sound from a pornomovie where its sound is blocked behind a paywall
Media Info
Media Info 8 dagar sedan
There is a spying methon, named *Lamphone* , ist about reading the sound from the bulb light, wihtout any laser systems, just olny recording camera with telesope , using and algorithm wihch is reading a changes as a queakes in a range from 300 to 950 micrometers and is generating a sound. This sound is attened in normialisations, noise reductions and badstop filtering to filnally acheiving a voice. This eavesdropper has been tested and work even in the distance up to 25 meters and may by use in real time. Quality and precision of sound depend of the optic sensor. Moreover, it may by improoved by involving a neuron nets into sound generating process.
J P 8 dagar sedan
If you perfect this, you could capture sound with a telescope
Revel Carlberg West
Revel Carlberg West 8 dagar sedan
Geez, get the Slow Mo Guys.
Michael O Neill
Michael O Neill 8 dagar sedan
I was thinking this the other day when I was watching ghost hunters, they say the past is scratched in to the walls like grooves in a record, so why not invent a machine that can read the walls?
Johan Faas
Johan Faas 9 dagar sedan
Billy Pacaccio
Billy Pacaccio 9 dagar sedan
that is VERY old tech! Russia had that in the 30's
fanfare100 9 dagar sedan
Isn't that somehow how Bin Laden was caught? At least a component thereof. There were other technologies at play too. But this was a crucial one.
MawaMaverick 9 dagar sedan
I love lastpass
X S 9 dagar sedan
freedomsteppa 9 dagar sedan
Kudos. Could not believe this would be possible. You guys are amazing. Especially mister expert there... can't even begin to imagine what you actually really work on, normally...
Carlos Correa
Carlos Correa 9 dagar sedan
Wow, Dr. Abe Davis maybe into something lol. This reminded me of the TV Show "Fringe" (Season 1 Ep 19 "The Road Not Taken") where people were been victims of spontaneous combustion and the lead character in order to investigate one of the incident scenes use an Electron Microscope to detect and record the tinny sound movements stored in glass from a window similar to what Dr. Abe Davis is trying to capture with cameras and then somehow feed the movements detected onto an Electron Microscope then into a computer where he can record the sound of people talking before the incident happened just like Dr. Abe Davis is trying to do here. The show honor it's name for pure entertaining purposes but the concept Dr. Abe Davis is doing reminded me of the episode. Maybe he can try an Electron Microscope. lol
ikillaaaful 9 dagar sedan
i just dont understand who disklikes this
Mr V
Mr V 9 dagar sedan
Impressive, nice video!
k j
k j 9 dagar sedan
But can a lightbulb act as a microphone?
Jim Lassiter
Jim Lassiter 9 dagar sedan
Can you scan an HD pic of a vinyl record (straight down perspective) and retrieve the groove data (laser scan I suppose). Pre supposing that you have already input standard data of vinyl LPs, thickness of disk, grove depth, avg. groove width & sonic frequencies of groove widths (if possible, I'm just tossing out an idea).
hani elgabou
hani elgabou 9 dagar sedan
A Neural Network trained on the sound and image would do wonders here. Your next project.
SB45 Saransh Kumar
SB45 Saransh Kumar 10 dagar sedan
This video was basically a very clever segway to LastPass. 😂
antartis 10 dagar sedan
Yes, Is called music sheet
Ѕimon de Danѕer
Ѕimon de Danѕer 10 dagar sedan
Use a laser beam instead of a camera.
BR1 10 dagar sedan
Nice Desk Tech
Thai Jackson
Thai Jackson 11 dagar sedan
Looks like Kanye might not be crazy after all
Quacer Productions Official Channel
Quacer Productions Official Channel 11 dagar sedan
now try this with silent film
Dave MacDonald
Dave MacDonald 11 dagar sedan
I wonder what you could do with a phantom camera?
jebes909090 11 dagar sedan
the best you would get is the sound at the exact moment the image was taken. what are you going to do with 1/100000 of a second of sound? it would be more useful and possibly interesting to see if you could recover sound from silent movies.
Brian Kerr
Brian Kerr 11 dagar sedan
videos not images
Christian Humer
Christian Humer 11 dagar sedan
BF: I didn´t say that! GF: 9:55
Adnan Bashour
Adnan Bashour 11 dagar sedan
Why didnt u use perhaps a silk thread ?
John Wilson
John Wilson 12 dagar sedan
'Mary had a little lamb'. Isn't that the same expression used by Alexander Bell?
Ralf Kath
Ralf Kath 12 dagar sedan
What is with lip reading? Or an AI that interpret s a rhythm stick? Or the movement of orchestra players?
Gary Reno
Gary Reno 12 dagar sedan
They did this with a WW2 Hitler film. My take is I know they took painstakingly high measures and a large budget to produce sound. You guys did it in a living room!
Reeve Pelts
Reeve Pelts 12 dagar sedan
those sounds are just scary. i'd prob end up with nightmares if i did those things.
I'm Just Me Too
I'm Just Me Too 12 dagar sedan
Cool video/study. Only I have to say that the title is a bit misleading. I was expecting a photo.
Aaron Hahn
Aaron Hahn 12 dagar sedan
NO you cant get sound with out sound, so stop being stupid!
teddy pendergrass
teddy pendergrass 12 dagar sedan
they sell lasers that can detect vibrations and convert to sound from miles away. highly illegal but..
Troy Bird
Troy Bird 13 dagar sedan
This is similar to the magic of collecting real time behaviour or psychographics of our everyday lives. Ultrasonic signals can be used with wifi, bluetooth ( and iBeacon) with the sensors that come with the devices. But, this stuff is amazing stuff, almost like throwing your voice to be recovered at a later date or learn some information that could not be said at a specific time.
Jeramie Curtice
Jeramie Curtice 13 dagar sedan
This was in an old ted talk. Nice follow up.
Rog Fusionkid
Rog Fusionkid 13 dagar sedan
I'm not sure there is much new information in this video, except for the new higher specs of "off the shelf" cameras'. Laser Microphones have existed since at least the 90s to my knowledge, I remember discussing how it worked in 2000 when i started a new job working with a very clever person, although wikipedia says it was demonstrated and patented in 2009. It absolutely existed at least a decade earlier. Exact same principle, only difference is the readily available equipment is now of a high enough spec to achieve the same as the laser mic did 20+ years ago. Nevertheless, it would be a very cool (but questionable) bit of technology. In the future it will probably be a smartphone app.
carl gangl
carl gangl 13 dagar sedan
Movement not brightness, slo mo guys.
Scarekrow 13 dagar sedan
how long does FBI needed to get in contact with this guy ?
10p6 13 dagar sedan
I love your videos, but I call foul on this one. This is not recovering sound from an image, just the differential between images or what we call Video. Also, laser microphones have been able to pick up very clear audio for over half a century.
Andrew Levine
Andrew Levine 13 dagar sedan
This might have been said already but I didn't want to read 6000+ comments ;-) Regarding your statement at 05:28: Sound to be digitized is in fact sampled at (at least) double the highest frequency of interest to avoid aliasing. See Nyquist-Shannon sampling theorem.
Templarablity 14 dagar sedan
Apparently you can do something like this sorta eavesdropping with Lidars on a Roomba now.
Thomas Lucas
Thomas Lucas 14 dagar sedan
Going to take a picture of a da Vinci. Brush strokes from painting should act like a recording/playback stylus.
Sebastian Lacki
Sebastian Lacki 14 dagar sedan
Apparently it’s possible to “see” other people’s conversation in another building based on the overhead light bulb flicked. That would be an interesting test
Softnumnums 14 dagar sedan
Easy answer. No you can not.
JSB Production
JSB Production 14 dagar sedan
those freaking MIT nerds...
Nonconcensusical 15 dagar sedan
Ve = Neutrino electron.
thejlucena 15 dagar sedan
I’ve been thinking about this since I was 5. Images that contain sound.
Mauricio Ribeiro
Mauricio Ribeiro 15 dagar sedan
can that explain that video without sound where you see energy towers jumping rope and some people "hear" their noise hitting the ground?
magnums_ferrari 15 dagar sedan
Science hero? Bay area? Oh I thought it was going to be Ben from applied science.
Ali Fatih Yılmaz
Ali Fatih Yılmaz 15 dagar sedan
This video sponsored by CIA
Annu Ranjan
Annu Ranjan 15 dagar sedan
My mind is blown!
Reborn In Rationality
Reborn In Rationality 15 dagar sedan
As long as we can't take a photo and reconstruct a conversation I don't care. If you really want to be safe from window vibration then someone will soon come up with a glass vibrator (a device that issues vibration through glass 'overwriting' vibration from aural sources.... not the other thing, they already exist.)
Pablo Wilkano
Pablo Wilkano 16 dagar sedan
The NSA will be contacting you shortly.
Benjamin Smith
Benjamin Smith 16 dagar sedan
Certainly higher frequencies could be captured with much more computation. "Compressive sensing" uses a single pixel camera to reconstruct images by focusing on a random reflection of the image. I don't remember the specifics, but they can reproduce wavelengths with fewer samples than the Nyquist rate. This required knowing the random reflection pattern, and calculating the probability of each pixel based on what the single sensed pixel recorded. This would work with images too, but would require the processing to take into account the movements at different parts of the image based on different frequencies. I suspect it would be very challenging, probably requiring a specially designed vibrating object. Small foils for higher frequencies, etc.
CarpeDiem13x 16 dagar sedan
*This is science. The real science. 99.9999% of failure, being laughed at, being doubted and then.... you taste that sweet 0.0001% glorious moment of victory*
Ivan Toman
Ivan Toman 16 dagar sedan
No need for headphones if you have proper speakers.
Andrew Duma
Andrew Duma 16 dagar sedan
I haven't watched more stupid video, sorry
T.K. Tronix
T.K. Tronix 17 dagar sedan
Wow nice .. please read my Vinyls optical with microscope
Gruuvin1 17 dagar sedan
60 FPS allows 60 samples per second, and Nyquist sampling theorem says you're going to recover sound frequencies up to 30 Hz. Good luck with that. If the video is 30 FPS, then all you recover is frequencies too low to hear.
JonIsPatented 17 dagar sedan
I once watched a muted video of nails on a chalkboard. I promise you that you can hear a silent image.
Omar Valentini
Omar Valentini 17 dagar sedan
One of the reasons you're getting so much noise is that the signal you're sampling is not band limited, so Nyquist's theorem doesn't apply.
Alejandro Sudiro
Alejandro Sudiro 17 dagar sedan
So you can’t hear picture but you may be able to hear a moving picture.
Antonio López
Antonio López 18 dagar sedan
LSD is the answer.... Hearing photos.... Smelling light..... Looking at voices.
Atharva Pande
Atharva Pande 18 dagar sedan
Does Doppler Effect due to moving source, affects the images/ sound. coz at micro level the pic is moving creating dark n light pixels, also the source is of sound is moving, so it may change the frequency of the recorded sound.,,,, I think this play a role in recording with precision.
Lush Interpolation
Lush Interpolation 18 dagar sedan
La tuya!!
Edwin Miranda
Edwin Miranda 18 dagar sedan
I bet the government has already been using this to listen to our conversations
Steve Stelly
Steve Stelly 18 dagar sedan
This is why bags of chips were made. And why snacks are always provided in meetings. 9:56
Gary McKinnon
Gary McKinnon 18 dagar sedan
EИDERS 19 dagar sedan
If this video does no give Humanity a really eye opening view as to the power that frequencies can have on inanimate, let alone animate objects (living cells)... I duno what will.
ׁ 19 dagar sedan
ᖴIᑎᗩᒪᒪY, ᗩ ᗯᗩY TO ᕼᕮᗩᖇ IᗰᗩGᕮS TᕼᗩT ᗯOᑎ'T GIᐯᕮ YOᑌ ᑌᑎᕮ᙭ᑭᕮᑕTᕮᗪ ᖴᒪᗩSᕼᗷᗩᑕKS ᒪᗩTᕮᖇ!
Mihail Valchev
Mihail Valchev 19 dagar sedan
And now imagine training a convolutional neural network to automate the whole process and even enhance the audio quality like you can do for images and videos!
Eric Pham
Eric Pham 19 dagar sedan
In theory yes, MIT had designed potato chip bag and if you work backward yes but very technical
M Kay
M Kay 20 dagar sedan
Could this tech be used to solve crimes?
Vanessa RA
Vanessa RA 20 dagar sedan
Awesome! I've loved your channel!
Tim Thuo
Tim Thuo 21 dag sedan
Found this video from an old Onion video
Helena Of Detroit
Helena Of Detroit 21 dag sedan
The beauty of this tech is that you can get sound from webcams and camera phones without taking control of the microphone. I'm sure plenty of agencies would like to have this available, and they probably do by now.
Karla Nováková
Karla Nováková 21 dag sedan
Say us what "three letter agency" is sponsoring that research
Samuel VanCuren
Samuel VanCuren 21 dag sedan
Twit - Security Now, a news about cyber security, prob one of the best podcast or tv shows about security, reported on this when they released the paper.
Logic Peanut
Logic Peanut 22 dagar sedan
I never knew my apple headphones had so much bass
musicalfringe 22 dagar sedan
Oh my god? Stop... uptalking?
Chad Thundercock
Chad Thundercock 22 dagar sedan
>"Can you recover sound from images?"[ >proceeds to recover sound from video please change the video title, it is misleading.
mukundan nair
mukundan nair 22 dagar sedan
I didn't understand the 3 Hz and 30 fps thing. Shouldn't it be 30 Hz and 30 fps for it to show the object in the same position
Demi 22 dagar sedan
You need the help of smartereveryday and his super duper camera
Sharan Ravigopal
Sharan Ravigopal 23 dagar sedan
Is there a paricular paper that I could read that details this? This is really cool!
Luis Mendo Tomás
Luis Mendo Tomás 24 dagar sedan
Matlab FTW!
Engineering with Origami
visningar 6mn
Asteroid Impact: What Are Our Chances?
Can Silence Actually Drive You Crazy?
The Infinite Pattern That Never Repeats
Could We Prevent An Impact?
visningar 854tn
Can Humans Sense Magnetic Fields?
visningar 1,9mn
How Was Video Invented?
visningar 2,1mn
The Art of Code - Dylan Beattie
NDC Conferences
visningar 1,2mn
I Waterproofed Myself With Aerogel!
visningar 27mn
Asteroid Impact: What Are Our Chances?
Phillipe Cohen
visningar 45tn