How Can Sound Be Recovered from Images?

TL;DR

It is possible to recover sound from images by analyzing the tiny movements of objects caused by sound vibrations. Light objects with damping, like crumpled tinfoil, perform best in this process. However, most cameras struggle due to limited framerates, which restrict the range of audible frequencies that can be captured.

Transcript

This video was filmed without sound. Is it possible to use only these images to reconstruct the sound? that is can you hear pictures? in this video I'm gonna try to demonstrate that it's possible to get sound from pictures but it's not gonna be easy so I'm gonna need some help. this episode was sponsored by LastPass which allowed me to fly to the B... Read More

Key Insights

• 👂 Sound can be extracted from images by analyzing the displacement of pixels, but the vibrations are incredibly small and require precise analysis.
• 🙂 Objects that have a lot of damping and are light, such as crumpled tinfoil or a bag of chips, work best for recovering sound.
• 🧡 The framerate of cameras limits the range of audible frequencies that can be captured.

Questions & Answers

Q: How do you extract sound from images?

To extract sound, the video analyzes the displacement of pixels by looking for changes in brightness and darkness on either side of an edge, providing an estimate of the displacement over time.

Q: What objects work best for recovering sound?

Objects that have a lot of damping and are light work best for recovering sound, such as crumpled tinfoil or a bag of chips. They need to move readily with changes in air pressure.

Q: Why does the framerate of cameras limit the range of audible frequencies?

Most cameras shoot at 30 frames per second, which is not enough to capture the full range of audible sound frequencies. To measure a frequency, at least twice the frequency needs to be sampled.

Q: How can this technology be applied in real life?

This technology can be applied to various fields such as security, where audio recordings of typing can reveal passwords, or in soundproof glass to recover human speech from outside.

Summary & Key Takeaways

• The video sets out to determine if it is possible to extract sound from images, explaining that sound vibrations are incredibly tiny and difficult to capture.

• The experiment involves recording various objects with a camera and analyzing the displacement of pixels to detect sound.

• The video concludes that while it is possible to recover sound from images, the framerate of most cameras limits the range of audible frequencies.