MIT researchers have developed a technique called Interactive Dynamic Video (IDV), which allows you to interact with the objects that appear in a video.
Simply explained, they create models from vibration patterns by analyzing the clips to capture the physical movement of objects. In this way, thanks to the work done by the algorithms, they can predict how certain objects will react to certain stimuli.
This will allow users to touch and interact with objects, and objects to react as they will in the real world. Although the simulation that is achieved is not perfect, the information collected in different videos could yield precise data on the effects of an activity or circumstance on the object.
This could have many applications, whether related to the world of entertainment or in different disciplines. An example given in the MIT post is that engineers will be able to use IDV to simulate how a bridge can react to a natural disaster. Or it could be used to simplify the process and cut costs for special effects in movies.
We can see the dynamics proposed by IDV in this video and how fascinating it can be: