Optical-flow for real-world vision-based interfaces

Johansson [1] filmed people in darkness wearing small light bulbs on their body. Moving cloud of identical bright dots on a dark field was enough to detect people in a scene, and even assess their activity, age and sex. Inspired by [1] we select a number of patches from an image. Then we estimate where these patches moved in the next image of a video sequence. This movement is called optical-flow. This elementary computer vision technique allows the basic user action of moving a virtual object using their body movements in a simple and natural way.

In applications developed by the Vivid Group (also Intel for Intel Play Me2Cam camera) image is segmented into background and the foreground object. This requires initialization. The Sony Eye Toy games don’t require initialization and use just simple difference between two successive images to detect the motion of the users. We calculate optical flow and get also the direction and speed of the movement. This simple extension allows much richer interaction.

1. Johansson, G., “Visual perception of biological motion and a model for its analysis”, Perception and Psychophysics, no 14, pages 201-211, 1973.