Archive for January, 2011

Kinect + Projector experiments

Wednesday, January 12th, 2011

Using Padé projection mapping to calibrate Kinect’s 3D world with a projector.

  1. Using the kinect camera, we can scan a 3D scene in realtime.
  2. Using a video projector, we can project onto a 3D scene in realtime.

Combining these, we re-project images onto geometry to create a new technique for augmented reality

Previous videos (for process)

YouTube Preview Image YouTube Preview Image YouTube Preview Image

The pipeline is:

  1. Capture Depth at CameraXY (OpenNI)
  2. Convert to image of WorldXYZ
  3. Pade transformation to create WorldXYZ map in ProjectorXY
  4. Calculate NormalXYZ  map in ProjectorXY
  5. Guassian Blur X of NormalXYZ in ProjectorXY
  6. Guassian Blur Y of NormalXYZ in ProjectorXY
  7. Light calculations on NormalXYZ, WorldXYZ maps in ProjectorXY