Notes from Art&&Code : Calibrating Projectors and Cameras: Practical Tools

Photo by Kyle McDonald

Overview

The Kinect device inputs a realtime 3D scan of a world scene.

A projector outputs a realtime 2D projection onto a 3D world scene.

Using OpenCV’s CalibrateCamera function, we are able to calculate the intrinsics (focal length, lens offset) and extrinsics (projector position, rotation) of a projector relative the 3D scan of the Kinect.

We project a 3D virtual world scene onto a 3D real world scene by presuming that they are geometrically consistent (thanks to the Kinect) and knowing the intrinsics and extrinsics of the projector.

We can think of this as either:

  • Calibrating a virtual camera inside the computer against the scanned 3D scene, such that the virtual camera exactly aligns with the real projector or
  • Calibrating a real projector in the real world against a real 3D scene using the Kinect to take measurements

Demo

YouTube Preview Image

Walkthrough

YouTube Preview Image

VVVV Patches

Patches and plugins are available at http://www.kimchiandchips.com/files/workshops/artandcode/ArtAndCode-CalibrateProjector.zip

(the old link went to github, but there seems to be some bugs with their download system at the moment https://github.com/elliotwoods/artandcode.Camera-and-projector-calibration/downloads)

Inside is a plugin which wraps EmguCV and OpenNI (you’ll need to have a recent version of OpenNI installed).

Also there are 2 patches:

  1. CalibrateCamera
  2. CalibrateProjector (WARNING : Renderer will open fullscreen on 'second' screen to right of main screen e.g. projector)

Wiki

Workshop notes are available here

Github

openFrameworks code here (will be adding / amending / breaking / creating in that repo. You might want to checkout the artandcode-end tag).

8 Responses to “Notes from Art&&Code : Calibrating Projectors and Cameras: Practical Tools”

  1. olympus digital camera Says:

    olympus digital camera…

    […]Kimchi and Chips' blog » Blog Archive » Notes from Art&&Code : Calibrating Projectors and Cameras: Practical Tools[…]…

  2. anthony Says:

    how do i get the vvvv calibrateprojector to work? on a vanilla install of windows 7, with latest versions of openni and vvvv.

  3. anthony Says:

    nevermind, works with vvvv26 not vvvv27

  4. zeos Says:

    Hi Elliot,

    I was just checking out the great stuff from http://www.kimchiandchips.com/blog/?p=725
    and I decided to try the patch, but I got this output:

    Element: VVVV.Nodes.EmguCV.IntrinsicsSplitNode (ContractName=”VVVV.PluginInterfaces.V1.IPluginBase”) –> VVVV.Nodes.EmguCV.IntrinsicsSplitNode –> TypeCatalog (Types=’VVVV.Nodes.EmguCV.IntrinsicsSplitNode’).

    at System.ComponentModel.Composition.Hosting.CompositionServices.GetExportedValueFromComposedPart(ImportEngine engine, ComposablePart part, ExportDefinition definition)
    at System.ComponentModel.Composition.Hosting.CatalogExportProvider.GetExportedValue(CatalogPart part, ExportDefinition export, Boolean isSharedPart)
    at System.ComponentModel.Composition.Hosting.CatalogExportProvider.CatalogExport.GetExportedValueCore()
    at System.ComponentModel.Composition.Primitives.Export.get_Value()
    at System.ComponentModel.Composition.ExportServices.GetCastedExportedValue[T](Export export)
    at System.ComponentModel.Composition.ExportServices.GetExportLifetimeContextFromExport[T](Export export)
    at System.ComponentModel.Composition.ExportServices.c__DisplayClass1c`1.b__1b()
    at System.ComponentModel.Composition.ExportFactory`1.CreateExport()
    at VVVV.Hosting.Factories.DotNetPluginFactory.CreatePlugin(INodeInfo nodeInfo, IPluginHost2 pluginHost)
    at VVVV.Hosting.Factories.DotNetPluginFactory.CreateNode(INodeInfo nodeInfo, IInternalPluginHost pluginHost)
    at VVVV.Hosting.Factories.AbstractFileFactory`1.Create(INodeInfo nodeInfo, INode host)
    at VVVV.Hosting.HDEHost.CreateNode(INode node)
    00:00:52 ERR : System.ComponentModel.Composition.CompositionException: The composition produced a single composition error. The root cause is provided below. Review the CompositionException.Errors property for more detailed information.

    1) Cannot create an instance of type ‘VVVV.Nodes.EmguCV.ExtrinsicsSplitNode’ because a constructor could not be selected for construction. Ensure that the type either has a default constructor, or a single constructor marked with the ‘System.ComponentModel.Composition.ImportingConstructorAttribute’.

    Resulting in: Cannot activate part ‘VVVV.Nodes.EmguCV.ExtrinsicsSplitNode’.
    Element: VVVV.Nodes.EmguCV.ExtrinsicsSplitNode –> VVVV.Nodes.EmguCV.ExtrinsicsSplitNode –> TypeCatalog (Types=’VVVV.Nodes.EmguCV.ExtrinsicsSplitNode’).

    Resulting in: Cannot get export ‘VVVV.Nodes.EmguCV.ExtrinsicsSplitNode (ContractName=”VVVV.PluginInterfaces.V1.IPluginBase”)’ from part ‘VVVV.Nodes.EmguCV.ExtrinsicsSplitNode’.
    Element: VVVV.Nodes.EmguCV.ExtrinsicsSplitNode (ContractName=”VVVV.PluginInterfaces.V1.IPluginBase”) –> VVVV.Nodes.EmguCV.ExtrinsicsSplitNode –> TypeCatalog (Types=’VVVV.Nodes.EmguCV.ExtrinsicsSplitNode’).

    Any ideas?
    all the best and thank you!

  5. elliot Says:

    hey!
    it looks like the dll’s have somehow been referenced incorrectly(?)
    can you try and hit me on skype?

  6. New link for resources | Music VJ Says:

    […] http://www.kimchiandchips.com/blog/?p=725 Share this:TwitterFacebookLike this:LikeBe the first to like this post. This entry was posted in Uncategorized by nikolatasic. Bookmark the permalink. […]

  7. Angelos Says:

    Hi Elliot,

    First of all deep thanks for posting this and everything else, you ‘re a true hero!..
    I tried to run it on latest vvvv (beta27.2) but it doesn’t run, so I suppose it only works with beta26.
    Also, Kinect works fine but when I toggle calibrate nothing appears on the Projector Out renderer. Am I doing something wrong? I tried to tweak the GenerateChessboard but with no luck. Since I am a vvvv rookie, is there something you would suggest that might fix this?

    All the best and thanks once more for your great contribution..

  8. elliot Says:

    i’m remaking this patch for multi-projector at
    https://github.com/elliotwoods/VVVV.Nodes.Mapping.Database
    i’m hoping to package this with a tutorial in the coming days.
    it’s in testing now and has some external dependencies that need wrapping up

Leave a Reply