Notes from Art&&Code : Calibrating Projectors and Cameras: Practical Tools

Photo by Kyle McDonald


The Kinect device inputs a realtime 3D scan of a world scene.

A projector outputs a realtime 2D projection onto a 3D world scene.

Using OpenCV’s CalibrateCamera function, we are able to calculate the intrinsics (focal length, lens offset) and extrinsics (projector position, rotation) of a projector relative the 3D scan of the Kinect.

We project a 3D virtual world scene onto a 3D real world scene by presuming that they are geometrically consistent (thanks to the Kinect) and knowing the intrinsics and extrinsics of the projector.

We can think of this as either:

  • Calibrating a virtual camera inside the computer against the scanned 3D scene, such that the virtual camera exactly aligns with the real projector or
  • Calibrating a real projector in the real world against a real 3D scene using the Kinect to take measurements


YouTube Preview Image


YouTube Preview Image

VVVV Patches

Patches and plugins are available at

(the old link went to github, but there seems to be some bugs with their download system at the moment

Inside is a plugin which wraps EmguCV and OpenNI (you’ll need to have a recent version of OpenNI installed).

Also there are 2 patches:

  1. CalibrateCamera
  2. CalibrateProjector (WARNING : Renderer will open fullscreen on 'second' screen to right of main screen e.g. projector)


Workshop notes are available here


openFrameworks code here (will be adding / amending / breaking / creating in that repo. You might want to checkout the artandcode-end tag).

8 Responses to “Notes from Art&&Code : Calibrating Projectors and Cameras: Practical Tools”

  1. olympus digital camera Says:

    olympus digital camera…

    […]Kimchi and Chips' blog » Blog Archive » Notes from Art&&Code : Calibrating Projectors and Cameras: Practical Tools[…]…

  2. anthony Says:

    how do i get the vvvv calibrateprojector to work? on a vanilla install of windows 7, with latest versions of openni and vvvv.

  3. anthony Says:

    nevermind, works with vvvv26 not vvvv27

  4. zeos Says:

    Hi Elliot,

    I was just checking out the great stuff from
    and I decided to try the patch, but I got this output:

    Element: VVVV.Nodes.EmguCV.IntrinsicsSplitNode (ContractName=”VVVV.PluginInterfaces.V1.IPluginBase”) –> VVVV.Nodes.EmguCV.IntrinsicsSplitNode –> TypeCatalog (Types=’VVVV.Nodes.EmguCV.IntrinsicsSplitNode’).

    at System.ComponentModel.Composition.Hosting.CompositionServices.GetExportedValueFromComposedPart(ImportEngine engine, ComposablePart part, ExportDefinition definition)
    at System.ComponentModel.Composition.Hosting.CatalogExportProvider.GetExportedValue(CatalogPart part, ExportDefinition export, Boolean isSharedPart)
    at System.ComponentModel.Composition.Hosting.CatalogExportProvider.CatalogExport.GetExportedValueCore()
    at System.ComponentModel.Composition.Primitives.Export.get_Value()
    at System.ComponentModel.Composition.ExportServices.GetCastedExportedValue[T](Export export)
    at System.ComponentModel.Composition.ExportServices.GetExportLifetimeContextFromExport[T](Export export)
    at System.ComponentModel.Composition.ExportServices.c__DisplayClass1c`1.b__1b()
    at System.ComponentModel.Composition.ExportFactory`1.CreateExport()
    at VVVV.Hosting.Factories.DotNetPluginFactory.CreatePlugin(INodeInfo nodeInfo, IPluginHost2 pluginHost)
    at VVVV.Hosting.Factories.DotNetPluginFactory.CreateNode(INodeInfo nodeInfo, IInternalPluginHost pluginHost)
    at VVVV.Hosting.Factories.AbstractFileFactory`1.Create(INodeInfo nodeInfo, INode host)
    at VVVV.Hosting.HDEHost.CreateNode(INode node)
    00:00:52 ERR : System.ComponentModel.Composition.CompositionException: The composition produced a single composition error. The root cause is provided below. Review the CompositionException.Errors property for more detailed information.

    1) Cannot create an instance of type ‘VVVV.Nodes.EmguCV.ExtrinsicsSplitNode’ because a constructor could not be selected for construction. Ensure that the type either has a default constructor, or a single constructor marked with the ‘System.ComponentModel.Composition.ImportingConstructorAttribute’.

    Resulting in: Cannot activate part ‘VVVV.Nodes.EmguCV.ExtrinsicsSplitNode’.
    Element: VVVV.Nodes.EmguCV.ExtrinsicsSplitNode –> VVVV.Nodes.EmguCV.ExtrinsicsSplitNode –> TypeCatalog (Types=’VVVV.Nodes.EmguCV.ExtrinsicsSplitNode’).

    Resulting in: Cannot get export ‘VVVV.Nodes.EmguCV.ExtrinsicsSplitNode (ContractName=”VVVV.PluginInterfaces.V1.IPluginBase”)’ from part ‘VVVV.Nodes.EmguCV.ExtrinsicsSplitNode’.
    Element: VVVV.Nodes.EmguCV.ExtrinsicsSplitNode (ContractName=”VVVV.PluginInterfaces.V1.IPluginBase”) –> VVVV.Nodes.EmguCV.ExtrinsicsSplitNode –> TypeCatalog (Types=’VVVV.Nodes.EmguCV.ExtrinsicsSplitNode’).

    Any ideas?
    all the best and thank you!

  5. elliot Says:

    it looks like the dll’s have somehow been referenced incorrectly(?)
    can you try and hit me on skype?

  6. New link for resources | Music VJ Says:

    […] Share this:TwitterFacebookLike this:LikeBe the first to like this post. This entry was posted in Uncategorized by nikolatasic. Bookmark the permalink. […]

  7. Angelos Says:

    Hi Elliot,

    First of all deep thanks for posting this and everything else, you ‘re a true hero!..
    I tried to run it on latest vvvv (beta27.2) but it doesn’t run, so I suppose it only works with beta26.
    Also, Kinect works fine but when I toggle calibrate nothing appears on the Projector Out renderer. Am I doing something wrong? I tried to tweak the GenerateChessboard but with no luck. Since I am a vvvv rookie, is there something you would suggest that might fix this?

    All the best and thanks once more for your great contribution..

  8. elliot Says:

    i’m remaking this patch for multi-projector at
    i’m hoping to package this with a tutorial in the coming days.
    it’s in testing now and has some external dependencies that need wrapping up

Leave a Reply