Capturing images with DirectX

The library that comes with OpenCV is rather limited. It supports only a few models of webcams/cameras. This can be inhibiting if you’re trying to work with some high quality camera.

DirectX is a much bigger thing (3D math + 3D audio + 3D graphics + input from joysticks/etc + loads of other things). One of the libraries in DirectX lets you access imaging hardware. And, DirectX supports a HUGE number of cameras. Rather, a huge number of cameras support DirectX.

And we’ll be using that to get images into our OpenCV programs.

Gathering dependencies

We won’t go into the details about how to use DirectX. That in itself would be another set of posts. So, we’ll use a different library, called VideoInput and use that to get the camera data.

You’ll need to download quite a bit of stuff for this to work.

First, you need the VideoInput library itself. So, go download the videoinput library and extract the files.

Now because this library uses DirectX, you need some stuff. Here’s what all you need

  1. Microsoft Platform SDK for Windows Server 2003 R2
  2. The DirectX SDK (obviously :P )
  3. The DirectShow SDK

As I write this post, I have these files downloaded:

  1. 5.2.3790.2075.51.PlatformSDK_Svr2003R2_rtm.img (409 MB)
  2. dxsdk_feb2005.exe (154MB)
  3. dxsdk_feb2005_extras.exe (35.6 MB)
  4. videoInput0.1995.zip (33.7 MB)

Getting things running

  1. Mount the .img file for the PlatformSDK (using something like PowerISO)
  2. Install the platform SDK
  3. Install the DirectX SDK
  4. The extracted files from the DirectX SDK extras go into the folder where you installed the DirectX SDK (just cut paste the extras folder)
  5. Extract the VideoInput archive into some folder

With all this done, we’re ready to start putting things together. So start Microsoft Visual Studio 2008 (or whatever IDE you use) and follow along.

Open up the Options dialog box:


Next, open up the directories tab:

See the “Show directories for” drop down on the top right? For each of those we’ll add some files.

Executable Files:

C:\Program Files\Microsoft Platform SDK for Windows Server 2003 R2\Bin
C:\Program Files\Microsoft DirectX 9.0 SDK (February 2005)\Utilities\Bin\x86

Include Files:

C:\videoInput0.1995\videoInput0.1995\compiledLib\compliedByVs2008\include
C:\Program Files\Microsoft Platform SDK for Windows Server 2003 R2\Include
C:\Program Files\Microsoft Platform SDK for Windows Server 2003 R2\Include\mfl
C:\Program Files\Microsoft Platform SDK for Windows Server 2003 R2\Include\atl
C:\Program Files\Microsoft DirectX 9.0 SDK (February 2005)\Include
C:\Program Files\Microsoft DirectX 9.0 SDK (February 2005)\Extras\DirectShow\Include
C:\Program Files\Microsoft DirectX 9.0 SDK (February 2005)\Extras\DirectShow\Samples\C++\DirectShow\BaseClasses

Library Files:

C:\videoInput0.1995\videoInput0.1995\compiledLib\compiledByVS2008
C:\Program Files\Microsoft Platform SDK for Windows Server 2003 R2\Lib
C:\Program Files\Microsoft Platform SDK for Windows Server 2003 R2\Samples\Multimedia\DirectShow\BaseClasses\Debug_Unicode
C:\Program Files\Microsoft DirectX 9.0 SDK (February 2005)\Lib\x86
C:\Program Files\Microsoft DirectX 9.0 SDK (February 2005)\Extras\DirectShow\Lib\x86
C:\Program Files\Microsoft DirectX 9.0 SDK (February 2005)\Extras\DirectShow\Samples\C++\DirectShow\BaseClasses\Debug_Unicode

With these settings done, we should be able to grab frames from the camera through DirectX.

Building Libraries

The DirectShow SDK is just source code. It comes with pre-built stuff. That would increase filesize. So the good people of Microsoft shifted the burden of building their libraries onto us.

Go here:

C:\Program Files\Microsoft DirectX 9.0 SDK (February 2005)\Extras\DirectShow\Samples\C++\DirectShow\BaseClasses\

You should see a solution file. Open that in Visual Studio. We need to build this solution to get all the files we need. Try building it as it is.

You would probably get some errors. If you get a linker error C2146 for winnt.h or error C4430 in ctlutil.h the problem here is that this member operator function does not have a return type:

operator=(LONG);

Change it to:

COARefTime& operator=(LONG);

If you want, you can read more about it over here.

Next, we need to make sure that it becomes a static library. To do that, Go to: Project Settings -> Librarian -> General and set Additional Dependencies to “dxguid.lib ole32.lib  strmiids.lib uuid.lib”, without the quotes. And also set Link Library Dependencies to Yes.

If you still get some errors, they’re probably about variables not being declared. Maybe a variable named Count, a static variable that needs a type (set its type to int), and iDone. These errors occur mostly because of for loops. So declaring them outside the for loop should be enough.

Once you’ve successfully built the solution (maybe with lots of warnings… but they don’t matter)… we’re done.

We now have all files required to capture from DirectX (using the DirectX library).

A sample project

With everything done, I’ll show a little demo of how to get frames from the videoinput library.

Create a new Win32 Project. Name it whatever you want, and accept the default settings.

In the main file of the project, we write the following code:

#include "stdafx.h"
#include "videoInput.h"
#include "cv.h"
#include "highgui.h"
 
int main()
{
    videoInput VI;
    int numDevices = VI.listDevices();
    int device1= 0;
    VI.setupDevice(device1);
    int width = VI.getWidth(device1);
    int height = VI.getHeight(device1);
    IplImage* image= cvCreateImage(cvSize(width, height), 8, 3);
    unsigned char* yourBuffer = new unsigned char[VI.getSize(device1)];
    cvNamedWindow("test");
    while(1)
    {
        VI.getPixels(device1, yourBuffer, false, false);
        image->imageData = (char*)yourBuffer;
        cvConvertImage(image, image, CV_CVTIMG_FLIP);
        cvShowImage("test", image);
        if(cvWaitKey(15)==27) break;
    }
 
    VI.stopDevice(device1);
    cvDestroyWindow("test");
    cvReleaseImage(&image);
 
    return 0;
}

Go to Project Properties -> Confuguration Properties -> Linker -> Input and add videoinput.lib along with the usual OpenCV library files.

Then try compiling this code. It should work perfectly and you should see a live video from the first camera the program detects.

If you get errors about some library names atlthunk.lib, you have two options:

  • Go to Project Settings -> Linker -> Command Line and Add /NODEFAULTLIB:”atlthunk”

Or

  • A little hack that works perfectly! Take a random .lib file, and rename it to atlthunk.lib. Place it in one of the Library directories we had added above.

Conclusion

With this, you’ve got access to all those cameras that you bought, but packed them up because OpenCV didn’t “support” them. They should all work now. Enjoy!

Oh, and do leave feedback… and if you face problems, do let me know!

Issues? Suggestions? Visit the Github issue tracker for AI Shack

Back to top

Summary

  • Several webcams don't work directly with OpenCV.
  • Most webcams do work with DirectX. They're made compatible with DirectX.
  • Using the videoInput library, we can use DirectX to fetch images.

46 Comments

  1. Posted July 9, 2010 at 12:50 pm | Permalink

    Hi Utkarsh, good article :-) But are you sure that you need to install the Platform SDK and DirectX SDK just to use videoInput Library? I thought it would work without those SDK’s since you just need DirectX on your computer, which comes with Windows anyway.

    Cheers,
    Shervin Emami.

    • Posted July 9, 2010 at 1:00 pm | Permalink

      Hi Shervin! videoInput uses the MFC and ATL libraries. They don’t come with the 2008 sdk, so you specifically need the 2003 SDK.

      And the SDKs are for compiling the library. If you already have the library compiled, you shouldn’t have a problem without them! :)

  2. Luca
    Posted November 2, 2010 at 10:39 pm | Permalink

    Hi, i get an error:
    LNK1104: cannot open file ‘atlthunk.lib’
    i use visual studio 2005…

    • Posted November 3, 2010 at 4:53 am | Permalink

      Just take a random file, rename it to atlthunk.lib and place it in your libraries folder. This file isn’t actually used.

  3. Nagakumar
    Posted November 24, 2010 at 8:43 pm | Permalink

    Could you tell me where to paste the extracted Video Input Library folders ????? My question might be silly, but am just starting to learn all these. Thanks in advance :) By the way, really good article man :) Just rocks :) I just can’t wait to get it working …………………. :)

    • Posted December 9, 2010 at 12:05 pm | Permalink

      Put it anywhere, it doesn’t matter. Just make sure you set appropriate paths in Visual Studio (or your IDE).

  4. Nagakumar
    Posted November 25, 2010 at 7:20 pm | Permalink

    Hey……….. I got over my problem mentioned in the last post. Now i got the same erroe Luca posted.

    LNK1104: cannot open file ‘atlthunk.lib’

    I tried renaming and pasting the file. I think am doing it wrongly. Could you be more specific as to where I should paste the file atlthunk.lib ???????????? Thanks in advance :)

    • Paritosh Parmar
      Posted May 28, 2011 at 12:03 pm | Permalink

      go to project properties > configuration properties > linker > input > …. and atlthunk.lib to ignore specific library option.

  5. Posted December 9, 2010 at 12:20 pm | Permalink

    Hi! Put the file in, say C:\videoInput0.1995\videoInput0.1995\compiledLib\compiledByVS2008. That way, when Visual Studio is looking for atlthunk.lib, it’ll go through this folder. And it will find it there too. And the program will compile! :)

  6. Supervan
    Posted January 18, 2011 at 2:47 pm | Permalink

    Hi,

    My software configuration
    ——————————–
    Windows 7 Pro 64bit
    Visual Studio 2010
    SDK for windows 7 and .net framework 4
    DirectX SDK (August 2009)
    OpenCV2.2

    Need to find out what to Include Directories,Executable Directories etc.
    I tried the following.

    Executable Directories
    C:\Program Files (x86)\Microsoft SDKs\Windows\v7.0A\Bin
    C:\Program Files (x86)\Microsoft DirectX SDK (August 2009)\Utilities\bin\x86
    Include Directories
    C:\Program Files (x86)\Microsoft SDKs\Windows\v7.0A\Include
    C:\Program Files (x86)\Microsoft SDKs\Windows\v7.0A\Include\gl
    C:\Program Files (x86)\Microsoft DirectX SDK (August 2009)\Include
    Library Directories
    C:\Program Files (x86)\Microsoft SDKs\Windows\v7.0A\Lib
    C:\Program Files (x86)\Microsoft DirectX SDK (August 2009)\Lib\x86
    C:\Program Files (x86)\Microsoft DirectX SDK (August 2009)\Extras\DirectSound\Debug\x86

    Thanks

  7. omeecd
    Posted February 7, 2011 at 10:03 am | Permalink

    Dear Utkarsh,
    Thanks for the nice post. However, I followed each and every one of your instruction but still stuck with the webcam :(
    I have a Toshiba Laptop with Microsoft Vista and I am using openCV 2.2 in Microsoft Visual Studio 2010 express edition.
    When I debug the code you posted here i get the following results in the console:
    ********* VIDEOINPUT LIBRARY -0.1995 – TFW07*************
    VIDEOINPUT SPY MODE!
    SETUP: LOOKING FOR CAPTURE DEVICE
    SETUP: 0) USB 2.0 CAMERA
    SETUP: 1 DEVICE(S) FOUND

    SETUP: SETTING UP DEVICE 0
    SETUP: USB 2.0 CAMERA
    SETUP: COULDN’T FIND PREVIEW PIN USING SMARTTREE
    SETUP: CAPTURE CALLBACK SET

    Then the program is terminated. I could not find out what the problem is.
    Please help me with it. Thanks.

    • Posted February 8, 2011 at 1:39 pm | Permalink

      Your webcam does get detected, but it appears that there’s some error while setting it up. Did you try connecting an external webcam?

  8. Helen
    Posted February 13, 2011 at 10:39 pm | Permalink

    My program stops running in “vi->setupDevice(0)” line…
    and doesn’t return any error…
    Could it be happened cause I’m using windows 7????
    I used the same program in WindowsXP, and I have no problem in it.
    I’m using a CCTV analog camera with a frame grabber conector.
    Help me please…

    • Posted February 14, 2011 at 9:17 pm | Permalink

      It could be an issue with the driver. But, I’m not sure.

  9. Sivadev
    Posted March 9, 2011 at 10:18 am | Permalink

    Hello Utkarsh,

    Nice article, helped me a lot. One question, I have two camera, modified the program and got the output as below

    ***** VIDEOINPUT LIBRARY – 0.1995 – TFW07 *****
    VIDEOINPUT SPY MODE!
    SETUP: Looking For Capture Devices
    SETUP: 0) USB2.0 ATV
    SETUP: 1) A4 TECH USB 2.0 Camera J
    SETUP: 2) UScreenCapture
    SETUP: 3) Basler GenICam Source
    SETUP: 4 Device(s) found

    SETUP: Setting up device 0
    SETUP: USB2.0 ATV
    SETUP: Couldn’t find preview pin using SmartTee
    SETUP: Checking crossbar
    SETUP: You are a webcam or snazzy firewire cam! No Crossbar needed
    SETUP: Default Format is set to 640 by 480
    SETUP: trying format RGB24 @ 320 by 240
    SETUP: trying format RGB32 @ 320 by 240
    SETUP: trying format RGB555 @ 320 by 240
    SETUP: trying format RGB565 @ 320 by 240
    SETUP: trying format YUY2 @ 320 by 240
    SETUP: Capture callback set
    SETUP: Device is setup and ready to capture.

    SETUP: Setting up device 1
    SETUP: A4 TECH USB 2.0 Camera J
    SETUP: Couldn’t find preview pin using SmartTee
    SETUP: Checking crossbar
    SETUP: You are a webcam or snazzy firewire cam! No Crossbar needed
    SETUP: Default Format is set to 640 by 480
    SETUP: trying format RGB24 @ 640 by 480
    SETUP: trying format RGB32 @ 640 by 480
    SETUP: trying format RGB555 @ 640 by 480
    SETUP: trying format RGB565 @ 640 by 480
    SETUP: trying format YUY2 @ 640 by 480
    SETUP: Capture callback set
    SETUP: Device is setup and ready to capture.

    My problem is with the device 1 – A4 TECH USB 2.0 Camera J, I got the image / video but with the device 0 – USB2.0 ATV, I just got a light gray display, no image/video. Really need some help here.

    • Posted March 11, 2011 at 4:07 pm | Permalink

      Hmm. Weird. Are you sure the code is correct?

  10. Eugene
    Posted March 10, 2011 at 8:19 pm | Permalink

    Hey sorry for the spam. Everything works perfectly fine. Awesome code!

    But one question, how i do get images from the webcam? all i see is the webcam is on, i see myself. so does it save every images it capture? or do i have to press a button or something? and where do they save the images to??

    Thanks

    • Posted March 11, 2011 at 4:06 pm | Permalink

      It doesn’t save images. It just captures them and stores them in the IplImage* variable (RAM). If you want, you can save the images to a file. Try cvSaveImage.

  11. Eugene
    Posted March 11, 2011 at 6:53 pm | Permalink

    Hey thanks for the reply.

    So basically I have to put in the cvsaveimage function yea. But exactly which line should I put it at? I am sorry I am a real noob in this. So after I put the command, the program will save images automatically or do I still have to press a button to activate it? And will it save all the pictures from the start of the video or at certain interval ?

    Sorry for asking so many questions.
    Thanks

    • Posted March 11, 2011 at 9:22 pm | Permalink

      image->imageData = (char*)yourBuffer;

      After that. No need to press some button. Files will be saved automatically. Files will be saved whenever the function gets called.

  12. gebs
    Posted March 15, 2011 at 3:00 am | Permalink

    hey I am running it in visual c++ 2008 express edition…i did everything you mentioned…but i still get the following error while building…can you please help me out?

    >C:\Program Files (x86)\Microsoft DirectX 9.0 SDK (February 2005)\DirectShow\Samples\C++\DirectShow\BaseClasses\streams.h(179): fatal error C1083: Cannot open include file: ‘atlbase.h’: No such file or directory

    • gebs
      Posted March 15, 2011 at 10:58 am | Permalink

      hey …so i figures out the prev error..bt i still cant figure these two errors?

      wxdebug.cpp(11): warning C4603: ‘_WINDLL’ : macro is not defined or definition is different after precompiled header use
      1> Add macro to precompiled header instead of defining here
      1> wxdebug.cpp(13) : use of precompiled header
      1>wxdebug.cpp(564): error C4430: missing type specifier – int assumed. Note: C++ does not support default-int

      can you plz help me out?

      • gebs
        Posted March 15, 2011 at 11:00 am | Permalink

        hey …so i figures out the prev error..bt i still cant figure these two errors?
        also am running it on visual c++ 2010 express edition…
        i typed wrongly in the prev query
        wxdebug.cpp(11): warning C4603: ‘_WINDLL’ : macro is not defined or definition is different after precompiled header use
        1> Add macro to precompiled header instead of defining here
        1> wxdebug.cpp(13) : use of precompiled header
        1>wxdebug.cpp(564): error C4430: missing type specifier – int assumed. Note: C++ does not support default-int

        can you plz help me out?

        • gebs
          Posted March 15, 2011 at 12:13 pm | Permalink

          Hey….sorry about all my previous comments! i made it work..and it works like a charm!! thanks for the amazing post!!

  13. Sujin Lee
    Posted April 5, 2011 at 12:50 pm | Permalink

    Dear Utkarsh

    Is it helpful using i-sight camera by bootcamp window7 in Macbook pro?
    Cause I could use this camera using OpenGL Api but if I use OpenCV Api, I couldn’t get real time video from this camera(i-sight, webcam…)
    I have no idea…..Could you consider these problems?

    Thanks~ ^^

    Sujin

    • Posted April 29, 2011 at 9:39 pm | Permalink

      Well, you could grab frames with OpenGL and then convert the image to OpenCV’s format – IplImage.

  14. Stanley007
    Posted April 14, 2011 at 8:52 pm | Permalink

    It’s great!!…I can use it fo my project…thank fo give an example to combine VI with OpenCV…But, actualy u no need to dload all the SDK mentioned above…jus dload VideoInput…Newer version 01995 has been compiled fo some IDE (CW, DevCPP, VS2005, and VS2008)…n my project in VS2010, using VI precompiled by VS2008…It’s works!!!…Thanksss….

  15. Posted April 28, 2011 at 10:35 pm | Permalink

    I’ve tried your code, it work great. I’m a very newbie in robo vision, i’m getting start . Can you help me a little?
    - How to get the reference manual of videoinput library (like Opencv).
    - Now i’m trying to load two views from two webcam (i use Logitech C200) to support robot detect obstacle to avoid, can you give me some idea (about algorithm, …). By using depth map is good for it?

    Sorry for not related question in this topic :(

    • Posted April 29, 2011 at 9:05 pm | Permalink

      1) There isn’t much documentation about videoInput.
      2) Yes i guess depth map would be a good way to do it.

  16. lee
    Posted May 12, 2011 at 2:25 pm | Permalink

    i’m sorry but however do you mount the .img file for the PlatformSDK? is there any guide on this?

    • Posted June 3, 2011 at 12:12 am | Permalink

      Download some software like PowerISO or DaemonTools Lite. They’ll create virtual CD drives. You can mount the image on those drives.

  17. Alabastern
    Posted May 14, 2011 at 7:56 am | Permalink

    Hi Utkarsh thanks for the articule everything works just perfect :D ,
    I’m trying to use this to capture images from an IP camera its a Panasonic BL-C20A I can actually see the stream in a web browser with this url http://192.168.1.253/nphMotionJpeg?Resolution=320×240&Quality=Standard but I cant make it work with OpenCV and now I have no clue about how to use the videoInput library to do that, any ideas?

    Thank you very much in advance.

  18. Adrienne
    Posted June 8, 2011 at 5:01 am | Permalink

    My software configuration
    ——————————–
    Windows 7 Pro 64bit
    Visual Studio 2008
    SDK for windows 7 and .net framework 4
    DirectX SDK (June 2010)
    OpenCV2.1

    I successfully compiled OpenCV 2.1 in 64-bit version in Debug mode.
    I successfully compiled the DirectShow baseclass solution in VS2008 as you said.

    I am stuck however. I opened a new Win32 project and copy and pasted you code into but I have 9 LNK2019 unresolved external symbol errors.

    Example: error LNK2019: unresolved external symbol "public: __cdecl videoInput::~videoInput(void)" (??1videoInput@@QEAA@XZ) referenced in function main CamOpen.cpp.obj

    I do not know what this means and I have been stuck on this problem for a while. Can you explain to me what this means and lead me towards what I can do to fix these errors.

    Thanks you are a life-saver!

    • Posted June 17, 2011 at 4:35 pm | Permalink

      Looks like you didn’t link to the videoInput.lib file.

  19. MRHB
    Posted June 18, 2011 at 4:20 pm | Permalink

    hello
    It woude be great if i can use it.coud you put here compiled library?these are hiuge file for download!

    • Posted June 18, 2011 at 6:11 pm | Permalink

      The download package already has compiled files. And it’s just a 35mb file! Shouldn’t take more than an hour even on a slow connection :P

  20. shdagger
    Posted July 3, 2011 at 8:00 pm | Permalink

    Thanks a lot. It’s working marvelously and helped me a lot in my project.
    Thanks again!

  21. Sodrohu
    Posted July 6, 2011 at 8:55 pm | Permalink

    Hey man, first of all, thanks for this nice tutorial.

    I got some problems I wanna share:

    1. I downloaded the Platform SDK and DirectX SDK 2005. The part where you need to link to Samples\Multimedia\DirectShow\BaseClasses\Debug_Unicode is missing in both SDKs. I can’t compile the baseclasses solution cuz it needsthe debug_unicode folder which I don’t have. Can you provide a link to a download for both Debug_Unicodes?

    2. User Stanley007 up there said that you only need VideoInput 01995 to make everything work. So,having linked everything(I didn’t compile baseclasses solution, just linked libs, includes and all), I ran the code. Got errors:

    Error 1 error C2733: second C linkage of overloaded function ‘_interlockedbittestandset’ not allowed c:\program files\microsoft visual studio 10.0\vc\include\intrin.h 1056 1 Test Program
    Error 2 error C2733: second C linkage of overloaded function ‘_interlockedbittestandreset’ not allowed c:\program files\microsoft visual studio 10.0\vc\include\intrin.h 1057 1 Test Program

    I’m using VS2010 Ultimate. Any Ides why is this happening?

  22. mrhb
    Posted July 12, 2011 at 11:25 am | Permalink

    helo,thanks alot for your guidance.is ther such good method fo IP cameras that work with Opencv well?

    • Posted July 13, 2011 at 9:12 am | Permalink

      I don’t think so. You’d have to learn about your IP camera’s interface. You could use standard HTTP/etc libraries to make it work somehow.

  23. Dave Taylor
    Posted July 31, 2011 at 2:45 am | Permalink

    Thanks again! I have to agree with one of your commenters though. I only needed the pre-compiled videoInput libraries and not the SDKs.

    Using Win 7, VS2010, OpenCV2.2, and videoInput 1995.

  24. Ranga
    Posted August 22, 2011 at 9:55 am | Permalink

    Hi Utkarsh,
    Is it possible to make 2 USB cameras to capture video from same PC using directX? If so, are the video captured from both the cameras are synchronized?

    • Posted September 2, 2011 at 12:23 pm | Permalink

      Yup – it’s possible. I’ve done it. What do you mean synced?

  25. Calcifer
    Posted August 28, 2011 at 12:46 pm | Permalink

    Great ariticle, Utkarsh.May I ask you a question? If the output data of a pixel of my camera is more than 24bit, whether I can get the data which has the same bits by your code?

One Trackback

  • By Week 5 | yonghaoteng on October 22, 2012 at 8:54 am

    [...] Early this week, I continued to work on my camera wrapper before integrating it into our project. There are some major changes in the project as we are using CvCapture in HighGui for getting IplImages which VideoInput has to do some conversion as shown by - http://www.aishack.in/2010/03/capturing-images-with-directx/. [...]

Post a Comment

Your email is never published nor shared. Required fields are marked *

*
*

You may use these HTML tags and attributes <a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code> <del datetime=""> <em> <i> <q cite=""> <strike> <strong>