Get camera frame from Vuforia in Unity

0
votes

I want to create a simple AR application using unity3d, vuforia package and opencv. Normally in unity AR app, selected 2d target is found and virtual 3d object is projected.

I want to change these scenario.

  1. Open AR camera
  2. Get frame from camera
  3. Process the frame using opencv functions(maybe opencvsharp)
  4. Find marker and project virtual object

To do this task, I did below steps :

  1. create a new project in unity
  2. import vuforia package
  3. delete main camera
  4. add AR camera

The AR camera has 3 c# files, I open and look them. But I don't see any code open camera and get frame. The screenshot is below :

enter image description here

EDIT

Accepted answer helps load user defined marker.

1
c#unity3dcameravuforia
The Vuforia SDK is a bit closed. Much of the scripts are nothing more than wrappers to the SDK DLLs. Not a specialist of Vuforia, but you may encounter trouble to get any intermediate values. - Kamalen
@Kamalen this work is really hard for beginners like me. But there is an example that controls unity game screen(not AR screen) from an opencv window. Maybe this example helps anyone. link - zakjma
I don't know about OpenCV. If you want to process a frame from camera before Vuforia, it may not be possible. I think you have to use a different camera GameObject, separated from Vuforia. - Kamalen

1 Answers

0
votes

This is the basic workflow of Vuforia using Unity.

Start from License Manager

  • On the License Manager page go to Add License Key

  • bPut in your details and if you don't plan to use a paid version in the Select Plan option click None

Now go to Target Manager

  • First Add a Database (if you don't have one already) and give it a Name, add License Key and click Create

  • Now inside your Database click Add Target and again put in all the details and upload your image.

  • Now Download Database and make sure you've set it's usage to Unity Editor

Now within Unity

  • Add your ARCamera

  • Import the Database you downlaoded into Unity through Assets - Import Package - Custom Package

  • Now in your Inspector Panel of ARCamera, you'll see Data Set Load Behaviour has your unity package name. Check it, and check Active as well

  • Now in Assets, Go to Qualcomm - Prefabs - ImageTarget and drag Image Target onto the scene.

  • In the ImageTarget Inspector you'll find ImageTarget behaviour and you can set the values to your Image.

From here on, what you do is completely up to you. You can add a Model or animate over is as you would on normal Unity applicatoins.

User Defined Targets can be found at: https://developer.vuforia.com/library/articles/Solution/Unity-Load-DataSet-from-SD-Card

Hope this helps.