Life Of Navin

Random Musings, Random Bullshit.

kc

OpenCV on Android with Java SDK and JNI - Part 1

Anyone who has even the slightest understanding of computer vision has at some point worked with OpenCV. It's a super cool (and fast) general purpose CV library. At a recent hackathon we were at, our hack required getting OpenCV running on an Android device, but because the documentation for doing this is very vague, we spent a lot of time just on the setup itself. What further complicated everything was that we had some code in C++ and some in Java and needed to bring all of this together, with rewriting the code of one language into the other not an option. We succeeded in the end, but we knew that our build flow was super hacky. But we're engineers and we don't like hacky code (not even at hackathons) so I thought it would be good to document how to do it the right way. So this is how you go about running OpenCV on Android:

Step 1: The Setup

  1. Download the following
  2. On first launch of Android Studio, it'll download all the latest SDK Components (Assuming you downloaded it without the SDK), platform-tools, build-tools etc. These are the components it offered to download for me. Note the build-tools version (Mine is 24.0.2, yours may differ), but you'll need it later on.
  3. Start a new project. Uncheck the checkbox that says "include C++ support". Next.
  4. On Target Android Devices, choose Phone and Tablet with Minimum SDK Version as 15 (That's Ice Cream Sandwich!!). Next.
  5. Add an Empty Activity. Next. On the Customize the Activity screen, just click Next.
  6. Your workspace is now ready! Go to Android -> SDK Manager.  Under SDK Platforms, check Android 6.0 Marshmallow, which is API Level 23 [1]. Under SDK Tools, check NDK (My version shows 12.1.2977051). Click on apply and it'll download the components.
    Open up the SDK Manager
    Download NDK
  7. Once this is done, go to File -> Project Settings and click on Select default NDK for Android NDK location. This will populate the text field with the path to the NDK you just downloaded.
  8. Unzip the OpenCV for Android zip file into some location.

Step 2: Adding OpenCV for Android SDK

  1. Go to File -> New -> Import Module and then add <OpenCV for Android Unzipped Path>/sdk/java as a module. 
    • You will probably get an error. Let's fix that. In the left menu listing the files of the project, you can switch from Android view to Project view (using the dropdown right at the top).  Open the build.gradle file for the module you just imported (openCVLibrary310 -> build.gradle) and edit values to what's shown below. Make sure the buildToolsVersion is the same as what you noted down in step 2 of setting up. Then select Sync Now and all errors should disappear.
  2. Right click on app in the left menu, and select Open Module Settings. Select the module app, click on Dependencies, and then add the openCVLibrary310 as a module dependency onto app.
    Open module settings for app
    After adding the dependency.
  3. Create a directory names jniLibs inside app -> src -> main. In this directory we add the architecture dependent .so file for OpenCV. You'll find all the architectures supported in <OpenCV for Android Unzipped Path>/sdk/native/libs/ . Choose your architecture (usually armeabi-v7a) and copy the directory into jniLibs.  We only need the libopencv_java3.so file so go ahead and delete all the other files.
    How it should look

Step 3: Test it out!

  1. Let's test if OpenCV loads properly in our Android app. Go to MainActivity.java and add the following code segment right at the start of the class:
  2. Now just connect your phone in USB Debugging mode and click on the Run icon at the top to build and push your application to your device [2].
  3. The application builds and pushes to the device and opens up the Android Monitor. In the filter, filter by OpenCV::Main and you should see OpenCV loaded. If instead you see OpenCV not loaded, it may be because you've use used the wrong architecture library in step 3 of Adding OpenCV. 
    OpenCV loaded. Yay!

Step 4: Let's Code!

We're all set to write our OpenCV code in our Android Application. Let's write a simple application which takes camera input, converts it to greyscale, performs a Gaussian on it and returns it to the screen.
  1. Create a file named camera.xml in src -> main -> res -> layout. Add the following in it.

  2. Open up src -> main -> AndroidManifest.xml and add the following lines just before the </manifest>

    • Also change the android:theme to @style/Theme.AppCompat.DayNight.NoActionBar to ensure no bar is seen at the top of the application
  3. Go to src -> main -> java and open MainActivity.java. Replace it's contents by:

    • The OpenCV code is actually all on the onCameraFrame method. As you can see, it takes a total of 2 lines of code to do what we want!
  4. Click the run icon to rebuild and push the application to your device. It should ask for Camera Permissions, which we just added. Give it access, and you should see a realtime Grayscale Gaussian-ed view of whatever the Camera sees!
    Success!
That's the first bit. In the second part of this post, I'll show how we can write native C++ OpenCV code to run on the device using JNI  (which is what the NDK we installed initially is for). The advantages of using JNI is both speed and flexibility. I'll also show how you can mix the Android OpenCV code and JNI code if you so wish to. Until then...

 Codex vincit omnia

----

[1] I usually download a couple of earlier SDKs as well for testing as you can see (Why?). Nougat (API level 24) also should work just fine, but I use the Marshmallow (API level 23)
[2] If you're new to all of this, read this tutorial!

0 comments :

Post a Comment

Subscribe to: Post Comments ( Atom )

Prologue

Finally after all these years, here's to the beginning of what was there, what is there and hopefully what will remain!! So here are my thoughts & words -Online!!

Blog Archive