Life Of Navin

In Part 1, we set up our Android Studio workspace and project to use the OpenCV4Android SDK. But what if some of my OpenCV code is not in Java but in C/C++? What we need to do is ensure that the C/C++ code is also loaded as part of your APK, and is linked correctly. We already set up the NDK in Part 1, so now we can use it to accomplish the task at hand. This is how you can do that:

1. Open up app -> build.gradle and add the following just before buildtypes
   

   	sourceSets.main {
   	jni.srcDirs = [] //disable automatic ndk-build call
   	jniLibs.srcDir 'src/main/jni'
   	}
   	task ndkBuild(type: Exec, description: 'Compile JNI source via NDK') {
   	commandLine "<path to your ndk>/ndk-build",
   	'NDK_PROJECT_PATH=build/intermediates/ndk',
   	'NDK_LIBS_OUT=src/main/jniLibs',
   	'APP_BUILD_SCRIPT=src/main/jni/Android.mk',
   	'NDK_APPLICATION_MK=src/main/jni/Application.mk'
   	}
   	tasks.withType(JavaCompile) {
   	compileTask -> compileTask.dependsOn ndkBuild
   	}

   view raw build.gradle hosted with ❤ by GitHub
   
   
• Basically this overwrites the default ndkBuild task and instead asks it to build from src/main/jni . Also, it defines a build script (Android.mk) and a NDK app Makefile (Application.mk) and outputs stuff into src/main/jniLibs. Remember jniLibs? That's where we had previously added the architecture specific opencv_java3.so files. Turns out that .so files written to that directory are included in the final build for a given architecture. So what we'll do is build our code into .so files and add them into the jniLibs directory for the given architecture
2. Create a directory jni inside app -> src - > main. This location should contain our native C++ source code. Go ahead and create 3 files. A C++ file here named native-lib.cpp, and 2 files named Application.mk and Android.mk respectively.
   
   
   
3. Add the following content into Android.mk, Application.mk and native-lib.cpp respectively.

   	LOCAL_PATH := $(call my-dir)
   	include $(CLEAR_VARS)
   	OPENCV_CAMERA_MODULES:=on
   	OPENCV_INSTALL_MODULES:=on
   	OPENCV_LIB_TYPE:=SHARED
   	include <PATH to opencv for android SDK>/sdk/native/jni/OpenCV.mk
   	LOCAL_MODULE := native-lib
   	LOCAL_SRC_FILES := native-lib.cpp
   	LOCAL_LDLIBS += -llog -ldl
   	include $(BUILD_SHARED_LIBRARY)

   view raw Android.mk hosted with ❤ by GitHub

   	APP_STL := gnustl_static
   	APP_CPPFLAGS := -frtti -fexceptions
   	APP_ABI := armeabi-v7a
   	APP_PLATFORM := android-24

   view raw Application.mk hosted with ❤ by GitHub

   	#include <jni.h>
   	#include <sstream>
   	#include <string>
   	#include <opencv2/imgproc/imgproc.hpp>
   	#include <opencv2/highgui/highgui.hpp>
   	#include <opencv2/core/core.hpp>
   	#include <opencv2/opencv.hpp>
   	#include <android/log.h>
   	#define LOG_TAG "Native_Lib_JNI"
   	#define LOGD(...) ((void)__android_log_print(ANDROID_LOG_DEBUG, LOG_TAG, __VA_ARGS__))
   	int const max_lowThreshold = 100;
   	int ratio = 3;
   	int kernel_size = 3;
   	int lowThreshold = 10;
   	extern "C" {
   	JNIEXPORT jboolean JNICALL
   	Java_navin_tuts_opencv_opencvapp_MainActivity_nativeCanny(JNIEnv *env, jobject instance, long iAddr) {
   	cv::Mat* blur = (cv::Mat*) iAddr;
   	cv::Canny(*blur, *blur, 80, 100, 3);
   	return true;
   	}
   	}

   view raw native-lib.cpp hosted with ❤ by GitHub
• You'll have to provide the path to the OpenCV4Android SDK on line 7 of Android.mk
• Also, in Application.mk , we mention which architecture we are building for (in my case, armeabi-v7a). You can change it as per your architecture. 
• The code in native-lib.cpp  simply takes an OpenCV Mat object (referenced by it's address) and runs Canny Edge detection on it, and saves the result back in the same address.
• If you want to know why the esoteric naming of methods in native-lib.cpp, it's because JNI needs to be given methods in that format. Read this to know more about this.
4. Okay, so we have native C++ code, and a way to build it. Finally, we just need to load this native code through Android and call it's methods. We first load the native library by loading the library statically. This can be done by using System.loadLibrary. Our static block will now look like this
   

   	static {
   	System.loadLibrary("native-lib");
   	if(!OpenCVLoader.initDebug()){
   	Log.d(TAG, "OpenCV not loaded");
   	} else {
   	Log.d(TAG, "OpenCV loaded");
   	}
   	}

   view raw MainActivity.java hosted with ❤ by GitHub
   
   
5. We do this by first defining the native method in MainActivity.java just before the closing parenthesis.
   

   public static native boolean nativeCanny(long iAddr);

   view raw MainActivity.java hosted with ❤ by GitHub
• Note that the name of the method in Java and C++ are strongly correlated. C++ names are basically named as Java_packagename_classname_methodname. Make sure you are following this convention, otherwise things WILL break.
6. Finally, all we have to do is make the call to the native method we've defined. Replace the onCameraFrame method we have with the following:
   

   	@Override
   	public Mat onCameraFrame(CameraBridgeViewBase.CvCameraViewFrame inputFrame) {
   	// This is where the magic happens. When we get a frame from the camera, we do the following:
   	// First, we convert it to grayscale and save it in mGray
   	mGray = inputFrame.gray();
   	// Return we pass mGray to a native method to do Canny Edge Detection on it
   	// The result is copied back to mGray itself
   	nativeCanny(mGray.getNativeObjAddr());
   	// Finally we retun mGray to the display
   	return mGray;
   	}

   view raw MainActivity.java hosted with ❤ by GitHub
   
   
• As can be seen, this code reads an image from the camera, converts it to greyscale, passes it to the native C++ code (explained above) and then returns the result back to the display. 
7. Now simply run MainActivity. If the build goes correctly, you should see libnative-lib.so is added to the jniLibs directory under the architecture you're building for, and when the app is pushed to your device, it shows you realtime edge detection.

   libnative-lib.so added to jniLibs

   Running the app. The thumbs up says it all!

That's it! So now we have Android + OpenCV4Android SDK + native C/C++ code working together to perform realtime computer vision tasks. In fact, there are ways in which you can further improve performance of JNI, but let's talk about that some other time.

codex pulchra est