Android Apply Effect on Camera frame

3
votes

From Android 17, it provides android.media.effect package for processing images with many effects. and with the official sample 'HelloEffect', it works for image only.

Now I want apply the effects for Android camera preview frame, I used SurfaceTexture and GL_TEXTURE_EXTERNAL_OES texture to render the frame from camera.

But the problem is how to apply the effect on this special texture: GL_TEXTURE_EXTERNAL_OES, it shows errors, it seems it cannot sample from this special texture.

error:

01-06 15:46:35.425: D/libEGL(14021): loaded /vendor/lib/egl/libEGL_adreno.so
    01-06 15:46:35.435: D/libEGL(14021): loaded /vendor/lib/egl/libGLESv1_CM_adreno.so
01-06 15:46:35.435: D/libEGL(14021): loaded /vendor/lib/egl/libGLESv2_adreno.so
01-06 15:46:35.435: I/Adreno-EGL(14021): : EGL 1.4 QUALCOMM build: _msm8974_refs/tags/AU_LINUX_ANDROID_JB_3.1.2.04.02.02.125.032_CL3849330_release_AU (CL3849330)
01-06 15:46:35.435: I/Adreno-EGL(14021): OpenGL ES Shader Compiler Version: 17.01.10.SPL
01-06 15:46:35.435: I/Adreno-EGL(14021): Build Date: 10/21/13 Mon
01-06 15:46:35.435: I/Adreno-EGL(14021): Local Branch: jb_3.1.2
01-06 15:46:35.435: I/Adreno-EGL(14021): Remote Branch: quic/4da73f3b691eea7a9356efff8d609916f1975107
01-06 15:46:35.435: I/Adreno-EGL(14021): Local Patches: NONE
01-06 15:46:35.435: I/Adreno-EGL(14021): Reconstruct Branch: NOTHING
01-06 15:46:35.565: D/OpenGLRenderer(14021): Enabling debug mode 0
01-06 15:46:36.295: W/Adreno-ES20(14021): : GL_INVALID_OPERATION
01-06 15:46:36.295: E/MCA(14021): GL Error: Operation 'Binding Texture' caused GL error (0x502)
01-06 15:46:36.295: E/MCA(14021): BindInputTextures failed
01-06 15:46:36.295: E/MCA(14021): Unable to render frame
01-06 15:46:36.295: E/MCA(14021): ShaderProgram: error processing shader!
01-06 15:46:36.305: W/dalvikvm(14021): threadid=11: thread exiting with uncaught exception (group=0x419fd898)
01-06 15:46:36.345: E/AndroidRuntime(14021): FATAL EXCEPTION: GLThread 97443
01-06 15:46:36.345: E/AndroidRuntime(14021): java.lang.RuntimeException: Error executing ShaderProgram!
01-06 15:46:36.345: E/AndroidRuntime(14021):    at android.filterfw.core.ShaderProgram.process(ShaderProgram.java:123)
01-06 15:46:36.345: E/AndroidRuntime(14021):    at android.filterfw.core.Program.process(Program.java:32)
01-06 15:46:36.345: E/AndroidRuntime(14021):    at android.filterpacks.imageproc.LomoishFilter.process(LomoishFilter.java:200)
01-06 15:46:36.345: E/AndroidRuntime(14021):    at android.filterfw.core.Filter.performProcess(Filter.java:474)
01-06 15:46:36.345: E/AndroidRuntime(14021):    at android.filterfw.core.FilterFunction.execute(FilterFunction.java:80)
01-06 15:46:36.345: E/AndroidRuntime(14021):    at android.filterfw.core.FilterFunction.executeWithArgList(FilterFunction.java:97)
01-06 15:46:36.345: E/AndroidRuntime(14021):    at android.media.effect.SingleFilterEffect.apply(SingleFilterEffect.java:76)
01-06 15:46:36.345: E/AndroidRuntime(14021):    at com.csz.camerafilters.activity.render.SurfaceRenderer.applyEffect(SurfaceRenderer.java:73)
01-06 15:46:36.345: E/AndroidRuntime(14021):    at com.csz.camerafilters.activity.render.SurfaceRenderer.onDrawFrame(SurfaceRenderer.java:63)
01-06 15:46:36.345: E/AndroidRuntime(14021):    at android.opengl.GLSurfaceView$GLThread.guardedRun(GLSurfaceView.java:1531)
01-06 15:46:36.345: E/AndroidRuntime(14021):    at android.opengl.GLSurfaceView$GLThread.run(GLSurfaceView.java:1248)
3
androidopengl-es
can you paste the errors? – Carlos Robles
@CarlosRobles Hi, thanks, plz check my log – John

3 Answers

3
votes

I was able to apply effects on camera frames using OpenCV SDK for Android. They have a ready sample showing that.

3
votes

Android effect bind texture as GL_TEXTURE2D, but your texture is GL_TEXTURE_EXTERNAL_OES, that's the reason. Gen another texture of GL_TEXTURE2D, draw camera frame on it, and use this texture all the way. Just try it, this is only a theory (base on my OpenGL knowledge).

1
votes

Yes in most devices you need to enable the GL_OES_EGL_image_external extension in the shader. Check out the pszFragEGLImageShader string in the sgxperf code at, https://github.com/prabindh/sgxperf/blob/master/sgxperf_gles20_vg.cpp

Also refer to Android OpenGL combination of SurfaceTexture (external image) and ordinary texture