How to properly take a screenshot, globally?

Question

Background

Since Android API 21, it's possible for apps to take screenshots globally and record the screen.

The problem

I've made a sample code out of all I've found on the Internet, but it has a few issues:

1. It's quite slow. Maybe it's possible to avoid it, at least for multiple screenshots, by avoiding the notification of it being removed till really not needed.
2. It has black margins on left and right, meaning something might be wrong with the calculations :

What I've tried

MainActivity.java

public class MainActivity extends AppCompatActivity {
    private static final int REQUEST_ID = 1;

    @Override
    protected void onCreate(Bundle savedInstanceState) {
        super.onCreate(savedInstanceState);
        setContentView(R.layout.activity_main);
        findViewById(R.id.checkIfPossibleToRecordButton).setOnClickListener(new OnClickListener() {
            @Override
            public void onClick(final View v) {
                ScreenshotManager.INSTANCE.requestScreenshotPermission(MainActivity.this, REQUEST_ID);
            }
        });
        findViewById(R.id.takeScreenshotButton).setOnClickListener(new OnClickListener() {
            @Override
            public void onClick(final View v) {
                ScreenshotManager.INSTANCE.takeScreenshot(MainActivity.this);
            }
        });
    }

    @Override
    protected void onActivityResult(int requestCode, int resultCode, Intent data) {
        if (requestCode == REQUEST_ID)
            ScreenshotManager.INSTANCE.onActivityResult(resultCode, data);
    }
}

layout/activity_main.xml

<LinearLayout
    android:id="@+id/rootView"
    xmlns:android="http://schemas.android.com/apk/res/android"
    android:layout_width="match_parent"
    android:layout_height="match_parent"
    android:gravity="center"
    android:orientation="vertical">


    <Button
        android:id="@+id/checkIfPossibleToRecordButton"
        android:layout_width="wrap_content"
        android:layout_height="wrap_content"
        android:text="request if possible"/>

    <Button
        android:id="@+id/takeScreenshotButton"
        android:layout_width="wrap_content"
        android:layout_height="wrap_content"
        android:text="take screenshot"/>

</LinearLayout>

ScreenshotManager

public class ScreenshotManager {
    private static final String SCREENCAP_NAME = "screencap";
    private static final int VIRTUAL_DISPLAY_FLAGS = DisplayManager.VIRTUAL_DISPLAY_FLAG_OWN_CONTENT_ONLY | DisplayManager.VIRTUAL_DISPLAY_FLAG_PUBLIC;
    public static final ScreenshotManager INSTANCE = new ScreenshotManager();
    private Intent mIntent;

    private ScreenshotManager() {
    }

    public void requestScreenshotPermission(@NonNull Activity activity, int requestId) {
        MediaProjectionManager mediaProjectionManager = (MediaProjectionManager) activity.getSystemService(Context.MEDIA_PROJECTION_SERVICE);
        activity.startActivityForResult(mediaProjectionManager.createScreenCaptureIntent(), requestId);
    }


    public void onActivityResult(int resultCode, Intent data) {
        if (resultCode == Activity.RESULT_OK && data != null)
            mIntent = data;
        else mIntent = null;
    }

    @UiThread
    public boolean takeScreenshot(@NonNull Context context) {
        if (mIntent == null)
            return false;
        final MediaProjectionManager mediaProjectionManager = (MediaProjectionManager) context.getSystemService(Context.MEDIA_PROJECTION_SERVICE);
        final MediaProjection mediaProjection = mediaProjectionManager.getMediaProjection(Activity.RESULT_OK, mIntent);
        if (mediaProjection == null)
            return false;
        final int density = context.getResources().getDisplayMetrics().densityDpi;
        final Display display = ((WindowManager) context.getSystemService(Context.WINDOW_SERVICE)).getDefaultDisplay();
        final Point size = new Point();
        display.getSize(size);
        final int width = size.x, height = size.y;

        // start capture reader
        final ImageReader imageReader = ImageReader.newInstance(width, height, PixelFormat.RGBA_8888, 1);
        final VirtualDisplay virtualDisplay = mediaProjection.createVirtualDisplay(SCREENCAP_NAME, width, height, density, VIRTUAL_DISPLAY_FLAGS, imageReader.getSurface(), null, null);
        imageReader.setOnImageAvailableListener(new OnImageAvailableListener() {
            @Override
            public void onImageAvailable(final ImageReader reader) {
                Log.d("AppLog", "onImageAvailable");
                mediaProjection.stop();
                new AsyncTask<Void, Void, Bitmap>() {
                    @Override
                    protected Bitmap doInBackground(final Void... params) {
                        Image image = null;
                        Bitmap bitmap = null;
                        try {
                            image = reader.acquireLatestImage();
                            if (image != null) {
                                Plane[] planes = image.getPlanes();
                                ByteBuffer buffer = planes[0].getBuffer();
                                int pixelStride = planes[0].getPixelStride(), rowStride = planes[0].getRowStride(), rowPadding = rowStride - pixelStride * width;
                                bitmap = Bitmap.createBitmap(width + rowPadding / pixelStride, height, Config.ARGB_8888);
                                bitmap.copyPixelsFromBuffer(buffer);
                                return bitmap;
                            }
                        } catch (Exception e) {
                            if (bitmap != null)
                                bitmap.recycle();
                            e.printStackTrace();
                        }
                        if (image != null)
                            image.close();
                        reader.close();
                        return null;
                    }

                    @Override
                    protected void onPostExecute(final Bitmap bitmap) {
                        super.onPostExecute(bitmap);
                        Log.d("AppLog", "Got bitmap?" + (bitmap != null));
                    }
                }.execute();
            }
        }, null);
        mediaProjection.registerCallback(new Callback() {
            @Override
            public void onStop() {
                super.onStop();
                if (virtualDisplay != null)
                    virtualDisplay.release();
                imageReader.setOnImageAvailableListener(null, null);
                mediaProjection.unregisterCallback(this);
            }
        }, null);
        return true;
    }
}

The questions

Well it's about the problems:

1. Why is it so slow? Is there a way to improve it?
2. How can I avoid, between taking screenshots, the removal of the notification of them? When can I remove the notification? Does the notification mean it constantly takes screenshots?
3. Why does the output bitmap (currently I don't do anything with it, because it's still POC) have black margins in it? What's wrong with the code in this matter?

NOTE: I don't want to take a screenshot only of the current app. I want to know how to use it globally, for all apps, which is possible officially only by using this API, as far as I know.

---

EDIT: I've noticed that on CommonsWare website (here), it is said that the output bitmap is larger for some reason, but as opposed to what I've noticed (black margin in beginning AND end), it says it's supposed to be in the end:

For inexplicable reasons, it will be a bit larger, with excess unused pixels on each row on the end.

I've tried what was offered there, but it crashes with the exception "java.lang.RuntimeException: Buffer not large enough for pixels" .

Answer 1

Why does the output bitmap (currently I don't do anything with it, because it's still POC) have black margins in it? What's wrong with the code in this matter?

You have black margins around your screenshot because you are not using realSize of the window you're in. To solve this:

1. Get the real size of the window:

    final Point windowSize = new Point();
    WindowManager windowManager = (WindowManager) context.getSystemService(Context.WINDOW_SERVICE);
    windowManager.getDefaultDisplay().getRealSize(windowSize);

2. Use that to create your image reader:

    imageReader = ImageReader.newInstance(windowSize.x, windowSize.y, PixelFormat.RGBA_8888, MAX_IMAGES);

3. This third step may not be required but I have seen otherwise in my app's production code (which runs on a variety of android devices out there). When you acquire an image for ImageReader and create a bitmap out of it. Crop that bitmap using the window size using below code.

    // fix the extra width from Image
    Bitmap croppedBitmap;
    try {
        croppedBitmap = Bitmap.createBitmap(bitmap, 0, 0, windowSize.x, windowSize.y);
    } catch (OutOfMemoryError e) {
        Timber.d(e, "Out of memory when cropping bitmap of screen size");
        croppedBitmap = bitmap;
    }
    if (croppedBitmap != bitmap) {
        bitmap.recycle();
    }

I don't want to take a screenshot only of the current app. I want to know how to use it globally, for all apps, which is possible officially only by using this API, as far as I know.

To capture screen/take screenshot you need an object of MediaProjection. To create such object, you need pair of resultCode (int) and Intent. You already know how these objects are acquired and cache those in your ScreenshotManager class.

Coming back to taking screenshots of any app, you need to follow the same procedure of getting these variables resultCode and Intent but instead of caching it locally in your class variables, start a background service and pass these variables to the same like any other normal parameters. Take a look at how Telecine does it here. When this background service is started it can provide a trigger (a notification button) to the user which when clicked, will perform the same operations of capturing screen/taking screenshot as you are doing in your ScreenshotManager class.

Why is it so slow? Is there a way to improve it?

How much slow is it to your expectations? My use case for Media projection API is to take a screenshot and present it to the user for editing. For me the speed is decent enough. One thing I feel worth mentioning is that the ImageReader class can accept a Handler to a thread in setOnImageAvailableListener. If you provide a handler there, onImageAvailable callback will be triggered on the handler thread instead of the one that created the ImageReader. This will help you in NOT creating a AsyncTask (and starting it) when an image is available instead the callback itself will happen on a background thread. This is how I create my ImageReader:

    private void createImageReader() {
            startBackgroundThread();
            imageReader = ImageReader.newInstance(windowSize.x, windowSize.y, PixelFormat.RGBA_8888, MAX_IMAGES);
            ImageHandler imageHandler = new ImageHandler(context, domainModel, windowSize, this, notificationManager, analytics);
            imageReader.setOnImageAvailableListener(imageHandler, backgroundHandler);
        }

    private void startBackgroundThread() {
            backgroundThread = new HandlerThread(NAME_VIRTUAL_DISPLAY);
            backgroundThread.start();
            backgroundHandler = new Handler(backgroundThread.getLooper());
        }