Efficient image to matrix conversion

Question

I am a beginner in c++ (mainly worked with Python) and I do not yet know how to properly do things. I want to process some color images as signals over time and, in order to do that, I want them to be in a double matrix.

A grayscale image would be 1d vector, from top left corner to bottom right, the color image would be a 2d vector, the second dimension being the 3 colors. That is, I want to flatten the image to a long vector, which would contain size 3 vectors with the rgb information.

I open the image using dlib like so:

#include <dlib/gui_widgets.h>
#include <dlib/image_io.h>
#include <dlib/image_transforms.h>

using namespace dlib;

array2d<rgb_pixel> img;
load_image(img, image_name);

Which gives me a dlib array2d containing pixel structs. Now, I want to change that to a flattened image. I figured that, since the images dimensions might change, I would use a

std::vector<std::vector<double>>

as my matrix.

The naive way to convert it would be the following:

#include <vector>
#include <dlib/gui_widgets.h>
#include <dlib/image_io.h>
#include <dlib/image_transforms.h>


std::vector<std::vector<double>> image_to_frame(array2d<rgb_pixel> const &image)
{
    const int total_num_of_px = image.nc() * image.nr();
    std::vector<std::vector<double>> frame = std::vector<std::vector<double>>(total_num_of_px);
    for (int i = 0; i < image.nr(); i++) 
    {
        for (int j = 0; j < image.nc(); j++)
        {
            frame[(i+1)*j] = std::vector<double>(3);
            frame[(i + 1)*j][0] = (double)image[i][j].red;
            frame[(i + 1)*j][1] = (double)image[i][j].green;
            frame[(i + 1)*j][2] = (double)image[i][j].blue;
        }
    }
    return frame;
}

But this takes 8 seconds for an 1280x720 image. Which seems to me to be a bit long. Is there a better way to do this? A more efficient way of converting the array2d to vector matrix?

Or is there a more efficient data structure than the vector matrix? Or should I not be using dlib and open the image in another way to be easier to convert?

In Python I can open the image directly as a numpy array then do a reshape, which is very fast. Is there some equivalent to this in c++ that I am not aware of?

What is a "flattened image"? What do you want to achieve using a vector what you cannot achieve with array2d<>? — Jodocus
Well, not "flattened image" in image processing sense, just, I don't know how to call it. I want to have image input as a signal over time, think of it like seeing sound, and the colors are equivalent to the channels (right channel and left channel). It is just something I am working on. Then... I do not know why I chose vector<vector>> for this instead of creating another array2d. Maybe because It seemed more familiar. Actually, I will try to reuse array2d to create the desired matrix to see if it works better. — Mihai Andrei

Leontyev Georgiy Leontyev Georgiy · Accepted Answer · 2017-10-26T08:37:52

From API it looks like that image inside dlib is stored exactly like it is done in OpenCV (dlib::toMat converts it by reusing the same memory). It means that you can take a pointer to the first element of array2d, then reinterpret_cast it to the pointer to the struct { uchar r, uchar g, uchar b } (or whatever you would like), its length will be nc*nr. Here you can copy the whole buffer using memcpy.

But I don't really get why you would need it because lines are stored continuosly, so you should not expect any cache misses.

UPDATE: also, cmon, half of the time your program is wasting by converting uchars to doubles. You shouldn't save RGB using double. There are unsigned chars by default.

UPDATE2:

struct rgb
{
    uchar r, g, b;
};

rgb* data = reinterpret_cast<rgb*>(&frame[0][0]);
std::vector<rgb> vect;
std::copy(data, data + nc * nr * sizeof(rgb), std::back_inserter(vect));

After that, you have flattened vector of the image that is stored directly in one piece of memory. If you don't need a copy, you can simply use your data pointer.

Also, if you want index-like access, you can use uchar[3] instead of rgb struct.

Efficient image to matrix conversion

1 Answers