Which is the best way to create a tensor of frames to classify videos with Keras CNN-LSTM

Question

everyone, i'm facing the following problem.

I have a CNN-LSTM Keras Model to video classification. I'm trying to create a tensor to store frames i've got with OpenCV, as you can see in this snippet of code:

for i in list1:
    #Video Path
    vid = str(path + i) #path to each video from list1 = os.listdir(path)
    #Reading the Video
    cap = cv2.VideoCapture(vid)
    #To Store Frames
    frames = []
        for j in range(15): #i want 15 frames from each video
        ret, frame = cap.read()
        if ret == True:
            print('Class 1 - Success! {0}'.format(count))
        frame = cv2.cvtColor(frame, cv2.COLOR_BGR2GRAY)
        frame = cv2.resize(frame,(28,28),interpolation=cv2.INTER_AREA)
        frames.append(frame)
X_data.append(frames)

But the problem is that i repeat this code for two classes of videos, and the shape of my final X_data is (2, 15, 28, 28). Should not that have more than 2 samples, once i have 145 videos in each folder?

My idea is to add another column to this X_data with the targets, 1 for the Class 1 of videos, and 0 for the Class 2 of videos, but with this shape i don't know what i have to do. :/

It is the best way to store 15 frames of each video in a tensor in order to use it to train a classifier (CNN-LSTM)?

Someone help me, please!

Thanks for the attention!

tRosenflanz tRosenflanz · Accepted Answer · 2018-07-26T16:15:20

    frames = []  <# You reset frames on each video
        loop to construct frames
X_data.append(frames) <#You add the frames into X_data but frames only
                        has frames from last video in list1

You might want to move the X_data into the loop or something along those lines

Something along these lines will work:

for class,video_list in enumerate([negative_videos, positive_videos]):
    for i in list1:
        #Video Path
        vid = str(path + i) #path to each video from list1 = os.listdir(path)
        #Reading the Video
        cap = cv2.VideoCapture(vid)
        #To Store Frames
        frames = []
        for j in range(15): #i want 15 frames from each video
            ret, frame = cap.read()
            if ret == True:
                print('Class 1 - Success! {0}'.format(count))
            frame = cv2.cvtColor(frame, cv2.COLOR_BGR2GRAY)
            frame = cv2.resize(frame,(28,28),interpolation=cv2.INTER_AREA)
            frames.append(frame)
        X_data.append(frames)
        y_data.append(class)

Which is the best way to create a tensor of frames to classify videos with Keras CNN-LSTM

1 Answers