How to create a video stream when I have raw H264 frames and timestamps?

4
votes

I receive from a third party application, a stream of packests containing raw H264 frames and a timestamp. I want to take the raw frames and the timestamp and re-stream, so I can see the video in a player like VLC

I tried to just take the raw H264 frames as they arrive and stream them over a TCP socket, and I'm actually able to see the video in VLC. But of course without timestamps, the video play at maximum speed and then it stop to wait other frames, and then it starts again at max speed. I guess I need to put those raw frames in a container and mark each frame with the timestamp, but I don't know how. Any help appreceated (if in C# even better)

Thanks

1
c#video-streamingh.264video-processing

1 Answers

0
votes

(1) Option 1: Timer

Simply use a timer to send your individual H264 frames. 

timer_delay = (1000/FPS); //if FPS is 30, then sends a frame every 33 milliseconds.

A frame (NAL unit) has a start code of four bytes as 00 00 00 00 so grab all bytes from one sequence of and including the 00 00 00 00 up to the last byte before another sequence of 00 00 00 00 begins. This should* be your frame data to send.

note:
I say "should" because H264 sometimes slices the images, so if you send a frame and get only half a picture that means you must send two or three sequences of NAL units as one whole item (eg: in one bytes array) to see a full image. You'll learn through active experimenting.

(2) Option 2: Pipes
Try FFmpeg a free command line tool. Actually it's also the audio/video decoder for VLC. You can use C#'s Standard in/out to send the received bytes to FFmpeg (which as a Process is running with encoder settings eg: the output format and resolution, etc).

See this article as a starting point:
https://mathewsachin.github.io/blog/2017/07/28/ffmpeg-pipe-csharp.html