passing real-time metadata synchronized with video frames using webrtc and text tracks

9
votes

I'm using WebRTC (win c++ native client) to broadcast real-time video to peers (hosted in Chrome).
Goal: send metadata along each video frame (metadata changes at frame level).

  1. Would it be possible to send the metadata within a text track to be consumed by a javascript at the peer side?
  2. If not, is there an alternative way of synchronizing WebRTC real-time video with metadata?
    e.g., using WebRTC DataChannel / WebSockets?
2
metadatawebrtc
Did you get a solution for this? – JMarc
Possible duplicate of Synchronization of data with video using WebRTC – bain

2 Answers

1
votes

I guess you need this features

https://webrtc.googlesource.com/src/+/77c8e65b88c9d2d95442b66ada504e0f1c553d66

Update multiplex encoder to support having augmenting data attached to the video

0
votes

No.

At the moment, WebRTC implementation (or specification) comes with no synchronization or the ability to synchronize data with video on a frame-by-frame basis. This is something being looked at for future WebRTC specification.

There are vendors who offer such a capability in their SDKs, but this is limited to their native SDKs and doesn't work in their browser based JS SDKs.