Can you pass Camera and Microphone objects between Flash instances?

Question

I'm working on an HTML5 app which needs to interface with Flash to get access to the local media (e.g., the webcam and microphone), and to transmit the audio-video between remote browsers. But in this app, I need the local webcam display to be on one part of the screen, separated by various HTML elements from the remote webcam display. I'm pretty sure this means that I need to have multiple instances of my Flash app running. But I think that you can only grab one instance of a webcam at a time, which means that I need to be able to share those webcam and microphone objects between Flash instances: one displaying the local webcam, and the other communicating with and displaying the remote webcam. Is it possible to do that? For instance, can I pass my Camera and Microphone instances out to JavaScript through the ExternalInterface and then pass them back into a separate instance of my Flash object?

In other words, I'm thinking of having an ActionScript class that looks like this (much simplified of course):

public class MediaController
{

    public function MediaController()
    {
        ExternalInterface.addCallback('getUserMedia', this.getUserMedia);
        ExternalInterface.addCallback('getCamera', this.getCamera);
        ExternalInterface.addCallback('setCamera', this.setCamera);
        ExternalInterface.addCallback('getMicrophone', this.getMicrophone);
        ExternalInterface.addCallback('setMicrophone', this.setMicrophone);
    }

    private var _mic:Microphone;
    private var _cam:Camera;

    public function getUserMedia()
    {
      _mic = Microphone.getMicrophone();
        _cam = Camera.getCamera();
    }

    public function getCamera():Camera
    {
        return this._cam;
    }

    public function setCamera(cam:Camera):void
    {
        this._cam = cam;
    }

    public function getMicrophone():Microphone
    {
        return this._mic;
    }

    public function setMicrophone(mic:Microphone):void
    {
        this._mic = mic;
    }
}

And I'd retrieve them in JavaScript like this:

var localUser = $('#localUser')[0];
localUser.getUserMedia();
var mic = localUser.getMicrophone();
var cam = localUser.getCamera();

And then pass them back into the instance that's actually communicating with the remote user like this:

var remoteUser = $('#remoteUser')[0];
remoteUser.setMicrophone(mic);
remoteUser.setCamera(cam);

Any known pitfalls associated with doing it that way? Is there a better way to handle this? (And before you ask, yes, in the absence of advice otherwise, I'm planning to code this up, and I'll let everyone know what I find - just want to know if there are any known pitfalls or alternatives before I get started. :-)

Answer 1

You cannot pass objects like Camera and Microphone to Javascript via ExternalInterface. When you use ExternalInterface to communicate with Javascript, any data you pass is marshaled into an XML format. So at that point, the camera/mic cease to be the Flash Camera and Microphone objects.

You might find that trying to access the same camera from two separate SWF's at the same time works in some browsers/operating systems. However, in others it fails. I've seen this behavior with two completely unrelated web sites that were accessing the camera.

It is possible for SWF's to communicate with each other using the LocalConnection class, though I've never tried anything like this with the camera or microphone.

Answer 2

For what it's worth, this is the approach I ended up taking (more-or-less), and it works. Complex, and a bit brittle, but it works:

// A typical 1:1 communication will involve four instances of the FlashMediaObject class:
// Instance1 (Initiator Sender): Displays local video, streams local video out to Instance4
// Instance2 (Initiator Receiver): Receives and displays video from Instance3
// Instance3 (Responder Sender): Displays local video, streams local video out to Instance2
// Instance4 (Responder Receiver): Receives and displays video from Instance1

// The workflow needs to go something like this:
// (1) Both: Room.onSessionAdded():     
//          SignalR makes both the JS clients for both the Initiator and the Responder aware of each other (i.e., their SessionId's).
// (2) Initiator: writeLocalMediaElement() -> fmo1.Connect():
//          Instance1 connects to Adobe's rtmfp service, gets its nearId, and passes it up to JS.
// (3) Responder: writeLocalMediaElement() -> fmo3.connect():
//          Instance3 connects to Adobe's rtmfp service, gets its peerId, and passes it up to JS.
// (4) Responder: prepareForCall() -> fmo4.connect():           
//          Instance4 connects to Adobe's rtmfp service, gets its peerId, and passes it up to JS.
// (5) Initiator: call() -> prepareForCall() -> fmo2.Connect():
//          Instance2 connects to Adbobe's rtmfp service, gets its nearId, and passes it up to JS.
// (6) Initiator: call() -> server.flashOffer():
//          The Initiator's JS controller contacts the Responder's JS (via SignalR), and passes it the two rtmfp ID's.
// (7) Responder: handleFlashOffer() -> fmo3.call():            
//          The Responder's JS controller passes the peerId for Instance2 (Initiator Receiver) to Instance3 (Responder Sender).
//          Instance3 begins publishing its video to Instance 2.
// (8) Responder: handleFlashOffer() -> fmo4.prepareForCall():
//          The Responder's JS controller passes the peerId for Instance1 (Initiator Sender) to Instance4 (Responder Receiver)
//          Instance4 prepares to receive a call from Instance1.
// (10) Responder: handleFlashOffer() -> server.flashAnswer():  
//          The Responder's JS controller contacts the Initiator's JS (via SignalR), and passes it the two peer ID's.
// (11) Initiator: handleFlashAnswer() -> fmo1.call():
//          The Initiator's JS controller passes the peerId for Instance4 (Responder Receiver) to Instance1 (Initiator Sender).
//          Instance1 connects to Instance4 and begins streaming video.
// (12) Initiator: handleFlashAnswer() -> fmo2.prepareForCall()
//          The Responder's JS controller passes the peerID for Instance3 (Responder Sender) to Instance2
//          Instance2 prepares to receive video from Instance3
// (9) Initiator: fmo2.onCall():
//          Instance2 begins playing video from Instance3.
// (13) Responder: fmo4.onCall():
//          Instance4 begins playing video from Instance1