Twilio Video + Text Chat with Angular

Question

I'm working with a doctor consultation application using Twilio Video Chat.

Following things are working in the application:

Node server that returns token
Android application that acts as client
Angular web application for doctors

Audio and video are working fine but I want to exchange text messages between the applications so that I can show connection or disconnection notification.

Here is my Angular code to make a connection:

/**
 * @description Connect to a room
 * @param accessToken 
 * @param options 
 */
connectToRoom(accessToken: string, options): void {
    connect(accessToken, options).then(room => {
        this.roomObj = room;

        if (!this.previewing && options['video']) {
            this.initializeLocalConnection();
        }

        this.roomParticipants = room.participants;

        room.participants.forEach(participant => {
            this.attachParticipantTracks(participant);
        });

        room.on('participantDisconnected', (participant) => {
            this.participantDisconnected(participant);
        });

        room.on('participantConnected', (participant) => {
            this.initializeRemoteConnection(room, participant);
        });

        // When a Participant adds a Track, attach it to the DOM.
        room.on('trackPublished', (track, participant) => {
            this.attachTracks([track]);
        });

        // When a Participant removes a Track, detach it from the DOM.
        room.on('trackRemoved', (track, participant) => {
            this.detachTracks([track]);
        });

        room.once('disconnected', room => {
            this.disconnectRoom(room);
        });
    }, (error) => {
        alert(error.message);
    });
}

And I'm calling this function with this code:

this.dataTrack = new LocalDataTrack();

this.connectToRoom(this.access_token, {
            name: this.room_name,
            //tracks: [this.dataTrack],
            audio: true,
            video: { height: 720, frameRate: 24, width: 1280 },
            bandwidthProfile: {
                video: {
                    mode: 'collaboration',
                    renderDimensions: {
                        high: { height: 1080, width: 1980 },
                        standard: { height: 720, width: 1280 },
                        low: { height: 176, width: 144 }
                    }
                }
            },
        });

I read that I need to use data track for this. To receive messages, I added following event:

participant.on('trackAdded', track => {
  console.log(`Participant "${participant.identity}" added ${track.kind} Track ${track.sid}`);
  if (track.kind === 'data') {
    track.on('message', data => {
      console.log(data);
    });
  }
});

But if I try to remove following comment from the code, audio and video stops working. There are no errors in the code.

//tracks: [this.dataTrack],

philnash philnash · Accepted Answer · 2021-04-19T07:25:28

Twilio developer evangelist here.

When you add the line tracks: [this.dataTrack] you are telling Twilio Video that those are the only tracks you want to include and that overrides the SDK asking for camera and microphone permission.

There are two things you could do here. Either ask for the video and audio tracks yourself, using navigator.mediaDevices.getUserMedia and pass the tracks in the array too.

Or, you could wait until the room has connected and then publish the data track.

connectToRoom(accessToken: string, options): void {
    connect(accessToken, options).then(room => {
        this.roomObj = room;

        this.roomObj.publishTrack(this.dataTrack);

        // etc

    })
}

Twilio Video + Text Chat with Angular

1 Answers