Record and play simultaneously on iOS (Phonegap build)

5
votes

I'm developing iOS and Android application using Phonegap Build version 3.3.0.

Main focus of application is audio recording with another audio / music playing in the background.

For both instances i'm using phonegap media api with correct audio files for recording (iOs - *.wav / Android - *.amr ) and playing (iOs and Android - *.mp3).

Example:

var audioRec = new Media(audioRecSrc, onSuccess, onError);
audioRec.startRecord();

var audioPlay = new Media(audioPlaySrc, onSuccess, onError);
audioPlay.play();

Example works on Android without any problem. Sound is recorded and music played normally. But in iOS only one is possible. Whichever is called last, play or record. The other one returns error with code - 4.

Is this limitation of Phonegap Media API on iOs or am I missing something?

2
javascriptiosaudiorecordphonegap-build
I believe this is a iOS limitation in itself. I don't know for sure as I'm not a huge apple dev but even trying to record in garageband with music playing on iPad will force the music to stop. Same thing occurs with skype sessions on the iPad. – Kyle
I don't believe that this is iOS limitation, because I played native (non-phonegap) music game (StarComposer - URL: itunes.apple.com/si/app/starcomposer/id670221315?mt=8) on iOS where this same thing is possible. Ofcourse using headphones, but nonetheless. – matjazu
Hmm maybe the answer does rely with the headphones. I remember reading somewhere that the reason iOS would do only 1 audio source at a time is so that the background sound wouldn't be recorded. Therefore implying it playing out loud. Try your app with headphones and see if it makes a difference – Kyle
I did that before posting question here, but did not make a difference. :/ – matjazu
Any news on this? Did you manage to solve it? – cherouvim

2 Answers

3
votes

In order to be able to simultaneously play media and record audio, one must set the category property of the AVAudioSession to AVAudioSessionCategoryPlayAndRecord. To do so you must deploy a custom iOS plugin that sets the corresponding value.

At the time speaking, this process is not quite straightforward because of a bug in the Cordova Media Plugin. Before starting to record, the plugin sets the category of the AVAudioSession to AVAudioSessionCategoryRecord indiscriminately. Because of that, after starting to record, playing back the desired media becomes impossible, unless you explicitly set the AVAudioSession category to play-and-record, after calling the record method. This clearly insinuates that you can only play your media, after recording has started.

However, this workaround may not be acceptable in many scenarios, which may require that media starts to play before sound is recorded. Thus, I have filed a bug report regarding the issue, which you may find here:

iOS Media plugin: cannot play video and record audio simultaneously

Plus, I have applied the fix and performed the following pull request:

Check if avSession.category is already set to "AVAudioSessionCategoryPlayAndRecord" before recording

It's really a minor fix (literally two lines of code), thus I believe that it will be soon applied to the master branch.

0
votes

Update

Upon even further investigation, I may have been wrong. According to the Cordova Media API documentation, when recording audio in iOS, the file that you are recording to must already exist and be a .wav file. Maybe you've already tried this (in which case, my previous answer is probably correct) but here's a simple script to create the file and then begin recording once the file handle is ready:

var audioRecSrc = "myrecording.wav";
var audioPlaySrc = "audiotoplay.mp3"

// Wait for PhoneGap to load
document.addEventListener("deviceready", onDeviceReady, false);

// PhoneGap is ready
function onDeviceReady() {
    window.requestFileSystem(LocalFileSystem.PERSISTENT, 0, gotFS, fail);
}

function gotFS(fileSystem) {
    fileSystem.root.getFile(audioRecSrc, {create: true, exclusive: false}, gotFileEntry, fail);
}

function gotFileEntry(fileEntry) {
    var audioRec = new Media(audioRecSrc, onSuccess, onError);
    audioRec.startRecord();

    var audioPlay = new Media(audioPlaySrc, onSuccess, onError);
    audioPlay.play();
}

function fail(error) {
    console.log("error : " + error.code);
}

Here's a link to the Cordova File API documentation in case you need to reference it. You'll also need to make sure that you have the appropriate permissions in your app configuration to use the File and Media APIs (everything you'll need is on the pages I linked)


Previous

That is a limitation of iOS. Here's a quote from Mobile Safari documentation (PhoneGap uses the native UIWebView when it's compiled with the iPhone SDK):

Multiple Simultaneous Audio or Video Streams

Currently, all devices running iOS are limited to playback of a single audio or video stream at any time. Playing more than one video—side by side, partly overlapping, or completely overlaid—is not currently supported on iOS devices. Playing multiple simultaneous audio streams is also not supported. You can change the audio or video source dynamically, however. See “Replacing a Media Source Sequentially” (page 41) for details

It would make sense to infer that if playing two streams at the same time isn't possible, neither would be playing and recording at the same time. As for the above comment about the StarComposer app, there's a very big difference between native iPhone apps and those using PhoneGap. PhoneGap uses Cordova to allow developers to access a subset of native device functions using JavaScript, while native apps have the full range of capabilities. The limited streaming capability you're experiencing is due to the HTML5 implementation on the iOS platform. Here's the document that I sourced the above quote from, it's on page 23:

https://developer.apple.com/library/safari/documentation/AudioVideo/Conceptual/Using_HTML5_Audio_Video/Using_HTML5_Audio_Video.pdf