Background:
I am developing an Android mobile application where users record audio in the app then upload it to a server, where it can be streamed back to the community.
Up until now we have used .WAV format which outputs a pretty large file size for uploading and streaming over mobile networks. I'm working on converting to .AAC format but I've learned about a few different problems that happen in different manufacturers.
Issues:
- All Android phones (API-15+) tested so far are able to record audio in AAC okay.
- However, Samsung devices will not directly play back AAC audio files using the MediaPlayer. Instead they need to be encoded in AAC but of the OutputFormat .MP4
- AAC audio files in .MP4 containers record perfectly and playback on all devices tested.
- Files upload fine to the server.
Then here is the main problem. When streaming the file to the audio player, the sound player fails to decode audio recorded on Samsung devices only. AAC files recorded on Motorola and Nexus phones are perfectly interchangeable with the previous WAV files. I am able to play the sound files directly on my desktop, and if I open the file directly in a web browser they will all play.
This problem happens when I stream and decode the audio using AudioContext. I believe the problem is related to missing header data from the Samsung audio codec, but I've invested a lot of time in this problem and haven't been able to find a solution that works yet. This problem seems straight forward but it's taken me a lot of work to figure out exactly what the problem is. Any help is MUCH appreciated.
Two approaches that I see are: 1. Fix the encoding problem at the source in the Android application 2. Correct for the error in the browser. I've seen posts about using a function attempt to re-sync the stream but I do not fully understand how to implement this.
I think the best solution is #1.
I downloaded two example files in the browser and looked at the raw data for comparison. (Snapshot of raw data http://i.imgur.com/tF6uHoO.png // I don't have enough reputation to embed images, but will revise later.)
Recording Code in Android RecordActiviy.java
// FILE NAME
mFileName = "file.mp4";
// INITIATE MEDIA RECORDER
mRecorder = new MediaRecorder();
mRecorder.setAudioSource(MediaRecorder.AudioSource.MIC);
mRecorder.setOutputFormat(MediaRecorder.OutputFormat.MPEG_4);
mRecorder.setOutputFile(mFileName);
mRecorder.setAudioEncoder(MediaRecorder.AudioEncoder.AAC);
mRecorder.setAudioSamplingRate(48000);
mRecorder.setAudioEncodingBitRate(64000);
try {
mRecorder.prepare();
mRecorder.start();
} catch (IOException e) {
Log.e("AUDIO RECORDING", "prepare() failed");
e.printStackTrace();
}
}
...
getSound.php
...
$File = $path . $filename;
switch($filetype){
case 'wav':
//FILE TYPE WAV
$mime_type = "audio/vnd.wave";
header("Content-type: ".$mime_type);
$test = readfile($file);
break;
case 'mp4':
//FILE TYPE MP4
//$mime_type = "audio/mp4";
$mime_type = "audio/aac";
header("Content-type: ".$mime_type);
$test = readfile($file);
break;
default:
break;
}
soundPlayer.js
...
var player;
function loadPlayer() {
player = new SoundPlayer();
..UI and other stuff..
}
// Build Sound Player Object
function SoundPlayer() {
this.arrayBuffer = null; //array buffer of audio data
this.context = new AudioContext(); //browser sound context
this.duration = null; //Length of audio clip
this.source = null; //Converted audio source
this.trueEnd = true; //Check to see if sound ended due to completion
this.isPlaying = false; //Check to see if sound is currently playing
this.t = 0; //Update variable for time counter
this.startTime = null; //Start time of audio is logged when playAudio()
this.byteData = null;
var sp = this; // sp = soundPlayer (Reference to SoundPlayer obj)
this.playAudio = function() {
this.source = null;
/* ArrayBuffer must be decoded each time play is called
* due to the nature of a decoded AudioBuffer only
* being able to be played once */
sp.context.decodeAudioData(sp.arrayBuffer, decoded);
sp.t = setInterval(updateProgress, 10);
};
this.loadSound = function(pid) {
$.ajax({
async: true,
url: "getSound.php",
data: { soundID: sID },
beforeSend: function(xhr) {
//Allow for binary data
xhr.overrideMimeType("text/plain; charset=x-user-defined");
},
success: function(dataAsString) {
dataAsString = dataAsString.trim();
if( dataAsString.localeCompare("") == 0 ){
alert("Sound File Not Found");
return;
}
// convert string to byte array
var bytes = [];
var i, l = dataAsString.length;
for (i = 0; i < l; ++i) {
bytes.push(dataAsString.charCodeAt(i) & 0xFF);
}
sp.byteData = bytes;
//Convert byte array into audio buffer
var arrayBuffer = new ArrayBuffer(bytes.length);
var bufferView = new Uint8Array(arrayBuffer);
for (var i = 0; i < bytes.length; i++) {
bufferView[i] = bytes[i];
}
sp.arrayBuffer = arrayBuffer;
//get sound duration info
sp.context.decodeAudioData(arrayBuffer, function(buffer) {
sp.duration = buffer.duration;
notify_UI_load_complete();
}, function(error){
**** // WE GET TO THIS BLOCK IN THE BROWSER
**** // RETURNS UNDEFINED
console.log("error decoding "+ arrayBuffer.length);
console.log(arrayBuffer);
console.log("error type "+ error);
});
},
error: function(jqXHR, textStatus, errorThrown) {
console.log("load error");
console.log(textStatus, errorThrown);
alert("Could not load specific sound file");
}
});
}
...
TL;DR Android Samsung AAC audio streaming error in Web Audio AudioContext. Web audio is hard. Can we just get a universal standard already.
