Audio recording delay

Question

I'm writting an application for Android where my goal is to record the user singing. After the recording, I play the record synced with the instrumental one (which is not a problem).

The problem is when I start recording at the same time as playing the audio I suspect there is a delay for the recording to start: (it is noticeable when I try to play the recording later when the recording recorded the instrumental too). The delay might be around 300ms on nexus 5.

My inquiries are:

• is the delay constant on all Android devices (4.0 +)?
• how to calculate delay without involving complex methods?
• is it acceptable on mobile device, to open the audio recording and scale it to instrumental (which may involve audio decoding/encoding)?

How to get rid of the delay?

Answer 1

is the delay constant on all Android devices (4.0 +)?

The delay is not constant, because at least the audio round-trip latency is not constant.

You can be sure that the delay is not constant because it also depends on:

• the actual audio hardware codec: some codec have a slow start. For example, look at the adc_ramp_work function in this code: it is doing a slow start on the codec.

• the way the audio is buffered in the processing chain: a buffer size of 4096 samples (the size used on most samsung phones), will obviously add more delay than a 1024 samples buffer.

how to calculate delay without involving complex methods? You can't. (without complex methods).

The audio latency reported by the AudioTrack API is only a software latency. And measuring the latency is not a simple task, as you can see in the official documentation.

how to get rid of the delay?

This is the good news here: you probably don't need to get rid of the delay, as long as you start playing once the recording is really started.

Jeffrey's solution is a great start. Note that you need to read() the data or your callbacks won't get called.

is it acceptable on mobile device, to open the audio recording and scale it to instrumental (which may involve audio decoding/encoding)?

AudioTrack.setVolume() gives you a simple way to scale your tracks. Finding the scaling factor automatically can be quite tricky: you will need to estimate the loudness of the recording, and also the loudness of your instrumental.

I suggest you use a much simpler solution: provide a slider on your UI which allows your user to adjust the mix between the instrumental and the voice recording (use AudioTrack.setVolume in the background). This way, you can be sure the instrumental/voice ratio really fits the user's taste.

Answer 2

As @Lukos already mentions, it is very likely that the delay depends on the processing speeds. It simply takes some time to start the recorder.

I believe your best solution would be to start the recorder first, and then start audio playback as soon as you know the recorder is started. This way you will have some delay in the recording, but at least you are in sync. You need to make sure the playback has been initialized, so it can be started as fast as possible.

This can be achieved using the AudioRecorder callbacks.

// Assuming you have a recorder objects defined somewhere
AudioRecord recorder;

recorder.setNotificationMarkerPosition(1);
recorder.setRecordPositionUpdateListener(new AudioRecord.OnRecordPositionUpdateListener() {
    @Override
    public void onMarkerReached(AudioRecord recorder) {
        // start audio playback
    }

    @Override
    public void onPeriodicNotification(AudioRecord recorder) {

    }
});

Note that I'm not sure about the marker position. You might be able to use 0 as well, not sure if that will give you a notification immediately after the recording started, or if it is before it is actually started. You have to do some tests there.