Distorted audio when playing raw PCM data buffer in iOS

Question

I'm trying to play audio buffers using Audio Unit in iOS. The audio buffer is from a C library which receives the audio from Playstation 4 and decode it using Opus. The format of the audio buffers is PCM int16_t.

With Audio Unit and TPCircularBuffer, I am done getting it to play the sound. However it is badly distorted and not clean.

Here is the setup of my Audio Unit class

init() {
    _TPCircularBufferInit(&buffer, 960
                          , MemoryLayout<TPCircularBuffer>.size)
    setupAudio()
}

func play(data: NSMutableData) {
    TPCircularBufferProduceBytes(&buffer, data.bytes, UInt32(data.length))
}

private func setupAudio() {
    do {
        try AVAudioSession.sharedInstance().setActive(true, options: [])
    } catch { }
    
    var audioComponentDesc = AudioComponentDescription(componentType: OSType(kAudioUnitType_Output),
                                                       componentSubType: OSType(kAudioUnitSubType_RemoteIO),
                                                       componentManufacturer: OSType(kAudioUnitManufacturer_Apple),
                                                       componentFlags: 0, componentFlagsMask: 0)
    let inputComponent = AudioComponentFindNext(nil, &audioComponentDesc)
    status = AudioComponentInstanceNew(inputComponent!, &audioUnit)
    
    if status != noErr {
        print("Audio Component Instance New Error \(status.debugDescription)")
    }
    
    var audioDescription = AudioStreamBasicDescription()
    audioDescription.mSampleRate = 48000
    audioDescription.mFormatID = kAudioFormatLinearPCM
    audioDescription.mFormatFlags = kAudioFormatFlagIsPacked | kAudioFormatFlagIsSignedInteger
    audioDescription.mChannelsPerFrame = 2
    audioDescription.mFramesPerPacket = 1
    audioDescription.mBitsPerChannel = 16
    audioDescription.mBytesPerFrame = (audioDescription.mBitsPerChannel / 8) * audioDescription.mChannelsPerFrame
    audioDescription.mBytesPerPacket = audioDescription.mBytesPerFrame * audioDescription.mFramesPerPacket
    audioDescription.mReserved = 0
    status = AudioUnitSetProperty(audioUnit, kAudioUnitProperty_StreamFormat,
                                  kAudioUnitScope_Input, 0, &audioDescription,
                                  UInt32(MemoryLayout<AudioStreamBasicDescription>.size))
    
    if status != noErr {
        print("Enable IO for playback error \(status.debugDescription)")
    }
    
    var flag: UInt32 = 0
    status = AudioUnitSetProperty(audioUnit, kAudioOutputUnitProperty_EnableIO,
                                  kAudioUnitScope_Input, 1, &flag,
                                  UInt32(MemoryLayout.size(ofValue: flag)))
    
    if status != noErr {
        print("Enable IO for playback error \(status.debugDescription)")
    }
    
    var outputCallbackStruct = AURenderCallbackStruct()
    outputCallbackStruct.inputProc = performPlayback
    outputCallbackStruct.inputProcRefCon = UnsafeMutableRawPointer(Unmanaged.passUnretained(self).toOpaque())
    status = AudioUnitSetProperty(audioUnit, kAudioUnitProperty_SetRenderCallback,
                                  kAudioUnitScope_Global, 0, &outputCallbackStruct,
                                  UInt32(MemoryLayout<AURenderCallbackStruct>.size))
    status = AudioUnitInitialize(audioUnit)
    if status != noErr {
        print("Failed to initialize audio unit \(status!)")
    }
    status = AudioOutputUnitStart(audioUnit)
    if status != noErr {
        print("Failed to initialize output unit \(status!)")
    }
}

The playback function

private func performPlayback(clientData: UnsafeMutableRawPointer,_ ioActionFlags: UnsafeMutablePointer<AudioUnitRenderActionFlags>, inTimeStamp: UnsafePointer<AudioTimeStamp>,
                         inBufNumber: UInt32, inNumberFrames: UInt32, ioData: UnsafeMutablePointer<AudioBufferList>?) -> OSStatus {
let player = Unmanaged<AudioPlayer>.fromOpaque(UnsafeRawPointer(clientData)!).takeUnretainedValue()
let buffer = ioData![0].mBuffers
let bytesToCopy = ioData![0].mBuffers.mDataByteSize
var bufferTail: UnsafeMutableRawPointer?
var availableBytes: UInt32 = 0
bufferTail = TPCircularBufferTail(&player.buffer, &availableBytes)
let bytesToWrite = min(bytesToCopy, availableBytes)
memcpy(buffer.mData, bufferTail, Int(bytesToWrite))
TPCircularBufferConsume(&player.buffer, bytesToWrite)
return noErr}

Here is where I call the play function

private func StreamAudioFrameCallback(buffers: UnsafeMutablePointer<Int16>?,
                                  samplesCount: Int, user: UnsafeMutableRawPointer?) {
let decodedData = NSMutableData()
if let buffer = buffers, samplesCount > 0 {
    let decodedDataSize = samplesCount * MemoryLayout<opus_int16>.size
    decodedData.append(buffer, length: decodedDataSize)
    AudioPlayer.shared.play(data: decodedData)
}

Is anyone familiar with this ? Any helps is appreciated.

I would confirm the input audio is good by playing it using another tool or dumping it into a file then copying that file over to your computer to play it using a known good tool — Scott Stensland
@ScottStensland do you know any good way to write data buffer (from pcm) to a file ? — gi_ba
a WAV file consists of two parts a header followed by a payload ... a 44 byte header where attributes like bit depth and sample rate are stored ... a payload is the raw audio in PCM format ... so just use a library to write to an output file in WAV format ... for your purposes do not get too weighted down trying to correctly specify all the particulars of populating the 44 byte header ... its lenient as long as you correctly store bit depth and sample rate — Scott Stensland
@ScottStensland Thank you so much for your response. I got it right. In my decoded data size, I set it to int16 size * samples count which is not correct. I need to use int32 size instead. Now I can play the audio quite well, however there are still some little crackling noises, any suggestion on that ? Im still playing the audio directly using the raw PCM data buffer, no WAV header added. — gi_ba

hotpaw2 hotpaw2 · Accepted Answer · 2021-05-01T19:54:47

You might not want to start playing until there's more in the circular buffer than the amount needed to cover the maximum time jitter in the incoming data rate. Try waiting until there's half a second of audio in the circular buffer before starting to play it. Then experiment with the amount of padding.

Distorted audio when playing raw PCM data buffer in iOS

1 Answers