Audio routing from multiple BT headsets on Mac, with real-time up-sampling from the mic's frequency

1
votes

I need to accept audio streams from multiple Bluetooth headsets, that can only sample from their microphones at low frequencies of 8KHz or 16KHz, record them to wav files while also mixing them in real-time with music playing at an acceptable sample rate of 44.1KHz or 48KHz and play that back to the users through the headset phones. Ideally, this would run on a Mac with macOS 10.14 Mojave, but I also have access to Raspberry Pi 3's and a NUC running Windows or Ubuntu Linux.

Had it not been for the need to upsample the microphone input I'd probably use Jack for this. However, Jack apparently needs to have all the audio cards at the same frequency and appears to provide no facilities for resamping.

So far the closest I got to a functioning solution is using SoX. However it creates a horrendous delay of about 1 second, and I haven't been able to control it using the --buffer option. Also, there seems to be no way on Mac to specify the audio input, essentially limiting me to a single headset.

rec -V -c 1 -r 48000 -p | play -c 1 -r 48000 -p

I've been trying to use PulseAudio's using module-loopback and module-pipe-source in conjunction with module-pipe-sink. So far I've been getting no audio or distorted audio. Also PulseAudio exits about a minute after I run it, regardless of what I do. PulseAudio's integration with macOS CoreAudio seems quite brittle.

I haven't been able to connect my headset to the Pi. According to this post microphone input from headset using the HSP BT profile is very poorly supported.

Some possibilities I've been considering but could still try:

  • Running Pulseaudio on a Raspberry Pi or inside a Docker container and sending audio back and forth from the Mac via RTP using Roc
  • Mix the audio from microphones into the built-in AUX output of the Mac. and sample it back in from an AUX input of a USB-attached sound-card.
  • Use Windows or Linux instead of Mac.
  • Write custom real-time code in Python.

I'd appreciate your advice on the best way to solve this audio live resampling and routing problem.

2
audio-streamingmacos-mojaveresamplingpulseaudioheadset
Roc doesn't work in this case, as it can only read from a stereo device. Anything mono or multi-channel will be rejected, at least on Mac. – Isac Casapu

2 Answers

1
votes

MAC OS X support the CoreAudio framework, the AudioToolkit framework, and the AudioUnit framework.So you can use the MAC frameworks to solve your requirement.

If you need to resample your input data, you have to use a separate AudioConverter unit.

If you need mixer, you can use audio unit for mixing different parties.

(Audio Unit)

1
votes

The solution has been under my nose the whole time. It turns out MacOS/X can combine multiple audio devices into an “aggregate device”, as explained on Roland's company blog. Care should be taken to enable drift correction for the audio channels coming from the headsets.