Mixing Audio Using OpenGL

Question

I want to mix two (or more) 16bit audio streams using OpenGL and I need a bit of help

Basically what I want to do is to put the audio data into texture which I draw to a frame buffer object and then read back. This is not a problem, however drawing the data in way that gives correct results is a bit more problematic.

I have basically two questions.

1. In order to mix the data by drawing i need to use blending (alpha = 0.5), however the result should not have any alpha channel. So if I render to e.g. a frame buffer with the format RGB will alpha blending still work as I expect and the resulting alpha will not be written to the fbo? (I want to avoid having to read back the fbo for each render pass)

texture |sR|sG|sB|

framebuffer(before) |dR|dG|dB|

framebuffer(after) |dR*0.5+sR*0.5|dG*0.5+sG*0.5|dB*0.5+sB*0.5|

1. The audio samples are signed 16bit integer values. Is it possible do signed calculations this way? Or will I need to first convert the values to unsigned on the cpu, draw them, and then make them signed again on the cpu?

EDIT:

I was a bit unclear. My hardware is restricted to OpenGL 3.3 hardware. I would prefer to not use CUDA or OpenCL, since I'm alrdy using OpenGL for other stuff.

Each audio sample will be rendered in seperate passes, which means that it has to "mix" with whats already been rendered to the frame buffer. The problem is how the output from the pixel shader is written to the framebuffer (this blending is not accessible through programmable shaders, as far as i know, and one has to use glBlendFunc).

EDIT2:

Each audio sample will be rendered in different passes, so only one audio sample will be available in the shader at a time, which means that they need to be accumulated in the FBO.

foreach(var audio_sample in audio_samples)
     draw(audio_sample);

and not

for(int n = 0; n < audio_samples.size(); ++n)
{
      glActiveTexture(GL_TEXTURE0 + n);
      glBindTexture(audio_sample);
}
draw_everything();

Answer 1

1. You should be able to do blending even if the destination buffer does not have alpha. That said, rendering to non-power-of-two sizes (rgb16 = 6bytes/pixel) usually incurs performance penalties.

2. Signed is not your typical render target format, but it does exist in the OpenGL 4.0 specification (Table 3.12, called RGB16_SNORM or RGB16I, depending on whether you want a normalized representation or not).

As a side note, you also have glBlendFunc(GL_CONSTANT_ALPHA,GL_ONE_MINUS_CONSTANT_ALPHA) to not even have to specify an alpha per-pixel. That may not be available on all GL implementations though.

Answer 2

Frankly, why wouldn't you just use programmable pixel shaders for that?

Do you have to use OpenGL 1 Fixed Functionality Pipeline?

I'd just go with a programmable shader operating on signed 16bit grayscale linear textures.

Edit:

foreach(var audio_sample in audio_samples) 
 blend FBO1 + audio_sample => FBO2
 swap FBO2, FBO1 

It ought to be just as fast, if not faster (thanks to streaming pipelines).

Answer 3

I agree with QDot. Could you however inform us a bit about the hardware restrictions you are facing? If you have reasonable modern hardware I might even suggest to go the CUDA or OpenCL route, in stead of going through OpenGL.