Three.js use framebuffer as texture

Question

I'm using an image in a canvas element as a texture in Three.js, performing image manipulations on the canvas using JavaScript, and then calling needsUpdate() on the texture. This works, but it's quite slow.

I'd like to perform the image calculations in a fragment shader instead. I've found many examples which almost do this:

Shader materials: http://mrdoob.github.io/three.js/examples/webgl_shader2.html This example shows image manipulations performed in a fragment shader, but that shader is functioning as the fragment shader of an entire material. I only want to use the shader on a texture, and then use the texture as a component of a second material.
Render to texture: https://threejsdoc.appspot.com/doc/three.js/examples/webgl_rtt.html This shows rendering the entire scene to a WebGLRenderTarget and using that as the texture in a material. I only want to pre-process an image, not render an entire scene.
Effects composer: http://www.airtightinteractive.com/demos/js/shaders/preview/ This shows applying shaders as a post-process to the entire scene.

Edit: Here's another one:

Render to another scene: http://relicweb.com/webgl/rt.html This example, referenced in Three.js Retrieve data from WebGLRenderTarget (water sim), uses a second scene with its own orthographic camera to render a dynamic texture to a WebGLRenderTarget, which is then used as a texture in the primary scene. I guess this is a special case of the first "render to texture" example listed above, and would probably work for me, but seems over-complicated.

As I understand it, ideally I'd be able to make a new framebuffer object with its own fragment shader, render it on its own, and use its output as a texture uniform for another material's fragment shader. Is this possible?

Edit 2: It looks like I might be asking something similar to this: Shader Materials and GL Framebuffers in THREE.js ...though the question doesn't appear to have been resolved.

Within the framework of three.js, the way to do what you want is to render (a scene containing a single plane and an orthographic camera) to a texture using your custom ShaderMaterial, and then use that texture as a uniform for a second ShaderMaterial. — WestLangley
Even your question was very educational; it has some really useful links to good examples. — Marquizzo

meetar meetar · Accepted Answer · 2014-02-08T20:38:51

Render to texture and Render to another scene as listed above are the same thing, and are the technique you want. To explain:

In vanilla WebGL the way you do this kind of thing is by creating a framebuffer object (FBO) from scratch, binding a texture to it, and rendering it with the shader of your choice. Concepts like "scene" and "camera" aren't involved, and it's kind of a complicated process. Here's an example:

http://learningwebgl.com/blog/?p=1786

But this also happens to be essentially what Three.js does when you use it to render a scene with a camera: the renderer outputs to a framebuffer, which in its basic usage goes straight to the screen. So if you instruct it to render to a new WebGLRenderTarget instead, you can use whatever the camera sees as the input texture of a second material. All the complicated stuff is still happening, but behind the scenes, which is the beauty of Three.js. :)

So: To replicate a WebGL setup of an FBO containing a single rendered texture, as mentioned in the comments, just make a new scene containing an orthographic camera and a single plane with a material using the desired texture, then render to a new WebGLRenderTarget using your custom shader:

// new render-to-texture scene
myScene = new THREE.Scene();

// you may need to modify these parameters
var renderTargetParams = {
  minFilter:THREE.LinearFilter,
  stencilBuffer:false,
  depthBuffer:false
};

myImage = THREE.ImageUtils.loadTexture( 'path/to/texture.png',
  new THREE.UVMapping(), function() { myCallbackFunction(); } );

imageWidth = myImage.image.width;
imageHeight = myImage.image.height;

// create buffer
myTexture = new THREE.WebGLRenderTarget( width, height, renderTargetParams );

// custom RTT materials
myUniforms = {
  colorMap: { type: "t", value: myImage },
};
myTextureMat = new THREE.ShaderMaterial({
  uniforms: myUniforms,
  vertexShader: document.getElementById( 'my_custom_vs' ).textContent,
  fragmentShader: document.getElementById( 'my_custom_fs' ).textContent
});

// Setup render-to-texture scene
myCamera = new THREE.OrthographicCamera( imageWidth / - 2,
  imageWidth / 2,
  imageHeight / 2,
  imageHeight / - 2, -10000, 10000 );

var myTextureGeo = new THREE.PlaneGeometry( imageWidth, imageHeight );
myTextureMesh = new THREE.Mesh( myTextureGeo, myTextureMat );
myTextureMesh.position.z = -100;
myScene.add( myTextureMesh );

renderer.render( myScene, myCamera, myTexture, true );

Once you've rendered the new scene, myTexture will be available for use as a texture in another material in your main scene. Note that you may want to trigger the first render with the callback function in the loadTexture() call, so that it won't try to render until the source image has loaded.

Three.js use framebuffer as texture

1 Answers