Storing data as a texture for use in Vertex Shader for Instanced Geometry (THREE JS / GLSL)

Question

I'm using a THREE.InstancedBufferGeometry, and I wish to access data in the Vertex Shader, encoded into a Texture.

What I want to do, is create a Data Texture with one pixel per instance, which will store position data for each instance (then at a later stage, I can update the texture using a simulation with a flow field to animate the positions).

I'm struggling to access the data from the texture in the Vertex Shader.

const INSTANCES_COUNT = 5000;

// FOR EVERY INSTANCE, GIVE IT A RANDOM X, Y, Z  OFFSET, AND SAVE IT IN DATA TEXTURE
const data = new Uint8Array(4 * INSTANCES_COUNT);
for (let i = 0; i < INSTANCES_COUNT; i++) {
    const stride = i * 4;
    data[stride] = (Math.random() - 0.5);
    data[stride + 1] = (Math.random() - 0.5);
    data[stride + 2] = (Math.random() - 0.5);
    data[stride + 3] = 0.0;
}

const offsetTexture = new THREE.DataTexture( data, INSTANCES, 1, THREE.RGBAFormat, THREE.FloatType );
offsetTexture.minFilter = THREE.NearestFilter;
offsetTexture.magFilter = THREE.NearestFilter;
offsetTexture.generateMipmaps = false;
offsetTexture.needsUpdate = true;

// CREATE MY INSTANCED GEOMETRY
const geometry = new THREE.InstancedBufferGeometry();
geometry.maxInstancedCount = INSTANCES_COUNT;
geometry.addAttribute( 'position', new THREE.Float32BufferAttribute([5, -5, 0, -5, 5, 0, 0, 0, 5], 3 )); // SIMPLE TRIANGLE


const vertexShader = `
    precision highp float;

    uniform vec3 color;
    uniform sampler2D offsetTexture;

    uniform mat4 modelViewMatrix;
    uniform mat4 projectionMatrix;

    attribute vec3 position;

    varying vec3 vPosition;
    varying vec3 vColor;


    void main(){
        vPosition = position;

        vec4 orientation = vec4(.0, .0, .0, .0);
        vec3 vcV = cross( orientation.xyz, vPosition );
        vPosition = vcV * ( 2.0 * orientation.w ) + ( cross( orientation.xyz, vcV ) * 2.0 + vPosition );

        vec2 uv = position.xy;
        vec4 data = texture2D( offsetTexture, uv );
        vec3 particlePosition = data.xyz * 1000.0;

        gl_Position = projectionMatrix * modelViewMatrix * vec4( vPosition + particlePosition, 1.0 );
    }
`;

const fragmentShader = `
    precision highp float;
    varying vec3 vColor;

    void main() {
        gl_FragColor = vec4(vColor, 1.0);
    }
`;

const uniforms = {
    size: { value: 1.0 },
    color: {
        type: 'c',
        value: new THREE.Color(0x3db230),
    },
    offsetTexture: {
        type: 't',
        value: offsetTexture,
    },
};

// CREATE MY MATERIAL
const material = new THREE.RawShaderMaterial({
    uniforms,
    vertexShader,
    fragmentShader,
    side: THREE.DoubleSide,
    transparent: false,
});

scene.add(new THREE.Mesh(geometry, material));

At the moment it seems that the data from the image isn't accessible in the vertex shader (if I just set the vUv to vec2(1.0, 0.0), for example, and change the offset positions, nothing changes), and also I'm not sure how to go about making sure that the instance can reference the correct texel in the texture.

So, my two issues are: 1) How to correctly set the Data Image Texture, and access that data in the Vertex Shader 2) How to correctly reference the texel storing the data for each particular instance (e.g, instance 1000 should use vec2(1000,1), etc

Also, do I have to normalize the data (0.0-1.0, or 0–255, or -1 – +1)

Thanks

Try replacing attribute offsetPosition with attribute uv. — WestLangley
@WestLangley whoops, the offsetPosition was an attribute which I removed for this example, it shouldn't be there. But regardless, I don't quite follow what replacing that with uv would achieve? (I haven't fully wrapped my head around UV's yet!) — Jack Wild
It appears you are trying to mix GPGPU and instancing. Make sure you understand each individually before you proceed. Each instance will have a uv attribute instead of a position offset. Use the uv to look up the position offset stored in the texture. Later, use GPGPU techniques to update the texture every frame. Conceptually, I do not see why that would not work, but I have not tried it. — WestLangley
Thanks for the tips. I've made some progress, but how the uv's are mapped to each instance isn't well documented. — Jack Wild

pailhead pailhead · Accepted Answer · 2018-01-30T17:03:24

You need to compute some kind of an index into the texture per instance. Meaning, you need an attribute that is going to be shared by each instance.

if your triangle is

[a,b,c]

your index should be

[0,0,0]

Lets say you have 1024 instances and a 1024x1 px texture.

attribute float aIndex;

vec2 myIndex = ((aIndex + 0.5)/1024.,1.);

vec4 myRes = texture2D( mySampler, myIndex);

Storing data as a texture for use in Vertex Shader for Instanced Geometry (THREE JS / GLSL)

1 Answers