How to share textures between a 2d Canvas and a WebGL canvas?

Question

I'm developing a browser game using WebGL and the recommended advice seems to be to use either straight-up HTML elements or a 2d canvas layered on top of your WebGL canvas to do the UI / HUD.

This is all well and good, but how do you handle texture sharing? For example, there might be items that drop on the ground in-game, so those exist in the world, and as a result would have to be WebGL textures, as they are items that exist on the ground.

However, when you pick them up, their icon is then used in the HUD as they take their place on the ability bar at the bottom of your screen. So you would have to have one Canvas2D texture to draw on your UI, and one WebGLTexture to draw on your WebGL canvas.

Currently, I am handling this by just loading the texture in twice, once as a WebGL texture in-game, and once as a canvas texture. But I don't think this solution is very scalable because in the worst case I would have to load every single texture in the game twice, and it would double the game's memory usage.

For reference, here are both ways I am loading the textures:

// Load canvas texture
static createCanvasFromImage(filename: string) {
  return new Promise((resolve, reject) => {
    const canvas = document.createElement("canvas");
    const context = canvas.getContext("2d");

    let newImg = new Image();

    newImg.onload = function() {
      canvas.width = newImg.width;
      canvas.height = newImg.height;

      context.drawImage(newImg, 0, 0);

      resolve(canvas);
    }

    newImg.src = filename;
  });
}

.

// Load WebGL texture
static loadImageAndCreateTextureInfo(gl: WebGLRenderingContext, url: string) {
  var tex = gl.createTexture();
  gl.bindTexture(gl.TEXTURE_2D, tex);
  // Fill the texture with a 1x1 blue pixel.
  gl.texImage2D(gl.TEXTURE_2D, 0, gl.RGBA, 1, 1, 0, gl.RGBA, gl.UNSIGNED_BYTE, new Uint8Array([0, 0, 255, 255]));

  gl.texParameteri(gl.TEXTURE_2D, gl.TEXTURE_MIN_FILTER, gl.NEAREST);
  gl.texParameteri(gl.TEXTURE_2D, gl.TEXTURE_MAG_FILTER, gl.NEAREST);
  gl.texParameteri(gl.TEXTURE_2D, gl.TEXTURE_WRAP_S, gl.CLAMP_TO_EDGE);
  gl.texParameteri(gl.TEXTURE_2D, gl.TEXTURE_WRAP_T, gl.CLAMP_TO_EDGE);

  // Don't know size until it loads
  var textureInfo = { width: 1, height: 1, texture: tex };

  var img = new Image();
  img.addEventListener('load', function() {
    textureInfo.width = img.width;
    textureInfo.height = img.height;

    gl.bindTexture(gl.TEXTURE_2D, textureInfo.texture);
    gl.texImage2D(gl.TEXTURE_2D, 0, gl.RGBA, gl.RGBA, gl.UNSIGNED_BYTE, img);
  });
  img.src = url;

  return textureInfo;
}

How is this normally handled? Is there any way to share these textures, such that each one is only loaded in memory once, and can be used on either canvas?

The data store of a WebGL texture object resides in GPU memory and is managed by the graphics driver. — Rabbid76
You can draw the image to a third canvas that you don't show to the user and use it in your canvas 2D via ctx.drawImage(textureCanvas, 0, 0) and WebGL via gl.texImage2D. — Georgi B. Nikolov

EthanKim8683 EthanKim8683 · Accepted Answer · 2021-05-02T05:50:34

I don't know if this really helps much, but maybe if you had a separate function that both functions call to create an image, which then pushes to an array of assets?

Like what if you had something like this:

async newImage (url) {

  const img = new Image();

  img.src = url;

  // Wait for the image to load

  await new Promise(resolve => img.onload = () => resolve());

  // Once loaded, push to "assets" (Or whatever you'd like to call it) and return image

  assets.push(img);

  return img;
}

// WebGL Texture, for example

static async loadImageAndCreateTextureInfo (gl: WebGLRenderingContext, url: string) {

  var tex = gl.createTexture();
  gl.bindTexture(gl.TEXTURE_2D, tex);

  // Fill the texture with a 1x1 blue pixel.

  gl.texImage2D(gl.TEXTURE_2D, 0, gl.RGBA, 1, 1, 0, gl.RGBA, gl.UNSIGNED_BYTE, new Uint8Array([0, 0, 255, 255]));

  gl.texParameteri(gl.TEXTURE_2D, gl.TEXTURE_MIN_FILTER, gl.NEAREST);
  gl.texParameteri(gl.TEXTURE_2D, gl.TEXTURE_MAG_FILTER, gl.NEAREST);
  gl.texParameteri(gl.TEXTURE_2D, gl.TEXTURE_WRAP_S, gl.CLAMP_TO_EDGE);
  gl.texParameteri(gl.TEXTURE_2D, gl.TEXTURE_WRAP_T, gl.CLAMP_TO_EDGE);

  // Don't know size until it loads

  var textureInfo = { width: 1, height: 1, texture: tex };

  // "await" makes the code wait for an action to be completed

  const img = await this.newImage(url);
  
  textureInfo.width = img.width;
  textureInfo.height = img.height;

  gl.bindTexture(gl.TEXTURE_2D, textureInfo.texture);
  gl.texImage2D(gl.TEXTURE_2D, 0, gl.RGBA, gl.RGBA, gl.UNSIGNED_BYTE, img);

  return textureInfo;
}

I haven't tested this code yet, so it may or may not work, but give it a try :)

How to share textures between a 2d Canvas and a WebGL canvas?

1 Answers