How does the paste image from clipboard functionality work in Gmail and Google Chrome 12+?

Question

I noticed a blog post from Google that mentions the ability to paste images directly from the clipboard into a Gmail message if you're using the latest version of Chrome. I tried this with my version of Chrome (12.0.742.91 beta-m) and it works great using control keys or the context menu.

From that behavior I need to assume that the latest version of webkit used in Chrome is able to deal with images in the Javascript paste event, but I have been unable to locate any references to such an enhancement. I believe ZeroClipboard binds to keypress events to trigger its flash functionality and as such wouldn't work through the context menu (also, ZeroClipboard is cross-browser and the post says this works only with Chrome).

So, how does this work and where the enhancement was made to Webkit (or Chrome) that enables the functionality?

Answer 1

I spent some time experimenting with this. It seems to sort of follow the new Clipboard API spec. You can define a "paste" event handler and look at event.clipboardData.items, and call getAsFile() on them to get a Blob. Once you have a Blob, you can use FileReader on it to see what's in it. This is how you can get a data url for the stuff you just pasted in Chrome:

document.onpaste = function (event) {
    var items = (event.clipboardData || event.originalEvent.clipboardData).items;
    console.log(JSON.stringify(items)); // might give you mime types
    for (var index in items) {
        var item = items[index];
        if (item.kind === 'file') {
            var blob = item.getAsFile();
            var reader = new FileReader();
            reader.onload = function (event) {
                console.log(event.target.result); // data url!
            }; 
            reader.readAsDataURL(blob);
        }
    }
};

Once you have a data url you can display the image on the page. If you want to upload it instead, you could use readAsBinaryString, or you could put it into an XHR using FormData.

Edit: Note that the item is of type DataTransferItem. JSON.stringify might not work on the items list, but you should be able to get mime type when you loop over items.

Answer 2

The answer by Nick seems to need small changes to still work :)

// window.addEventListener('paste', ... or
document.onpaste = function (event) {
  // use event.originalEvent.clipboard for newer chrome versions
  var items = (event.clipboardData  || event.originalEvent.clipboardData).items;
  console.log(JSON.stringify(items)); // will give you the mime types
  // find pasted image among pasted items
  var blob = null;
  for (var i = 0; i < items.length; i++) {
    if (items[i].type.indexOf("image") === 0) {
      blob = items[i].getAsFile();
    }
  }
  // load image if there is a pasted image
  if (blob !== null) {
    var reader = new FileReader();
    reader.onload = function(event) {
      console.log(event.target.result); // data url!
    };
    reader.readAsDataURL(blob);
  }
}

Example running code: http://jsfiddle.net/bt7BU/225/

So the changes to nicks answer were:

var items = event.clipboardData.items;

to

var items = (event.clipboardData  || event.originalEvent.clipboardData).items;

Also I had to take the second element from the pasted items (first one seems to be text/html if you copy an image from another web page into the buffer). So I changed

  var blob = items[0].getAsFile();

to a loop finding the item containing the image (see above)

I didn't know how to answer directly to Nick's answer, hope it is fine here :$ :)

Answer 3

As far as I know -

With HTML 5 features(File Api and the related) - accessing clipboard image data is now possible with plain javascript.

This however fails to work on IE (anything less than IE 10). Don't know much about IE10 support also.

For IE the optiens that I believe are the 'fallback' options are either using Adobe's AIR api or using a signed applet