Storing rotated / flipped images in Google App Engine

Question

I'm trying to rotate / flip an image that is already sitting inside Google Cloud Storage.

This is the blobkey:

BlobKey blobKey = new BlobKey(upload.getBlobKey());

Then I retrieve the image and apply an image transform:

ImagesService imagesService = ImagesServiceFactory.getImagesService();
Image image = ImagesServiceFactory.makeImageFromBlob(blobKey);
Transform transform = ImagesServiceFactory.makeRotate(90);
Image newImage = imagesService.applyTransform(transform, image);

The RAW image data I can get in a byte array using:

newImage.getImageData()

Using GCS directly, I can write it to the cloud storage service:

GcsFilename fileName = new GcsFilename("bucket-name", upload.getFileName());
GcsOutputChannel outputChannel = GcsServiceFactory.createGcsService().createOrReplace(fileName, GcsFileOptions.getDefaultInstance());
outputChannel.write(ByteBuffer.wrap(newImage.getImageData()));
outputChannel.close();

Locally I can see the rotated image sitting inside the appengine-generated folder, how do I get the new serving url or what should the objectName be in order to overwrite the original image? Is the objectName the same as the original filename when it was uploaded, is it the blobkey, ...?

Update:

I'm storing the GcsFilename upon upload:

BlobstoreService blobstoreService = BlobstoreServiceFactory.getBlobstoreService();
Map<String,List<FileInfo>> finfos = blobstoreService.getFileInfos(req);
String gcsFileName = finfos.get("uploadKey").get(0).getGsObjectName();
upload.setGcsFileName(gcsFileName);
upload.persist();

The GcsFileName looks like this (locally):

/gs/bucket-name/fake-encoded_gs_key:YXVjdGlvbi1wb3J0YWwtdXBsb2Fkcy84SXp5VGpIUWlZSDY5X1Ytck5TbGtR-0f969f5b4f53a01479ff2d5eaf02fa1a (unindexed)

Then inside my rotate method, I do the following:

GcsFilename fileName = new GcsFilename( "bucket-name", upload.getGcsFileName());
GcsOutputChannel outputChannel = GcsServiceFactory.createGcsService().createOrReplace(fileName, GcsFileOptions.getDefaultInstance());
outputChannel.write(ByteBuffer.wrap(newImage.getImageData()));
outputChannel.close();

BlobstoreService bs = BlobstoreServiceFactory.getBlobstoreService();
BlobKey bk = bs.createGsBlobKey(fileName.getObjectName());

When I print the new blobKey using bk.getKeyString(), I get an enormous blobkey:

encoded_gs_key:L2dzL2F1Y3Rpb24tcG9ydGFsLXVwbG9hZHMvZmFrZS1lbmNvZGVkX2dzX2tleTpZWFZqZEdsdmJpMXdiM0owWVd3dGRYQnNiMkZrY3k4NFNYcDVWR3BJVVdsWlNEWTVYMVl0Y2s1VGJHdFItMGY5NjlmNWI0ZjUzYTAxNDc5ZmYyZDVlYWYwMmZhMWE

When I try:

imagesService.getServingUrl(ServingUrlOptions.Builder.withBlobKey(bk);

I now get a INVALID_BLOB_KEY: Could not read blob.

Same with:

imagesService.getServingUrl(ServingUrlOptions.Builder.withGoogleStorageFileName(fileName.getObjectName()));

INVALID_BLOB_KEY: Could not read blob.

Update - Testing On Live:

I've done some logging on live to see if I can pinpoint where things are going wrong.

When I print out the GcsFilename, this is what I'm getting:

GcsFilename fileName = new GcsFilename("bucket-name", upload.getGcsFileName());
System.out.println("fileName.getObjectName(): " + fileName.getObjectName());
System.out.println("fileName.getBucketName(): " + fileName.getBucketName());

2014-03-08 09:31:35.076 [s~_____/live-1-0-2.374269833496060124].<stdout>: fileName.getObjectName(): /gs/bucket-name/L2FwcGhvc3RpbmdfcHJvZC9ibG9icy9BRW5CMlI 
2014-03-08 09:31:35.076 [s~_____/live-1-0-2.374269833496060124].<stdout>: fileName.getBucketName(): bucket-name

Then generating the new blobkey:

BlobstoreService bs = BlobstoreServiceFactory.getBlobstoreService();
BlobKey bk = bs.createGsBlobKey(fileName.getObjectName());

System.out.println("GSBlobKey: " + bk.getKeyString());

2014-03-08 09:31:37.993 [s~_____/live-1-0-2.374269833496060124].<stdout>: GSBlobKey: AMIfv94Tj3xO9hPWwMm314Id0obQs1gVWOHYVuGaaAjjMbieBUD6x0nNGbqiY8o5Jk9iLga4pPe9-8L1

Then using 3 different methods for getting the new serving urls:

String newImageUrl1 = imagesService.getServingUrl(ServingUrlOptions.Builder.withGoogleStorageFileName(fileName.getObjectName())); 
System.out.println("newImageUrl1: " + newImageUrl1);

String newImageUrl2 = imagesService.getServingUrl(ServingUrlOptions.Builder.withBlobKey(bk));
System.out.println("newImageUrl2: " + newImageUrl2);

String newImageUrl3 = imagesService.getServingUrl(bk);
System.out.println("newImageUrl3: " + newImageUrl3);


2014-03-08 09:31:38.116 [s~_____/live-1-0-2.374269833496060124].<stdout>: newImageUrl1: http://lh3.ggpht.com/My7zTT79xOltI3cMzHidx7-Us91Qlp13jdv-vJceQ_OpX3FONSr9YzI2...
2014-03-08 09:31:38.152 [s~_____/live-1-0-2.374269833496060124].<stdout>: newImageUrl2: http://lh3.ggpht.com/My7zTT79xOltI3cMzHidx7-Us91Qlp13jdv-vJceQ_OpX3FONSr9YzI2... 
2014-03-08 09:31:38.218 [s~_____/live-1-0-2.374269833496060124].<stdout>: newImageUrl3: http://lh3.ggpht.com/My7zTT79xOltI3cMzHidx7-Us91Qlp13jdv-vJceQ_OpX3FONSr9YzI2...

When I access these URLs, all I'm getting is the original image.

When I look in the Cloud Storage Dashboard, the original image is sitting in: Home/bucket-name/

The rotated image however is sitting in: Home/bucket-name//gs/bucket-name/

Hence the reason it's not overwriting the original. I'm guessing either the fileName I'm creating is incorrect or the GcsOutputChannel I'm creating is changing the path? I'm also curious why I'm getting the original image's url instead of the new image's url?

If I split the original GcsFileName and only take the last part, I'm getting an error:

GcsFilename fileName = new GcsFilename("bucket-name", upload.getGcsFileName().split("/")[3]);

Google storage filenames must be prefixed with /gs/

Any ideas?

Solution

Retreive blobKey using:

BlobKey blobKey = blobstoreService.createGsBlobKey("/gs/" + fileName.getBucketName() + "/" + fileName.getObjectName());

... even though fileName.getObjectName() already contains "/gs/bucket-name/" in front of the objectname. When printing it to standard out, your fileName should contain /gs/bucket-name/ twice, weird, but it works.

Using that blobKey, the correct serving URL is created

Hey Jan. From the "fake-encoded_gs_key" I understand that you are making tests on the local development server. Some GCS and ImageService features are not available there. Did you make the same tests on a GAE production application ? — David
The rotated image is being written in the incorrect location, if I can fix that, then rotate would be 100%, I've updated the question. — Jan Vladimir Mostert
Hey Jan, "getObjectName()" should not return the bucket name, nor the "/gs" at the beginning. Are you sure that your logs output the object name and not the result of the toString() method ? — David
Both getObjectName and getBucketName returns a string type and the objectName has the /gs/bucket-name/... in front of the name even locally when running in debug mode. I've done what the Google Docs recommend and prefix it with another /gs/bucket-name/ and now I'm able to rotate and store an image once, the second rotate doesn't work, but that'll look into further. Thanks for all the help!! — Jan Vladimir Mostert
Thanks for the solution listed above! It works for me too for a similar issue. — Peter Kelley

David David · Accepted Answer · 2014-03-04T18:23:33

To override the original image, you simply need to specify the same Cloud Storage filename.

I understand that you have uploaded the image to Cloud Storage through App Engine via the BlobStore API. To get the actual filename of the original image, you must add the following code in the upload servlet :

  Map<String,List<BlobInfo>> uploadInfos = getFileInfos(httpServletRequest);
  String theGCSFilename = uploadInfos.get("yourFileName").getGsObjectName();

Note that if you do not want to overwrite the original file, but want to provide a serving URL to the customer, you can either :

Use the ImageService like this :

ImagesService imagesService = ImagesServiceFactory.getImagesService();
String url = imagesService.getServingUrl(ServingUrlOptions.Builder.withGoogleStorageFileName(java));

Create a custom GCSServingServlet who will use the BlobstoreService.serve() method to serve a file from the Blobstore or Cloud Storage. In that case, you must check the permissions yourself. Note that you can create a BlobKey from a GCSFilename using BlobStoreService.createGSBlobKey().
Use the Cloud Storage serving URLs :
- Private : http://commondatastorage.googleapis.com/bucket/filename
- Public : http://storage.googleapis.com//bucket/filename

Storing rotated / flipped images in Google App Engine

1 Answers