store and access image file paths when templating (from cloudinary or other service)

Question

I’m using gulp and nunjucks to automate some basic email templating tasks.

I have a chain of tasks which can be triggered when an image is added to the images folder e.g.:

1. images compressed
2. new image name and dimensions logged to json file
3. json image data then used to populate template when template task is run

So far so good.

I want to be able to define a generic image file path for each template which will then concatenate to each image name (as stored in the json file). So something like:

<img src="{{data.path}}{{data.src}}" >

If I want to nominate a distinct folder to contain the images for each template generated then cloudinary requires a mandatory unique version component to be applied in the file path. So the image path can never be consistent throughout a template.

if your public ID includes folders (elements divided by '/'), the version component is mandatory, (but you can make it shorter. )

For example: http://res.cloudinary.com/demo/image/upload/v1312461204/sample_email/hero_image.jpg

http://res.cloudinary.com/demo/image/upload/v1312461207/sample_email/footer_image.jpg

Same folder. Different path.

So it seems I would now need to create a script/task that can log and store each distinct file path (with its unique id generated by cloudinary) for every image any time an image is uploaded or updated and then rerun the templating process to publish them.

This just seems like quite a convoluted process so if there’s an easier approach I’d love to know?

Else if that really is the required route it would great if someone could point me to an example of the kind of script that achieves something similar.

Presumably some hosting services will not have the mandatory unique key which makes life easier. I have spent some time getting to know cloudinary and it’s a free service with a lot of scope so I guess I'm reluctant to abandon ship but open to all suggestions.

Thanks

Answer 1

Note that the version component (e.g., v1312461204) isn't mandatory anymore for most use-cases. The URL could indeed work without it, e.g.,: http://res.cloudinary.com/demo/image/upload/sample_email/hero_image.jpg

Having said that, it is very recommended to include the version component in the URL in cases where you'd like to update the image with a new one while keeping the exact same public ID. In that case, if you'd access the exact same URL, you might get a CDN cached version of the image, which may be the old one. Therefore, when you upload, you can get the version value from Cloudinary's upload response, and store it in your DB, and the next time you update your image, also update the URL with the new version value.

Alternatively, you can also ask Cloudinary to invalidate the image while uploading. Note that while including the version component "busts" the cache immediately, invalidation may take a while to propagate through the CDN. For more information: http://cloudinary.com/documentation/image_transformations#image_versions

Answer 2

This is the solution I came up with. It's based on adapting the generic script I use to upload images from a folder to cloudinary and now stores the updated file paths from cloudinary and generates a json data file to publish the hosted src details to a template.

I'm sure it could be a lot better semantically so welcome any revisions offered if someone stumbles on this but it seems to do the job:

// points to the config file where we are defining file paths
var path = require('./gulp.path')();
// IMAGE HOSTING
var fs = require('fs'); // !! not installed !! Not required?? 
var cloudinary = require('cloudinary').v2;
var uploads = {};
var dotenv = require('dotenv');
dotenv.load();
// Finds the images in a specific folder and retrurns an array
var read = require('fs-readdir-recursive');
// Set location of images
var imagesInFolder = read(path.images);

// The array that will be populated with image src data
var imgData = new Array();

(function uploadImages(){

// Loop through all images in folder and upload
for(var i = 0; i < imagesInFolder.length;i++){
       cloudinary.uploader.upload(path.images + imagesInFolder[i], {folder: path.hosted_folder, use_filename: true, unique_filename: false, tags: 'basic_sample'}, function(err,image){
          console.log();
          console.log("** Public Id");
          if (err){ console.warn(err);}
          console.log("* Same image, uploaded with a custom public_id");
          console.log("* "+image.public_id);

          // Generate the category title for each image. The category is defined within the image name. It's the first part of the image name i.e. anything prior to a hyphen:
          var title = image.public_id.substr(image.public_id.lastIndexOf('/') + 1).replace(/\.[^/.]+$/, "").replace(/-.*$/, "");

          console.log("* "+title); 
          console.log("* "+image.url);

          // Add the updated src for each image to the output array 
          imgData.push({ 
                        [title] : {"src" : image.url}
                      });

            // Stringify data with no spacing so .replace regex can easily remove the unwanted curly braces
            var imgDataJson = JSON.stringify(imgData, null, null);

            // Remove the unwanted [] that wraps the json imgData array
           var imgDataJson = imgDataJson.substring(1,imgDataJson.length-1);

           // Delete unwanted braces "},{" replace with "," otherwise what is output is not valid json
           var imgDataJson =  imgDataJson.replace(/(},{)/g, ',');

          var outputFilename = "images2-hosted.json"

        // output the hosted image path data to a json file
        // (A separate gulp task is then run to merge and update the new 'src' data into an existing image data json file)
        fs.writeFile(path.image_data_src + outputFilename, imgDataJson, function(err) {
          if(err) {
            console.log(err);
          } else {
            console.log("JSON saved to " + outputFilename);
          }
         });

        });
      }
 })();

A gulp task is then used to merge the newly generated json to overide the existing json data file:

// COMPILE live image hosting data
var merge = require('gulp-merge-json');
gulp.task('imageData:comp', function() {
  gulp
    .src('src/data/images/*.json')
    .pipe(merge('src/data/images.json'))
    .pipe(gulp.dest('./'))
    .pipe(notify({ message: 'imageData:comp task complete' }));
});