How to auto-update Google Spreadsheet with import data each month?

Question

I want to store UK police crime data for my local area. UK police update police.uk with new data each month. This is accessible two ways: 1. CSV. Crime for my area is this This is downloadable as CSV ( 1. & 2. 2. JSON via API. Info here - I haven't tackled this.

The problem: The web page and CSV only show data for the last six months. I want to create a Google Drive Spreadsheet that updates each month to add latest data on an ongoing basis, so as to retain ALL data over time.

I don't really know how to tackle this, especially on the API side. Should I be using standard Spreadsheet functions like ImportData, Google Spreadsheets' Script editor or even use a programming language like Python, which I don't know?

Thanks.

dev dev · Accepted Answer · 2013-12-08T22:14:14

How interesting! I never knew the police had a data API we could use! I would suggest using Google Apps Script, you can easily import the JSON data using that.

For example to first fetch the data you can use the URL Fetch Service like so:

var data = UrlFetchApp.fetch("http://data.police.uk/api/crimes-street/all-crime?lat=52.629729&lng=-1.131592&date=2013-01");

This will return the raw JSON data for you to play with how you see fit.

You can then iterate over the objects, adding them to an array to add to the spreadsheet. A complete function might look something like the below. I know it's extremely messy. Now I know this police API exists I plan (when I get some free time) to set up something similar, I'll update this answer when I have a better solution but I have tested the below and it does write all the data from a URL to the spreadsheet.

function importPoliceData() {
  var data = UrlFetchApp.fetch("http://data.police.uk/api/crimes-street/all-crime?lat=52.629729&lng=-1.131592&date=2013-01");
  var jsonData = JSON.parse(data.getContentText());
  var dataArray = [];
  for(var i = 0; i < jsonData.length; i++){
    var thisRow = [];
    for(var a in jsonData[i]){
      if(typeof jsonData[i][a] == 'object'){
        for(var o in jsonData[i][a]){
          if(typeof jsonData[i][a][o] == 'object'){
            for( var p in jsonData[i][a][o]){
              thisRow.push(jsonData[i][a][o][p]);
            }
          } else {
            thisRow.push(jsonData[i][a][o]);
          }
        }
      } else {
        thisRow.push(jsonData[i][a]);
      }
    }
    while(thisRow.length != 13){
      thisRow.push("");
    }
    dataArray.push(thisRow);
  }
  var sheet = SpreadsheetApp.getActiveSheet();
  var range = sheet.getRange(1,1,dataArray.length, dataArray[0].length);
  range.setValues(dataArray);
}

How to auto-update Google Spreadsheet with import data each month?

1 Answers