Google Cloud Dataflow UDF adding numeric schema

Question

I am using Dataflow to connect Google Cloud Pub-Sub and BigQuery. My Pub-Sub messages are in JSON form and have a field called Temp with the temperature in degrees Celsius. I am trying to add schema for the temperature in Celsius and Fahrenheit called TempC and TempF to the BigQuery table as well as the timestamp. The code correctly adds the timestamp, but not TempC and TempF.

I copied the code into an HTML document with some sample JSON text and it works fine there.

Is there something weird where the UDF can't create numeric schema?

My UDF in javascript is:

 /**
 * A transform function which adds the timestamp and converts temp from c to f
 * @param {string} inJson
 * @return {string} outJson
 */
function transform_udf1(inJson) {
  var obj = JSON.parse(inJson);

  var timestamp = getTime();
  var tempc = parseFloat(obj.Temp);
  var tempf = tempc * (9/5) + 32;

  obj.Timestamp = timestamp;
  obj.TempC = tempc;
  obj.TempF = tempf;

  return JSON.stringify(obj);

}

function getTime() {
    var d = new Date();
    return d;
}

Elizabeth Elizabeth · Accepted Answer · 2019-04-10T16:44:07

I was able to work around it by performing the conversions in javascript on my website.

Google Cloud Dataflow UDF adding numeric schema

1 Answers