ML Engine Google Cloud Platform, parse features from a string inside the deployed model

Question

I am using TensorFlow on ML Engine - Google Cloud Platform to solve a regression problem. I need to send a tensor string that contains a date to ML Engine, something like "2018/06/05 23:00" and from there have my deployed model extract the features which basically are (year, month, day, hour). For the example above would be (2018, 06, 05, 23). The thing is that I need to get this performed in the deployed model in ML Engine and not in an in-between API.

First what I did is to adapt the Census Model Tutorial to my regression problem. https://cloud.google.com/ml-engine/docs/tensorflow/getting-started-training-prediction

In this tutorial, they deploy the model in ML Engine using the gcloud command gcloud ml-engine models create $MODEL_NAME ... through the terminal.

Below you will find the way I found to manipulate a string tensor that contains a date to obtain the features

import tensorflow as tf
import numpy as np 
date_time = tf.placeholder(shape=(1,), dtype=tf.string, name="ph_date_time")

INPUT_COLUMNS=["year", "month", "day", "hour"]


split_date_time = tf.string_split(date_time, ' ')

date = split_date_time.values[0]
time = split_date_time.values[1]

split_date = tf.string_split([date], '-')
split_time = tf.string_split([time], ':')

year = split_date.values[0]
month = split_date.values[1]
day = split_date.values[2]
hours = split_time.values[0]
minutes = split_time.values[1]

year = tf.string_to_number(year, out_type=tf.int32, name="year_temp")
month = tf.string_to_number(month, out_type=tf.int32, name="month_temp")
day = tf.string_to_number(day, out_type=tf.int32, name="day_temp")
hours = tf.string_to_number(hours, out_type=tf.int32, name="hour_temp")
minutes = tf.string_to_number(minutes, out_type=tf.int32, name="minute_temp")

year = tf.expand_dims(year, 0, name="year")
month = tf.expand_dims(month, 0, name="month")
day = tf.expand_dims(day, 0, name="day")
hours = tf.expand_dims(hours, 0, name="hours")
minutes = tf.expand_dims(minutes, 0, name="minutes")

features = []
features = np.append(features, year)
features = np.append(features, month)
features = np.append(features, day)
features = np.append(features, hours)

# this would be the actual features to the deployed model
actual_features = dict(zip(INPUT_COLUMNS, features))



with tf.Session() as sess:
    year, month, day, hours, minutes = sess.run([year, month, day, hours, minutes], feed_dict={date_time: ["2018-12-31 22:59"]})
    print("Year =", year)
    print("Month =", month)
    print("Day =", day)
    print("Hours =", hours)
    print("Minutes =", minutes)

The thing is that I don't know how to tell the ML Engine to use this parsing above. I know it has to do with either the input_fn to define the model or the serving_input_fn used to export the model, but I am not sure if I have to paste my code in both or one of them, any recommendation would be highly appreciate it, and sorry if the explanation is not clear.

Answer 1

The general pattern to follow is (see this doc):

1. Create an input_fn used in training, typically using tf.data.Dataset. The input_fn should call helper functions to do data transformations, like those in your code. The output will be a dictionary of feature names to batches of values.
2. Define FeatureColumns for the items in the output of your input_fn. If necessary, do things like feature crosses, bucketization, etc.
3. Instantiate the estimator (e.g. DnnRegressor), passing the FeatureColumns to the constructor
4. Create an input_fn specifically for serving, that has one or more tf.Placeholder with None (variable batch size) as the outer dimension. Call the same helper functions from (1) to do the transformations. Return a tf.estimator.export.ServingInputReceiver with the placeholders as inputs and a dict that should look the same as the dict in (1).

Your particular case warrants a few additional details. First, you have hardcoded a batch size of 1 into your placeholder and the corresponding code continues that assumption. Your placeholder must have shape=[None].

Unfortunately, your code has been written under the assumption that shape was 1, e.g., split_date_time.values[0] will no longer be valid. I've added a helper function to address that in the code below.

Here's some code that should hopefully work for you:

import tensorflow as tf

# tf.string_split returns a SparseTensor. When using a variable batch size,
# this can be difficult to further manipulate. In our case, we don't need
# a SparseTensor, because we have a fixed number of elements each split.
# So we do the split and convert the SparseTensor to a dense tensor.
def fixed_split(batched_string_tensor, delimiter, num_cols):
    # When splitting a batch of elements, the values array is row-major, e.g.
    # ["2018-01-02", "2019-03-04"] becomes ["2018", "01", "02", "2019", "03", "04"].
    # So we simply split the string then reshape the array to create a dense
    # matrix with the same rows as the input, but split into columns, e.g.,
    # [["2018", "01", "02"], ["2019", "03", "04"]]
    split = tf.string_split(batched_string_tensor, delimiter)
    return tf.reshape(split.values, [-1, num_cols])


def parse_dates(dates):  
    split_date_time = fixed_split(dates, ' ', 2)

    date = split_date_time[:, 0]
    time = split_date_time[:, 1]

    # The values of the resulting SparseTensor will alternate between year, month, and day
    split_date = fixed_split(date, '-', 3)
    split_time = fixed_split(time, ':', 2)

    year = split_date[:, 0]
    month = split_date[:, 1]
    day = split_date[:, 2]
    hours = split_time[:, 0]
    minutes = split_time[:, 1]

    year = tf.string_to_number(year, out_type=tf.int32, name="year_temp")
    month = tf.string_to_number(month, out_type=tf.int32, name="month_temp")
    day = tf.string_to_number(day, out_type=tf.int32, name="day_temp")
    hours = tf.string_to_number(hours, out_type=tf.int32, name="hour_temp")
    minutes = tf.string_to_number(minutes, out_type=tf.int32, name="minute_temp")

    return {"year": year, "month": month, "day": day, "hours": hours, "minutes": minutes}


def training_input_fn():
    filenames = ["/var/data/file1.txt", "/var/data/file2.txt"]
    dataset = tf.data.TextLineDataset(filenames)    
    dataset.batch(BATCH_SIZE)
    return parse_dates(iterator.get_next())


def serving_input_fn():
    date_strings = tf.placeholder(dtype=tf.string, shape=[None], name="date_strings")
    features = parse_dates(date_strings)
    return tf.estimator.export.ServingInputReceiver(features, date_strings)


with tf.Session() as sess:
    date_time_list = ["2018-12-31 22:59", "2018-01-23 2:09"]

    date_strings = tf.placeholder(dtype=tf.string, shape=[None], name="date_strings")
    features = parse_dates(date_strings)


    fetches = [features[k] for k in ["year", "month", "day", "hours", "minutes"]]
    year, month, day, hours, minutes = sess.run(fetches, feed_dict={date_strings: date_time_list})
    print("Year =", year)
    print("Month =", month)
    print("Day =", day)
    print("Hours =", hours)
    print("Minutes =", minutes)