Querying FIX protocol message files using Apache Drill

1
votes

I was able to query FIX messages as a csv using delimiter as '\u0001', but the results had tag=value in each of the columns, like so:

Expected:

---------
 8      |
---------
 FIX.4.4|
 FIX.4.4|
 FIX.4.4|
 FIX.4.4|
 FIX.4.4|
---------

Actual:

-----------
 EXPR$1   |
-----------
 8=FIX.4.4|
 8=FIX.4.4|
 8=FIX.4.4|
 8=FIX.4.4|
 8=FIX.4.4|
-----------

How do I query FIX protocol message files using Apache Drill to achieve the above expected result? Will this need a custom storage format implementation?

1
apachecsvapache-drill
It looks like you would need to create custom storage format. - Arina Yelchiyeva

1 Answers

1
votes

You can contribute to Apache Drill and develop "FIX Protocol" storage format plugin.

Also you can try to parse your string values and extract the result from it by SQL:

0: jdbc:drill:> SELECT split(a, '=')[0] as `key`, split(a, '=')[1] as `value` FROM (VALUES('8=FIX.4.4')) t(a);
+------+----------+
| key  |  value   |
+------+----------+
| 8    | FIX.4.4  |
+------+----------+