This is the record that fluentd write to my log.
2016-02-22 14:38:59 {"login_id":123,"login_email":"[email protected]"}
The date time is the time key of fluentd. How can i place that time inside the json string ?
This is the record that fluentd write to my log.
2016-02-22 14:38:59 {"login_id":123,"login_email":"[email protected]"}
The date time is the time key of fluentd. How can i place that time inside the json string ?
My friend helped me this. He used this fluentd plugin:
http://docs.fluentd.org/articles/filter-plugin-overview
This is the config:
<filter trackLog>
type record_modifier
<record>
fluentd_time ${Time.now.strftime("%Y-%m-%d %H:%M:%S")}
</record>
</filter>
<match trackLog>
type record_modifier
tag trackLog.finished
</match>
<match trackLog.finished>
type webhdfs
host localhost
port 50070
path /data/trackLog/%Y%m%d_%H
username hdfs
output_include_tag false
remove_prefix trackLog.finished
output_include_time false
buffer_type file
buffer_path /mnt/ramdisk/trackLog
buffer_chunk_limit 4m
buffer_queue_limit 50
flush_interval 5s
</match>