Apache Ignite Hadoop Accelerator MapReduce Jobs are not in the JobHistory Server

0
votes

I'm currently running Apache Ignite Hadoop accelerator for MapReduce. The jobs run, but I am unable to see them in the JobHistoryServer. I wouldn't expect to see the jobs in Yarn's Resource Manager (and don't).

I'm running my MapReduce jobs like hadoop --config path/to/config/ jar path/to/jar ....

In the mapred-site.xml, I've added 

<property>
  <name>mapreduce.framework.name</name>
  <value>ignite</value>
</property>
<property>
  <name>mapreduce.jobtracker.address</name>
  <value>[your_host]:11211</value>
</property>

My mapreduce.jobhistory.* settings have not been changed.

In the core-site.xml I've added

<property>
  <name>fs.default.name</name>
  <value>igfs://igfs@/</value>
</property>
<property>
  <name>fs.igfs.impl</name>
  <value>org.apache.ignite.hadoop.fs.v1.IgniteHadoopFileSystem</value>
</property>
<property>
  <name>fs.AbstractFileSystem.igfs.impl</name>
  <value>org.apache.ignite.hadoop.fs.v2.IgniteHadoopFileSystem</value>
</property>

I've also added ignite-core-1.6.0.jar, ignite-hadoop-1.6.0.jar, and ignite-shmem-1.0.0.jar to the $HADOOP_HOME path. Similarly, I've exported HADOOP_HOME, HADOOP_COMMON_HOME, HADOOP_HDFS_HOME, and HADOOP_MAPRED_HOME.

Is this functionality not supported by Ignite or am I doing something wrong?

Also, is there a way to track the MapReduce job running on Ignite?

1
hadoopmapreduceignite

1 Answers

0
votes

Currently Ignite does not integrate anyhow with Hadoop History server, issue https://issues.apache.org/jira/browse/IGNITE-3766 requests that.