I am executing a hive query and its failing. I have used the below set parameters.
SET hive.execution.engine=tez ;
SET tez.am.resource.memory.mb=4096;
SET hive.tez.container.size=4096;
set hive.compute.query.using.stats=true;
set hive.stats.fetch.column.stats=true;
set hive.stats.fetch.partition.stats=true;
SET hive.execution.engine=tez ;
SET hive.vectorized.execution.enabled=false;
SET hive.auto.convert.join=false;
I am getting the error.
Vertex failed, vertexName=Map 38,
vertexId=vertex_1479183638971_463139_1_22, diagnostics=[Task failed,
taskId=task_1479183638971_463139_1_22_000036,
diagnostics=[TaskAttempt 0 failed, info=[Container container_1479183638971_463139_01_000556 finished with diagnostics set to [Container failed, exitCode=-104.
Container [pid=31713,containerID=container_1479183638971_463139_01_000556] is running beyond physical memory limits. Current usage: 4.2 GB of 4 GB physical memory used;
So that I have tried with parallel option the below parameters still I am getting some different error.
SET hive.execution.engine=tez ;
SET tez.am.resource.memory.mb=4096;
SET hive.tez.container.size=4096;
set hive.compute.query.using.stats=true;
set hive.stats.fetch.column.stats=true;
set hive.stats.fetch.partition.stats=true;
SET hive.execution.engine=tez ;
SET hive.vectorized.execution.enabled=false;
SET hive.auto.convert.join=false;
SET tez.am.java.opts=-server -Xmx3276m -Djava.net.preferIPv4Stack=true -XX:+UseNUMA -XX:+UseParallelGC;
SET hive.tez.java.opts=-server -Xmx3276m -Djava.net.preferIPv4Stack=true -XX:+UseNUMA -XX:+UseParallelGC;
I am getting the error.
Status: Failed
Counters limit exceeded: Too many counters: 2001 max=2000 FAILED: Execution Error, return code 2 from org.apache.hadoop.hive.ql.exec.tez.TezTask. Counters limit exceeded: Too many counters: 2001 max=2000
Please help me. How to make my query to execute.
Thanks in Advance.
