Not able to join postgres table through apache drill

1
votes

Not able to join multiple tables of postgres database through apache drill. When trying the same below error is coming.

org.apache.drill.common.exceptions.UserRemoteException: SYSTEM ERROR: IllegalStateException: Memory was leaked by query. Memory leaked: (40960) Allocator(op:0:0:6:JdbcSubScan) 1000000/40960/2228224/10000000000 (res/actual/peak/limit) Fragment 0:0 [Error Id: b05fe30e-cc3a-4e7f-b81e-46ecfd1a9466 on INBBRDSSVM300.india.tcs.com:31010] (java.lang.IllegalStateException) Memory was leaked by query. Memory leaked: (40960) Allocator(op:0:0:6:JdbcSubScan) 1000000/40960/2228224/10000000000 (res/actual/peak/limit) org.apache.drill.exec.memory.BaseAllocator.close():492 org.apache.drill.exec.ops.OperatorContextImpl.close():124 org.apache.drill.exec.ops.FragmentContext.suppressingClose():416 org.apache.drill.exec.ops.FragmentContext.close():405 org.apache.drill.exec.work.fragment.FragmentExecutor.closeOutResources():346 org.apache.drill.exec.work.fragment.FragmentExecutor.cleanup():179 org.apache.drill.exec.work.fragment.FragmentExecutor.run():290 org.apache.drill.common.SelfCleaningRunnable.run():38 java.util.concurrent.ThreadPoolExecutor.runWorker():1145 java.util.concurrent.ThreadPoolExecutor$Worker.run():615 java.lang.Thread.run():745

1
apache-drill
share drill version , OS and join query, data types of tables on which join is performed - Dev
apache-drill-1.5.0,Linux - Smruti Ranjan
select f1.* from postgres.table1 f1,postgres.table2 d1 where f1.field_fk = d1.field_skey field_fk -> bigint field_skey -> bigint - Smruti Ranjan
There is an open issue - issues.apache.org/jira/browse/DRILL-4398 - Dev
You can try a work around - create one more plugin postgres1 with same configurations as postgres. Modify your query as select f1.* from postgres.table1 f1,postgres1.table2 d1 where f1.field_fk = d1.field_skey Let me know if this works - Dev

1 Answers

0
votes

Try increasing DRILLBIT_MAX_PROC_MEM which should be higher than DRILL_MAX_DIRECT_MEMORY+DRILL_HEAP