Apache phoenix concurrent queries failing with exception

Question

We are trying a bunch of operations like SELECT -> store row key into a collection ->then split the collection into each worker thread -> Each thread again created connection using phoenix jdbc -> perform SELECT then depending on the result UPSERT into a different phoenix table.

I am using ExecutorService with a fixed thread pool of 4 I am seeing exceptions as below.

org.apache.phoenix.exception.PhoenixIOException: org.apache.phoenix.exception.PhoenixIOException: The system cannot find the path specified
       at org.apache.phoenix.util.ServerUtil.parseServerException(ServerUtil.java:108)
       at org.apache.phoenix.iterate.BaseResultIterators.getIterators(BaseResultIterators.java:538)
       at org.apache.phoenix.iterate.ConcatResultIterator.getIterators(ConcatResultIterator.java:50)
       at org.apache.phoenix.iterate.ConcatResultIterator.currentIterator(ConcatResultIterator.java:97)
       at org.apache.phoenix.iterate.ConcatResultIterator.next(ConcatResultIterator.java:117)
       at org.apache.phoenix.jdbc.PhoenixResultSet.next(PhoenixResultSet.java:764)
       at com.vonage.test.PopulateStagingGWCDRWorker.run(MyCode.java:74)
       at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
       at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
       at java.lang.Thread.run(Thread.java:745)
Caused by: java.util.concurrent.ExecutionException: org.apache.phoenix.exception.PhoenixIOException: The system cannot find the path specified
       at java.util.concurrent.FutureTask.report(FutureTask.java:122)
       at java.util.concurrent.FutureTask.get(FutureTask.java:206)
       at org.apache.phoenix.iterate.BaseResultIterators.getIterators(BaseResultIterators.java:534)
       ... 8 more
Caused by: org.apache.phoenix.exception.PhoenixIOException: The system cannot find the path specified
       at org.apache.phoenix.util.ServerUtil.parseServerException(ServerUtil.java:108)
       at org.apache.phoenix.iterate.SpoolingResultIterator.<init>(SpoolingResultIterator.java:122)
       at org.apache.phoenix.iterate.SpoolingResultIterator.<init>(SpoolingResultIterator.java:73)
       at org.apache.phoenix.iterate.SpoolingResultIterator$SpoolingResultIteratorFactory.newIterator(SpoolingResultIterator.java:67)
       at org.apache.phoenix.iterate.ChunkedResultIterator.<init>(ChunkedResultIterator.java:92)
       at org.apache.phoenix.iterate.ChunkedResultIterator$ChunkedResultIteratorFactory.newIterator(ChunkedResultIterator.java:72)
       at org.apache.phoenix.iterate.ParallelIterators$1.call(ParallelIterators.java:92)
       at org.apache.phoenix.iterate.ParallelIterators$1.call(ParallelIterators.java:83)
       at java.util.concurrent.FutureTask.run(FutureTask.java:266)
       ... 3 more
Caused by: java.io.IOException: The system cannot find the path specified
       at java.io.WinNTFileSystem.createFileExclusively(Native Method)
       at java.io.File.createTempFile(File.java:2024)
       at org.apache.commons.io.output.DeferredFileOutputStream.thresholdReached(DeferredFileOutputStream.java:176)
       at org.apache.phoenix.iterate.SpoolingResultIterator$1.thresholdReached(SpoolingResultIterator.java:98)
       at org.apache.commons.io.output.ThresholdingOutputStream.checkThreshold(ThresholdingOutputStream.java:224)
       at org.apache.commons.io.output.ThresholdingOutputStream.write(ThresholdingOutputStream.java:92)
       at java.io.DataOutputStream.writeByte(DataOutputStream.java:153)
       at org.apache.hadoop.io.WritableUtils.writeVLong(WritableUtils.java:273)
       at org.apache.hadoop.io.WritableUtils.writeVInt(WritableUtils.java:253)
       at org.apache.phoenix.util.TupleUtil.write(TupleUtil.java:146)
       at org.apache.phoenix.iterate.SpoolingResultIterator.<init>(SpoolingResultIterator.java:107)
       ... 10 more
enter code here

But If I am using a pool size of 2 or less it works fine. I was wondering if there is a property at the client side that can be changed ?

This can be closed. Adding the value for phoenix.spool.directory in the hbase-site.xml fixes this. — hopeIsTheonlyWeapon

Indrajeet Gour Indrajeet Gour · Accepted Answer · 2017-08-15T11:54:23

I my case I solved this by using below dependency in pom.xml

<dependency>
            <groupId>org.apache.hbase</groupId>
            <artifactId>hbase-protocol</artifactId>
            <version>1.1.11</version>
</dependency>

Just to update you i am having Hbase version : 1.1 and phoenix is at 4.7

Apache phoenix concurrent queries failing with exception

2 Answers