Presto-Glue-EMR integration: presto-cli giving NullPointerException

Question

I am trying to connect my Glue catalog to Presto and Hive in EMR. While running the queries in presto-cli, I am getting NullPointerException whereas the same query succeeds in hive-cli.

Started the cli like below

presto-cli --catalog hive

Exception on executing a query:

Query 20180814_174636_00003_iika5 failed: java.lang.NullPointerException: parameters is null

EMR Configuration looks like this:

[
    {
        "classification": "presto-connector-hive",
        "properties": {
            "hive.metastore": "glue"
        },
        "configurations": []
    },
    {
        "classification": "hive-site",
        "properties": {
            "hive.metastore.client.factory.class": "com.amazonaws.glue.catalog.metastore.AWSGlueDataCatalogHiveClientFactory"
        },
        "configurations": []
    }
]

EMR version: 5.16.0 Presto version: 0.203

Reference Doc: https://docs.aws.amazon.com/emr/latest/ReleaseGuide/emr-presto-glue.html

Debug logs

Query 20180816_060942_00001_m9i52 failed: java.lang.NullPointerException: parameters is null
com.google.common.util.concurrent.UncheckedExecutionException: java.lang.NullPointerException: parameters is null
    at com.google.common.cache.LocalCache$Segment.get(LocalCache.java:2052)
    at com.google.common.cache.LocalCache.get(LocalCache.java:3943)
    at com.google.common.cache.LocalCache.getOrLoad(LocalCache.java:3967)
    at com.google.common.cache.LocalCache$LocalLoadingCache.get(LocalCache.java:4952)
    at com.google.common.cache.LocalCache$LocalLoadingCache.getUnchecked(LocalCache.java:4958)
    at com.facebook.presto.hive.metastore.CachingHiveMetastore.get(CachingHiveMetastore.java:207)
    at com.facebook.presto.hive.metastore.CachingHiveMetastore.getPartitionNamesByParts(CachingHiveMetastore.java:499)
    at com.facebook.presto.hive.metastore.SemiTransactionalHiveMetastore.doGetPartitionNames(SemiTransactionalHiveMetastore.java:467)
    at com.facebook.presto.hive.metastore.SemiTransactionalHiveMetastore.getPartitionNamesByParts(SemiTransactionalHiveMetastore.java:445)
    at com.facebook.presto.hive.HivePartitionManager.getFilteredPartitionNames(HivePartitionManager.java:284)
    at com.facebook.presto.hive.HivePartitionManager.getPartitions(HivePartitionManager.java:146)
    at com.facebook.presto.hive.HiveMetadata.getTableLayouts(HiveMetadata.java:1305)
    at com.facebook.presto.spi.connector.classloader.ClassLoaderSafeConnectorMetadata.getTableLayouts(ClassLoaderSafeConnectorMetadata.java:73)
    at com.facebook.presto.metadata.MetadataManager.getLayouts(MetadataManager.java:346)
    at com.facebook.presto.sql.planner.iterative.rule.PickTableLayout.planTableScan(PickTableLayout.java:203)
    at com.facebook.presto.sql.planner.iterative.rule.PickTableLayout.access$200(PickTableLayout.java:61)
    at com.facebook.presto.sql.planner.iterative.rule.PickTableLayout$PickTableLayoutWithoutPredicate.apply(PickTableLayout.java:186)
    at com.facebook.presto.sql.planner.iterative.rule.PickTableLayout$PickTableLayoutWithoutPredicate.apply(PickTableLayout.java:153)
    at com.facebook.presto.sql.planner.iterative.IterativeOptimizer.transform(IterativeOptimizer.java:168)
    at com.facebook.presto.sql.planner.iterative.IterativeOptimizer.exploreNode(IterativeOptimizer.java:141)
    at com.facebook.presto.sql.planner.iterative.IterativeOptimizer.exploreGroup(IterativeOptimizer.java:104)
    at com.facebook.presto.sql.planner.iterative.IterativeOptimizer.exploreChildren(IterativeOptimizer.java:193)
    at com.facebook.presto.sql.planner.iterative.IterativeOptimizer.exploreGroup(IterativeOptimizer.java:106)
    at com.facebook.presto.sql.planner.iterative.IterativeOptimizer.exploreChildren(IterativeOptimizer.java:193)
    at com.facebook.presto.sql.planner.iterative.IterativeOptimizer.exploreGroup(IterativeOptimizer.java:106)
    at com.facebook.presto.sql.planner.iterative.IterativeOptimizer.optimize(IterativeOptimizer.java:95)
    at com.facebook.presto.sql.planner.LogicalPlanner.plan(LogicalPlanner.java:140)
    at com.facebook.presto.sql.planner.LogicalPlanner.plan(LogicalPlanner.java:129)
    at com.facebook.presto.execution.SqlQueryExecution.doAnalyzeQuery(SqlQueryExecution.java:327)
    at com.facebook.presto.execution.SqlQueryExecution.analyzeQuery(SqlQueryExecution.java:312)
    at com.facebook.presto.execution.SqlQueryExecution.start(SqlQueryExecution.java:268)
    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
    at java.lang.Thread.run(Thread.java:748)
Caused by: java.lang.NullPointerException: parameters is null
    at java.util.Objects.requireNonNull(Objects.java:228)
    at com.facebook.presto.hive.metastore.Partition.<init>(Partition.java:54)
    at com.facebook.presto.hive.metastore.Partition$Builder.build(Partition.java:180)
    at com.facebook.presto.hive.metastore.glue.converter.GlueToPrestoConverter.convertPartition(GlueToPrestoConverter.java:141)
    at com.facebook.presto.hive.metastore.glue.GlueHiveMetastore.lambda$getPartitions$8(GlueHiveMetastore.java:558)
    at java.util.ArrayList.forEach(ArrayList.java:1257)
    at com.facebook.presto.hive.metastore.glue.GlueHiveMetastore.getPartitions(GlueHiveMetastore.java:558)
    at com.facebook.presto.hive.metastore.glue.GlueHiveMetastore.getPartitionNamesByParts(GlueHiveMetastore.java:541)
    at com.facebook.presto.hive.metastore.CachingHiveMetastore.loadPartitionNamesByParts(CachingHiveMetastore.java:504)
    at com.google.common.cache.CacheLoader$FunctionToCacheLoader.load(CacheLoader.java:165)
    at com.google.common.cache.CacheLoader$1.load(CacheLoader.java:188)
    at com.google.common.cache.LocalCache$LoadingValueReference.loadFuture(LocalCache.java:3524)
    at com.google.common.cache.LocalCache$Segment.loadSync(LocalCache.java:2273)
    at com.google.common.cache.LocalCache$Segment.lockedGetOrLoad(LocalCache.java:2156)
    at com.google.common.cache.LocalCache$Segment.get(LocalCache.java:2046)
    ... 33 more

Answer 1

Seems like presto 0.203 has this bug, I faced it too, I switched to a newer version and it worked.

In the time I am writing this answer, EMR 5.17 is released and it has presto 0.206 which has this problem resolved.