Connecting to bigtable from cloud dataflow

Question

I am trying to connect to BigTable from beam api. I am constantly facing an error :

Exception in thread "BigtableSession-startup-1" 
java.lang.IllegalArgumentException: Jetty ALPN/NPN has not been 
properly configured.

Here os the POM i am using : http://maven.apache.org/xsd/maven-4.0.0.xsd"> 4.0.0

<groupId>com.dataflow.example</groupId>
<artifactId>dataflowTest</artifactId>
<version>0.0.1-SNAPSHOT</version>
<packaging>jar</packaging>

<name>dataflowTest</name>
<url>http://maven.apache.org</url>

<properties>
    <project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>
    <maven-compiler-plugin.version>3.6.2</maven-compiler-plugin.version>
    <maven-exec-plugin.version>1.6.0</maven-exec-plugin.version>
    <beam.version>2.1.0</beam.version>
    <bigtable.version>1.0.0</bigtable.version>
    <slf4j.version>1.7.21</slf4j.version>
    <bigtable.table>Dataflow_test</bigtable.table>
    <pubsubTopic>projects/${bigtable.projectID}/topics/shakes</pubsubTopic>
    <bigtable.hbase.version>${bigtable.version}</bigtable.hbase.version>
</properties>

<build>
    <pluginManagement>
        <plugins>
            <plugin>
                <groupId>org.apache.maven.plugins</groupId>
                <artifactId>maven-compiler-plugin</artifactId>
                <version>3.5.1</version>
                <configuration>
                    <source>1.8</source>
                    <target>1.8</target>
                </configuration>
            </plugin>
        </plugins>
    </pluginManagement>
    <extensions>
        <!-- Use os-maven-plugin to initialize the "os.detected" properties -->
        <extension>
            <groupId>kr.motd.maven</groupId>
            <artifactId>os-maven-plugin</artifactId>
            <version>1.4.0.Final</version>
        </extension>
    </extensions>
    <plugins>
        <plugin>
            <groupId>org.apache.maven.plugins</groupId>
            <artifactId>maven-compiler-plugin</artifactId>
            <configuration>
                <source>1.8</source>
                <target>1.8</target>
            </configuration>
        </plugin>
        <plugin>
            <groupId>org.apache.maven.plugins</groupId>
            <artifactId>maven-shade-plugin</artifactId>
            <executions>
                <execution>
                    <id>bundle-and-repackage</id>
                    <phase>package</phase>
                    <goals>
                        <goal>shade</goal>
                    </goals>
                    <configuration>
                        <filters>
                            <filter>
                                <artifact>*:*</artifact>
                                <excludes>
                                    <exclude>META-INF/*.SF</exclude>
                                    <exclude>META-INF/*.DSA</exclude>
                                    <exclude>META-INF/*.RSA</exclude>
                                </excludes>
                            </filter>
                        </filters>
                        <transformers>
                            <transformer
                                implementation="org.apache.maven.plugins.shade.resource.ServicesResourceTransformer" />
                        </transformers>
                    </configuration>
                </execution>
            </executions>
        </plugin>
    </plugins>
</build>


<dependencies>
    <dependency>
        <groupId>org.apache.beam</groupId>
        <artifactId>beam-runners-google-cloud-dataflow-java</artifactId>
        <version>2.2.0</version>
    </dependency>
    <dependency>
        <groupId>org.apache.beam</groupId>
        <artifactId>beam-sdks-java-extensions-google-cloud-platform-core</artifactId>
        <version>2.2.0</version>
        <exclusions>
            <exclusion>
                <groupId>io.netty</groupId>
            </exclusion>
        </exclusions>
    </dependency>
    <!-- https://mvnrepository.com/artifact/org.mortbay.jetty.alpn/alpn-boot -->
    <!-- https://mvnrepository.com/artifact/io.netty/netty-tcnative-boringssl-static -->
    <dependency>
        <groupId>io.netty</groupId>
        <artifactId>netty-tcnative-boringssl-static</artifactId>
        <version>2.0.7.Final</version>
    </dependency>

    <dependency>
        <groupId>com.google.cloud.bigtable</groupId>
        <artifactId>bigtable-hbase-beam</artifactId>
        <version>${bigtable.version}</version>
        <exclusions>
            <exclusion>
                <groupId>io.netty</groupId>
            </exclusion>
        </exclusions>
    </dependency>

    <dependency>
        <groupId>com.google.cloud.dataflow</groupId>
        <artifactId>google-cloud-dataflow-java-sdk-all</artifactId>
        <version>2.2.0</version>
        <exclusions>
            <exclusion>
                <groupId>io.netty</groupId>
            </exclusion>
        </exclusions>
    </dependency>
    <dependency>
        <groupId>mysql</groupId>
        <artifactId>mysql-connector-java</artifactId>
        <version>6.0.6</version>
    </dependency>
    <dependency>
        <groupId>junit</groupId>
        <artifactId>junit</artifactId>
        <version>3.8.1</version>
        <scope>test</scope>
    </dependency>
</dependencies>

And the code base where i am getting this error :

try (BigtableSession session = new BigtableSession(optionsBuilder
        .setCredentialOptions(CredentialOptions.credential(options.as(GcpOptions.class).getGcpCredential())).build())) {

      BigtableTableAdminClient tableAdminClient = session.getTableAdminClient();

I have tried fork26, fork13, 2.0.7.Final and various other version of netty.

Answer 1

Try this link - https://github.com/grpc/grpc-java/blob/master/SECURITY.md#troubleshooting . You might need to exclude all the netty dependencies from each of the dependencies and add netty as a dependency separately to avoid the conflict. I had faced a similar issue while using Spark and Big Table together.

Answer 2

There seems to be a conflict between the versions of netty that you are using. Can you use something like depedencymanagement to resolve this.

There might be some other versions of netty that is being used in the some other dependencies, try removing the other versions.