Chained apply return type seems determined at compile time

Question

Can someone please explain the following lines of code to me:

import org.apache.beam.sdk.Pipeline;
[..]
Pipeline p = Pipeline.create(options);
p.apply(TextIO.read().from("gs://apache-beam-samples/shakespeare/*"))
.apply("ExtractWords", ParDo.of(new DoFn<String, String>() {/* etc */}

What I don't get is why this compiles. Apply method in Pipeline returns a T extends POutput. Interface POutput does not have any apply method.

In this case, it just so happens that TextIO.read().from(...) returns a PCollection as POutput, and THAT has an apply method.

But as far as the Pipeline contract goes, we know that just a POutput will be returned. So how does the compiler get to check the type of the argument passed to the first apply? From what I remember from when I was coding Java, that would be seen only at runtime.

that would contradict what I think I knew - that at compile time T is replaced with it's bounded type - POutput in our case — Andrei

jkff jkff · Accepted Answer · 2017-12-12T01:32:32

The signature is:

public <OutputT extends POutput> OutputT apply(
    String name, PTransform<? super PBegin, OutputT> root) {
  ...
}

This means that, for any possible type OutputT that extends POutput, this method is applicable to (String, PTransform<? super PBegin, OutputT>) and returns OutputT.

TextIO.read() returns a TextIO.Read which extends PTransform<PBegin, PCollection<String>>; it matches PTransform<? super PBegin, OutputT>:

The type PBegin is trivially a supertype of PBegin
The type PCollection<String> extends POutput

So, the .apply() method is applicable, with the substitution OutputT = PCollection<String>. After substituting the actual value of OutputT into the signature of .apply(), the return value is PCollection<String>.

What you're saying (OutputT being replaced with POutput) happens during type erasure - the type-erased signature of Pipeline.apply() is POutput apply(String, PTransform); and that is its signature in the compiled bytecode of Pipeline as far as the JVM is concerned when actually executing it. However, type erasure happens after type checking, not before - otherwise almost all Java programs using generics would fail to compile.

This is exactly the same thing that happens when you invoke Arrays.asList(1, 2, 3) - it returns a List<Integer> rather than List<Object> or List, even though the type-erased signature is List asList(Object[]).

Chained apply return type seems determined at compile time

1 Answers