Apache beam Dataflow SDK error with example

Question

I'm trying one of the beam google dataflow pipeline examples, but i'm bumping into a exception regarding MapElements and methods SingleFunction / SerializableFunction calls. The code snippet is the following:

static class ParseTableRowJson extends SimpleFunction<String, TableRow> {
    @Override
    public TableRow apply(String input) {
        try {
            return Transport.getJsonFactory().fromString(input, TableRow.class);
        } catch (IOException e) {
            throw new RuntimeException("Failed parsing table row json", e);
        }
    }
}
......
p.apply(TextIO.read().from(options.getInput()))
                .apply(MapElements.via(new ParseTableRowJson()))
                .apply(new ComputeTopSessions(samplingThreshold))
                .apply("Write", 
TextIO.write().withoutSharding().to(options.getOutput()));

The exception in that its an ambiguous call to the methods:

Ambiguous method call. Both
via (SimpleFunction<String, TableRow>) in MapElements and
via (SerializableFunction)             in MapElements match

Has someone else bumped into the same exception and got a way around it?

The full example is in github (https://github.com/apache/beam/blob/master/examples/java/src/main/java/org/apache/beam/examples/complete/TopWikipediaSessions.java).

Thanks,

Fernando

Ben Chambers Ben Chambers · Accepted Answer · 2017-10-31T22:37:42

This seems to have been fixed in the code at HEAD. Specifically, MapElements no longer has two static versions of via. Short-term, you can either install Beam from HEAD or update the example to use ParDo directly by making the ParseTableRowJson a DoFn instead of a SimpleFunction.

Apache beam Dataflow SDK error with example

1 Answers