Avoid multiple calls to API using spring webclient with flux

Question

Both those calls are client http api:

Flux<SavingsViewFilter> views = savingsApi.getViewFilterSavings(viewId);
            Flux<Group> groups = groupsApi.getAllGroups();

and return a Flux

requestBodySpec.retrieve().bodyToFlux(returnType);

I need to filter element inside the Flux<Group>, based on values from eacg view.getGroupId()

        return views.flatMap(view ->
                groups
                        .filter(group -> Objects.equals(group.getGroupId(), view.getGroupId()))
                        .flatMap(group -> Flux.just(DepositAccount.builder()
                                .agencyName(group.getGroupName())
                                .settlementAccount(view.getName())
                                .build())));

It is working, but the problem is that it's doing for each view object, one more HTTP request to getAllGroups.

How can I avoid multiple requests to getAllGroups?

Answer 1

The issue is you are subscribing to groups for every element of views.

You can use join to only subscribe to each once, and then join element by element.

views
    //Join fluxes and create tuple for each pair
    .join(groups, s-> Flux.never(),s-> Flux.never(),Tuples::of)
    //Filter out any that don't have matching groupIds
    .filter(t -> t.getT1().getGroupId().equals(t.getT2().getGroupId()))
    //Use map, not flatMap since builder is not blocking
    .map(t -> DepositAccount.builder()
        .agencyName(t.getT2().getGroupName())
        .settlementAccount(t.getT1().getName())
        .build()
     );

Answer 2

Really this is not the correct question and needs to be thought through a little better.

The end result is that you are trying to match the ids for views with the ids for groups and create a join object. This is not the purpose that the Flux API is meant for. In the end it looks like you are trying to do that task with two loops which is a very inefficient way to do that.

It is much better if you investigate alternative approaches.

1) If both group and view and database tables then use SQL to join. Create a new endpoint that is efficient.

2) Put one collection into a map and use it to combine the two entities.

@AllArgsConstructor
@Getter
class View {
    Integer id;
    String view;
}

@AllArgsConstructor
@Getter
class Group {
    Integer id;
    String group;
}

@Data
@AllArgsConstructor
class Combined {
    String view;
    String group;
}

private void run() {
    Flux<View> views = Flux.just(new View(1, "v1"), new View(2, "v2"), new View(3, "v3"));
    Flux<Group> groups = Flux.just(new Group(2, "g2"), new Group(3, "g3"), new Group(4, "g4"));

    views.collectMap(View::getId)
            .flatMapMany(viewMap -> groups.filter(group -> viewMap.containsKey(group.getId()))
                    .map(group ->
                            new Combined(viewMap.get(group.getId()).getView(), group.getGroup()))
            )
            .subscribe(System.out::println);
}