Merge two observables in angular for CSV to JSON conversion

Question

I have a csv file and I want to conver it into a JSON.

I' m reading the CSV file with HttpClient and then I'm using csvToJson to convert it.

This test code works:

this.httpClient
      .get('assets/csv/results.csv', { responseType: 'text' })
      .subscribe(
        (data) => {
          csv()
            .fromString(data)
            .subscribe((jsonObj) => console.log(jsonObj));
        }
      );

But when I try to merge the two observables to create a function:

  convert() {
    this.httpClient
      .get('assets/csv/results.csv', { responseType: 'text' })
      .pipe(switchMap((d) => csv().fromString(d)))
      .subscribe((c) => console.dir(c));
  }

I get this error:

core.js:4352 ERROR TypeError: You provided an invalid object where a stream was expected. You can provide an Observable, Promise, Array, or Iterable.
    at subscribeTo (subscribeTo.js:27)
    at innerSubscribe (innerSubscribe.js:69)
    at SwitchMapSubscriber._innerSub (switchMap.js:44)
    at SwitchMapSubscriber._next (switchMap.js:34)
    at SwitchMapSubscriber.next (Subscriber.js:49)
    at MapSubscriber._next (map.js:35)
    at MapSubscriber.next (Subscriber.js:49)
    at FilterSubscriber._next (filter.js:33)
    at FilterSubscriber.next (Subscriber.js:49)
    at MergeMapSubscriber.notifyNext (mergeMap.js:70)
    at SimpleInnerSubscriber._next (innerSubscribe.js:10)
    at SimpleInnerSubscriber.next (Subscriber.js:49)
    at XMLHttpRequest.onLoad (http.js:1678)
    at ZoneDelegate.invokeTask (zone-evergreen.js:399)
    at Object.onInvokeTask (core.js:27474)
    at ZoneDelegate.invokeTask (zone-evergreen.js:398)

Update

I've created a stackblitz to experiment.

In the stackblitz I get an error I don't get on localhost:

Error in src/app/csv-2-json.service.ts (18:24)
This expression is not callable.
Type '{ default: (param?: Partial<CSVParseParam>, options?: any) => Converter; }' has no call signatures.

In the library source code I see that the fromString() returns a Converter that implements PromiseLike<any[]> so I thought it should work.

What I'm doing wrong?

Answer 1

I think the issue lays with the csvtojson package.

First of all, if you import it via import * as csv from "csvtojson", what you get in the csv variable is an Object which has 4 properties, 'csv' 'Converter' and 'default', each pointing to a function. So the error that you get in the stackblitz, Type '{ default: (param?: Partial<CSVParseParam>, options?: any) => Converter; }' has no call signatures., means that you are trying to treat an Object as a function and invoke it. But you can not invoke an Object in Javascript.

You can make a step forward by retrieving the 'csv' function out of the csv' Object, like this const csvFunc = csv["csv"]. Now in csvFunc` you have a function you can invoke.

But at this point we step into another issue. if we do const csvInvocationResult = csvFunc().fromString(d) what we get is not a standard Promise, but an Object which accepts the then method, just like a regular Promise. Therefore Object.getPrototypeOf.then(jsonFroCsv => // do something) actually works.

Unfortunately in the Stackbliz I can not navigate through the prototype chain (I get an error while trying to execute Object.getPrototypeOf) so I do not know where this object inherits from. Anyways, not being a real Promise, the switchMap operator does not work.

I suggest you dig into csvtojson library to see how it works and see whether you can adjust it to return a real Promise or to find a way to convert the result it returns to a real Promise.

This answer does not resolve your issue but I hope helps cast some light into it.

Answer 2

Error

If we read the error message the following part is essential: You provided an invalid object where a stream was expected

Explanation

Your code returns an object somewhere in your pipe where instead an observable was expected. I expect this to happen in your switchMap. I have not worked with the given library but I expect csv().fromString(d) to return an object and not an observable.

Solution

I expect your code to work if you change the object to an observable by creating one via of.

switchMap(d => of(csv().fromString(d)))

Possible better solution

In my opinion the switchMap is nonsence in your case.

1. If you want to stick to the above code you can just use a map instead:

map(d => csv().fromString(d))

2. If you want to merge all the previous incoming data use scan to accumulate all the previous values. Then afterwards you can use map on all your previous data

scan((acc, curr) => ([...acc, ...curr]), [])
// I don't know how to accumulate your data as I did not work with the library. I now pushed all the values to an array