Parsing XML response using 'servant-client' and 'servant-xml'

1
votes

I want to parse an API response into a data type using servant-client, servant-xml and xmlbf libraries.

This is an example API response

<GoodreadsResponse>
   <Request>
      <authentication>true</authentication>
      <key>api_key</key>
      <method>search_index</method>
   </Request>
   <search>
      <query>Ender's Game</query>
      <results-start>1</results-start>
      <results-end>20</results-end>
   </search>
</GoodreadsResponse>

and this is the data type I want to parse it into

data GoodreadsRequest = 
        GoodreadsRequest { authentication :: Text
                         , key            :: Text
                         , method         :: Text
                         }


data GoodreadsSearch = 
        GoodreadsSearch { query        :: Text
                        , resultsStart :: Int
                        , resultsEnd   :: Int
                        }


data GoodreadsResponse = 
        GoodreadsResponse { goodreadsRequest :: GoodreadsRequest
                          , goodreadsSearch  :: GoodreadsSearch
                          }

This is the servant API type I want to use it with

type API
  = "search" :> "index.xml" :> QueryParam "key" Key :> QueryParam "q" Query :> Get '[XML] GoodreadsResponse

which constructs an endpoint like this 

https://www.goodreads.com/search/index.xml?key=api_key&q=Ender%27s+Game

and after writing the rest of the scaffolding code (clientM, baseURL, client environment, etc), the error I get is 

No instance for (FromXml GoodreadsResponse) arising from a use of 'client'

Writing 

instance FromXml GoodreadsResponse where
    fromXml = undefined

suppresses the error so I think I'm on the right track, but I don't know how to go about writing the parser.

Edit: Result from a different end-point that contains a list of 'works'

<GoodreadsResponse>
   <Request>
      <authentication>true</authentication>
      <key>api_key</key>
      <method>search_index</method>
   </Request>
   <search>
      <query>Ender's Game</query>
      <results-start>1</results-start>
      <results-end>20</results-end>
      <results>
            <work>
                <id type="integer">2422333</id>
                <average_rating>4.30</average_rating>
                <best_book type="Book">
                    <id type="integer">375802</id>
                    <title>Ender's Game (Ender's Saga, #1)</title>
                </best_book>
            </work>
            <work>
                <id type="integer">4892733</id>
                <average_rating>2.49</average_rating>
                <best_book type="Book">
                    <id type="integer">44687</id>
                    <title>Enchanters' End Game (The Belgariad, #5)</title>
                </best_book>
            </work>
            <work>
                <id type="integer">293823</id>
                <average_rating>2.30</average_rating>
                <best_book type="Book">
                    <id type="integer">6393082</id>
                    <title>Ender's Game, Volume 1: Battle School (Ender's Saga)</title>
                 </best_book>
            </work>
      </results>
   </search>
</GoodreadsResponse>

to be parsed into

data GoodreadsResponse = 
        GoodreadsResponse { goodreadsRequest :: GoodreadsRequest
                          , goodreadsSearch  :: GoodreadsSearch
                          }

data GoodreadsRequest = 
        GoodreadsRequest { authentication :: Text
                         , key            :: Text
                         , method         :: Text
                         }

data GoodreadsSearch = 
        GoodreadsSearch { query        :: Text
                        , resultsStart :: Int
                        , resultsEnd   :: Int
                        , results      :: GoodreadsSearchResults
                        }

data GoodreadsSearchResults = GooreadsSearchResults { works :: [Work] }

data Work = Work { workID               :: Int
                 , workAverageRating    :: Double
                 , workBestMatchingBook :: Book
                 }

data Book = Book { bookID    :: Int
                 , bookTitle :: Text
                 }
1
xmlhaskellservant
Re <id type="integer">, what else can type ever be? Can you add examples? - Joseph Sible-Reinstate Monica
Looking at several API responses, it seems like type can be either integer or Book. - atis
I've abridged both occurrence in that snippet. - atis
With your latest edit, do you just want best_book to be ignored? - Joseph Sible-Reinstate Monica
No sorry I was still editing. Should be fixed now. - atis

1 Answers

1
votes

Wow, there's no examples or predefined instances in xmlbf, and its documentation also has multiple mistakes. Anyway, after playing with it for a bit, it looks like this is how you do it:

{-# LANGUAGE OverloadedStrings #-}

import Data.Text.Lazy (unpack)
import Text.Read (readEither)
import Xmlbf

instance FromXml GoodreadsRequest where
  fromXml = pElement "Request" $ do
    a <- pElement "authentication" pText
    k <- pElement "key" pText
    m <- pElement "method" pText
    pure GoodreadsRequest{ authentication = a, key = k, method = m }

instance FromXml GoodreadsSearch where
  fromXml = pElement "search" $ do
    q <- pElement "query" pText
    s <- pElement "results-start" pText
    s' <- either fail return . readEither $ unpack s
    e <- pElement "results-end" pText
    e' <- either fail return . readEither $ unpack e
    pure GoodreadsSearch{ query = q, resultsStart = s', resultsEnd = e' }

instance FromXml GoodreadsResponse where
  fromXml = pElement "GoodreadsResponse" $ do
    r <- fromXml
    s <- fromXml
    pure GoodreadsResponse{ goodreadsRequest = r, goodreadsSearch = s }

And here it is working with your example XML:

GHCi, version 8.8.2: https://www.haskell.org/ghc/  :? for help
Prelude> :l Main.hs
[1 of 1] Compiling Main             ( Main.hs, interpreted )
Ok, one module loaded.
*Main> :set -XOverloadedStrings
*Main> import Xmlbf.Xeno
*Main Xmlbf.Xeno> fromRawXml "<GoodreadsResponse>\n   <Request>\n      <authentication>true</authentication>\n      <key>api_key</key>\n      <method>search_index</method>\n   </Request>\n   <search>\n      <query>Ender's Game</query>\n      <results-start>1</results-start>\n      <results-end>20</results-end>\n   </search>\n</GoodreadsResponse>" >>= runParser fromXml :: Either String GoodreadsResponse
Right (GoodreadsResponse {goodreadsRequest = GoodreadsRequest {authentication = "true", key = "api_key", method = "search_index"}, goodreadsSearch = GoodreadsSearch {query = "Ender's Game", resultsStart = 1, resultsEnd = 20}})
*Main Xmlbf.Xeno>

Edit: Here's how you use it on lists, with your other endpoint:

{-# LANGUAGE OverloadedStrings #-}

import Control.Applicative (Alternative(many))
import Data.Text.Lazy (unpack)
import Text.Read (readEither)
import Xmlbf

instance FromXml GoodreadsResponse where
  fromXml = pElement "GoodreadsResponse" $ do
    r <- fromXml
    s <- fromXml
    pure GoodreadsResponse{ goodreadsRequest = r, goodreadsSearch = s }

instance FromXml GoodreadsRequest where
  fromXml = pElement "Request" $ do
    a <- pElement "authentication" pText
    k <- pElement "key" pText
    m <- pElement "method" pText
    pure GoodreadsRequest{ authentication = a, key = k, method = m }

instance FromXml GoodreadsSearch where
  fromXml = pElement "search" $ do
    q <- pElement "query" pText
    s <- pElement "results-start" pText
    s' <- either fail return . readEither $ unpack s
    e <- pElement "results-end" pText
    e' <- either fail return . readEither $ unpack e
    r <- fromXml
    pure GoodreadsSearch{ query = q, resultsStart = s', resultsEnd = e', results = r }

instance FromXml GoodreadsSearchResults where
  fromXml = pElement "results" $ do
    w <- many fromXml
    pure GooreadsSearchResults{ works = w }

instance FromXml Work where
  fromXml = pElement "work" $ do
    i <- pElement "id" pText -- the type attribute is ignored
    i' <- either fail return . readEither $ unpack i
    r <- pElement "average_rating" pText
    r' <- either fail return . readEither $ unpack r
    b <- fromXml
    pure Work{ workID = i', workAverageRating = r', workBestMatchingBook = b }

instance FromXml Book where
  fromXml = pElement "best_book" $ do -- the type attribute is ignored
    i <- pElement "id" pText -- the type attribute is ignored
    i' <- either fail return . readEither $ unpack i
    t <- pElement "title" pText
    pure Book{ bookID = i', bookTitle = t }

And the result:

GHCi, version 8.8.2: https://www.haskell.org/ghc/  :? for help
Prelude> :l Main.hs
[1 of 1] Compiling Main             ( Main.hs, interpreted )
Ok, one module loaded.
*Main> :set -XOverloadedStrings
*Main> import Xmlbf.Xeno
*Main Xmlbf.Xeno> fromRawXml "<GoodreadsResponse>\n   <Request>\n      <authentication>true</authentication>\n      <key>api_key</key>\n      <method>search_index</method>\n   </Request>\n   <search>\n      <query>Ender's Game</query>\n      <results-start>1</results-start>\n      <results-end>20</results-end>\n      <results>\n            <work>\n                <id type=\"integer\">2422333</id>\n                <average_rating>4.30</average_rating>\n                <best_book type=\"Book\">\n                    <id type=\"integer\">375802</id>\n                    <title>Ender's Game (Ender's Saga, #1)</title>\n                </best_book>\n            </work>\n            <work>\n                <id type=\"integer\">4892733</id>\n                <average_rating>2.49</average_rating>\n                <best_book type=\"Book\">\n                    <id type=\"integer\">44687</id>\n                    <title>Enchanters' End Game (The Belgariad, #5)</title>\n                </best_book>\n            </work>\n            <work>\n                <id type=\"integer\">293823</id>\n                <average_rating>2.30</average_rating>\n                <best_book type=\"Book\">\n                    <id type=\"integer\">6393082</id>\n                    <title>Ender's Game, Volume 1: Battle School (Ender's Saga)</title>\n                 </best_book>\n            </work>\n      </results>\n   </search>\n</GoodreadsResponse>" >>= runParser fromXml :: Either String GoodreadsResponse
Right (GoodreadsResponse {goodreadsRequest = GoodreadsRequest {authentication = "true", key = "api_key", method = "search_index"}, goodreadsSearch = GoodreadsSearch {query = "Ender's Game", resultsStart = 1, resultsEnd = 20, results = GooreadsSearchResults {works = [Work {workID = 2422333, workAverageRating = 4.3, workBestMatchingBook = Book {bookID = 375802, bookTitle = "Ender's Game (Ender's Saga, #1)"}},Work {workID = 4892733, workAverageRating = 2.49, workBestMatchingBook = Book {bookID = 44687, bookTitle = "Enchanters' End Game (The Belgariad, #5)"}},Work {workID = 293823, workAverageRating = 2.3, workBestMatchingBook = Book {bookID = 6393082, bookTitle = "Ender's Game, Volume 1: Battle School (Ender's Saga)"}}]}}})
*Main Xmlbf.Xeno>

The new key concept in this one is Control.Applicative.many. It keeps running an Alternative until it fails, and then puts all of the successful results into a list. In this case, that means repeating fromXml :: Parser Work until it starts to fail (hopefully because there's no <work>s left). Note that there's one flaw in how many works in this context (IMO, because xmlbf's parser interface isn't very good), namely that a malformed <work> element will just cause everything from it through </results> to be ignored, instead of the error bubbling up. You could use slightly more complicated code involving pChildren to fix that if you want.