Parse XML in Haskell

Question

I'm trying to get data from a webpage that serves a XML file periodically with stock market quotes (sample data). The structure of the XML is very simple, and is something like this:

<?xml version="1.0"?>
<Contents>
  <StockQuote Symbol="PETR3" Date="21-12-2010" Time="13:20" Price="23.02" />
</Contents>

(it's more than that but this suffices as an example).

I'd like to parse it to a data structure:

 data Quote = Quote { symbol :: String, 
                      date   :: Data.Time.Calendar.Day, 
                      time   :: Data.Time.LocalTime.TimeOfDay,
                      price  :: Float}

I understand more or less how Parsec works (on the level of the Real World Haskell book), and I tried a bit the Text.XML library but all I could develop was a code that worked but is too big for such a simple task and looks like a half baked hack and not the best one could do.

I don't know a lot about parsers and XML (I know basically what I read in the RWH book, I never used parsers before) (I just do statistical and numerical programming, I'm not a computer scientist). Is there a XML parsing library where I could just tell what is the model and extract the information right away, without having to parse each element by hand and without having to parse pure string?

I'm thinking about something like:

  myParser = do cont  <- openXMLElem "Contents"
                quote <- openXMLElem "StockQuote" 
                symb <- getXMLElemField "Symbol"
                date <- getXMLElemField "Date"
                (...) 
                closequote <- closeXMLElem "StockQuote"
                closecont  <- closeXMLElem "Contents"
                return (symb, date)


  results = parse myParser "" myXMLString

where I wouldn't have to deal with the pure string and create the combinators myself (I suck at it).

EDIT: I probably need to read a bit (just enough to get this done the right way) about parsers in general (not only Parsec) and the minimum about XML. Do you guys recomend something?

The real string I have to parse is this:

 stringTest = "<?xml version=\"1.0\"?>\r\n<ComportamentoPapeis><Papel Codigo=\"PETR3\" 
 Nome=\"PETROBRAS ON\" Ibovespa=\"#\" Data=\"05/01/201100:00:00\" 
 Abertura=\"29,80\" Minimo=\"30,31\" Maximo=\"30,67\" Medio=\"30,36\" 
 Ultimo=\"30,45\" Oscilacao=\"1,89\" Minino=\"29,71\"/></ComportamentoPapeis>\r\n"

EDIT2:

I tried the following (readFloat, readQuoteTime, etc... are just functions to read things from strings).

bvspaParser :: (ArrowXml a) => a XmlTree Quote
bvspaParser = hasName "ComportamentoPapeis" /> hasName "Papel" >>> proc x -> do
   (hour,date) <- readQuoteTime ^<< getAttrValue "Data" -< x
   quoteCode   <- getAttrValue "Codigo" -< x
   openPrice   <- readFloat ^<< getAttrValue "Abertura" -< x
   minim       <- readFloat ^<< getAttrValue "Minimo" -< x
   maxim       <- readFloat ^<< getAttrValue "Maximo" -< x
   ultimo      <- readFloat ^<< getAttrValue "Ultimo" -< x
   returnA     -< Quote quoteCode (LocalTime date hour) openPrice minim maxim ultimo

docParser :: String -> IO [Quote]
docParser  str = runX $ readString [] str >>> (parseXmlDocument False) >>> bvspaParser

When I call it in ghci:

*Main> docParser stringTest >>= print
[]

Is anything wrong?

Answer 1

There are plenty of XML libraries written for Haskell that can do the parsing for you. I recommend the library called xml (see http://hackage.haskell.org/package/xml). With it, you can simply write e.g.:

let contents = parseXML source
    quotes   = concatMap (findElements $ simpleName "StockQuote") (onlyElems contents)
    symbols  = map (findAttr $ simpleName "Symbol") quotes
    simpleName s = QName s Nothing Nothing
print symbols

This snippet prints [Just "PETR3"] as a result for your example XML, and it's easy to extend for collecting all the data you need. To write the program in the style you describe you should use the Maybe monad, as the xml lookup functions often return a Maybe String, signaling whether the tag, element or attribute could be found. Also see a related question: Which Haskell XML library to use?

Answer 2

For simple xml parsing, you can't go wrong with tagsoup. http://hackage.haskell.org/package/tagsoup

Answer 3

The following snippet uses xml-enumerator. It leaves date and time as text (parsing those is left as an exercise to the reader):

{-# LANGUAGE OverloadedStrings #-}
import Text.XML.Enumerator.Parse
import Data.Text.Lazy (Text, unpack)

data Quote = Quote { symbol :: Text
                   , date   :: Text
                   , time   :: Text
                   , price  :: Float}
  deriving Show

main = parseFile_ "test.xml" (const Nothing) $ parseContents

parseContents = force "Missing Contents" $ tag'' "Contents" parseStockQuote
parseStockQuote = force "Missing StockQuote" $ flip (tag' "StockQuote") return $ do
    s <- requireAttr "Symbol"
    d <- requireAttr "Date"
    t <- requireAttr "Time"
    p <- requireAttr "Price"
    return $ Quote s d t (read $ unpack p)

Answer 4

I've used Haskell XML Toolbox in the past. Something along the lines of

{-# LANGUAGE Arrows #-}

quoteParser :: (ArrowXml a) => a XmlTree Quote
quoteParser =
    hasName "Contents" /> hasName "StockQuote" >>> proc x -> do
    symbol <- getAttrValue "Symbol" -< x
    date <- readTime defaultTimeLocale "%d-%m-%Y" ^<< getAttrValue "Date" -< x
    time <- readTime defaultTimeLocale "%H:%M" ^<< getAttrValue "Time" -< x
    price <- read ^<< getAttrValue "Price" -< x
    returnA -< Quote symbol date time price

parseQuoteDocument :: String -> IO (Maybe Quote)
parseQuoteDocument xml =
    liftM listToMaybe . runX . single $
    readString [] xml >>> getChildren >>> quoteParser

Answer 5

There are other ways to use this library, but for something simple like this I threw together a sax parser.

import Prelude as P
import Text.XML.Expat.SAX
import Data.ByteString.Lazy as L

parsexml txt = parse defaultParseOptions txt :: [SAXEvent String String]

main = do
  xml <- L.readFile "stockinfo.xml"
  return  $ P.filter stockquoteelement (parsexml xml)

  where
    stockquoteelement (StartElement "StockQuote" attrs) = True
    stockquoteelement _ = False

From there you can figure out where to go. You could also use Text.XML.Expat.Annotated in order to parse it into a structure that is more like what you are looking for above:

parsexml txt = parse defaultParseOptions txt :: (LNode String String, Maybe XMLParseError)

And then use Text.XML.Expat.Proc to surf the structure.