0
votes

I'm using the Python lxml library to parse my xml, but I'm having a hard time parsing one specific text. Checkout the following code:

>>> print type(raw_text_xml)
<type 'unicode'>
>>> from lxml import etree
>>> article_xml_root = etree.fromstring(raw_text_xml, parser)
Traceback (most recent call last):
  File "<input>", line 1, in <module>
    article_xml_root = etree.fromstring(raw_text_xml, parser)
  File "lxml.etree.pyx", line 3032, in lxml.etree.fromstring (src/lxml/lxml.etree.c:68121)
  File "parser.pxi", line 1786, in lxml.etree._parseMemoryDocument (src/lxml/lxml.etree.c:102470)
  File "parser.pxi", line 1667, in lxml.etree._parseDoc (src/lxml/lxml.etree.c:101229)
  File "parser.pxi", line 1035, in lxml.etree._BaseParser._parseUnicodeDoc (src/lxml/lxml.etree.c:96139)
  File "parser.pxi", line 582, in lxml.etree._ParserContext._handleParseResultDoc (src/lxml/lxml.etree.c:91290)
  File "parser.pxi", line 683, in lxml.etree._handleParseResult (src/lxml/lxml.etree.c:92476)
  File "parser.pxi", line 622, in lxml.etree._raiseParseError (src/lxml/lxml.etree.c:91772)
XMLSyntaxError: Start tag expected, '<' not found, line 1, column 1

so it says the first character is not a <, which by inspection is true:

>>> print raw_text_xml[:20]
ďťż<?xml version="1.

it has 3 weird characters in front of the xml. So to clean these I tried the following:

>>> article_xml_root = etree.fromstring(raw_text_xml[3:], parser)
Traceback (most recent call last):
  File "<input>", line 1, in <module>
    article_xml_root = etree.fromstring(raw_text_xml[3:], parser)
  File "lxml.etree.pyx", line 3032, in lxml.etree.fromstring (src/lxml/lxml.etree.c:68121)
  File "parser.pxi", line 1781, in lxml.etree._parseMemoryDocument (src/lxml/lxml.etree.c:102435)
ValueError: Unicode strings with encoding declaration are not supported. Please use bytes input or XML fragments without declaration.

And now it suddenly complains about it being a unicode string with encoding declaration, while if you look all the way up to my first line of code, it was Unicode all along.

Does anybody know why after slicing it suddenly gives a whole different error? And most importantly, does anybody know how I can solve this?

3

3 Answers

3
votes

why after slicing it suddenly gives a whole different error?

Because after the slicing the first error vanishes and the parsing can progress until the second one is found.

And most importantly, does anybody know how I can solve this?

Maybe the error message is right (it happens) and you can solve it by converting the unicode to bytes. I guess that's better than removing the encoding declaration.

raw_text_xml.encode('utf8')

Or instead of 'utf8' whatever encoding is declared in the xml fragment.

2
votes

The first error was caused by wrong characters. Once you have fixed it, you fall in second which is that your raw_text_xml is unicode.

You can know what will be a proper encoding (ASCII, latin1, utf8, ...). I cannot without seeing the actual content.

Assuming it is the content of encoding variable, you should be able to do:

article_xml_root = etree.fromstring(raw_text_xml.encode(encoding), parser)

(but I strongly advice you to first control what shows print raw_text_xml[3:160] ...)

0
votes

Where ever you decoded that original Unicode, it was done incorrectly. It looks like iso-8859-2, where it was originally UTF-8 with BOM signature. The following backs out the bad decoding and re-decodes correctly:

>>> s.encode('iso-8859-2').decode('utf-8-sig')
'<?xml version="1.'