How to parse a WordPress CSV export using Python

2
votes

I need to import content from WordPress into Plone, a Python-based CMS, and I have a dump of the posts table as a huge CSV vanilla file using ";" as a delimiter.

The problem is the standard CSV reader from the csv module is not smart enough to parse the HTML content inside a row (the post_content field).

For instance, when the parser encounters something like <p>&nbsp;</p> it interprets the semicolon as a field delimiter and I end up with more items than fields and with fields with wrong content.

Is there any other option to solve this kind of issues? Processing the row with a regex seems pretty scary to me.

3
pythonwordpresscsvplonetransmogrifier
Hum. Would it be okay if you first converted all the HTML &nbsp; into spaces and then tried csv.reader? - NightShadeQueen
can you add a sample of your input? - Padraic Cunningham

3 Answers

2
votes

After some additional research, I discovered the excel-tab dialect by reading the text of the PEP 0305 (which proposed the addition of the cvs module to Python); this is mentioned in the module documentation, but I haven't noticed at first.

I then re-exported the posts using a tab as a delimiter (\t).

enter image description here

I made a test reading a batch of 1,000 rows and found no errors at all.

1
votes

The CSV module provides the escapechar format parameter, which allows you to escape the delimiter (which you have set to semicolon). If you can provide escapechar='\\' in the call to csv.reader(), you could then replace the character \ in your CSV file with \\, and replace &nbsp; with &nbsp\; (using a text editor's find/replace option).

1
votes

Another option, for smaller sites, could be using pywordpress, a pythonic interface to WordPress XML-RPC API.