Moving chunks of data in a file with awk

Question

I'm moving my bookmarks from kippt.com to pinboard.in.

I exported my bookmarks from Kippt and for some reason, they were storing tags (preceded by #) and description within the same field. Pinboard keeps tags and description separated.

This is what a Kippt bookmark looks like after export:

<DT><A HREF="http://www.example.org/" ADD_DATE="1412337977" LIST="Bookmarks">This is a title</A>
<DD>#tag1 #tag2 This is a description

This is what it should look like before importing into Pinboard:

<DT><A HREF="http://www.example.org/" ADD_DATE="1412337977" LIST="Bookmarks" TAGS="tag1,tag2">This is a title</A>
<DD>This is a description

So basically, I need to replace #tag1 #tag2 by TAGS="tag1,tag2" and move it on the first line within <A>.

I've been reading about moving chunks of data here: sed or awk to move one chunk of text betwen first pattern pair into second pair?

I haven't been to come up with a good recipe so far. Any insight?

Edit:

Here's an actual example of what the input file looks like (3 entries out of 3500):

<DT><A HREF="http://phabricator.org/" ADD_DATE="1412973315" LIST="Bookmarks">Phabricator</A>
<DD>#bug #tracking 

<DT><A HREF="http://qz.com/261426/the-hidden-commands-for-diagnosing-and-improving-your-netflix-streaming-quality/" ADD_DATE="1412838293" LIST="Inbox">The hidden commands for diagnosing and improving your Netflix streaming quality – Quartz</A>

<DT><A HREF="http://www.farmholidays.is/" ADD_DATE="1412337977" LIST="Bookmarks">Icelandic Farm Holidays | Local experts in Iceland vacations</A>
<DD>#iceland #tour #car #drive #self Self-driving tour of Iceland

Does it have to be used with awk/sed or is e.g. python possible as well? — Tim Zimmermann
It could be Python, which I'm starting to learn. It could be another language as well (Ruby, etc.) but I'd like to learn on a language I already know :-) Thanks. — Sebastien Wains
What is supposed to happen to <DD>#bug #tracking in your example? Shall the line be deleted completely or shall it result in <DD>? — Tim Zimmermann
<DD>#iceland #tour #car #drive #self Self-driving tour of Iceland should become <DD>Self-driving tour of Iceland — Sebastien Wains
Yes, sure, I got that. But the second line only contains tags and no further description. So my question is whether the result shall have a line that only contains <DD> or whether that line should be deleted? — Tim Zimmermann

Tim Zimmermann Tim Zimmermann · Accepted Answer · 2014-10-11T22:46:06

This might not be the most beautiful solution, but since it seems to be a one-time-thing it should be sufficient.

import re
dt = re.compile('^<DT>')
dd = re.compile('^<DD>')


with open('bookmarks.xml', 'r') as f:
    for line in f:
        if re.match(dt, line):
            current_dt = line.strip()
        elif re.match(dd, line):
            current_dd = line
            tags = [w for w in line[4:].split(' ') if w.startswith('#')]
            current_dt = re.sub('(<A[^>]+)>', '\\1 TAGS="' + ','.join([t[1:] for t in tags]) + '">', current_dt)
            for t in tags:
                current_dd = current_dd.replace(t + ' ', '')
            if current_dd.strip() == '<DD>':
                current_dd = ""
        else:
            print current_dt
            print current_dd
            current_dt = ""
            current_dd = ""

    print current_dt
    print current_dd

If some parts of the code are not clear, just tell me. You can of course use python to write the lines to a file instead of printing them, or even modify the original file.

Edit: Added if-clause so that empty <DD> lines won't show up in the result.

Moving chunks of data in a file with awk

2 Answers