Extract highlight text on webpage

1
votes

I would like to know if there is anyway to Extract highlighted text from a paragraph on a webpage.

After a long search.I have come across this module https://python-docx.readthedocs.io/en/latest/ but its for documents.

For example lets say we have the following paragraph:

"Stack Overflow is a privately held website, the flagship site of the Stack Exchange Network,created in 2008 by Jeff Atwood and Joel Spolsky. It was created to be a more open alternative to earlier question and answer sites such as Experts-Exchange. The name for the website was chosen by voting in April 2008 by readers of Coding Horror, Atwood's popular programming blog. It features questions and answers on a wide range of topics in computer programming"

Now in the above paragraph lets say the bold string of words are the ones I highlighted and I want to extract plus output the highlighted ones. Is there a way I can do this on a webpage.

So the output should be: privately held website ; Experts-Exchange ; wide range of topics.

3
javascriptpythonweb-scrapingpython-requests
You need to give us input data in a format that we can feed to python. How is it stored? as HTML? - Jonas Adler
sorry for posting a comment into answer, a bit overstretched today. Can You please specify how do you want to extract this text? Do you want to extract it using javascript or python? Do you want to extract bold (<b>) text only? Or do you want to be able to specify what counts as a highlighted text? - JKurcik
@JonasAdler I'm mostly looking into performing this task on webpage and expecting the format to be HTML or any other comparable markup language. - sreenan
@JKurcik No problem. Here I used bold as a reference but in actual I'm trying to find ways were I can highlight text on webpage using a mouse, color tag them and extract them. Hope I answered your question let me know if you need more info. - sreenan
in that case you need to first implement mechanisms for selecting a range of selected text upon some event, be it a keyboard shortcut or a button somewhere. Then you will need to implement color selection and wrap this text in <span style="color:#yourcolor" class="highlighted"> tag. When you need to extract highlighted text, you will search for class .highlighted in parent element, depending on the framework you use. It's too broad to provide a simple answer :) - JKurcik

3 Answers

0
votes

I think that this solution would be better applied to what you are looking for:

const req = require('tinyreq');

req('http://www.treepad.com/docs/tpp/manual/documents/127A901E40BA449B3C4359B720246BA3B2E67362.html', (err, body) => {
    if (err) { return console.log(err); }
    body.split('<body')[1].split('<span').map(textBold => {
        if(textBold.includes('background-color:')){
            console.log(textBold.split('>')[1].split('</SPAN')[0]);
            console.log('────────────────────');
        }
    });
});

output of treepad Highlighting text example link:

white against a dark blue
────────────────────
background
────────────────────
black against a gray background
────────────────────
0
votes

What I would do would be to use tinyreq and go through the body in search of the tag. Able this can be useful:

const req = require('tinyreq');

const start = '<mark>'; const end = '</mark>';
// const start = '<b>'; const end = '</b>';
// const start = '<strong>'; const end = '</strong>';

req('https://en.wikipedia.org/wiki/Language_code', (err, body) => {
    if (err) { return console.log(err); }
    body.split('<body')[1].split(start).map(textBold => {
        if(textBold.includes(end)){
            console.log(textBold.split(end)[0]);
            console.log('────────────────────');
        }
    });
});
0
votes

you can simply do this with bs4. first make sure you have installed bs4 and requests and if you want to install them just run these two commands

pip install requests
pip install bs4

then you have to write a python script like this

from bs4 import BeautifulSoup
import requests

page_url = 'http://127.0.0.1:1234'
source_code = requests.get(page_url)
plain_text = source_code.text
soup = BeautifulSoup(plain_text, features="lxml")
for bold in soup.findAll('b'):
    print(bold.contents)