Web scraping number

1
votes
import bs4, requests

url = requests.get(f'https://www.marketwatch.com/investing/stock/aapl?mod=over_search')
soup = bs4.BeautifulSoup(url.text,'lxml')
print(soup.find("div",{"class": "intraday__data"}).find("h3",{"class": "intraday__price"}).find("bg-quote"))

OUTPUT

< bg-quote channel="/zigman2/quotes/202934861/composite,/zigman2/quotes/202934861/lastsale" class="value" field="Last" format="0,0.00" session="pre">141.97< /bg-quote>

It gives me an error when I type .text in the last of the last line and here is the error

AttributeError Traceback (most recent call last) ~/Desktop/Python/playground.py in 2 url = requests.get(f'https://www.marketwatch.com/investing/stock/aapl?mod=over_search') 3 soup = bs4.BeautifulSoup(url.text,'lxml') ----> 4 print(soup.find("div",{"class": "intraday__data"}).find("h3",{"class": "intraday__price"}).find("bg-quote")).text

AttributeError: 'NoneType' object has no attribute 'text'

I just want to extract the price, any ideas?

2
python-3.xweb-scrapingtextbeautifulsouprequest

2 Answers

2
votes

You're typing .text after print() and print returns always None. Do this instead:

print(
    soup.find("div", {"class": "intraday__data"})
    .find("h3", {"class": "intraday__price"})
    .find("bg-quote")
    .text
) # <-- NOT .text here!

Prints:

141.77
1
votes
import requests
from bs4 import BeautifulSoup


def main(url):
    r = requests.get(url)
    soup = BeautifulSoup(r.text, 'lxml')
    print(soup.select('.value')[6].text)


main('https://www.marketwatch.com/investing/stock/aapl?mod=over_search')

Output:

141.76