1
votes

I am going through python class and I wrote this script to web scrape and getting the following error and I cant see what I did wrong. It is supposed to get all the two star rated book titles into a list.

AttributeError: ResultSet object has no attribute 'select'. You're probably treating a list of elements like a single element. Did you call find_all() when you meant to call find()?

     import requests
     import bs4
     import lxml

     base_url = 'https://books.toscrape.com/catalogue/page-{}.html'
     #how to scrape

     res = requests.get(base_url.format(1))
     soup = bs4.BeautifulSoup(res.text, "lxml")

     products = soup.select(".product_pod")

     #example.select('a') [1] ['title']

     two_star_title = []
     for n in range (1,51):

       scrape_url = base_url.format(n)
       res = requests.get(scrape_url)
       soup = bs4.BeautifulSoup(res.text, "lxml")
       books = soup.select(".product_pod")

       for book in books:
           if len(books.select('.star-rating.Two')) != 0:
             book_title = books.select('a') [1] ('title')
             two_star_title.append(book.title)
    print(two_star_title)
books.select should be book.select - Wondercricket