How to webscrape the correct element from a stat tracking website (cod.tracker.gg) using Python

2
votes

On this specific page (or any 'matches' page) there are names you can select to view individual statistics for a match. How do I grab the 'kills' stat for example using webscraping?

In most of the tutorials I use the webscraping seems simple. However, when inspecting this site, specifically the 'kills' item, you see something like

<span data-v-71c3e2a1 title="Kills" class ="name".

Question 1.) What is the 'data-v-71c3e2a1'? I've never seen anything like this in my html,css, or webscraping tutorials. It appears in different variations all over the site.

Question 2.) More importantly, how do I grab the number of kills in this section? I've tried using scrapy and grabbing by xpath:

scrapy shell https://cod.tracker.gg/warzone/match/1424533688251708994?handle=PatrickPM

response.xpath("//*[@id="app"]/div[3]/div[2]/div/main/div[3]/div[2]/div[2]/div[6]/div[2]/div[3]/div[2]/div[1]/div/div[1]/span[2]").get()

but this raises a syntax error

response.xpath("//*[@id="app"]

SyntaxError: invalid syntax

Grabbing by response.css("").get() is also difficult. Should I be using selenium? Or just regular requests/bs4? Nothing I do can grab it.

Thank you.

1
pythonhtmlweb-scrapingstatistics

1 Answers

1
votes

Does this return the data you need?

import requests


endpoint = "https://api.tracker.gg/api/v1/warzone/matches/1424533688251708994"
r = requests.get(endpoint, params={"handle": "PatrickPM"})
data = r.json()["data"]

In any way I suggest using API if there's one available. It's much easier than using BeautifulSoup or selenium.