Issue with piping and rvest

Question

I'm trying to scrape options prices from the ASX with rvest and I'd like some help piping my code. I want to pipe and end up with a data frame.

The page at the link above has two tables, the first with share price information and the second with all options. When I run the following code I get a dataframe of the second table:

html <- read_html("http://www.asx.com.au/asx/markets/optionPrices.do?by=underlyingCode&underlyingCode=ANZ&expiryDate=&optionType=B")
nodes <- html_nodes(html, "table.options")
df <- html_table(nodes)[[2]]

But when I try to pipe that same code with the following:

html <- read_html("http://www.asx.com.au/asx/markets/optionPrices.do?by=underlyingCode&underlyingCode=ANZ&expiryDate=&optionType=B")
html %>% 
 html_nodes("table.options") %>% 
 html_table()[[2]]

I get an error reading 'Error in UseMethod("html_table") : no applicable method for 'html_table' applied to an object of class "NULL"'

Can anyone tell me what I'm doing wrong?

At least have the decency to inform potential answerers that you're willing to put them at risk of civil & criminal penalties for your financial gain. asx.com.au/about/terms-use.htm — hrbrmstr

EmilHvitfeldt EmilHvitfeldt · Accepted Answer · 2018-01-03T00:48:02

If you want to index that directly you would have to index in a seperate pipe like so

read_html("http://www.asx.com.au/asx/markets/optionPrices.do?by=underlyingCode&underlyingCode=ANZ&expiryDate=&optionType=B") %>%
html_nodes("table.options") %>%
html_table() %>%
.[[2]]

where . acts as the output of the previous pipe.

Issue with piping and rvest

1 Answers