Text mining with tm.plugin.webmining package using GoogleFinanceSource function

Question

I am studying text mining on the online book http://tidytextmining.com/. In the fifth chapter: http://tidytextmining.com/dtm.html#financial

the following code:

library(tm.plugin.webmining)
library(purrr)

company <- c("Microsoft", "Apple", "Google", "Amazon", "Facebook",
             "Twitter", "IBM", "Yahoo", "Netflix")
symbol <- c("MSFT", "AAPL", "GOOG", "AMZN", "FB", "TWTR", "IBM", "YHOO", "NFLX")

download_articles <- function(symbol) {
    WebCorpus(GoogleFinanceSource(paste0("NASDAQ:", symbol)))
}
stock_articles <- data_frame(company = company,
                             symbol = symbol) %>%
    mutate(corpus = map(symbol, download_articles))

gives me the error:

StartTag: invalid element name
Extra content at the end of the document
Error: 1: StartTag: invalid element name
2: Extra content at the end of the document

Any hints? Someone suggested to remove company and symbol related to "Twitter", but it still doesn't work and returns the same error. Many thanks in advance

stomper stomper · Accepted Answer · 2018-01-17T02:44:01

I am having the same issue, however, have narrowed it down slightly. This snippet of the code results in the same error.

GoogleFinanceSource("NASDAQ:MSFT")

StartTag: invalid element name
Extra content at the end of the document
Error: 1: StartTag: invalid element name
2: Extra content at the end of the document

I also saw where others have suggested removing Twitter. I get the point it would have failed as Twitter is not on NASDAQ. I tried the suggested "NYSE:TWTR" and got the same result, however.

I attempted to use GoogleNewsSource to see if I would get the same issue and got a different error which this article on github suggests is being caused by the parser. I wonder if these two issues could be related. github.com/mannau/tm.plugin.webmining/issues/14.

GoogleNewsSource("Microsoft")

Unknown IO error failed to load external entity "http://news.google.com/news?hl=en&q=Microsoft&ie=utf-8&num=100&output=rss"
Error: 1: Unknown IO error2: failed to load external entity "http://news.google.com/news?hl=en&q=Microsoft&ie=utf-8&num=100&output=rss"

That all being said, I have found a work around using a modified ticker list and YahooFinanceSource as follows:

company <- c("Microsoft", "Apple", "Google")
symbol <- c("MSFT", "AAPL", "GOOG")

download_articles <- function(symbol) {
    WebCorpus(YahooFinanceSource(symbol))
}

stock_articles <- data_frame(company = company,
                         symbol = symbol) %>%
mutate(corpus = map(symbol, download_articles))

Text mining with tm.plugin.webmining package using GoogleFinanceSource function

3 Answers