Remove chararcters in text corpus

0
votes

I'm analyzing a corpus of emails. Some emails contain URLs. When I apply the removePunctuation function from the tm library, I get httpwww, and then I lose the info of a web address. What I would like to do, is to replace the "://" with " " across all of the corpus. I tried gsub, but then I the datatype of the corpus changes and I can't continue to process it with tm package.

Here is an example:

As you can see, gsub changes the class of the corpus to an array of characters, causing tm_map to fail.

> corpus
# A corpus with 4257 text documents
> corpus1 <- gsub("http://","http ",corpus)
> class(corpus1)
# [1] "character"
> class(corpus)
# [1] "VCorpus" "Corpus"  "list"   
> cleanSW <- tm_map(corpus1,removeWords, stopwords("english"))
# Error in UseMethod("tm_map", x) : 
# no applicable method for 'tm_map' applied to an object of class "character"
> cleanSW <- tm_map(corpus,removeWords, stopwords("english"))
> cleanSW
# A corpus with 4257 text documents

How can I bypass it? Maybe there's a way to convert it back to corpus from array of characters?

1
regexrtext-miningtm
You can't paste("://", ...) back in successfully? - Rich Scriven
how do I use it across the whole corpus? - Yoav
what about the other / and the possible . or : in the web address? - James
Same issue, I just gave the :// as an example, but as you mentioned it applies to some more characters as well. - Yoav
Can you show the code you tried and provide data? - Tyler Rinker

1 Answers

2
votes

Found a solution to this problem here: Removing non-English text from Corpus in R using tm(), Corpus(VectorSource(dat1)) worked for me.