Ruby Mechanize get href attribute value

Question

I am fairly new to Ruby but am muddling my way through a scraper. I am using Mechanize and so far it's looking quite good. Though I have now got a bit stuck with grabbing the href attribute of a bunch of links.

I need to get the href attribute so that I can then open each of those pages and scrape some more information.

Is this possible?

Here's an example.

all_results.search("table.mcsResultsTable tr").each do |tablerow|
    installer_link = tablerow.search("td:first-child a").href
    puts installer_link + "\n"

Arup Rakshit Arup Rakshit · Accepted Answer · 2013-09-27T10:01:47

Here is an example to help you out,about how to extract the href atttribute :

require 'nokogiri'

doc = Nokogiri::HTML.parse <<-eot
<a name="html" href = "http://foo">HTML Tutorial</a><br>
<a name="css" href = "http://fooz">CSS Tutorial</a><br>
<a name="xml" href = "http://fiz">XML Tutorial</a><br>
<a href="/js/">JavaScript Tutorial</a>
eot

doc.search("//a").class # => Nokogiri::XML::NodeSet
doc.search("//a").each {|nd| puts nd['href'] }
doc.search("//a").map(&:class)
# => [Nokogiri::XML::Element, Nokogiri::XML::Element, Nokogiri::XML::Element,
#    Nokogiri::XML::Element]

output :

http://foo
http://fooz>CSS Tutorial</a><br>
<a name=
/js/

Basically doc.search("//a") will give you nodesets,which is nothing but a collection of Nokogiri::XML::Node(s).You can use then the method Nokogiri::XML::Node#[] to get the attribute value of any specific node. Nokogiri holds the attribute/value pairs as a Hash. Look below :

require 'nokogiri'

doc = Nokogiri::HTML.parse <<-eot
<a target="_blank" class="tryitbtn" href="tryit.asp?filename=try_methods">Try it yourself &raquo;</a>
eot

doc.at('a').keys 
# => ["target", "class", "href"]
doc.at('a').values
# => ["_blank", "tryitbtn", "tryit.asp?filename=try_methods"]
doc.at('a')['target'] # => "_blank"
doc.at('a')['class']  # => "tryitbtn"

Ruby Mechanize get href attribute value

1 Answers