Geek Answers Handbook

How to get HTML body from URL using Nokogiri in Rails?

I want to parse the body attributes from a URL.

For example:

url = 'http://rca.yandex.com/?key=rca.1.1.20140120T051507Z.3db118ab435efdff.6c84331313b6b7d66abd191410f72e0e1c3c8795&url=http://endtimeheadlines.wordpress.com/2014/01/17/think-tank-extraordinary-crisis-needed-to-preserve-new-world-order/#comment-36708?utm_source=twitterfeed&utm_medium=facebook[&callback=http://64.191.99.245:3023/posts][&full=1]'

When I try:

page = Nokogiri::HTML(html)

I get:

#<Nokogiri::HTML::Document:0x52fd6d6 name="document" children=[#<Nokogiri::XML::DTD:0x52fd1f4 name="html">, #<Nokogiri::XML::Element:0x52fc6aa name="html" children=[#<Nokogiri::XML::Element:0x5301f56 name="body" children=[#<Nokogiri::XML::Element:0x53018d0 name="p" children=[#<Nokogiri::XML::Text:0x53015f6 "http://rca.yandex.com/?key=rca.1.1.20140120T051507Z.3db118ab435efdff.6c84331313b6b7d66abd191410f72e0e1c3c8795&url=http://endtimeheadlines.wordpress.com/2014/01/17/think-tank-extraordinary-crisis-needed-to-preserve-new-world-order/#comment-36708?utm_source=twitterfeed&utm_medium=facebook[&callback=http://64.191.99.245:3023/posts][&full=1]">]>]>]>]>

How to get attributes inside this URL?

For example: page.css("div"). I want to get the value from HTML body.

+4
source share
4 answers

It’s not entirely clear what you are trying to do, but this may help:

require 'nokogiri'

html = '<html><head><title>foo</title><body><p>bar</p></body></html>'

doc = Nokogiri::HTML(html)

Using at, you will find the first occurrence of a tag that is reasonable in an HTML document, since you should only have one <body>tag.

doc.at('body') # => #<Nokogiri::XML::Element:0x3ff194d24cd4 name="body" children=[#<Nokogiri::XML::Element:0x3ff194d24acc name="p" children=[#<Nokogiri::XML::Text:0x3ff194d248c4 "bar">]>]>

If you need child elements of a tag, use childrento get them:

doc.at('body').children # => [#<Nokogiri::XML::Element:0x3ff194d24acc name="p" children=[#<Nokogiri::XML::Text:0x3ff194d248c4 "bar">]>]

If you want to get child nodes as HTML:

doc.at('body').children.to_html # => "<p>bar</p>"
doc.at('body').inner_html # => "<p>bar</p>"

body:

doc.at('body').content # => "bar"
doc.at('body').text # => "bar"

"" attributes <body>:

require 'nokogiri'

html = '<html><head><title>foo</title><body on_load="do_something()"><p>bar</p></body></html>'

doc = Nokogiri::HTML(html)
doc.at('body').attributes # => {"on_load"=>#<Nokogiri::XML::Attr:0x3fdc3d923ca0 name="on_load" value="do_something()">}
doc.at('body')['on_load'] # => "do_something()"

attributes , . Nokogiri:: XML:: Node [], Hash.

+18

page.css('body') . to_s

+2

to_xml to_html . , Nokogiri .

page = Nokogiri::HTML(html)
page.to_xml

div document, :

divs = page.css('div') # returns either string or array depending upon the number of divs in your document.
divs.to_xml
0

Of course, you get the parsed HTML / XML tree obtained from the link. And here in your example:

page.css("div")

just try to get everything divin the analyzed document as Array. You can list them one by one and get the text of each of div:

page.css("div").each do| div |
   p div.text
end
-1
source

More articles:


All Articles