Geek Answers Handbook

Get a DOCTYPE document using BeautifulSoup

I just started redoing scrapy in combination with BeautifulSoup , and I wonder if I am missing something very obvious, but I can’t figure out how to get the doctype of the returned html document from the resulting soup object.

Given the following html:

<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01//EN" "http://www.w3.org/TR/html4/strict.dtd"> <html lang="en"> <head> <meta charset=utf-8 /> <meta name="viewport" content="width=620" /> <title>HTML5 Demos and Examples</title> <link rel="stylesheet" href="/css/html5demos.css" type="text/css" /> <script src="js/h5utils.js"></script> </head> <body> <p id="firstpara" align="center">This is paragraph <b>one</b> <p id="secondpara" align="blah">This is paragraph <b>two</b>. </html>

Can someone tell me if there is a way to extract the declared doctype from it using BeautifulSoup?

+7
python parsing scrapy beautifulsoup
source share
3 answers

Beautiful Soup 4 has a class for DOCTYPE declarations, so you can use them to extract all the ads at the top level (although you are no doubt expecting one or nothing!)

 def doctype(soup): items = [item for item in soup.contents if isinstance(item, bs4.Doctype)] return items[0] if items else None
+4
source share

You can go through the top-level items and check each one to see if this is a declaration. Then you can check it to find out what the announcement is:

 for child in soup.contents: if isinstance(child, BS.Declaration): declaration_type = child.string.split()[0] if declaration_type.upper() == 'DOCTYPE': declaration = child
+3
source share

You can simply get the first element in the contents of the soup:

 >>> soup.contents[0] u'DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01//EN" "http://www.w3.org/TR/html4/strict.dtd"'
0
source share

More articles:


All Articles