Geek Answers Handbook

How to find gaps with a specific class containing specific text using beautiful soup and re?

how can I find the whole range with the class 'blue' which contains text in the format:

 04/18/13 7:29pm

which could be like this:

 04/18/13 7:29pm

or

 Posted on 04/18/13 7:29pm

in terms of building logic for this, this is what I got so far:

 new_content = original_content.find_all('span', {'class' : 'blue'}) # using beautiful soup find_all pattern = re.compile('<span class=\"blue\">[data in the format 04/18/13 7:29pm]</span>') # using re for _ in new_content: result = re.findall(pattern, _) print result

I was referring to https://stackoverflow.com/a/2126149/ and to https://stackoverflow.com/questions/524432/... to try to figure out a way to do this, but above is all I got so far.

change

to clarify the scenario, there is a range with:

 <span class="blue">here is a lot of text that i don't need</span>

and

 <span class="blue">this is the span i need because it contains 04/18/13 7:29pm</span>

and note that I need 04/18/13 7:29pm not the rest of the content.

change 2:

I also tried:

 pattern = re.compile('<span class="blue">.*?(\d\d/\d\d/\d\d \d\d?:\d\d\w\w)</span>') for _ in new_content: result = re.findall(pattern, _) print result

and got the error:

 'TypeError: expected string or buffer'
+7
source share
3 answers
 import re from bs4 import BeautifulSoup html_doc = """ <html> <body> <span class="blue">here is a lot of text that i don't need</span> <span class="blue">this is the span i need because it contains 04/18/13 7:29pm</span> <span class="blue">04/19/13 7:30pm</span> <span class="blue">Posted on 04/20/13 10:31pm</span> </body> </html> """ # parse the html soup = BeautifulSoup(html_doc) # find a list of all span elements spans = soup.find_all('span', {'class' : 'blue'}) # create a list of lines corresponding to element texts lines = [span.get_text() for span in spans] # collect the dates from the list of lines using regex matching groups found_dates = [] for line in lines: m = re.search(r'(\d{2}/\d{2}/\d{2} \d+:\d+[a|p]m)', line) if m: found_dates.append(m.group(1)) # print the dates we collected for date in found_dates: print(date)

exit:

 04/18/13 7:29pm 04/19/13 7:30pm 04/20/13 10:31pm
+13
source

This is a flexible regex that you can use:

 "(\d\d?/\d\d?/\d\d\d?\d?\s*\d\d?:\d\d[a|p|A|P][m|M])"

Example:

 >>> import re >>> from bs4 import BeautifulSoup >>> html = """ <html> <body> <span class="blue">here is a lot of text that i don't need</span> <span class="blue">this is the span i need because it contains 04/18/13 7:29pm</span> <span class="blue">04/19/13 7:30pm</span> <span class="blue">04/18/13 7:29pm</span> <span class="blue">Posted on 15/18/2013 10:00AM</span> <span class="blue">Posted on 04/20/13 10:31pm</span> <span class="blue">Posted on 4/1/2013 17:09aM</span> </body> </html> """ >>> soup = BeautifulSoup(html) >>> lines = [i.get_text() for i in soup.find_all('span', {'class' : 'blue'})] >>> ok = [m.group(1) for line in lines for m in (re.search(r'(\d\d?/\d\d?/\d\d\d?\d?\s*\d\d?:\d\d[a|p|A|P][m|M])', line),) if m] >>> ok [u'04/18/13 7:29pm', u'04/19/13 7:30pm', u'04/18/13 7:29pm', u'15/18/2013 10:00AM', u'04/20/13 10:31pm', u'4/1/2013 17:09aM'] >>> for i in ok: print i 04/18/13 7:29pm 04/19/13 7:30pm 04/18/13 7:29pm 15/18/2013 10:00AM 04/20/13 10:31pm 4/1/2013 17:09aM
+2
source

This pattern seems to satisfy what you are looking for:

 >>> pattern = re.compile('<span class="blue">.*?(\d\d/\d\d/\d\d \d\d?:\d\d\w\w)</span>') >>> pattern.match('<span class="blue">here is a lot of text that i dont need</span>') >>> pattern.match('<span class="blue">this is the span i need because it contains 04/18/13 7:29pm</span>').groups() ('04/18/13 7:29pm',)
+1
source

More articles:


All Articles