Geek Answers Handbook

How to scroll page to end of page using selenium in python

I am trying to scroll to the end so that I can make all the data visible and extract it. I tried to find a command for it, but it is available in java (driver.executeScript) but not found for python. Now I make the computer press the end key a thousand times:

while i<1000: scroll = driver.find_element_by_tag_name('body').send_keys(Keys.END) i+=1

And I also tried driver.execute_script ("window.scrollTo (0, document.body.scrollHeight);"), but it scrolls to the end of the loaded page and the same END key. As soon as the following content is loaded at the bottom of the page. But now it does not scroll again.

I know there will be a very good alternative for this. Please help.

+10
source share
4 answers

Well, finally, I understood the solution:

 lenOfPage = driver.execute_script("window.scrollTo(0, document.body.scrollHeight);var lenOfPage=document.body.scrollHeight;return lenOfPage;") match=False while(match==False): lastCount = lenOfPage time.sleep(3) lenOfPage = driver.execute_script("window.scrollTo(0, document.body.scrollHeight);var lenOfPage=document.body.scrollHeight;return lenOfPage;") if lastCount==lenOfPage: match=True
+17
source

This can be done in one line by scrolling to document.body.scrollHeight

 driver.execute_script("window.scrollTo(0, document.body.scrollHeight);")
+4
source

You can use scrollingElement with scrollTop and scrollHeight to to scroll to the end of the page.

 driver.execute_script("var scrollingElement = (document.scrollingElement || document.body);scrollingElement.scrollTop = scrollingElement.scrollHeight;")

Recommendations:

  1. Scroll automatically to the bottom of the page.
  2. Document.scrollingElement - Web API | MDN
  3. Element.scrollHeight - Web API | MDN
  4. Element.scrollTop - Web API | MDN
0
source

None of this helped me, but the solution below worked:

 driver.get("https://www.youtube.com/user/teachingmensfashion/videos") def scroll_to_bottom(driver): old_position = 0 new_position = None while new_position != old_position: # Get old scroll position old_position = driver.execute_script( ("return (window.pageYOffset !== undefined) ?" " window.pageYOffset : (document.documentElement ||" " document.body.parentNode || document.body);")) # Sleep and Scroll time.sleep(1) driver.execute_script(( "var scrollingElement = (document.scrollingElement ||" " document.body);scrollingElement.scrollTop =" " scrollingElement.scrollHeight;")) # Get new position new_position = driver.execute_script( ("return (window.pageYOffset !== undefined) ?" " window.pageYOffset : (document.documentElement ||" " document.body.parentNode || document.body);")) scroll_to_bottom(driver)
0
source

More articles:


All Articles