Effective Method for Scrolling Pages Using Selenium

Question

I'm currently hard-coded for scrolling using Selenium, the Firefox Driver in Python, is there any way I can scroll it up and down the page? A page can grow 20 times more than usual when images on the page load.

I need to scroll up and down several times without skipping a single part of the page.

The current method is inefficient and sometimes does not scroll to some part of the page. Next is my current code

browser = webdriver.Firefox() browser.get(link) browser.execute_script("window.scrollTo(0, document.body.scrollHeight/3.4);") time.sleep(0.2) browser.execute_script("window.scrollTo(0, document.body.scrollHeight/3.5);") time.sleep(0.2) browser.execute_script("window.scrollTo(0, document.body.scrollHeight/3.7);") time.sleep(0.2) browser.execute_script("window.scrollTo(0, document.body.scrollHeight/3.8);") time.sleep(0.2) browser.execute_script("window.scrollTo(0, document.body.scrollHeight/4);") time.sleep(0.2) browser.execute_script("window.scrollTo(0, document.body.scrollHeight/4.2);") time.sleep(0.2) browser.execute_script("window.scrollTo(0, document.body.scrollHeight/4.3);") time.sleep(0.2) browser.execute_script("window.scrollTo(0, document.body.scrollHeight/4.5);") time.sleep(0.2) browser.execute_script("window.scrollTo(0, document.body.scrollHeight/4.7);") time.sleep(0.2) browser.execute_script("window.scrollTo(0, document.body.scrollHeight/4.9);") time.sleep(0.2) browser.execute_script("window.scrollTo(0, document.body.scrollHeight/5.2);") time.sleep(0.2) browser.execute_script("window.scrollTo(0, document.body.scrollHeight/5.1);") time.sleep(0.2) browser.execute_script("window.scrollTo(0, document.body.scrollHeight/5.8);") time.sleep(0.2) browser.execute_script("window.scrollTo(0, document.body.scrollHeight/3.7);") time.sleep(0.2) browser.execute_script("window.scrollTo(0, document.body.scrollHeight/50);") time.sleep(0.2) 

Answer 1

You can use this if using Selenium in Python ...

 scheight = .1 while scheight < 9.9: driver.execute_script("window.scrollTo(0, document.body.scrollHeight/%s);" % scheight) scheight += .01 

Or something like that. You can add if brakes to capture data at specific scheight locations within while . Just do += or iterate what you like. I used .01 so that I can visually see its scrolling, otherwise you would need timeouts, and this would seem intermittent. Depends on what you need.

Answer 2

Can you try just scrolling with one shot?

  browser.execute_script("window.scrollTo(0, document.body.scrollHeight/3.5);window.scrollTo(0, document.body.scrollHeight/3.7);") 

Just list all scroll calls in the same script call.