Selenium is slow, or is my code wrong?

Question

So, I am trying to log into Quora using Python, and then clear some things.

I use Selenium to enter the site. Here is my code:

from selenium import webdriver from selenium.webdriver.common.keys import Keys driver = webdriver.Firefox() driver.get('http://www.quora.com/') username = driver.find_element_by_name('email') password = driver.find_element_by_name('password') username.send_keys('email') password.send_keys('password') password.send_keys(Keys.RETURN) driver.close() 

Now questions:

• It took ~ 4 minutes to find and fill out the registration form, which painfully slows down. Is there something I can do to speed things up?

• When he logged in, how can I make sure there were no errors? In other words, how to check the response code?

• How to save cookies with selenium so that I can continue to clear after logging in?

• If there is no way to make selenium faster, is there another alternative for entry? (Quora does not have an API)

Answer 1

I had a similar problem with very slow calls to find_elements_xxx in Selenium Python using ChromeDriver. As a result, I found a problem with calling driver.implicitly_wait (), which I did before calls to find_element_xxx (); when I pulled it out, my calls to find_element_xxx () were quick.

Now I know that these elements were there when I called the find_elements_xxx () calls. Therefore, I cannot imagine why implicit_wait should have affected the speed of these operations, but it did.

Answer 2

• I was there, selenium slowly. It may not be as slow as 4 minutes to fill out the form. Then I started using phantomjs, which is much faster than firefox, since it is headless. You can simply replace Firefox () with PhantomJS () in the webdriver line after installing the latest phantomjs.

• To verify that you have a login, you can claim for some item that appears after logging in.

• If you do not leave your driver, cookies will be available for links.

• You can try using urllib and send directly to the login link. You can use cookiesjar to save cookies. You can even just save a cookie, after all, a cookie is just a line in the http header

Answer 3

Running firefox with javascript disabled will help:

 from selenium import webdriver from selenium.webdriver.common.keys import Keys firefox_profile = webdriver.FirefoxProfile() firefox_profile.set_preference("browser.download.folderList",2) firefox_profile.set_preference("javascript.enabled", False) driver = webdriver.Firefox(firefox_profile=firefox_profile) driver.get('http://www.quora.com/') username = driver.find_element_by_name('email') password = driver.find_element_by_name('password') username.send_keys('email') password.send_keys('password') password.send_keys(Keys.RETURN) driver.close() 

Answer 4

You can fix the filling of the form using your own setAttribute method, here is the code for java for it

 public void setAttribute(By locator, String attribute, String value) { ((JavascriptExecutor) getDriver()).executeScript("arguments[0].setAttribute('" + attribute + "',arguments[1]);", getElement(locator), value); } 

Answer 5

For Windows 7 and IEDRIVER with Python Selenium, terminating the Windows command prompt and restarting it cured my problem.

I am having problems with find_element..clicks. They took 30 seconds plus a bit. Here's the type of code I have, including capturing, how long to work.

 timeStamp = time.time() elem = driver.find_element_by_css_selector(clickDown).click() print("1 took:",time.time() - timeStamp) timeStamp = time.time() elem = driver.find_element_by_id("cSelect32").click() print("2 took:",time.time() - timeStamp) 

It was a record of about 31 seconds for each click. After finishing the command line and restarting it (which ends any IEDRIVERSERVER.exe processes), it was 1 second per click.