How to use a browser with python selenium?

Question

I want to use browserermob to monitor network connections when running a GUI test with selenium. I found the information and documentation here and here and here , but it is completely unclear how to actually use it.

In the documentation, it reads:

server = Server("path/to/browsermob-proxy") 

But what is this way? Where to find him?

Also i see

 java -jar browsermob.jar --port 9090 

but no explanation as to whether this file is jar if it is part of a browser installation, or something unrelated.

I would appreciate it if someone could provide an example of COMPLETE and WORKING on how to use the browser and that ALL that I need to install ...

Answer 1

You can try below code

 from browsermobproxy import Server import psutil import time for proc in psutil.process_iter(): # check whether the process name matches if proc.name() == "browsermob-proxy": proc.kill() dict = {'port': 8090} server = Server(path="./BrowserMobProxy/bin/browsermob-proxy", options=dict) server.start() time.sleep(1) proxy = server.create_proxy() time.sleep(1) from selenium import webdriver profile = webdriver.FirefoxProfile() selenium_proxy = proxy.selenium_proxy() profile.set_proxy(selenium_proxy) driver = webdriver.Firefox(firefox_profile=profile) proxy.new_har("google") driver.get("http://www.google.co.uk") print (proxy.har) # returns a HAR JSON blob server.stop() driver.quit() 

Two things, if your code does not work, you can sometimes leave the process open. So I added below for the same

 import psutil import time for proc in psutil.process_iter(): # check whether the process name matches if proc.name() == "browsermob-proxy": proc.kill() 

Also 1 second sleep before and after creating the proxy server

 server.start() time.sleep(1) proxy = server.create_proxy() time.sleep(1) 

This helps to get rid of some periodic problems that you may encounter during server startup.

Answer 2

BrowserMob Proxy

BrowserMob Proxy is an open source tool that is used to collect performance data for web applications in HAR format. It also allows you to manipulate browser behavior and traffic, such as simulating network traffic, rewriting HTTP requests and responses, etc., as well as manipulate network traffic from your AJAX applications. In short, the BrowserMob proxy helps us BrowserMob proxy client-side performance data for a web application using the Selenium WebDriver automated tests.

You can find more information about BrowserMob Proxy in the Python documentation and this guide .

Demo BrowserMob Proxy 2.0 with Python Client for Windows

• Install browsermob-proxy via CLI:

   C:\Users\your_user>pip install browsermob-proxy Collecting browsermob-proxy Downloading browsermob-proxy-0.8.0.tar.gz Collecting requests>=2.9.1 (from browsermob-proxy) Downloading requests-2.18.4-py2.py3-none-any.whl (88kB) 100% |¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦| 92kB 148kB/s Collecting idna<2.7,>=2.5 (from requests>=2.9.1->browsermob-proxy) Downloading idna-2.6-py2.py3-none-any.whl (56kB) 100% |¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦| 61kB 639kB/s Collecting urllib3<1.23,>=1.21.1 (from requests>=2.9.1->browsermob-proxy) Downloading urllib3-1.22-py2.py3-none-any.whl (132kB) 100% |¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦| 133kB 250kB/s Collecting certifi>=2017.4.17 (from requests>=2.9.1->browsermob-proxy) Downloading certifi-2017.11.5-py2.py3-none-any.whl (330kB) 100% |¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦| 337kB 401kB/s Collecting chardet<3.1.0,>=3.0.2 (from requests>=2.9.1->browsermob-proxy) Downloading chardet-3.0.4-py2.py3-none-any.whl (133kB) 100% |¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦| 143kB 296kB/s Installing collected packages: idna, urllib3, certifi, chardet, requests, browse rmob-proxy Running setup.py install for browsermob-proxy ... done Successfully installed browsermob-proxy-0.8.0 certifi-2017.11.5 chardet-3.0.4 id na-2.6 requests-2.18.4 urllib3-1.22 

• Download to browsermob-proxy The browsermob-proxy-2.1.4-bin binaries form the following URL:

   https://bmp.lightbody.net/ 

• Extract and Save directory in C:\Utility

• Manually launch the Browsermobproxy server using the CLI command:

   C:\Utility\browsermob-proxy-2.1.4\lib>java -jar browsermob-dist-2.1.4.jar --port 9090 Running BrowserMob Proxy using LittleProxy implementation. To revert to the legacy implementation, run the proxy with the command-line option '--use-littleproxy false'. [INFO 2018-01-17T19:01:30,276 net.lightbody.bmp.proxy.Main] (main) Starting BrowserMob Proxy version 2.1.4 [INFO 2018-01-17T19:01:30,388 org.eclipse.jetty.util.log] (main) jetty-7.xy-SNAPSHOT [INFO 2018-01-17T19:01:30,471 org.eclipse.jetty.util.log] (main) started oejsServletContextHandler{/,null} [INFO 2018-01-17T19:01:30,871 org.eclipse.jetty.util.log] (main) Started SelectChannelConnector@0.0.0.0:9090 

• You can also launch Browsermobproxy Server through your code, as shown below.

• Create a new PyDev module (if you are using Eclipse ) and write the main program through your IDE as follows:

   from browsermobproxy import Server server = Server("C:\\Utility\\browsermob-proxy-2.1.4\\bin\\browsermob-proxy") server.start() proxy = server.create_proxy() from selenium import webdriver profile = webdriver.FirefoxProfile() profile.set_proxy(proxy.selenium_proxy()) driver = webdriver.Firefox(firefox_profile=profile) proxy.new_har("google") driver.get("http://www.google.co.in") proxy.har # returns a HAR JSON blob server.stop() driver.quit() 

• Snapshot:

• Run your Test like running Python
• During the execution of your Program you will see that the Firefox Quantum browser is initializing, and the url http://www.google.co.in opens and closes at the end of the test.

• When Test Execution you will find the following files in your workspace that provide you with all the details of Test Execution :

   bmp.log geckodriver.log server.log 

• Snapshot:

Answer 3

You need to configure the driver to use BMP as a proxy server so that it can record network activity. Here is an example ...

 from browsermobproxy import Server from selenium import webdriver server = Server('/path/to/bmp/bin/browsermob-proxy') #Local path to BMP server.start() proxy = server.create_proxy() #Proxy is used to generate a HAR file containing the connection URLS that the MP3s are loaded from. chrome_options = webdriver.ChromeOptions() chrome_options.add_argument("--proxy-server={0}".format(proxy.proxy)) #Configure chrome options driver = webdriver.Chrome(chrome_options=chrome_options) proxy.new_har('filename') 

Then all actions will be written to this file.

In my case, the path for the binary was C:\Python27\Lib\site-packages\browsermobproxy\browsermob-proxy-2.1.0-beta-3\bin\browsermob-proxy on Windows with Python 2.7

Answer 4

This problem is due to the fact that installing browsermob-proxy does not just do:

 pip install browsermob-proxy 

After you have executed the above code, you need to go to https://bmp.lightbody.net , download zip, unzip it, and then when calling Server () in your python script, determine the path to the executable from zip, which you just completed. created. It is located in the bin folder and is called browsermob-proxy.

In my case, it was:

 server = Server("/anaconda3/lib/python3.7/site-packages/browsermobproxy/browsermob-proxy-2.1.4/bin/browsermob-proxy")