Open URL in Chrome and save its source code using command line

Question

I find it difficult to find how to save the page as html or .txt using the command line in the Chrome browser,

This is what I have done so far,

C:\Users\Cipher\AppData\Local\Google\Chrome\Application>chrome.exe --new-window http://google.com 

This command will open a new Chrome browser window and visit google.com, but I could not understand how I can save google.com as an html or as a txt file, is there any way to do this using the command line?

Answer 1

use http://en.wikipedia.org/wiki/Chromium_Embedded_Framework (create the webbrowser component) to load and display the page.

after that save it as you want.

Answer 2

Do you really need to open Google Chrome? You can get the page source using Wget (available for UNIX systems or for Windows in this post on SuperUser ). After installation, simply use the following command:

 wget http://google.com -O yourfilename.html 

And that should be all :) I don’t think there is a way to tell Chrome to load HTML from the command line :(

UPDATE:. There is a repo on GitHub called chrome-cli that allows the user to control Chrome from the command line. The downside is that it only works on Mac OS X.

Answer 3

You cannot complete the task that you describe manually, but you can complete it using WebDriver automation.

Chrome can be remotely controlled using an API called WebDriver (included with the Selenium 2 automation kit). WebDrive has bindings for various programming languages, including, for example, JavaScript and Python.

Here is a sample code for Python (not tested):

 from selenium import webdriver driver = webdriver.Chrome('/path/to/chromedriver') # Optional argument, if not specified will search path. driver.get('http://www.google.com/'); html = driver.page_source f = open("myhtml", "wt") f.write(html) f.close() 

Orignal example

Answer 4

I created a small script to accomplish just this task: https://github.com/abiyani/automate-save-page-as . See Demonstration of gif in README.

It automates keyboard actions that you would otherwise have performed to save the page manually (literally transmits these key signals to the OS). As a side effect that he used in another mine project, he was tested on various versions of linux: Ubuntu, Mint, Fedora, etc. - and works great on all of them. It probably won’t work (at least unchanged) on the Mac and, of course, not on Windows.