Geek Answers Handbook

Screen scraper

Now I am learning cURL, I come across one complex one, connected with entering the page by username and password directly

+3
source share
3 answers

For standard HTTP authentication, you can try:

curl http://username:password@url

It should work!

+1
source

The method you need to use will depend on how the username / password verification of the webpage is implemented, but this may help you:
http://curl.haxx.se/mail/archive-2008-05/0113.html

+1
source

, , , CAPTCHA. ,

  • POST URL- (. HTML).
  • cookie
  • cookie ( )

I do it with wget. curlshould be similar (see his manual).

12:

wget --keep-session-cookies --save-cookies "mycookies" \
     --post-data "login=mylogin&password=mypass" submit_URL

3:

wget --load-cookies "mycookies" --keep-session-cookies --save-cookies "mycookies" \
     another_URL_behind_login_form

From what I see in man curl1-2, there should be something like this (not verified):

curl -F "login=mylogin;password=mypass" -c "mycookies" submit_URL

and 3:

curl -b "mycookies" -c "mycookies" another_URL

But I have not tried it with help curl.

0
source

More articles:


All Articles