Download large file using curl

Question

I need to upload a remote file using curl.

Here is an example of the code I have:

$ch = curl_init(); curl_setopt($ch, CURLOPT_URL, $url); curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1); $st = curl_exec($ch); $fd = fopen($tmp_name, 'w'); fwrite($fd, $st); fclose($fd); curl_close($ch); 

But it cannot process large files, as it first reads into memory.

Is it possible to transfer a file directly to disk?

Answer 1

 <?php set_time_limit(0); //This is the file where we save the information $fp = fopen (dirname(__FILE__) . '/localfile.tmp', 'w+'); //Here is the file we are downloading, replace spaces with %20 $ch = curl_init(str_replace(" ","%20",$url)); curl_setopt($ch, CURLOPT_TIMEOUT, 50); // write curl response to file curl_setopt($ch, CURLOPT_FILE, $fp); curl_setopt($ch, CURLOPT_FOLLOWLOCATION, true); // get curl response curl_exec($ch); curl_close($ch); fclose($fp); ?> 

Answer 2

I use this handy feature:

By downloading it in increments of 4094 bytes, it will not fill your memory

 function download($file_source, $file_target) { $rh = fopen($file_source, 'rb'); $wh = fopen($file_target, 'w+b'); if (!$rh || !$wh) { return false; } while (!feof($rh)) { if (fwrite($wh, fread($rh, 4096)) === FALSE) { return false; } echo ' '; flush(); } fclose($rh); fclose($wh); return true; } 

Using:

  $result = download('http://url','path/local/file'); 

Then you can check if everything is ok:

  if (!$result) throw new Exception('Download error...'); 

Answer 3

Find the code below if you want to download the contents of the specified URL, also want to save it in a file.

 <?php $ch = curl_init(); /** * Set the URL of the page or file to download. */ curl_setopt($ch, CURLOPT_URL,'http://news.google.com/news?hl=en&topic=t&output=rss'); $fp = fopen('rss.xml', 'w+'); /** * Ask cURL to write the contents to a file */ curl_setopt($ch, CURLOPT_FILE, $fp); curl_exec ($ch); curl_close ($ch); fclose($fp); ?> 

If you want to download a file from an FTP server, you can use the php FTP extension. Code below:

 <?php $SERVER_ADDRESS=""; $SERVER_USERNAME=""; $SERVER_PASSWORD=""; $conn_id = ftp_connect($SERVER_ADDRESS); // login with username and password $login_result = ftp_login($conn_id, $SERVER_USERNAME, $SERVER_PASSWORD); $server_file="test.pdf" //FTP server file path $local_file = "new.pdf"; //Local server file path ##----- DOWNLOAD $SERVER_FILE AND SAVE TO $LOCAL_FILE--------## if (ftp_get($conn_id, $local_file, $server_file, FTP_BINARY)) { echo "Successfully written to $local_file\n"; } else { echo "There was a problem\n"; } ftp_close($conn_id); ?> 

Answer 4

when curl used to load a large file, then CURLOPT_TIMEOUT is the main parameter that you must set.

CURLOPT_RETURNTRANSFER should be true if you get a file, such as pdf / csv / image, etc.

Here you can find additional information (correct url) Curl Doc

From this page:

 curl_setopt($request, CURLOPT_TIMEOUT, 300); //set timeout to 5 mins curl_setopt($request, CURLOPT_RETURNTRANSFER, true); // true to get the output as string otherwise false 

Answer 5

You can use this function, which creates a temporary file in the file system and returns the path to the downloaded file if everything works fine:

 function getFileContents($url) { // Workaround: Save temp file $img = tempnam(sys_get_temp_dir(), 'pdf-'); $img .= '.' . pathinfo($url, PATHINFO_EXTENSION); $fp = fopen($img, 'w+'); $ch = curl_init(); curl_setopt($ch, CURLOPT_SSL_VERIFYHOST, 0); curl_setopt($ch, CURLOPT_SSL_VERIFYPEER, 0); curl_setopt($ch, CURLOPT_URL, $url); curl_setopt($ch, CURLOPT_FILE, $fp); curl_setopt($ch, CURLOPT_HEADER, false); curl_setopt($ch, CURLOPT_FOLLOWLOCATION, true); $result = curl_exec($ch); curl_close($ch); fclose($fp); return $result ? $img : false; }