Server error while loading large files using PHP

I use the PHPSecLib SFTP functions to upload files from an FTP server.

Line

$sftp->get($fname); 

It works if the file is up to 200 MB, but if it is 300 MB, the browser says "Firefox cannot find the file in [download.php]". That is, he says that he cannot find the php file that I use to download the remote file.

At first I thought that this was due to setting memory_limit in php.ini, but it doesn’t matter if it is set to 128M or 350M; 200 MB files still work, and 300 MB files fail. And it fails after ten seconds, so max_execution_time or max_input_time does not seem to be the culprits. What could be wrong?

0
source share
2 answers

First, I highly recommend that you add the code set_time_limit(0); to the very top of your php file (even before it is included), since you are dealing with operations, you do not know how much time they will take.

I would say that this is a case of a web server / browser exit in the absence of sending / receiving any data for a "long" period of time. To fix this, we need to slightly modify the SFTP.php file, namely the Net_SFTP class, then go to the get method (line 1482, if you have phpseclib 0.3.1) and add some things inside the single "while" control structure (I will introduce entire function below) to add the following code:

 if (strtolower(PHP_SAPI) != 'cli') { // run this if request is handled by a webserver (like your case) $my_iter++; if($my_iter > 1024){ $my_iter = 0; // reset the counter echo "transferring ... " . date("G:i:s") . "<br />"; // send something to the buffer } // flush the buffers and prevent the timeout by actually outputting something to the browser ob_end_flush(); ob_flush(); flush(); ob_start(); usleep(100); // just in case, try removing this delay } 

Which basically outputs something from time to time (1024 iterations of that time) and flushes the buffers to actually output something to the browser. Feel free to change the values. Because of these issues, this is code (SFTP class) that should not be run from the web server. I mean, you CAN, but you will run into some problems, like this one.

In addition, if you try to send (), you will have to make a similar modification using the appropriate method, but I hope this will solve your problem (at least this fixed my timeout problems here in my local dev block).

Now a complete modification of the method goes below, as promised; -)

 function get($remote_file, $local_file = false) { if (!($this->bitmap & NET_SSH2_MASK_LOGIN)) { return false; } $remote_file = $this->_realpath($remote_file); if ($remote_file === false) { return false; } $packet = pack('Na*N2', strlen($remote_file), $remote_file, NET_SFTP_OPEN_READ, 0); if (!$this->_send_sftp_packet(NET_SFTP_OPEN, $packet)) { return false; } $response = $this->_get_sftp_packet(); switch ($this->packet_type) { case NET_SFTP_HANDLE: $handle = substr($response, 4); break; case NET_SFTP_STATUS: // presumably SSH_FX_NO_SUCH_FILE or SSH_FX_PERMISSION_DENIED $this->_logError($response); return false; default: user_error('Expected SSH_FXP_HANDLE or SSH_FXP_STATUS', E_USER_NOTICE); return false; } if ($local_file !== false) { $fp = fopen($local_file, 'wb'); if (!$fp) { return false; } } else { $content = ''; } $read = 0; while (true) { if (strtolower(PHP_SAPI) != 'cli') { // run this if request is handled by a webserver (like your case) $my_iter++; if($my_iter > 1024){ $my_iter = 0; // reset the counter echo "transferring ... " . date("G:i:s") . "<br />"; // send something to the buffer } // flush the buffers and prevent the timeout by actually outputting something to the browser ob_end_flush(); ob_flush(); flush(); ob_start(); usleep(100); // just in case, try removing this delay } $packet = pack('Na*N3', strlen($handle), $handle, 0, $read, 1 << 20); if (!$this->_send_sftp_packet(NET_SFTP_READ, $packet)) { if ($local_file !== false) { fclose($fp); } return false; } $response = $this->_get_sftp_packet(); switch ($this->packet_type) { case NET_SFTP_DATA: $temp = substr($response, 4); $read+= strlen($temp); if ($local_file === false) { $content.= $temp; } else { fputs($fp, $temp); } break; case NET_SFTP_STATUS: $this->_logError($response); break 2; default: user_error('Expected SSH_FXP_DATA or SSH_FXP_STATUS', E_USER_NOTICE); if ($local_file !== false) { fclose($fp); } return false; } } if ($local_file !== false) { fclose($fp); } if (!$this->_send_sftp_packet(NET_SFTP_CLOSE, pack('Na*', strlen($handle), $handle))) { return false; } $response = $this->_get_sftp_packet(); if ($this->packet_type != NET_SFTP_STATUS) { user_error('Expected SSH_FXP_STATUS', E_USER_NOTICE); return false; } $this->_logError($response); // check the status from the NET_SFTP_STATUS case in the above switch after the file has been closed if ($status != NET_SFTP_STATUS_OK) { return false; } if (isset($content)) { return $content; } return true; } 
+1
source

Something else you can do ...

 <?php include('Net/SFTP.php'); $sftp = new Net_SFTP('www.domain.tld'); $sftp->login('username', 'password'); $start = 0; while (true) { $response = $sftp->get('1mb', false, $start, 1024); $start+= 1024; if (empty($response)) { break; } echo $response; } 

T. Upload the file in several pieces.

+1
source

All Articles