Migrating a large file via SFTP to PHP

Question

I have a large file (200 MB up). I need to migrate it through PHP cron job. Using Phpseclib gives the following error:

Allowed memory size of 134217728 bytes has been exhausted (trying to allocate 4133 bytes) in / app / vendor / phpseclib / phpseclib / phpseclib / Net / SSH 2.php

Is there any way to do this with PHP cron job ?

Simple one-line code, where $ localFile is an existing CSV file

 $sftp->put('/Import/coupons/coupons_import_test.csv', $localFile, NET_SFTP_LOCAL_FILE); 

PS . This needs to be done after PHP created this file in the /tmp folder, so the script transition time should come into play.

[Change] I do not intend to increase the memory limit, since later files may have higher sizes. A solution in which I can transfer the file in parts (add mode) or use some shell script with PHP cron may be useful

The file size on the remote server is 111.4 MB, and the actual file is much larger on the local one.

[Change after fixing] The problem disappeared after switching to version 2.0.2 from version 1.0 I had to change the code for put

 $sftp->put('/Import/coupons/coupons_import.csv', $localFile, $sftp::SOURCE_LOCAL_FILE); 

Answer 1

Phpseclib should be able to transfer large files perfectly without the need to increase the available memory.

I think that you probably got into the old error "SSH2: do not consider the length of the data by the size of the window." Most likely, you can use the older version of Phpseclib (the older error version is even related to relatively new software, for example, Magento 1.9. *)

Check your version if it is not the latest version from https://github.com/phpseclib/phpseclib

Answer 2

Instead of seeing the code that you are using, I assume that you are trying to load a 200mb file as a string, and you are trying to download it as a string. eg. $sftp->put('filename.remote', file_get_contents('filename.local')); . If yes, try this instead:

 $sftp->put('filename.remote', 'filename.local', NET_SFTP_LOCAL_FILE); 

Answer 3

This has changed a bit with phpseclib> = 2.0.5. You must enable the namespace use phpseclib\Net\SFTP; and then do something like this:

 $sftp->put('filename.remote', 'filename.local', SFTP::SOURCE_LOCAL_FILE); 

Answer 4

Phpseclib does not work for me in Drupal7 and PHP7. This generated an error: "Net_SFTP class not found." Even other classes that I can use are included just like I included the SFTP.php class. I am not sure that I am wrong.