Geek Answers Handbook

How to use AJAX to download a large CSV file?

We have several CSV files with a lot of records. You must upload these files to the MySQL database using the web interface. These files are collected from various field work and uploaded to the server through a web application. The web application is developed using PHP.

Initially, the number of entries was very small. But now it has become very large, and users were not able to upload the file to the server.

Problems -

  • We need to wait for the entire file to load to begin processing.
  • We can see data validation errors after downloading the entire file.
  • Downloading large files is not supported by default.
  • PHP run time is limited.

So, I am looking for a solution to download the contents of a CSV file as a fragment. This will help me process a small amount of data for each request.

My idea is to read data from a CSV file using JavaScript. Then POST this data using AJAX.

Want to see data validation errors for a small amount of data and, if necessary, stop execution.

0
source share
1 answer

After doing some research on the Internet, I found the HTML5 JavaScript API files very useful for reading content from the local file system. I wrote a small script and it works great.

In this example, I created two files -

  • index.htm
  • service.php

index.htm

<!DOCTYPE html>
<html>
	<head>
		<meta charset="UTF-8">
		<title>Upload large CSV file</title>
		<style>
			body{font-size:8pt; color:#333}
			#wrap{width:500px; margin:5px auto}
			#responce{height:200px; overflow-y:scroll; border:1px solid #ddd;}
		</style>
	</head>

	<body>
		<div id="wrap">
			<ul id="responce">
				
			</ul><!-- // response -->
			
			<input type="file" id="fileSelected"/>
			<button id="btnUpload">Upload</button>
		</div><!-- // wrap -->
		
		<script
			  src="https://code.jquery.com/jquery-3.1.1.min.js"
			  integrity="sha256-hVVnYaiADRTO2PzUGmuLJr8BLUSjGIZsDYGmIJLv2b8="
			  crossorigin="anonymous"></script>
		<script>
			var reader_offset = 0;		//current reader offset position
			var buffer_size = 1024;		//
			var pending_content = '';
			
			/**
			* Read a chunk of data from current offset.
			*/
			function readAndSendChunk()
			{
				var reader = new FileReader();
				var file = $('#fileSelected')[0].files[0];
				reader.onloadend = function(evt){
					
					//check for end of file
					if(evt.loaded == 0) return;
					
					//increse offset value
					reader_offset += evt.loaded;
					
					//check for only complete line
					var last_line_end = evt.target.result.lastIndexOf('\n');
					var content = pending_content + evt.target.result.substring(0, last_line_end);
					
					pending_content = evt.target.result.substring(last_line_end+1, evt.target.result.length);
					
					//upload data
					send(content);
				};
				var blob = file.slice(reader_offset, reader_offset + buffer_size);
				reader.readAsText(blob);
			}
			
			/**
			* Send data to server using AJAX
			*/
			function send(data_chank)
			{
				$.ajax({
					url: "service.php",
					method: 'POST',
					data: data_chank
				}).done(function(response) {
				
					//show responce message received from server
					$( '#responce' ).append( '<li>' + response + '</li>');
					
					//try for next chank
					readAndSendChunk();
				});
			}
			
			/**
			* On page load
			*/
			$(function(){
				$('#btnUpload').click(function(){
					reader_offset = 0;
					pending_content = '';
					readAndSendChunk();
				});
			});
		</script>
	</body>
</html>
Run codeHide result

service.php

<?php
$content = file_get_contents('php://input');
$lines = explode("\n", $content);
foreach($lines as $line){
    $csv_row = str_getcsv($line);
    //save data into database
    //----
}
+1
source

More articles:


All Articles