How can I parse data using Symfony2?

Question

I am writing a web application in PHP using Symfony2. The user can upload a CSV file with data that is stored in the database. Parsing each line of a CSV file takes about 0.2 seconds because I make some requests to the Google Maps API.

Therefore, when you download a CSV file with 5000 lines, which is the real case in my application, it may take 16 minutes to analyze the entire file.

I do not want the user to wait 16 minutes until he can continue to use my application. So my question is: how can I parse a CSV file in the background so that the user can continue browsing?

Answer 1

Load the load into the job into the job queue table and Command , regularly run by cron , which handles any work in the job queue table.

As processing continues during the task, you can update the task table so that the user can check and see the progress (for example, you could have an ajax progress bar) and find out when the task will be completed.

This way you also remove the download from processing, and you can control how many jobs are processed at once. Having long runs of jobs directly from user input without using a throttling system / queues is a great way to open yourself up to a denial of service attack ...

Answer 2

You can create a kernel.terminate event kernel.terminate and parse there. This event is triggered when a response is sent to the browser. An example implementation will be

Service Announcement,

 //services.yml csv_parse_listener: class: FQCN\Of\CsvParseListener tags: - { name: kernel.event_listener, event: kernel.terminate, method: onKernelTerminate } 

Listener class

 namespace Your\namespace; use Symfony\Component\HttpKernel\Event\PostResponseEvent; class CsvParseListener { public function onKernelTerminate(PostResponseEvent $event) { $request = $event->getRequest(); if($request->get('_route') !== "Your_route"){ return; } $csvFile = $request->files->get('file_field_name'); //move file using $csvFile->move() //read and parse } } 

Answer 3

You can write a script that is only intended for processing a CSV file, and exec (), which is a script from a script that controls loading. On * IX systems, you can make a command run by exec () running in the background by adding the and character.

You probably also want to include a script that allows the user to check the progress of processing.