Guidelines for using a production server with php application

Question

I created a PHP application, and I read that it is better to use a “working” server + queue when calling api or performing lengthy operations.

A quick search for a textbook turned out to be dry. I created my application using codeigniter and I make various calls on facebook api + using php based image processing throughout the application. The only thing that interests me is how to help me the queue server + worker if I make api calls or resize my image and the user usually does not want to receive a response from my server until it is complete.

What situations would be good candidates for a server with a working + server, and are there any guidelines for including them in my application? I recently included memcache in my application, it was trivially easy. I just wrapped my sql queries with a memcache handler.

Answer 1

In the example you described (image resizing), you basically keep the Apache connection open for the time it takes to resize the image. Apache processes are expensive and in order to make your system as scalable as possible, you should strive to keep your web requests / responses as short as possible. Another idea is that with a queue you can control concurrency. What if 100 users upload an image at the same time? can your server handle this? If you have a working (server) server to process these requests, you can only allow X parallel jobs to run.

The same goes for web service requests: instead of staying open, you basically unload the web service call execution into the workflow, this frees up the apache process, and you can implement an AJAX polling mechanism that checks if the request is complete, provided by the server server for the web service. In the long run, the system will scale better, and users usually do not like to wait for an operation to complete without any feedback on where it is located. Queuing allows you to execute a task asynchronously and provide the visitor with feedback on where the task completion status is located.

I usually work with the Zend Server Job queue ( http://devzone.zend.com/article/11907 and http://devzone.zend.com/article/11907 ), which is available with the full version of Zend Server (commercial). However, Gearman also does a great job of this and has a PHP extension: http://php.net/manual/en/book.gearman.php and an example: http://www.php.net/manual/en/gearmanclient .do.php .

Hope this helps.

- EDIT -

@Casey, I started adding a comment, but realized that it would quickly become too long a response, so I edited the answer. I just read a document for cloud management that I do not know. However, fortunately, I used Codeigniter quite widely, so I will try to hack the answer for you:

1- The concept of Cloudcontrol for the desktop is to run a php script from the command line. Therefore, you need Codeigniter to accept the script call from the command line and send it to the dispatcher. You probably want to limit this to one controller. See code: http://pastebin.com/GZigWbT3 This file essentially does what the CI index.php file does, except that it emulates a request using the $_REQUEST['SERVER_URI'] parameter. Be sure to place this file outside your document root and adjust the $system_folder variable $system_folder .

2- You need a script.php controller in the controllers folder from which you disable web requests. You can do something:

 <?php class script extends CI_Controller { public function __construct() { if(php_sapi_name() !== 'cli') { show_404(); } parent::__construct(); } public function resizeImage($arg1, $arg2) { //Whatever logic to resize image, or library call to do so. } } 

3. The last part is designed to develop a wrapper library in CI (in your system / application / libraries folder) that would effectively wrap the functionality of calling the CloudController working body

  public function _construct() { $ci = get_instance(); //add check to make sure that the value is set in the configuration //Ideally since this is a library, pass the app_name in a setter to avoid creating a dependancy on the config object. //Somewhere in one of your config files add $config['app_name'] = 'YOUR_APP_NAME/YOUR_DEP_NAME'; //where APP_NAME and DEP_NAME are cloud controller app_name and dep_name $this->_app_name = $ci->config->item('app_name'); //Also add: $config['utilities_script'] = 'path/to/utilities.php'; //This is the script created in step 1 $this->_utilities_script = $ci->config->item('utilities_script'); } public function run() { $args = func_get_args(); if(count($args) < 1 ) { //We expect at least one arg which would be the command name trigger_error('Run expects at least one argument', E_USER_ERROR); } $method = array_shift($args); //utilities.php is the file created in step 1 $command = "cctrlapp " . $this->_app_name . " worker.add ".$this->_utilities_script; //Add arguments if any $command .= ' "'.implode(' ', $args).'"'; //finally... exec($command); } } 

4 Now from anywhere in your code where you really want to queue the task, if from the controller:

 $this->load->library('Worker'); //resizeImage will call the method resizeImage in the script controller. $this->worker->run('resizeImage', $width, $height); 

Note that:
1- It could be polished further, it was really to give you an idea of ​​how this can be done
2- Since I do not have a cloudcontroller account, I have no way to verify the code, so configuration may be required. Utilities.phph script I use in my projects so that it is good.
Good luck

Answer 2

If you do not need special configuration of the work / queue server, you can create a small library for your codeigniter installation to manage a simple work queue.

During the initial client request, you verify that the generated image or the deleted file in the cache does not need to (re) generate and maintain the files. If you need to build the file or image, you will tell the queue library to add it to the queue, and then close the connection to the browser. However, you are still processing the queue at the end of your controller during the same request. Thus, you do not need a separate queue and a working server.

For me, comments are at http://www.php.net/manual/en/features.connection-handling.php , where it is very useful. You basically do something like the following: (proof of concept, see Link for details)

 header("Connection: close\r\n"); // close the connection ob_end_flush(); // flush everything ob_flush(); flush(); set_time_limit(300); // set a nicer time-out for the queue-worker $this->queue_lib->process(); // do processing sleep(5); // or get some of that much needed sleep echo 'Text user will never see'; 

During development and debugging, you can temporarily disconnect part of the closed connection and see any output. For production, you can use log_message ().

Queue-library functionality (notes on the / self encoder): when adding a file to the queue, the queue library should check if the file can already be in the queue. Since in this setting workers start asynchronously (many different browser connections), when the worker starts processing the task, he must set the work status as “processing” so that no other worker starts working on the same job. Alternatively, you can configure the next one the queue by setting the general status of the queue to “processing queue” (one employee at a time). Timeouts for jobs (or for the general queue) are probably a good idea, too, and the timeout should be slightly larger than set_time_limit (). This way you can find out when the task might fail, and update the error log. Pre-clean the process queue to make sure that they are processed and do not go beyond the timeout.

Note. From the same linked page, if you are working with files on the local file system and at the same time want to use ignore_user_abort (true) or register_shutdown_function (), it seems advisable to save the working directory first. $ cwd = getcwd ();

change
found a good starting point for the job library: http://www.andy-russell.com/job-scheduler-library