Server-side processing with Datatables v1.10.0

Question

Server-side processing with Datatables v1.10.0

Hi, I am having some problems related to the fact that the server-side functions for processing data tables work with SQL Server.

I have a test page that should display two columns from a database table (for now).

HTML:

<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd"> <html xmlns="http://www.w3.org/1999/xhtml"> <head> <title></title> <link rel="Stylesheet" type="text/css" href="DataTables-1.10.0/media/css/jquery.dataTables.min.css" /> </head> <body> <table id="example" class="display" cellspacing="0" width="100%"> <thead> <tr> <th align="center">PK</th> <th align="center">Network</th> </tr> </thead> <tfoot> <tr> <th align="center">PK</th> <th align="center">Network</th> </tr> </tfoot> </table> </body> <script type="text/javascript" src="DataTables-1.10.0/media/js/jquery.js"></script> <script type="text/javascript" src="DataTables-1.10.0/media/js/jquery.dataTables.min.js"> </script> <script type="text/javascript" charset="utf-8"> $(document).ready(function () { $('#example').dataTable({ "processing": true, "bServerSide": true, "ajax": "PHP/testGetArchive.php" }); }); </script> </html>

I am using the sample code found on the website. Here for the server side functions:

http://next.datatables.net/examples/server_side/simple.html

This is my version of the php called page:

 <?php /* * DataTables example server-side processing script. * * Please note that this script is intentionally extremely simply to show how * server-side processing can be implemented, and probably shouldn't be used as * the basis for a large complex system. It is suitable for simple use cases as * for learning. * * See http://datatables.net/usage/server-side for full details on the server- * side processing requirements of DataTables. * * @license MIT - http://datatables.net/license_mit */ /* * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * Easy set variables */ // DB table to use $table = 'tblViews'; // Table primary key $primaryKey = 'PK'; // Array of database columns which should be read and sent back to DataTables. // The `db` parameter represents the column name in the database, while the `dt` // parameter represents the DataTables column identifier. In this case simple // indexes $columns = array( array( 'db' => 'PK', 'dt' => 0 ), array( 'db' => 'Network', 'dt' => 1 ) ); // SQL server connection information $sql_details = array( 'user' => '******', 'pass' => '******', 'db' => '******db', 'host' => '******\SQLEXPRESS' ); /* * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * If you just want to use the basic configuration for DataTables with PHP * server-side, there is no need to edit below this line. */ require( 'ssp.class.php' ); echo json_encode( SSP::simple( $_GET, $sql_details, $table, $primaryKey, $columns ) );

He then calls the second PHP example found here:

https://github.com/DataTables/DataTables/blob/master/examples/server_side/scripts/ssp.class.php

Here is my copy. The only Modification that I performed was to remove the block of code that is required for the examples.

 <?php /* * Helper functions for building a DataTables server-side processing SQL query * * The static functions in this class are just helper functions to help build * the SQL used in the DataTables demo server-side processing scripts. These * functions obviously do not represent all that can be done with server-side * processing, they are intentionally simple to show how it works. More complex * server-side processing operations will likely require a custom script. * * See http://datatables.net/usage/server-side for full details on the server- * side processing requirements of DataTables. * * @license MIT - http://datatables.net/license_mit */ class SSP { /** * Create the data output array for the DataTables rows * * @param array $columns Column information array * @param array $data Data from the SQL get * @return array Formatted data in a row based format */ static function data_output ( $columns, $data ) { $out = array(); for ( $i=0, $ien=count($data) ; $i<$ien ; $i++ ) { $row = array(); for ( $j=0, $jen=count($columns) ; $j<$jen ; $j++ ) { $column = $columns[$j]; // Is there a formatter? if ( isset( $column['formatter'] ) ) { $row[ $column['dt'] ] = $column['formatter']( $data[$i][ $column['db'] ], $data[$i] ); } else { $row[ $column['dt'] ] = $data[$i][ $columns[$j]['db'] ]; } } $out[] = $row; } return $out; } /** * Paging * * Construct the LIMIT clause for server-side processing SQL query * * @param array $request Data sent to server by DataTables * @param array $columns Column information array * @return string SQL limit clause */ static function limit ( $request, $columns ) { $limit = ''; if ( isset($request['start']) && $request['length'] != -1 ) { $limit = "LIMIT ".intval($request['start']).", ".intval($request['length']); } return $limit; } /** * Ordering * * Construct the ORDER BY clause for server-side processing SQL query * * @param array $request Data sent to server by DataTables * @param array $columns Column information array * @return string SQL order by clause */ static function order ( $request, $columns ) { $order = ''; if ( isset($request['order']) && count($request['order']) ) { $orderBy = array(); $dtColumns = SSP::pluck( $columns, 'dt' ); for ( $i=0, $ien=count($request['order']) ; $i<$ien ; $i++ ) { // Convert the column index into the column data property $columnIdx = intval($request['order'][$i]['column']); $requestColumn = $request['columns'][$columnIdx]; $columnIdx = array_search( $requestColumn['data'], $dtColumns ); $column = $columns[ $columnIdx ]; if ( $requestColumn['orderable'] == 'true' ) { $dir = $request['order'][$i]['dir'] === 'asc' ? 'ASC' : 'DESC'; $orderBy[] = '`'.$column['db'].'` '.$dir; } } $order = 'ORDER BY '.implode(', ', $orderBy); } return $order; } /** * Searching / Filtering * * Construct the WHERE clause for server-side processing SQL query. * * NOTE this does not match the built-in DataTables filtering which does it * word by word on any field. It possible to do here performance on large * databases would be very poor * * @param array $request Data sent to server by DataTables * @param array $columns Column information array * @param array $bindings Array of values for PDO bindings, used in the * sql_exec() function * @return string SQL where clause */ static function filter ( $request, $columns, &$bindings ) { $globalSearch = array(); $columnSearch = array(); $dtColumns = SSP::pluck( $columns, 'dt' ); if ( isset($request['search']) && $request['search']['value'] != '' ) { $str = $request['search']['value']; for ( $i=0, $ien=count($request['columns']) ; $i<$ien ; $i++ ) { $requestColumn = $request['columns'][$i]; $columnIdx = array_search( $requestColumn['data'], $dtColumns ); $column = $columns[ $columnIdx ]; if ( $requestColumn['searchable'] == 'true' ) { $binding = SSP::bind( $bindings, '%'.$str.'%', PDO::PARAM_STR ); $globalSearch[] = "`".$column['db']."` LIKE ".$binding; } } } // Individual column filtering for ( $i=0, $ien=count($request['columns']) ; $i<$ien ; $i++ ) { $requestColumn = $request['columns'][$i]; $columnIdx = array_search( $requestColumn['data'], $dtColumns ); $column = $columns[ $columnIdx ]; $str = $requestColumn['search']['value']; if ( $requestColumn['searchable'] == 'true' && $str != '' ) { $binding = SSP::bind( $bindings, '%'.$str.'%', PDO::PARAM_STR ); $columnSearch[] = "`".$column['db']."` LIKE ".$binding; } } // Combine the filters into a single string $where = ''; if ( count( $globalSearch ) ) { $where = '('.implode(' OR ', $globalSearch).')'; } if ( count( $columnSearch ) ) { $where = $where === '' ? implode(' AND ', $columnSearch) : $where .' AND '. implode(' AND ', $columnSearch); } if ( $where !== '' ) { $where = 'WHERE '.$where; } return $where; } /** * Perform the SQL queries needed for an server-side processing requested, * utilising the helper functions of this class, limit(), order() and * filter() among others. The returned array is ready to be encoded as JSON * in response to an SSP request, or can be modified if needed before * sending back to the client. * * @param array $request Data sent to server by DataTables * @param array $sql_details SQL connection details - see sql_connect() * @param string $table SQL table to query * @param string $primaryKey Primary key of the table * @param array $columns Column information array * @return array Server-side processing response array */ static function simple ( $request, $sql_details, $table, $primaryKey, $columns ) { $bindings = array(); $db = SSP::sql_connect( $sql_details ); // Build the SQL query string from the request $limit = SSP::limit( $request, $columns ); $order = SSP::order( $request, $columns ); $where = SSP::filter( $request, $columns, $bindings ); // Main query to actually get the data $data = SSP::sql_exec( $db, $bindings, "SELECT SQL_CALC_FOUND_ROWS `".implode("`, `", SSP::pluck($columns, 'db'))."` FROM `$table` $where $order $limit" ); // Data set length after filtering $resFilterLength = SSP::sql_exec( $db, "SELECT FOUND_ROWS()" ); $recordsFiltered = $resFilterLength[0][0]; // Total data set length $resTotalLength = SSP::sql_exec( $db, "SELECT COUNT(`{$primaryKey}`) FROM `$table`" ); $recordsTotal = $resTotalLength[0][0]; /* * Output */ return array( "draw" => intval( $request['draw'] ), "recordsTotal" => intval( $recordsTotal ), "recordsFiltered" => intval( $recordsFiltered ), "data" => SSP::data_output( $columns, $data ) ); } /** * Connect to the database * * @param array $sql_details SQL server connection details array, with the * properties: * * host - host name * * db - database name * * user - user name * * pass - user password * @return resource Database connection handle */ static function sql_connect ( $sql_details ) { try { $db = @new PDO( "mysql:host={$sql_details['host']};dbname={$sql_details['db']}", $sql_details['user'], $sql_details['pass'], array( PDO::ATTR_ERRMODE => PDO::ERRMODE_EXCEPTION ) ); } catch (PDOException $e) { SSP::fatal( "An error occurred while connecting to the database. ". "The error reported by the server was: ".$e->getMessage() ); } return $db; } /** * Execute an SQL query on the database * * @param resource $db Database handler * @param array $bindings Array of PDO binding values from bind() to be * used for safely escaping strings. Note that this can be given as the * SQL query string if no bindings are required. * @param string $sql SQL query to execute. * @return array Result from the query (all rows) */ static function sql_exec ( $db, $bindings, $sql=null ) { // Argument shifting if ( $sql === null ) { $sql = $bindings; } $stmt = $db->prepare( $sql ); //echo $sql; // Bind parameters if ( is_array( $bindings ) ) { for ( $i=0, $ien=count($bindings) ; $i<$ien ; $i++ ) { $binding = $bindings[$i]; $stmt->bindValue( $binding['key'], $binding['val'], $binding['type'] ); } } // Execute try { $stmt->execute(); } catch (PDOException $e) { SSP::fatal( "An SQL error occurred: ".$e->getMessage() ); } // Return all return $stmt->fetchAll(); } /* * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * Internal methods */ /** * Throw a fatal error. * * This writes out an error message in a JSON string which DataTables will * see and show to the user in the browser. * * @param string $msg Message to send to the client */ static function fatal ( $msg ) { echo json_encode( array( "error" => $msg ) ); exit(0); } /** * Create a PDO binding key which can be used for escaping variables safely * when executing a query with sql_exec() * * @param array &$a Array of bindings * @param * $val Value to bind * @param int $type PDO field type * @return string Bound key to be used in the SQL where this parameter * would be used. */ static function bind ( &$a, $val, $type ) { $key = ':binding_'.count( $a ); $a[] = array( 'key' => $key, 'val' => $val, 'type' => $type ); return $key; } /** * Pull a particular property from each assoc. array in a numeric array, * returning and array of the property values from each item. * * @param array $a Array to get data from * @param string $prop Property to read * @return array Array of property values */ static function pluck ( $a, $prop ) { $out = array(); for ( $i=0, $len=count($a) ; $i<$len ; $i++ ) { $out[] = $a[$i][$prop]; } return $out; } }

I keep making mistakes saying that the code cannot find the driver, although I have the sqlserv and pdo_sqlsrv drivers installed in my php environment. Is there something wrong with the code causing this error? Are my drivers wrong? Any help with this would be greatly appreciated. I got 65,000 rows of data to process and it would be impossible to send everything to the client at a time.

+7

jquery sql php datatable

Tinydan May 07 '14 at 11:26

source share

1 answer

Tinydan · Accepted Answer · 2014-05-20T16:00:10+0000

It took me some time. But I found out where I was wrong, and now I have DataTables working with SQL Server through server-side scripts. I am posting this solution in the hope that it will help someone else like me, I am having problems. I have broken my answer into pieces.

PHP environment

SQLSRV drivers for php can be found here . Download the SQLSRV30.EXE installation package. You may find that when you try to run this executable, you will receive the error message "This is not a valid win32 application." If so, unzip the executable with something like 7-zip. As a result, the file will have the files you need inside.

When you unpacked the package, you need to select the correct driver. Most Windows installations use unsafe drivers:

php version 5.3:

php_sqlsrv_53_nts.dll

php_pdo_sqlsrv_53_nts.dll

php version 5.4:

php_sqlsrv_54_nts.dll

php_pdo_sqlsrv_54_nts.dll

Copy the appropriate files to the ext folder in your php directory. Now modify the php.ini file to have a link to these files. Do this by adding an entry to the dynamic extensions section. The result will be something like this:

 extension=php_sqlsrv_54_nts.dll

Then add the section for the driver in the settings section of the module section, for example:

 [sqlsrv] sqlsrv.LogSubSystems=-1 sqlsrv.LogSeverity=-1 sqlsrv.WarningsReturnAsErrors=0

Documentation on these parameters can be found here .

After you add these drivers and add a link to them in the php.ini file, you should also make sure that the Microsoft SQL Server 2012 client profile is also installed.

These links were taken from PHP.net:

Microsoft SQL Server 2012 x86 Client Profile Microsoft SQL Server 2012 x64 Client Profile

After completing these steps, restart the web server. The driver should now be installed and ready to use. You can verify this using the info.php page.

Server side Script:

Now that the web server is configured to use the SQL SRV driver, we can now use it to query the SQL Server database. I used the server side of the script here . Here are a few issues I found with him:

 <?php /* Indexed column (used for fast and accurate table cardinality) */ $sIndexColumn = ""; /* DB table to use */ $sTable = ""; /* Database connection information */ $gaSql['user'] = ""; $gaSql['password'] = ""; $gaSql['db'] = ""; $gaSql['server'] = ""; /* * Columns * If you don't want all of the columns displayed you need to hardcode $aColumns array with your elements. * If not this will grab all the columns associated with $sTable */ $aColumns = array(); /* * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * If you just want to use the basic configuration for DataTables with PHP server-side, there is * no need to edit below this line */ /* * ODBC connection */ $connectionInfo = array("UID" => $gaSql['user'], "PWD" => $gaSql['password'], "Database"=>$gaSql['db'],"ReturnDatesAsStrings"=>true); $gaSql['link'] = sqlsrv_connect( $gaSql['server'], $connectionInfo); $params = array(); $options = array( "Scrollable" => SQLSRV_CURSOR_KEYSET ); /* Ordering */ $sOrder = ""; if ( isset( $_GET['iSortCol_0'] ) ) { $sOrder = "ORDER BY "; for ( $i=0 ; $i<intval( $_GET['iSortingCols'] ) ; $i++ ) { if ( $_GET[ 'bSortable_'.intval($_GET['iSortCol_'.$i]) ] == "true" ) { $sOrder .= $aColumns[ intval( $_GET['iSortCol_'.$i] ) ]." ".addslashes( $_GET['sSortDir_'.$i] ) .", "; } } $sOrder = substr_replace( $sOrder, "", -2 ); if ( $sOrder == "ORDER BY" ) { $sOrder = ""; } } /* Filtering */ $sWhere = ""; if ( isset($_GET['sSearch']) && $_GET['sSearch'] != "" ) { $sWhere = "WHERE ("; for ( $i=0 ; $i<count($aColumns) ; $i++ ) { $sWhere .= $aColumns[$i]." LIKE '%".addslashes( $_GET['sSearch'] )."%' OR "; } $sWhere = substr_replace( $sWhere, "", -3 ); $sWhere .= ')'; } /* Individual column filtering */ for ( $i=0 ; $i<count($aColumns) ; $i++ ) { if ( isset($_GET['bSearchable_'.$i]) && $_GET['bSearchable_'.$i] == "true" && $_GET['sSearch_'.$i] != '' ) { if ( $sWhere == "" ) { $sWhere = "WHERE "; } else { $sWhere .= " AND "; } $sWhere .= $aColumns[$i]." LIKE '%".addslashes($_GET['sSearch_'.$i])."%' "; } } /* Paging */ $top = (isset($_GET['iDisplayStart']))?((int)$_GET['iDisplayStart']):0 ; $limit = (isset($_GET['iDisplayLength']))?((int)$_GET['iDisplayLength'] ):10; $sQuery = "SELECT TOP $limit ".implode(",",$aColumns)." FROM $sTable $sWhere ".(($sWhere=="")?" WHERE ":" AND ")." $sIndexColumn NOT IN ( SELECT $sIndexColumn FROM ( SELECT TOP $top ".implode(",",$aColumns)." FROM $sTable $sWhere $sOrder ) as [virtTable] ) $sOrder"; $rResult = sqlsrv_query($gaSql['link'],$sQuery) or die("$sQuery: " . sqlsrv_errors()); $sQueryCnt = "SELECT * FROM $sTable $sWhere"; $rResultCnt = sqlsrv_query( $gaSql['link'], $sQueryCnt ,$params, $options) or die (" $sQueryCnt: " . sqlsrv_errors()); $iFilteredTotal = sqlsrv_num_rows( $rResultCnt ); $sQuery = " SELECT * FROM $sTable "; $rResultTotal = sqlsrv_query( $gaSql['link'], $sQuery ,$params, $options) or die(sqlsrv_errors()); $iTotal = sqlsrv_num_rows( $rResultTotal ); $output = array( "sEcho" => intval($_GET['sEcho']), "iTotalRecords" => $iTotal, "iTotalDisplayRecords" => $iFilteredTotal, "aaData" => array() ); while ( $aRow = sqlsrv_fetch_array( $rResult ) ) { $row = array(); for ( $i=0 ; $i<count($aColumns) ; $i++ ) { if ( $aColumns[$i] != ' ' ) { $v = $aRow[ $aColumns[$i] ]; $v = mb_check_encoding($v, 'UTF-8') ? $v : utf8_encode($v); $row[]=$v; } } If (!empty($row)) { $output['aaData'][] = $row; } } echo json_encode( $output ); ?>

Indexed column

When you specify an indexed column to use for searches, make sure it is included in the column array! if you do not specify which columns to use paging, this will not work. Data paging with this code works by executing a query to select all primary keys, if not in the TOP X results from another query.

Connection options

Make sure the connection settings are complete and correct. They are needed to allow the script to connect to the database. If there are no parameters or the parameters are incorrect for entering the SQL server, then the script will never be able to connect to the database.

Column array

I found that using this code without the specified columns returns invalid or NULL data. The best way to stop this is to populate the array with the column names that I would like to select each, enclosed in quotation marks and separated by commas. This is also reasonable, because why send the client anything but the required data?

Client side

HTML

DataTables require a well-formed html table. This means a table with full tags. If all tags are missing for the returned data, then DataTables will return an error. If you have columns that you want to return but not show, you can use exVort ColVis and set the default column setting in java script.

Datatable uses its own CCS file, so be sure to include it!

java script

DataTables uses jQuery and its own Javascrpt file, so make sure you include references to them in your script tags!

 //Datatables Basic server side initilization $(document).ready(function () { //Datatable var table = $('#tableID').DataTable({ "bProcessing": true, "bServerSide": true, "sAjaxSource": "serverSideScript.php" }); });

These are the basic functions necessary for this server side script to work. It will receive the first 10 lines at the initial draw, using the database options that you specify on the php page. Here you can add the necessary extensions, such as ColVis and TableTools . Full documentation for these extensions and other data table initialization options can be found here .

I hope this answer helps everyone who has the same problems as mine.

Server-side processing with Datatables v1.10.0

More articles: