Geek Answers Handbook

Checking UTF-8 in PHP without using preg_match ()

I need to check some user data that is encoded in UTF-8. Many of them recommended using the following code:

preg_match('/\A(
     [\x09\x0A\x0D\x20-\x7E]
   | [\xC2-\xDF][\x80-\xBF]
   |  \xE0[\xA0-\xBF][\x80-\xBF]
   | [\xE1-\xEC\xEE\xEF][\x80-\xBF]{2}
   |  \xED[\x80-\x9F][\x80-\xBF]
   |  \xF0[\x90-\xBF][\x80-\xBF]{2}
   | [\xF1-\xF3][\x80-\xBF]{3}
   |  \xF4[\x80-\x8F][\x80-\xBF]{2}
  )*\z/x', $string);

This is a regular expression taken from http://www.w3.org/International/questions/qa-forms-utf-8 . Everything was fine until I discovered an error in PHP, which, apparently, has been since at least 2006. Preg_match () throws a seg error if the string $ is too long. There seems to be no workaround. You can see the error message here: http://bugs.php.net/bug.php?id=36463

, preg_match, , , . , Stack Overflow, , . :

[13.01.2010]: - , , , . .

function check_UTF8_string(&$string) {
    $len = mb_strlen($string, "ISO-8859-1");
    $ok = 1;

    for ($i = 0; $i < $len; $i++) {
        $o = ord(mb_substr($string, $i, 1, "ISO-8859-1"));

        if ($o == 9 || $o == 10 || $o == 13 || ($o >= 32 && $o <= 126)) {

        }
        elseif ($o >= 194 && $o <= 223) {
            $i++;
            $o2 = ord(mb_substr($string, $i, 1, "ISO-8859-1"));
            if (!($o2 >= 128 && $o2 <= 191)) {
                $ok = 0;
                break;
            }
        }
        elseif ($o == 224) {
            $o2 = ord(mb_substr($string, $i + 1, 1, "ISO-8859-1"));
            $o3 = ord(mb_substr($string, $i + 2, 1, "ISO-8859-1"));
            $i += 2;
            if (!($o2 >= 160 && $o2 <= 191) || !($o3 >= 128 && $o3 <= 191)) {
                $ok = 0;
                break;
            }
        }
        elseif (($o >= 225 && $o <= 236) || $o == 238 || $o == 239) {
            $o2 = ord(mb_substr($string, $i + 1, 1, "ISO-8859-1"));
            $o3 = ord(mb_substr($string, $i + 2, 1, "ISO-8859-1"));
            $i += 2;
            if (!($o2 >= 128 && $o2 <= 191) || !($o3 >= 128 && $o3 <= 191)) {
                $ok = 0;
                break;
            }
        }
        elseif ($o == 237) {
            $o2 = ord(mb_substr($string, $i + 1, 1, "ISO-8859-1"));
            $o3 = ord(mb_substr($string, $i + 2, 1, "ISO-8859-1"));
            $i += 2;
            if (!($o2 >= 128 && $o2 <= 159) || !($o3 >= 128 && $o3 <= 191)) {
                $ok = 0;
                break;
            }
        }
        elseif ($o == 240) {
            $o2 = ord(mb_substr($string, $i + 1, 1, "ISO-8859-1"));
            $o3 = ord(mb_substr($string, $i + 2, 1, "ISO-8859-1"));
            $o4 = ord(mb_substr($string, $i + 3, 1, "ISO-8859-1"));
            $i += 3;
            if (!($o2 >= 144 && $o2 <= 191) ||
                !($o3 >= 128 && $o3 <= 191) ||
                !($o4 >= 128 && $o4 <= 191)) {
                $ok = 0;
                break;
            }
        }
        elseif ($o >= 241 && $o <= 243) {
            $o2 = ord(mb_substr($string, $i + 1, 1, "ISO-8859-1"));
            $o3 = ord(mb_substr($string, $i + 2, 1, "ISO-8859-1"));
            $o4 = ord(mb_substr($string, $i + 3, 1, "ISO-8859-1"));
            $i += 3;
            if (!($o2 >= 128 && $o2 <= 191) ||
                !($o3 >= 128 && $o3 <= 191) ||
                !($o4 >= 128 && $o4 <= 191)) {
                $ok = 0;
                break;
            }
        }
        elseif ($o == 244) {
            $o2 = ord(mb_substr($string, $i + 1, 1, "ISO-8859-1"));
            $o3 = ord(mb_substr($string, $i + 2, 1, "ISO-8859-1"));
            $o4 = ord(mb_substr($string, $i + 3, 1, "ISO-8859-1"));
            $i += 5;
            if (!($o2 >= 128 && $o2 <= 143) ||
                !($o3 >= 128 && $o3 <= 191) ||
                !($o4 >= 128 && $o4 <= 191)) {
                $ok = 0;
                break;
            }
        }
        else {
            $ok = 0;
            break;
        }
    }

    return $ok;
}

, . , , . , .

!

+5
5

:

, , - :

1) ,

/* Set internal character encoding to UTF-8 */
mb_internal_encoding("UTF-8");

2)

if(mb_check_encoding($string))
{
    // do something
}

, , :

if(mb_check_encoding($string, 'UTF-8'))
{
    // do something
}
+7

, PHP isUtf8(), , UTF-8 PHP PHP.

UTF-8 - , :

function isValid($string)
{
    return preg_match(
        '/\A(?>
            [\x00-\x7F]+                       # ASCII
          | [\xC2-\xDF][\x80-\xBF]             # non-overlong 2-byte
          |  \xE0[\xA0-\xBF][\x80-\xBF]        # excluding overlongs
          | [\xE1-\xEC\xEE\xEF][\x80-\xBF]{2}  # straight 3-byte
          |  \xED[\x80-\x9F][\x80-\xBF]        # excluding surrogates
          |  \xF0[\x90-\xBF][\x80-\xBF]{2}     # planes 1-3
          | [\xF1-\xF3][\x80-\xBF]{3}          # planes 4-15
          |  \xF4[\x80-\x8F][\x80-\xBF]{2}     # plane 16
        )*\z/x',
        $string
    ) === 1;
}

, W3C. "+" . PCRE - , . , PCRE ( segfault). ( ), .

mb_check_encoding(), mbstring. UTF-8 , :

function isValid($string)
{
    return mb_check_encoding($string, 'UTF-8') === true;
}

, PHP 5.4.0, :

  • 5.4.0 Unicode. , 5 6 UTF-8.
  • 5.3.0 UTF-8.
  • 5.2.5 - , .

UTF-8, . , .

mb_detect_encoding() UTF-8. PHP 5.4.0, :

function isValid($string)
{
    return mb_detect_encoding($string, 'UTF-8', true) === 'UTF-8';
}

, 5.4.0. , , . , , true ( ).

UTF-8 - "u" PCRE. , . :

function isValid($string)
{
    return preg_match('//u', $string) === 1;
}

, "u" UTF-8. , 5.5.10. :

  • 5.5.10 3 4 UTF-8. Unicode, .
  • 5.2.5 (, 5 6 )

"u" : . , PHP, .

UTF-8 - json_encode(), , UTF-8. 5.5.0, false . :

function isValid($string)
{
    return json_encode($string) !== false;
}

, , , . PHP , , .

+2

ereg() preg_match? , , .

+1

You can use iconv to verify reality. Just try converting it to UTF-16 and make sure you get the error message.

+1
source

Here is a solution based on string functions:

http://www.php.net/manual/en/function.mb-detect-encoding.php#85294

<?php
function is_utf8($str) {
    $c=0; $b=0;
    $bits=0;
    $len=strlen($str);
    for($i=0; $i<$len; $i++){
        $c=ord($str[$i]);
        if($c > 128){
            if(($c >= 254)) return false;
            elseif($c >= 252) $bits=6;
            elseif($c >= 248) $bits=5;
            elseif($c >= 240) $bits=4;
            elseif($c >= 224) $bits=3;
            elseif($c >= 192) $bits=2;
            else return false;
            if(($i+$bits) > $len) return false;
            while($bits > 1){
                $i++;
                $b=ord($str[$i]);
                if($b < 128 || $b > 191) return false;
                $bits--;
            }
        }
    }
    return true;
}
?>
0
source

More articles:


All Articles