PHP: how to encode U + FFFD to replace?

Question

I am trying to display a data feed on a page. We encounter strange coding problems. For some reason, the feed has the symbol U+FFFD . And htmlentities() cannot escape the character, so I need to replace it manually. (I am using PHP 5.3)

I tried the following:

 $string = str_replace( "\xFFFD", "_", $string ); $string = str_replace( "\XFFFD", "_", $string ); $string = str_replace( "\uFFFD", "_", $string ); $string = str_replace("\x{FFFD}", "_", $string ); $string = str_replace("\X{FFFD}", "_", $string ); $string = str_replace("\P{FFFD}", "_", $string ); $string = str_replace("\p{FFFD}", "_", $string ); 

None of the above works.

After reading this page http://php.net/manual/en/regexp.reference.unicode.php - I'm not sure what I'm doing wrong. Do I need to compile UTF-8 support in PCRE?

Answer 1

Use preg_replace instead:

 $string = preg_replace('@\x{FFFD}@u', '_', $string); 

Answer 2

You should try to fix the original problem, FFFD ( Unicode character replacement character ) is in most cases not a real text character, but a sign that something was trying to decode in UTF encoding, but something was not actually encoded in UTF encoding. This is an alternative to silently discarding invalid bytes or completely stopping the decoding process, in any case, if you see this, an error has occurred.

There is no way to find out what the original character is. Especially with your decision, since you are replacing the _ character, you may not even know that the original source was incorrectly decoded. You must go back to the source and decode it correctly.

Note. It is possible that the source text uses as a literal, an ordinary character, for example, when it talks about it, and then there is no error. I exclude this possibility in my answer.