Python: Check if string contains Chinese character?

Question

Maybe this is a string

ipath= "./data/NCDC/上海/虹桥/9705626661750dat.txt" 

or

 ipath = './data/NCDC/ciampino/6240476818161dat.txt' 

How to find out that the first line contains Chinese ?

I find this answer to be useful: Find all Chinese text in a string using Python and Regex

but this did not work:

 import re ipath= "./data/NCDC/上海/虹桥/9705626661750dat.txt" re.findall(ur'[\u4e00-\u9fff]+', ipath) # => [] 

Answer 1

The corresponding line should also be in Unicode

 >>> import re >>> ipath= u"./data/NCDC/上海/虹桥/9705626661750dat.txt" >>> re.findall(r'[\u4e00-\u9fff]+', ipath) [u'\u4e0a\u6d77', u'\u8679\u6865'] 

Answer 2

If you just want to know if there is a Chinese character in your string, you do not need re.findall , use re.search , and the fact that the matching objects are true.

 >>> import re >>> ipath= u'./data/NCDC/上海/虹桥/9705626661750dat.txt' >>> ipath2 = u'./data/NCDC/ciampino/6240476818161dat.txt' >>> for x in (ipath, ipath2): ... if re.search(u'[\u4e00-\u9fff]', x): ... print 'found chinese character in ' + x ... found chinese character in ./data/NCDC/上海/虹桥/9705626661750dat.txt 

Answer 3

And for those of us who don't care about re :

 >>> ipath= u"./data/NCDC/上海/虹桥/6240476818161dat.txt" >>> for i in range(len(ipath)): ... if ipath[i] > u'\u4e00' and ipath[i] < u'\u9fff': ... print ipath[i] ...上海虹桥 

Edit: for a complete list of Chinese characters, this link is worth a look, since the range U + 4E00..U + 9FFF is not complete. What is the full range of Chinese characters in Unicode?

Answer 4

 import re ipath= raw_input() print re.findall(ur'[\u4e00-\u9fff]+', ipath.decode("utf-8")) 

Output: ./data/NCDC/上海/虹桥/9705626661750dat.txt [u'\u4e0a\u6d77', u'\u8679\u6865']

You need to decode the input to make it unicode.

or

  import re ipath= unicode(raw_input(),encoding="utf-8") print re.findall(ur'[\u4e00-\u9fff]+', ipath) 

Answer 5

'' is a byte string in Python 2. Either add from __future__ import unicode_literals to the beginning of the module, or use literals from Unicode: u'' :

 >>> import re >>> ipath= u"./data/NCDC/上海/虹桥/9705626661750dat.txt" >>> re.findall(ur'[\u4e00-\u9fff]+', ipath) [u'\u4e0a\u6d77', u'\u8679\u6865'] 

Answer 6

Using these is_cjk ranges , we can write the is_cjk function:

 # list of cjk codepoint ranges # tuples indicate the bottom and top of the range, inclusive cjk_ranges = [ ( 0x4E00, 0x62FF), ( 0x6300, 0x77FF), ( 0x7800, 0x8CFF), ( 0x8D00, 0x9FCC), ( 0x3400, 0x4DB5), (0x20000, 0x215FF), (0x21600, 0x230FF), (0x23100, 0x245FF), (0x24600, 0x260FF), (0x26100, 0x275FF), (0x27600, 0x290FF), (0x29100, 0x2A6DF), (0x2A700, 0x2B734), (0x2B740, 0x2B81D), (0x2B820, 0x2CEAF), (0x2CEB0, 0x2EBEF), (0x2F800, 0x2FA1F) ] def is_cjk(char): char = ord(char) for bottom, top in cjk_ranges: if char >= bottom and char <= top: return True return False 

Then we can use it to process the text, using functions such as filter , any , all and map to process the text character by character or to make more complex functions:

 txt = "./data/NCDC/上海/虹桥/9705626661750dat.txt" txt_sanitized = "./data/NCDC/9705626661750dat.txt" any(map(is_cjk, txt)) # True any(map(is_cjk, txt_sanitized)) # False ''.join(filter(is_cjk, txt)) # '上海虹桥' 

Please note that CJK ranges will include not only Chinese characters, but may also include Korean and Japanese characters. For more complex use, try a special library, for example, cjklib .

Answer 7

In python 3.6, I used this

 def find_china_symbols(text): """ :param text: input text with wrong symbols :return: True if incorrect char exists in text """ for char in text: if ord(char) > 10000: print(char, ': ', ord(char)) return True 

Answer 8

According to this question, the range should be [\u2E80-\u2FD5\u3190-\u319f\u3400-\u4DBF\u4E00-\u9FCC]