What is the best way to replace smart quotes, smart apostrophe, and ellipsis in Rails 3?

Question

My application allows the user to enter text. When they copy and paste from MS Word, it inserts smart quotes, smart apostrophes, and ellipses. These characters are stored in the database and cause problems. What is the best way to replace these non-UTF-8 characters with normal quotation marks ("), apostrophes (') and periods (...)?

Also, how do you test this functionality? I added a test with these special characters and # encoding: ISO-8859-1 at the top of the file. Special characters made the tests stop: /home/george/.rvm/gems/ruby-1.9.2-p180/gems/redgreen-1.2.2/lib/redgreen.rb:62:in 'sub': invalid byte sequence in UTF-8 (ArgumentError) ... Obviously, a reddish stone is incompatible with these characters ...?

Thanks.

Answer 1

you can add a before_save method that converts your text to the corresponding UTF-8 characters. if you have only one field that can contain characters other than UTF8, then it is simple, if you have many fields, it would be better if you dynamically iterated over the changed text / string fields and fixed the UTF-8 problem. In any case, you need to use String # encode . Here is an example

 before_save :fix_utf8_encoding def fix_utf8_encoding columns = self.class.columns.select{|col| [:text,:string].include?(col.type)}.map{|col| col.name.to_sym} columns.each do |col| self[col] = self.self[col].encode('UTF-8', :invalid => :replace, :undef => :replace) if self[col].kind_of?(String) #Double checking just in case. end end private :fix_utf8_encoding 

And for bonus points, you can also check if the field has been changed using the modified rudders? assistants before correcting it.