Umlauts (ä,ö,ü, ....) break Qiita

[sjanssen2]

When I edit my user profile and enter my affiliation in German, i.e. "Universität Gießen", I get the following error, due to the use of ä and ß:

Is there an easy way to extend the DB schema to allow for these characters? If not, I think we need more speaking error messages to let the user know what the actual problem is.

[antgonza]

I'm thinking that this might be an issue in some pages and not in others; do you have the URL that is broken?

[sjanssen2]

this affects inputs in the profile site https://qiita.ucsd.edu/profile/. I haven't tested other pages. I guess this is due to wrong restrictive encoding of the according DB columns

[antgonza]

Thank you. FWIW the issue is with decode:

File "/home/qiita/qiita/qiita_pet/handlers/user_handlers.py", line 53, in post
profile = {name: data[0].decode('ascii')

File "/home/qiita/qiita/qiita_pet/handlers/user_handlers.py", line 53, in <dictcomp>
profile = {name: data[0].decode('ascii')

UnicodeDecodeError: 'ascii' codec can't decode byte 0xc3 in position 0: ordinal not in range(128)

[sjanssen2]

whats our way forward? Provide better error message to the user OR use a less limited codec?

[antgonza]

Well, not sure; depends on what we want to support and how long we want to spend on this. One option is to change the ascii encoding to unicode or something similar and check how this plays in the DB.