Perl 5.8 and Unicode problems
Below is what I asked to autrijus, but if you have any suggestions on this issue, I’m open to them.
- It’s widely known that Jcode.pm has Unicode map problem that Full-Width-Tilde (U+FF5E) doesn’t map well to euc-jp. It’s due to the mistake of Unicode.org’s own mapping table.
But with the recent Encode.pm, it still has problem:> % perl -MEncode -e ‘print encode(“euc-jp” “\x{ff5e}”, Encode::FB_CROAK)’ “\x{ff5e}” does not map to euc-jp at /usr/lib/perl/5.8.2/Encode.pm line 149. What does it mean? grepping ucm files shows:> % grep -i FF5E ucm/euc-jp.ucm \xA2\xB2 |3 # 1-2-18 Doesn’t it mean that UFF5E maps to \xA2\xB2 in euc-jp?
- What’s the best practice in develop application in multi-encoding environment, like web+db+xml applications?
It’d make me a mess while developing in such enviroment that:
TT Template is written in euc-jp or utf-8 one data is fetched via XML (RSS) in utf-8 or euc-jp another data is stored to and fetched from MySQL in utf-8 HTTP requests come from mobile phones in Shift_JIS
Concatinating non-Unicode strings with Unicode strings raise UTF-8 Auto Upgrading and thus raw UTF-8 Strings get corrupted.
For example at least, how do I tell Template-Toolkit that template is written in euc-jp? It calls open() in its own modules, so binmode or encoding.pm, unless you open template files and pass its filehandle explicitly, which is not the case of mine.
I tend to think there should be encoding layers to all data-stream-handling modules like DBI, Template-Toolkit, CGI.pm (or Apache::Request) etc. Am I thinking right here?