How so? These emoji characters are in widespread use. The initial characters had competing standards amongst Japanese providers. The unicode adoption allows indexing and searching without encoding problems. The competing standards also defined different glyphs, which feels like going back to the 80s or 90s with character pages.
Also, we can't jump a shark in Unicode, but we do have a few lovely 🐋 🐳 whales for you to jump if you'd like.
The only time I've seen whale here, it looked like this:🍣
I'm glad emoji have been standardized. Back in the dumbphone days each carrier had their own set of characters, meaning the smiley face you sent😘 might have come out as a piece of poo...
Interesting. In Google Chome, I just see squares where the character should be. However, if I copy and paste it into the url bar, or anywhere else in OSX, I see a color picture of a pizza.
OS X's/Cocoa's text handling figures out you're using a character from that range and automatically uses a font that has the emoji characters defined - Apple Color Emoji for that one character. I'm guessing this does or does not show up depending on the degree of explicit control one is exercising over the fonts when rendering text and what APIs are being used.
I think this is because V8, the JavaScript engine for Chrome uses USC-2 encoding. USC-2 characters can take up to 16 bits, not enough room to hold most of Unicode 6.0 characters.
These are to support emoji symbols in Unicode. There's a lot in there beside pizza - party poppers, wedding chapels, game console controllers, "information desk person" and the ever mysterious "Japanese goblin".
GNU Unifont covers the entire BMP. Every character in the BMP has an 8x16 or 16x16 bitmap in the font.
A "unicode complete" font is more of a novelty than anything else. For real work, pick a collection of fonts that cover the space you need. Good Arabic or Chinese fonts are not going to be great Latin fonts. For that matter, good Chinese fonts are not good Japanese fonts, even if coverage is good enough. In fact, the order of precedence that these fonts have will depend on which language is the primary language. If you're typesetting Japanese with Latin characters, then you can just pick a nice Mincho font, but Mincho has unnatural spacing if your audience has native English readers. (So if you quote Japanese in an English text, pick a Latin font and use Mincho for the Japanese; but if you quote English in a Japanese text, you can use Mincho for everything.)
It's hard enough just to find a good Latin font that also has a good set of mathematical symbols. "Computer Modern" of course, and "Times", but these sets are rare.
Adobe InDesign and Illustrator support composite fonts. Designers can mix and match Latin and Japanese / CJK glyphs, tweak them individually, and use overrides for tricky code points.
Web typography is a different beast because fonts can be bloat-y.
Emoji is such a weird thing. On one hand, I use them constantly for IMing. A quick 👍 is 10x better than the single-character "K" response in terms of expressiveness and playfulness.
On the other hand, the symbol coverage seems very random to my western brain. For example, there are three different mounds of rice, but no tacos. Nine different commuter trains, but no motorcycle. Also, the seemingly random assortment of flags.
I know this is a result of their Japanese origins, but if I had my druthers, emoji would be a little more inclusive, and then phones would have a little emoji-editor where you could edit your keyboards so they only included the symbols you wanted to use.
Yes you need a font on your system that has a symbol for that codepoint. I know I had to install 'symbola' font to get `love hotel` to display a few months ago.
It's not a character set issue. It's just that none of your fonts have that glyph.
I'm on Linux, but after installing the symbola font (Arch package: `ttf-symbola`) and restarting firefox, the square turns into the proper thumbs up sign.
I don't think Unicode emoji make sense in the long run - even if they're handy in SMS, and fun to scatter around normal text boxes, it would be better to use chat protocols with their own sets of consistent emoticons (as, indeed, many mobile messaging apps have). Why?
(a) Details matter. It doesn't matter if my 'A' displays in a different font on your phone, but the difference between Apple's colorful, pixelated, Space Invaders-like rendering of http://www.fileformat.info/info/unicode/char/1F47E/index.htm and the server-generated image's outlined round tentacle monster is fairly significant.
(b) Emoticons (at least those beyond the core set) are things of fashion: we always want more, but old ones go out of style. Though depictions can change, we're going to be stuck with the current list of Unicode emoji forever.
I have a writing app I've developed for iOS, and a while ago received a bug report from someone saying the app crashed every time they tried to open their document. I couldn't figure out what was going on but after eventually receiving a copy of the document, I discovered that the crash was due to my incorrect handling of emoji characters.
Most string classes, including NSString in iOS, typically represent strings as sequences of 16-bit characters. With 65,536 possible combinations, this is perfectly adequate to support the characters from all major languages in use worldwide [1], plus emoji. Unicode does however allow for values outside of this range, which are represented by "surrogate" pairs of 16-bit values in the range D800-DFFF. It was these I was handling incorrectly, and had missed in my testing.
But if there's space in the 0-65,535 range to represent emoji, why would it be stored outside of this? Well, it turns out that the initial implementation of emoji in iOS used the "private use area" of this range to encode the character values [2]. NTT DoCoMo also had their own (incompatible) way of encoding emoji in the private use area [3]. When attempts were made to formally standardise on emoji in unicode, they couldn't use the private use area, and ended up going with values above 65,535.
If unicode had stuck to representing existing characters and symbols and said no to requests for stuff like emoji and klingon, string representation in modern software could have been kept a whole lot simpler.
Joel Spolsky has an excellent explanation of unicode and encoding issues here:
this is perfectly adequate to support the characters from all major languages in use worldwide, plus emoji
Emoji characters are in the range U+1F300..U+1F5FF, which is outside the BMP (up to U+FFFF), and will be encoded with surrogate pairs in UTF-16.
Also, even within the BMP, assuming one 16-bit UTF-16 block is the same as one "character" is dangerous, as combining characters (for example) should arguably not be separated from their base character.
If unicode had stuck to representing existing characters and symbols and said no to requests for stuff like emoji and klingon, string representation in modern software could have been kept a whole lot simpler.
Emoji and Klingon are trivial to deal with in comparison to the issues that can appear with "real" languages when you start making assumptions (cf. combining characters, normalisation forms, CJK unification and compatibility blocks, presentation forms, collation, etc.)
> With 65,536 possible combinations, this is perfectly adequate to support the characters from all major languages in use worldwide [1]
It depends on how you represent characters. Korean Hangul represents each square shape separately so requires over 10,000 characters. If it hadn't been like that, Korea wouldn't have accepted Unicode for use because it would have multiplied the number of bytes required to represent characters from their existing system. There's 10,000 Japanese/Chinese Kanji in modern use, plus another less commonly used 65,000 encoded, and there's no consistent way to represent them as components.
> If unicode had stuck to representing existing characters and symbols and said no to requests for stuff like emoji and klingon, string representation in modern software could have been kept a whole lot simpler.
Klingon was rejected for encoding by the Unicode Consortium in 1997, and remains excluded to this day.
emoji was included as a political nod to the Japanese, who even today are the main holdouts to Unicode being the exclusive code for worldwide use.
47 comments
[ 3.1 ms ] story [ 112 ms ] thread(It's a "love hotel", one character away from a regular hotel, 🏨)
Also, we can't jump a shark in Unicode, but we do have a few lovely 🐋 🐳 whales for you to jump if you'd like.
I'm glad emoji have been standardized. Back in the dumbphone days each carrier had their own set of characters, meaning the smiley face you sent😘 might have come out as a piece of poo...
I haven't touched it in years as you can see, but it was fun while it lasted.
There have been previous discussions about the topic here on HN: https://news.ycombinator.com/item?id=4833631
Btw I'm on Windows 7 x64 SP1, I installed KB2729094 but no effects on IE idk why
PDF chart at
http://www.unicode.org/charts/PDF/U1F300.pdf
for the full glory of it all.
A "unicode complete" font is more of a novelty than anything else. For real work, pick a collection of fonts that cover the space you need. Good Arabic or Chinese fonts are not going to be great Latin fonts. For that matter, good Chinese fonts are not good Japanese fonts, even if coverage is good enough. In fact, the order of precedence that these fonts have will depend on which language is the primary language. If you're typesetting Japanese with Latin characters, then you can just pick a nice Mincho font, but Mincho has unnatural spacing if your audience has native English readers. (So if you quote Japanese in an English text, pick a Latin font and use Mincho for the Japanese; but if you quote English in a Japanese text, you can use Mincho for everything.)
It's hard enough just to find a good Latin font that also has a good set of mathematical symbols. "Computer Modern" of course, and "Times", but these sets are rare.
Web typography is a different beast because fonts can be bloat-y.
On the other hand, the symbol coverage seems very random to my western brain. For example, there are three different mounds of rice, but no tacos. Nine different commuter trains, but no motorcycle. Also, the seemingly random assortment of flags.
I know this is a result of their Japanese origins, but if I had my druthers, emoji would be a little more inclusive, and then phones would have a little emoji-editor where you could edit your keyboards so they only included the symbols you wanted to use.
It is because of the encoding format on HN? Or because my browser's built in fonts don't have all the Unicodes?
I'm on Linux, but after installing the symbola font (Arch package: `ttf-symbola`) and restarting firefox, the square turns into the proper thumbs up sign.
http://apple.stackexchange.com/questions/41228/why-do-emoji-...
(a) Details matter. It doesn't matter if my 'A' displays in a different font on your phone, but the difference between Apple's colorful, pixelated, Space Invaders-like rendering of http://www.fileformat.info/info/unicode/char/1F47E/index.htm and the server-generated image's outlined round tentacle monster is fairly significant.
(b) Emoticons (at least those beyond the core set) are things of fashion: we always want more, but old ones go out of style. Though depictions can change, we're going to be stuck with the current list of Unicode emoji forever.
I have a writing app I've developed for iOS, and a while ago received a bug report from someone saying the app crashed every time they tried to open their document. I couldn't figure out what was going on but after eventually receiving a copy of the document, I discovered that the crash was due to my incorrect handling of emoji characters.
Most string classes, including NSString in iOS, typically represent strings as sequences of 16-bit characters. With 65,536 possible combinations, this is perfectly adequate to support the characters from all major languages in use worldwide [1], plus emoji. Unicode does however allow for values outside of this range, which are represented by "surrogate" pairs of 16-bit values in the range D800-DFFF. It was these I was handling incorrectly, and had missed in my testing.
But if there's space in the 0-65,535 range to represent emoji, why would it be stored outside of this? Well, it turns out that the initial implementation of emoji in iOS used the "private use area" of this range to encode the character values [2]. NTT DoCoMo also had their own (incompatible) way of encoding emoji in the private use area [3]. When attempts were made to formally standardise on emoji in unicode, they couldn't use the private use area, and ended up going with values above 65,535.
If unicode had stuck to representing existing characters and symbols and said no to requests for stuff like emoji and klingon, string representation in modern software could have been kept a whole lot simpler.
Joel Spolsky has an excellent explanation of unicode and encoding issues here:
http://www.joelonsoftware.com/articles/Unicode.html
[1] To the best of my knowledge; correct me if I'm wrong
[2] http://openradar.appspot.com/6402446
[3] http://web.archive.org/web/20080216230900/http://www.nttdoco...
Emoji characters are in the range U+1F300..U+1F5FF, which is outside the BMP (up to U+FFFF), and will be encoded with surrogate pairs in UTF-16.
Also, even within the BMP, assuming one 16-bit UTF-16 block is the same as one "character" is dangerous, as combining characters (for example) should arguably not be separated from their base character.
If unicode had stuck to representing existing characters and symbols and said no to requests for stuff like emoji and klingon, string representation in modern software could have been kept a whole lot simpler.
Emoji and Klingon are trivial to deal with in comparison to the issues that can appear with "real" languages when you start making assumptions (cf. combining characters, normalisation forms, CJK unification and compatibility blocks, presentation forms, collation, etc.)
It depends on how you represent characters. Korean Hangul represents each square shape separately so requires over 10,000 characters. If it hadn't been like that, Korea wouldn't have accepted Unicode for use because it would have multiplied the number of bytes required to represent characters from their existing system. There's 10,000 Japanese/Chinese Kanji in modern use, plus another less commonly used 65,000 encoded, and there's no consistent way to represent them as components.
> If unicode had stuck to representing existing characters and symbols and said no to requests for stuff like emoji and klingon, string representation in modern software could have been kept a whole lot simpler.
Klingon was rejected for encoding by the Unicode Consortium in 1997, and remains excluded to this day.
emoji was included as a political nod to the Japanese, who even today are the main holdouts to Unicode being the exclusive code for worldwide use.