Interesting, looks like Firefox automatically resolves these to the real letters in the address bar, but leaves them as is for google search. Nice job aggregating these into an easy-to-use font.
What fonts are present (and therefore what characters will render successfully) depends not just on the version of Android, but also on the decisions of the device vendor about what to include.
I have an Android One phone running 8.1.0. I would normally expect font issues to be related to the underlying OS, but then I don't understand why Chrome works and Firefox doesn't.
And when the post is not flagged, the mods will change the text to normal letters. (If they take too much time, you can send a fancy font removal request to them hn@ycombinator.com)
In the early 2000s, unicode added a block of Mathematical Alphanumeric Symbols. The styles stay constant because 𝖊, ℯ, and 𝓮 might mean different things.
Unicode has a lot of characters that look like English letters. For example it has ℝ, the standard symbol for the set of real numbers. All together they form alphabets.
They're included in unicode for whatever reason [1]. It'll work in everything that handles unicode properly and has a font that includes those characters.
And yet it works in the weirdest places. Saving some of the text in a file and opening it in a Python interpreter works fine, it even counts the numbers of letters right.
What would be an example of a string like "𝕙𝕖𝕝𝕝𝕠" appearing in a context where the screen reader should not attempt to pronounce it? Note that in math usage, a lone ℝ should be pronounced exactly the same way as a lone R.
There are no natural languages that use mathematical blackboard bold script, although you are right that screen readers probably shouldn't try to read Cyrillic according to what English letter the characters look the most like.
In math usage an ℝ might be "the Regular set" and involved in equations that use a variable R. It's not unusual, and in fact, sometimes important, when reciting math as speech to make it clear which is the "bold" or "set" or "group" R, versus which is the ordinary R.
Depending on context it may not actually mean "F". Screen readers are not yet capable of fully understanding the meaning behind the text they are reading.
If for example someone uses this to write a mathematical formula, having a screen reader says "F" changes the entire meaning.
Could this be done, absolutely, but it is something to be aware of and is something I noticed on Twitter where blind users were complaining that they were unable to "read" Twitter messages using this, thereby making them second class citizens all over again.
If I were to write a math formula in ASCII, it would probably have strings of multiplied variables that would be indistinguishable from English words to a screenreader that didn't know anything about meaning. The only way it would know to sound out the letters is if they had spaces or asterisks between them. The additional glyphs would do nothing to change this.
Screen readers should faithfully render characters as sound so people with limited/no eyesight can engage with written language, not make a guess that you meant to write cursed/fancy text instead of what those characters are for.
I think mapping common lookalikes to their corresponding Latin-1 glyphs would be a broad improvement, and with websites like this collecting them it may not even be all that much work.
Although, like you say, it would be nice if screenreaders could produce verbal descriptions of iconographic symbols. That would be a lot of work but it would be helpful.
I wonder actually. Google seems to understand these, you can search with them. I wouldn't be surprised if screen readers understand these symbols. They're originally for Mathematics, and I'm sure there's plenty of blind mathematicians.
Some of them are outside the mathematical range, that just happened to be the one I picked.
Google has a lot of resources to do normalization, when IDNA in the URL bar became common they and other browser manufacturers had to put resources behind similar looking glyph attacks to make sure that you were actually on google.com and not on some site that was using a homoglyph attack.
In chromium, the glyph attack defense works by normalizing to a "confusability skeleton" -- the ICU library actually does the normalization. Even within latin-1 this does some mangling: m gets normalized to rn, w to vv, etc. The query rewriting is a different problem, but it's possible it uses the confusables list as an input.
blind mathematicians don't write english to communicate in these unicode blocks... These blocks don't exist to be used for writing words in the english language.
Yes, I just mean to say, if you were creating a screen reader, it wouldn't be that hard to take this sort of thing into account. So, maybe they do.
Either way, I'll add a little warning to the site.
It's less "can they render them" and more "do they maintain their meaning in alternate renderings".
For example, it's common in these to use non-latin characters that visually resemble latin characters. But if you were to try and substitute them literally, either as their actual phonetic sound, or a latin equivalent that is semantically similar but visually different, you risk breaking the meaning.
It's a bit like when old manuscripts use letters that look like "s" and "f" in places that seem nonsensical today.
In google's case, it's certainly worth it to write a little interpreter that maps the visual meaning to its latin equivalent so you can get better search results.
>It's less "can they render them" and more "do they maintain their meaning in alternate renderings".
I just entered "this is a test" on the site in Safari and enabled VoiceOver. It reads it as "Mathematical Bold Fraktur Small t. Mathematical Bold Fraktur Small h" and so on.
Completely broken for those who rely on VoiceOver as a screenreader.
I wouldn't be surprised if screen readers understand these symbols.
I develop web sites with screen readers in mind, and I would be very surprised if they could handle this sort of thing consistently.
Google spent billions of dollars learning how to search and interpret the web. Screen reading companies don't have that kind of scratch.
Also, it's important to remember that screen readers are about more than just the blind. There are screen readers that help people learn a new language, or translate text into simplified forms for people with low education, or low attention (think of the Mac's built-in text summarizing service).
For people who have a hard time reading custom fonts because of limited sight, or limited attention, they often override custom CSS fonts with something easier to read. This sort of thing will make the page unusable.
I'm not trying to suggest screen reading is easy, but the billions of dollars Google invest are not comparable here as they obviously do a huge range of things and the reader tools need to do a much narrower range.
This came up at Halloween with all those pleas not to post tweets heavy with emoji due to the issues with screen readers. I get the concern, and I'd typically do my best to be inclusive with personal content and compliant with accessibility on professional content, but there is a balance to be struck - we don't need a Procrustean restriction on what are now reasonably established forms of communication, what we need is for screen reader efforts to step up and work in these cases. It may seem a challenge (especially for legacy coded readers) but other posters already linked to basic solutions that can help on the fonts and this isn't beyond the wit of human ingenuity. This is an opportunity.
I'm not saying that we need to prohibit anyone from using crazy fonts. I'm merely responding to a postulation that screen readers could probably handle the text. My response, from experience, is that they probably can't, and most people misunderstand the myriad of uses for screen reading technology.
Odd fonts like this have a place. For example, I don't expect any screen reader ever to be able to interpret a PETSCII drawing.
This is called unicode normalisation, specifically NKFD form. You can do it in two lines of Python. This was covered last time a similar website came up (plagiarising myself):
Nobody in that thread tried it on an actual screen reader then, either. Someone did mention that iOS got well confused by it, so there's one data point.
Could also have issues with systems that have broken 16-bit-only Unicode support (Java, Windows, ...) in which code points beyond U+FFFF have to be encoded with some surrogate pair nonsense that is likely untested in many text-handling situations.
That's like saying UTF-8 requires "nonsense" pair, triplet, or quadruplet chars. UTF-16 handles all Unicode code points just fine. UCS-2 does not. Windows transitioned from UCS-2 to UTF-16 long ago.
The problem is Windows programs, not Windows per se.
The difference is that UTF-8 gets tested in this regard; multi-byte encoding situations actually occurring in UTF-8 are not rare occurrences that only trigger on funny characters that nobody uses.
(For that matter, four-byte UTF-8 situations are in the same boat, of course, but not two- or three-.)
I'd kind of expect screen readers to be able to apply Unicode compatibility decomposition http://unicode.org/reports/tr15/#Canon_Compat_Equivalence since there are many characters that are just visual variants. Ligatures like ffl or the like are at least somewhat common e.g. in PDFs. On the other hand, maybe that breaks other stuff and only whitelisted characters are converted.
Maybe a HN reader using a screen reader can describe how theirs handles these characters.
NVDA: oss, people have hacked in normalization that they can flip on when they hear something that sounds like nonsense, and then flip back off after reading that particular part.
JAWS: people have to listen to a bunch of crap if there's no alttext and will not be able to understand your content
VoiceOver OSX: people have to listen to a bunch of crap if there's no alttext and will not be able to understand your content
That implies that PDFs work with glyph indices, not Unicode code points. How do PDFs work for shaped languages like Hindi? In a script like that, you may have glyphs without any corresponding Unicode code point. Or does the PDF perhaps store the original unshaped text?
It might just be the output of some particular programs that pre-bake their ligatures. All I know is that sometimes, I try to copy-paste text out of a PDF only to end up with some annoying ligatures interspersed throughout.
Tt's possible for a PDF to be annotated with the original Unicode text (look up the `/ActualText` feature in the spec), to support extraction of the underlying text rather than the shaped glyph stream for purposes such as copy/paste and search.
However, few PDF generators do this, and not all PDF readers have good support for it. So results vary depending on the specific tools and use-case.
I'm not a regular screen reader user but VoiceOver (in macOS High Sierra) will not read the whole words, it will only say "Show HN." Because I can see there's more there, I can delve deeper and navigate character-by-character at which point it will say "f," "a," "n," "c," "y," "space," etc.
Things like screen readers can still try the Unicode decomposition techniques to try to make sense of the nonsense. The Fraktur F does decompose to "F", in this particular example. A better example is something like Lowercase Greek Letter Alpha which does not decompose to Latin "a", despite the readable similarity to most Latin character form audiences.
Though there too, there are patterns screen readers can attempt to find to figure out when alpha is pretending to be Latin a.
That said, it's still not a great idea to use them for text anywhere. It puts a lot of burden on the reader's pattern matching skills. Not just screen readers, but human readers too; everyone reads them a bit slower, and that's before you consider the usual human skill/ability modifiers such as dyslexia that make these things so much worse, too.
I've been using http://qaz.wtf/u/convert.cgi for ages. Little bit more dated interface, but same idea. That utility also does some pseudo-alphabets using characters from other scripts.
Very cool. I don't know if this counts as a bug as it's not really the intended use, but if I copy one of the results and then paste it into the "Write something here..." input, all the results then match the pasted style rather than being re-styled.
because, if i can work it out, the script takes ascii char 69 (E) and maps it to say unicode 2107 which is the Euler Constant and looks like a scripted E - so when you then paste unicode 2107 into the app again, it has no map for 2107 back to 65 and so cannot "change" fonts
Thanks, this is intended. It makes the code less complicated, but also means you can add the different fonts together more easily, which is particularly good for mobile.
There's a program/extension somewhere that allows these extra "fonts" in facebook comments sections... One of my family members started typing in some weird math font that broke on my old tablet.
Note: To see what Unicode characters are in a piece of text, one can paste them into a tool like Uniview: https://r12a.github.io/uniview/ (Developed by one of the W3C i18n people)
Interesting - I thought using Unicode characters would break text search capability - but when I did the old Ctrl-F 'find' in Chrome and looked for the word 'fancy', it detected all of the above. Intriguing, because in another thread here, there is mention of a gothic 'a' not being treated as a standard letter 'a'...
Please don't use this. Concrete example: if you use the OpenDyslexia font and have configured your browser to override individual website's fonts, this is what you see: https://i.imgur.com/aCk2ShW.png
171 comments
[ 2.2 ms ] story [ 179 ms ] thread[1] https://unicode-table.com/en/1D573/
https://www.compart.com/en/unicode/U+1D571
This is terrible for screen readers and the like which are unable to read or understand these unicode characters making accessibility a real concern.
If for example someone uses this to write a mathematical formula, having a screen reader says "F" changes the entire meaning.
Could this be done, absolutely, but it is something to be aware of and is something I noticed on Twitter where blind users were complaining that they were unable to "read" Twitter messages using this, thereby making them second class citizens all over again.
Although, like you say, it would be nice if screenreaders could produce verbal descriptions of iconographic symbols. That would be a lot of work but it would be helpful.
at any rate, both budgets are not infinite, and it's infuriating to hear the described as such.
Google has a lot of resources to do normalization, when IDNA in the URL bar became common they and other browser manufacturers had to put resources behind similar looking glyph attacks to make sure that you were actually on google.com and not on some site that was using a homoglyph attack.
https://blog.malwarebytes.com/101/2017/10/out-of-character-h...
This may explain why Google is able to discern these use cases.
For example, it's common in these to use non-latin characters that visually resemble latin characters. But if you were to try and substitute them literally, either as their actual phonetic sound, or a latin equivalent that is semantically similar but visually different, you risk breaking the meaning.
It's a bit like when old manuscripts use letters that look like "s" and "f" in places that seem nonsensical today.
In google's case, it's certainly worth it to write a little interpreter that maps the visual meaning to its latin equivalent so you can get better search results.
I just entered "this is a test" on the site in Safari and enabled VoiceOver. It reads it as "Mathematical Bold Fraktur Small t. Mathematical Bold Fraktur Small h" and so on.
Completely broken for those who rely on VoiceOver as a screenreader.
Anyone know if there are any existing libraries that do this conversion?
http://minaret.info/test/normalize.msp
"𝕙𝕖𝕝𝕝𝕠".Normalize(NormalizationForm.FormKC);
I develop web sites with screen readers in mind, and I would be very surprised if they could handle this sort of thing consistently.
Google spent billions of dollars learning how to search and interpret the web. Screen reading companies don't have that kind of scratch.
Also, it's important to remember that screen readers are about more than just the blind. There are screen readers that help people learn a new language, or translate text into simplified forms for people with low education, or low attention (think of the Mac's built-in text summarizing service).
For people who have a hard time reading custom fonts because of limited sight, or limited attention, they often override custom CSS fonts with something easier to read. This sort of thing will make the page unusable.
This came up at Halloween with all those pleas not to post tweets heavy with emoji due to the issues with screen readers. I get the concern, and I'd typically do my best to be inclusive with personal content and compliant with accessibility on professional content, but there is a balance to be struck - we don't need a Procrustean restriction on what are now reasonably established forms of communication, what we need is for screen reader efforts to step up and work in these cases. It may seem a challenge (especially for legacy coded readers) but other posters already linked to basic solutions that can help on the fonts and this isn't beyond the wit of human ingenuity. This is an opportunity.
Odd fonts like this have a place. For example, I don't expect any screen reader ever to be able to interpret a PETSCII drawing.
https://news.ycombinator.com/item?id=17752680
unicode normalisation: http://unicode.org/reports/tr15/
Nobody in that thread tried it on an actual screen reader then, either. Someone did mention that iOS got well confused by it, so there's one data point.
The difference is that UTF-8 gets tested in this regard; multi-byte encoding situations actually occurring in UTF-8 are not rare occurrences that only trigger on funny characters that nobody uses.
(For that matter, four-byte UTF-8 situations are in the same boat, of course, but not two- or three-.)
Yeah. Notorious example here is MySQL's "utf8" column type only supporting 3-byte UTF-8 sequences.
Maybe a HN reader using a screen reader can describe how theirs handles these characters.
NVDA: oss, people have hacked in normalization that they can flip on when they hear something that sounds like nonsense, and then flip back off after reading that particular part.
JAWS: people have to listen to a bunch of crap if there's no alttext and will not be able to understand your content
VoiceOver OSX: people have to listen to a bunch of crap if there's no alttext and will not be able to understand your content
However, few PDF generators do this, and not all PDF readers have good support for it. So results vary depending on the specific tools and use-case.
https://www.compart.com/en/unicode/U+03B1
Though there too, there are patterns screen readers can attempt to find to figure out when alpha is pretending to be Latin a.
That said, it's still not a great idea to use them for text anywhere. It puts a lot of burden on the reader's pattern matching skills. Not just screen readers, but human readers too; everyone reads them a bit slower, and that's before you consider the usual human skill/ability modifiers such as dyslexia that make these things so much worse, too.
𝕋𝕙𝕚𝕤 𝕥𝕖𝕤𝕥 𝕥𝕖𝕩𝕥 𝕝𝕠𝕠𝕜 𝕝𝕚𝕜𝕖 𝔼𝕟𝕘𝕝𝕚𝕤𝕙 𝕓𝕦𝕥 𝕚𝕤 𝕦𝕤𝕚𝕟𝕘 𝕟𝕠𝕟-𝕒𝕤𝕔𝕚𝕚 𝕔𝕙𝕒𝕣𝕒𝕔𝕥𝕖𝕣𝕤 𝕒𝕟𝕕 𝕤𝕠 𝕚𝕗 𝕥𝕙𝕚𝕤 𝕣𝕖𝕞𝕒𝕚𝕟𝕤 𝕣𝕖𝕒𝕕𝕒𝕓𝕝𝕖 𝕥𝕠 𝕒𝕟 𝔼𝕟𝕘𝕝𝕚𝕤𝕙 𝕤𝕡𝕖𝕒𝕜𝕖𝕣 𝕨𝕖 𝕔𝕒𝕟 𝕓𝕖 𝕣𝕖𝕒𝕤𝕠𝕟𝕒𝕓𝕝𝕪 𝕔𝕠𝕟𝕗𝕚𝕕𝕖𝕟𝕥 𝕨𝕖 𝕒𝕣𝕖 𝕙𝕒𝕟𝕕𝕝𝕚𝕟𝕘 𝕦𝕟𝕚𝕔𝕠𝕕𝕖 𝕡𝕣𝕠𝕡𝕖𝕣𝕝𝕪 𝕥𝕙𝕣𝕠𝕦𝕘𝕙𝕠𝕦𝕥 𝕥𝕙𝕖 𝕧𝕒𝕣𝕚𝕠𝕦𝕤 𝕤𝕪𝕤𝕥𝕖𝕞𝕤 𝕠𝕦𝕣 𝕕𝕒𝕥𝕒 𝕥𝕣𝕒𝕧𝕖𝕝𝕤.
Yes, Firefox for Android doesn't render it properly.
...
日本の作品も (🅙🅐🅟🅐🅝🅔🅢🅔 also works)
中国也有作品 (ℭ𝔥𝔦𝔫𝔢𝔰𝔢 too)
etc.
╔═════════════════════════════════════════════════════════════════════════════════════════╗
║ Those have been around for a very long time and were very popular on teen blogging platforms like skyrock, myspace, etc. ║
╚═════════════════════════════════════════════════════════════════════════════════════════╝
ヾ(＾-＾)ノ
But you aren’t going to be able to read that either. Try your phone?
Everything started on SomethingAwful many many years ago.
That's what I assume anyway.
I haven’t delved into the symbol area of Unicode you’re talking about here but I’d bet those all evaluate to “fancy”.