Unicode in identifiers is an exceptionally bad idea, with a miniscule upside and horrible downsides:
1) As the article demonstrates, this allows obfuscation contests in easy mode. Strangely, the author claims:
> Nor do I think most of these tricks would get by a code review or go unnoticed using something as simple as syntax coloring.
How would syntax highlighting catch that? Those are valid identifiers! And yes, these "tricks" would almost certainly go unnoticed in a code review, at least when you don't assume that your coworker is maliciously inserting bugs.
2) Even in the absence of malicious intent, Unicode identifiers open the door to bugs and head-scratchers: different characters looking identical, or the same character encoded in different ways.
3) ASCII letters (a-z, A-Z), digits (0-9), and underscore are kind of the lowest common demoninator. When you allow Unicode identifiers, you don't "internationalize" your code, you're doing the exact opposite! If you're using ä, ö, ü, ß in your API, how is a Canadian supposed to easily use it?
The one big upside is, for quite a good amount of people, being able to use their native language instead of being stuck with the latin alphabet
When I was in france, all the software were written using french identifiers. It was much better this way since none of the code was meant to be shared outside of france and it made the code much more readable for people who were supposed to work on it. Why should it be possible only for latin based languages?
All keywords are in English. The standard library is in English. All external library APIs, open source or otherwise, are in English. But your own functions and variables have to be different? Why? If your grasp on English is sufficient for using a programming language, than it is also sufficient for using English identifiers. And I'm saying that as a native German speaker.
> it made the code much more readable
"Much more readable"? It made the code base inconsistent with the rest of the language. Are you saying that they were lacking even basic English language skills?
I very much agree with you. Having some strange words in the middle of a large code that is mostly english is just, well, weird. I think inconsistency is a serious issue.
The language keywords are just keywords; you simply learn their meaning. As a native English speaker, though 'while' is a somewhat intuitively-named keyword, the 'for' keyword in c/c++ has to be learned. After a while you don't notice, any more than musicians have to learn Italian to play music (quick: what do 'fortissimo' and 'da capo ala fine' mean).
I don't see any problem with source code that says something like 'auto maßtab = waage.m_maßtab();' 'auto' and '=' are simply language tokens.
This isn't a very convincing argument to me. "We fail at source code localization in 4 other ways already" is not a good argument for "let's force a failure in a 5th way, too, even though that's the one place where there's no technical issue with it".
To me, using English keywords everywhere is a historical accident, not a design goal. In next-generation systems like Subtext [1], it looks entirely possible to add localization to source code, too.
Once you throw out the Lisp way of having one namespace for all symbols (or, OK, like 7 in CL), to be used as functions or values, and go to a system where you want to be able to statically type-check everything, why not fully commit to it and let users give local names to identifiers? Then we won't be limited to legacy C-style naming rules, or have to argue about how many space characters to add to make all their names line up on a high-resolution VT100 emulator, either.
If you work on a purely technical framework, yes it can make sense, even if i don't see why using french would be an issue for french people. Actually, i always liked the fact that since all the keywords are english, i can use mostly all the french language.
When you work on a specialised field (for example some veterinarian software I worked in), learning the specific terms in your own language is already a lot. Learning how to translate those is adding much more complexity. It gets even worse when your specifications are written in one language (french in my example) and the software is written in another, so not re-using the syntax of the specifications. Add that most of the people will then have no understanding of what the code does without a list of translation, you end up in a quite messy situation...
Yes i'm saying that a lot of colleagues were lacking basic english skills, and i don't see the issue with that. English is a nice to have to communicate with people abroad, but absolutely not necessary to live in France.
Similar experience while working for Sony in Japan: all the code is in English, all the comments are in Kanji. Oddly, the variable names were always and only the variable names used in the code examples, as if they thought they were the only allowed variables.
In Russia it's considered very bad and novice style to use Russian words as identifiers (they usually spell it with latin letters anyway, constantly switching between latin and cyrillic layouts is very cumbersome). There's one exception: 1C platform, but that software is used internally for accounting and basically a DSL with heavy use of application domain terminology.
Sometimes I'm tempted to use Russian identifiers for application domain terminology, because very often there's no adequate translation or I just don't know English enough to translate some terms. I like the following example: every man here has name combined from last name (Фамилия) which is used for all family members, first name (Имя), that's pretty standard. But there's also third name: Отчество, which is basically father's name with some suffix, e.g. father Иван will have child with third name Иванович. Now I've seen all kinds of translations for those terms, including plain wrong. Last name, family name for Фамилия, first name, name for имя, father name, second name, patronymic, patronymic name for отчество. I've seen bugs because one developer interpreted those terms differently from another.
And that's just for name. There are plenty of other terms. Add to that that many developers are bad at English and at best will use first result from Google Translate. So may be national identifiers have its place.
> every man here has name combined from last name (Фамилия) which is used for all family members, first name (Имя), that's pretty standard.
Sounds like you get to skirt a few of the problems outlines in Falsehoods Programmers Believe About Names[1]. That's nice for programmers in Russia that deal with Russia only, if true. :)
I think Russians do get to skirt a few of the problems... if they believe their systems are only used by native Russians with standard Russian names.
Of course the whole point of the article you mentioned is that making such assumptions leads to odd behavior. I (an American) lived in Russia for a while. One time I was having a couch delivered to my apartment and had to give the store my full name. I was delighted to see my middle name show up on the receipt in the отчество (patronymic) field with the Russian patronymic suffix ович attached. I still got the couch no problem, but I got a laugh out of their attempt to shoehorn my (clearly not Russian) names into their naming paradigm.
Much better that's debatable: 1) you can't really be a programmer without good English knowledge 2) I'm French and worked on a software in French: eventually the source code had to be translated in English (a requirement from one customer): what a waste of time & money..
Unicode in identifiers is an exceptionally bad idea
Unicode identifiers can be perfectly well-defined, and many languages have them. The general idea is an identifier can start with any character that has derived property XID_Start, and the remainder can be any characters that have derived property XID_Continue.
different characters looking identical, or the same character encoded in different ways
Follow the recommendation of UAX #31 and normalize identifiers to NFKC prior to comparing them. For example, the ligature variants in the article should not -- if recommendations are followed -- be different identifiers. Here's some Python (3):
>>> import unicodedata
>>> raw = ['vpnTrafficPort', 'vpnTrafficPort', 'vpnTrafficPort', 'vpnTrafficPort']
>>> normalized = set(unicodedata.normalize('NFKC', s) for s in raw)
>>> normalized
{'vpnTrafficPort'}
So there wouldn't be a "surprise" lurking in Python -- all four strings are legal identifiers, but all four of them are also the same identifier (because Python applies normalization).
The real difficulty with Unicode identifiers is in places where you really can't avoid Unicode: user inputs. Those who don't read Unicode technical reports are doomed to suffer the moment they build, say, a user-account system that comes into contact with the real world.
When you allow Unicode identifiers, you don't "internationalize" your code
No, you let other people localize their code. We (the tech world) spent decades making everyone else learn English as a prerequisite to learning a programming language. Now we have the ability to lessen that burden and let the non-ASCII world (which does in fact include English!) spend less time learning English and more time writing code. We probably should do that, even if it seems icky to you.
So instead of having to learn English as a prerequisite to understand code bases, you propose that we learn a dozen or more different natural languages instead?
Can you imagine what it would be like if every library had to be released in N different translations?
Sounds like an exceptionally bad idea. A huge step back for our field.
If you take the position that every codebase everywhere must be available to and understood by every programmer everywhere, sure, you have to settle on a common language. Luckily, I don't think anybody takes that position, or at least takes it seriously.
But there are plenty of monolingual dev teams out there whose shared language isn't English and isn't written in ASCII. Why should they be forced to write code in English if nobody who works with it is an English speaker?
I already regularly see people post questions to Stack Overflow and mailing lists and IRC where names of classes and functions and variables are ASCII but not English (i.e., someone writes a class called "Utilisateur", not "User"). Why not let them just use their language fully?
English will continue to be the lingua franca of programming for the foreseeable future, and full professional competence will probably require being able to read APIs that are in English.
However, for the sake of learners everywhere, it is important that it be possible to program using non-ascii identifiers. Asking a 10 year old who doesn't speak English to learn what "while" means is quite different from requiring their variables to be in a language they don't understand.
> Follow the recommendation of UAX #31 and normalize identifiers to NFKC prior to comparing them.
You're proposing that we replace the rules for identifier equality (comparing case-sensitive ASCII strings composed of A-Z, a-z, 0-9, _ is very easy to understand) with a complex system which hardly anyone can remember in its entirety, and even fewer people will want to learn? All so that according to you, some people need to learn a little less English? I sure hope that those normalization rules have been translated to lots of languages, otherwise those people would still need to learn English...
> Now we have the ability to lessen that burden and let the non-ASCII world (which does in fact include English!) spend less time learning English and more time writing code.
Do you seriously think that a programmer needs lower English skills just because they can use non-ASCII characters in their function names? That programming language's standard library is in English, its documentation is in English, external libraries are in English, their documentation is in English, StackOverflow is in English, but you need to "spend less time learning English" because you can name your variable "über_schwellwert" instead of "exceeds_threshold"?! Come on.
I think it would be great if people don't have to learn English to program. There are tons of people out there using computers, programming with very very poor English knowledge. Besides technical implementation issues, I see no reason to oppose allowing them to name things in a language they are more familiar with.
Of course you can keep using ASCII strings for your program but they should be able to name things however they like too.
The rules for unicode normalization only make a difference when you use them. As an English language programmer, they'll probably never affect you, and you're free to use your linter to require all code in your projects use only ASCII identifiers, just to prevent any potential problems.
Perl apparently handles this well: you have to declare that a file uses a particular script if you're using non-ASCII identifiers.
> you have to declare that a file uses a particular script
So, no calling functions in other files that use other scripts?
A reasonable restriction is to demand that all characters in any one identifier are from the same script, and that purely typographical devices such as ligatures are banned.
If you’re calling from another file you edit the pragmas in the file you’re calling from. Seems easy enough.
I actually am not quite clear on the details: one source made it sound like a limited set of scripts was enabled by default. Perhaps someone who knows Perl better can clarify.
If I'm in a Japanese file and want to call a function in a Greek file, I don't see how I can avoid writing a Greek identifier in the middle of the Japanese.
The rejection of XID_Start/XID_Continue linked from the Rust issue is not, as far as I can tell, due to "weird" issues -- it's because the sets of characters with those properties can change over time as Unicode adds more characters.
Other languages tend to solve this by stating that version X of the language uses identifiers having those properties in version Y of the Unicode database. This is a problem for C and C++ because of how infrequently those languages update their standards and how glacially slowly implementations adopt new versions of the standards. It's less of a problem for languages like Rust where the language spec and standard toolchain evolve on a much faster (at least one update per year) cadence.
The tweet you link complains about U+2800. That character is not in XID_Start or XID_Continue.
So a language adopting XID_Start/XID_Continue would not allow U+2800 in an identifier. And in fact if I try it in Python I get "SyntaxError: invalid character in identifier".
Thanks, I hadn’t realized that covered U+2800—my mistake.
I haven’t fully digested the links to the C working group, but as those Perl docs point out, you have to do some work even after restricting to XID_Start/Continue, even if you don’t have to forbid other characters.
The Perl doc mostly refers back to suggestions from UAX#31, along with specific notes from UTR#36 and UTS#39.
The suggestion it cares most about from UAX#31 is to restrict to scripts that actually are in use (i.e., don't let people name variables using Linear B characters). If you want to layer that on top of a base pool of identifier characters taken from XID_Start/XID_Continue, you can, and that's not the same as "invent your own base pool of identifier characters".
The big thing you get from reading UTR#36 and UTS#39 is learning how to detect or prevent homoglyph attacks (like people registering "paypal.com" but with Cyrillic "a").
And UTS#39 takes the ideas from UAX#31 all the way and gives you an example of defining profiles on top of the base set of identifier characters to deal with specific issues. It's OK to do that!
Finally, the things that, in a programming language's allowed identifier syntax, would be prevented by going to a more restrictive profile on top of XID_Start/XID_Continue, are vanishingly rare and tend to be exploitable only if you're already completely owned. For example, if someone can slip a mixed-script confusable identifier into your source, they can already slip things into your source; you're owned in so many ways at that point that it starts seeming silly to focus with laser intensity on just this one issue. Which means that if you just want an easy-ish to implement baseline recommendation, XID_Start/XID_Continue isn't that bad.
> The real difficulty with Unicode identifiers is in places where you really can't avoid Unicode: user inputs. Those who don't read Unicode technical reports are doomed to suffer the moment they build, say, a user-account system that comes into contact with the real world.
Simply reject any piece of input, such as a user display name, which uses characters from two or more different scripts, or purely typographical devices such as ligatures.
Reject any mixed-script confusable (and for certain types of inputs, like email addresses, validate sub-components separately -- someone might get to choose their local-part but not their domain).
For strings which aren't mixed-script confusable, if you need to use them as identifiers and perform comparisons with them, normalize to NFKC and case fold. Doing so will eliminate any ligatures, stylistic variants, composed versus decomposed forms, or other things that people use to call Unicode "weird".
Anyone can type them using a modifier key. I have a UK keyboard and to get ß I hold option and press s. It's no different to using the shift key. If you've got no problem with capital letters you should have no problem with these letters.
But then you need to learn and remember how to type those letters. There's such a big difference between capital letters, which you need and use constantly, and badly discoverable key combinations, which you need and use infrequently, that your comparison is downright disingenuous.
Sometimes I boot into a machine where the keyboard layout is set to English. Finding the right keys on my non-English keyboard to even just type "ls -l /" or similar, is a major pain in the ass. It would be the same for a Canadian having to type 'ß' on an irregular basis.
PCs have “alt codes”, which is a combo of alt+numbers to represent some (most?) Unicode characters. I believe the German characters are in the alt+0250 (+-10) zone
I think it's useful, if you feel it's a burden to have to interoperate with a codebase using non-ASCII identifiers, to consider how literally every person in the world whose native language is non-ASCII must have felt being forced to use it for programming.
Let people who have to interoperate with multilingual colleagues settle on conventions for doing so. Let people who don't use their own language already.
I don’t think the fact that Unicode identifiers can be confusing is a big problem.
If you trust some third party enough to download code they wrote and run it, the added risk that you confuse a few identifiers and run code you didn’t intend to run isn’t that big.
If this happens in code you wrote, you fix it when you discover it or learn to live with it, just as when you find the same problems with ASCII identifiers. Those also can be confusing (Il0 vs I1O, color vs colour, misspellings, etc)
Also, inconsistent transliteration to ASCII can introduce problems. Using native strings can prevent that.
The importance of Unicode is in letting people write programs in foreign languages. I had a Pascal textbook written in the 80s in which all the variables had German names. That's no longer possible today. Shitty variable naming using ASCII, on the other hand, is still possible and is actually quite prevalent.
> How would syntax highlighting catch that? Those are valid identifiers!
Author here, you're absolutely right. When I wrote this, I was thinking of some of the more obvious tricks like the alternate semicolon, parens or curlies. I'm gonna edit the article to qualify that, and add a note about maybe using a SwiftLint exclusion rule containing all these unicode characters.
“Zalgo-style” function names for dangerous or weird functions is kind of a cute idea.
In dense scientific code there’s always been a place for special characters. (apl, anyone?) To the right kind of programmer, using the proper mathematical notation for operations and variables in a function can greatly improve readability and clarity.
Unicode for use in theorem proving languages also has a role — agda comes to mind here.
I made a similar post when I actually fixed unicode identifiers for cperl (the perl5 replacement), according to the unicode security recommendations which everybody else besides me and java chose to ignore. Most languages are horribly insecure. But I eventually complained to rust, and they are adopting these recommendations.
54 comments
[ 1.5 ms ] story [ 98.7 ms ] thread1) As the article demonstrates, this allows obfuscation contests in easy mode. Strangely, the author claims:
> Nor do I think most of these tricks would get by a code review or go unnoticed using something as simple as syntax coloring.
How would syntax highlighting catch that? Those are valid identifiers! And yes, these "tricks" would almost certainly go unnoticed in a code review, at least when you don't assume that your coworker is maliciously inserting bugs.
2) Even in the absence of malicious intent, Unicode identifiers open the door to bugs and head-scratchers: different characters looking identical, or the same character encoded in different ways.
3) ASCII letters (a-z, A-Z), digits (0-9), and underscore are kind of the lowest common demoninator. When you allow Unicode identifiers, you don't "internationalize" your code, you're doing the exact opposite! If you're using ä, ö, ü, ß in your API, how is a Canadian supposed to easily use it?
When I was in france, all the software were written using french identifiers. It was much better this way since none of the code was meant to be shared outside of france and it made the code much more readable for people who were supposed to work on it. Why should it be possible only for latin based languages?
> it made the code much more readable
"Much more readable"? It made the code base inconsistent with the rest of the language. Are you saying that they were lacking even basic English language skills?
I don't see any problem with source code that says something like 'auto maßtab = waage.m_maßtab();' 'auto' and '=' are simply language tokens.
There's no potential for confusion if you allow ASCII letters and 'ß'. Confusion can arise, however, when you allow all Unicode letters.
To me, using English keywords everywhere is a historical accident, not a design goal. In next-generation systems like Subtext [1], it looks entirely possible to add localization to source code, too.
Once you throw out the Lisp way of having one namespace for all symbols (or, OK, like 7 in CL), to be used as functions or values, and go to a system where you want to be able to statically type-check everything, why not fully commit to it and let users give local names to identifiers? Then we won't be limited to legacy C-style naming rules, or have to argue about how many space characters to add to make all their names line up on a high-resolution VT100 emulator, either.
[1]: https://en.wikipedia.org/wiki/Subtext_(programming_language)
Yes i'm saying that a lot of colleagues were lacking basic english skills, and i don't see the issue with that. English is a nice to have to communicate with people abroad, but absolutely not necessary to live in France.
Oddly (to me, anyway), the software house that made the program was located in Africa.
Sometimes I'm tempted to use Russian identifiers for application domain terminology, because very often there's no adequate translation or I just don't know English enough to translate some terms. I like the following example: every man here has name combined from last name (Фамилия) which is used for all family members, first name (Имя), that's pretty standard. But there's also third name: Отчество, which is basically father's name with some suffix, e.g. father Иван will have child with third name Иванович. Now I've seen all kinds of translations for those terms, including plain wrong. Last name, family name for Фамилия, first name, name for имя, father name, second name, patronymic, patronymic name for отчество. I've seen bugs because one developer interpreted those terms differently from another.
And that's just for name. There are plenty of other terms. Add to that that many developers are bad at English and at best will use first result from Google Translate. So may be national identifiers have its place.
Sounds like you get to skirt a few of the problems outlines in Falsehoods Programmers Believe About Names[1]. That's nice for programmers in Russia that deal with Russia only, if true. :)
1: https://www.kalzumeus.com/2010/06/17/falsehoods-programmers-...
Of course the whole point of the article you mentioned is that making such assumptions leads to odd behavior. I (an American) lived in Russia for a while. One time I was having a couch delivered to my apartment and had to give the store my full name. I was delighted to see my middle name show up on the receipt in the отчество (patronymic) field with the Russian patronymic suffix ович attached. I still got the couch no problem, but I got a laugh out of their attempt to shoehorn my (clearly not Russian) names into their naming paradigm.
Unicode identifiers can be perfectly well-defined, and many languages have them. The general idea is an identifier can start with any character that has derived property XID_Start, and the remainder can be any characters that have derived property XID_Continue.
different characters looking identical, or the same character encoded in different ways
Follow the recommendation of UAX #31 and normalize identifiers to NFKC prior to comparing them. For example, the ligature variants in the article should not -- if recommendations are followed -- be different identifiers. Here's some Python (3):
So there wouldn't be a "surprise" lurking in Python -- all four strings are legal identifiers, but all four of them are also the same identifier (because Python applies normalization).The real difficulty with Unicode identifiers is in places where you really can't avoid Unicode: user inputs. Those who don't read Unicode technical reports are doomed to suffer the moment they build, say, a user-account system that comes into contact with the real world.
When you allow Unicode identifiers, you don't "internationalize" your code
No, you let other people localize their code. We (the tech world) spent decades making everyone else learn English as a prerequisite to learning a programming language. Now we have the ability to lessen that burden and let the non-ASCII world (which does in fact include English!) spend less time learning English and more time writing code. We probably should do that, even if it seems icky to you.
Can you imagine what it would be like if every library had to be released in N different translations? Sounds like an exceptionally bad idea. A huge step back for our field.
But there are plenty of monolingual dev teams out there whose shared language isn't English and isn't written in ASCII. Why should they be forced to write code in English if nobody who works with it is an English speaker?
I already regularly see people post questions to Stack Overflow and mailing lists and IRC where names of classes and functions and variables are ASCII but not English (i.e., someone writes a class called "Utilisateur", not "User"). Why not let them just use their language fully?
English will continue to be the lingua franca of programming for the foreseeable future, and full professional competence will probably require being able to read APIs that are in English.
However, for the sake of learners everywhere, it is important that it be possible to program using non-ascii identifiers. Asking a 10 year old who doesn't speak English to learn what "while" means is quite different from requiring their variables to be in a language they don't understand.
You're proposing that we replace the rules for identifier equality (comparing case-sensitive ASCII strings composed of A-Z, a-z, 0-9, _ is very easy to understand) with a complex system which hardly anyone can remember in its entirety, and even fewer people will want to learn? All so that according to you, some people need to learn a little less English? I sure hope that those normalization rules have been translated to lots of languages, otherwise those people would still need to learn English...
> Now we have the ability to lessen that burden and let the non-ASCII world (which does in fact include English!) spend less time learning English and more time writing code.
Do you seriously think that a programmer needs lower English skills just because they can use non-ASCII characters in their function names? That programming language's standard library is in English, its documentation is in English, external libraries are in English, their documentation is in English, StackOverflow is in English, but you need to "spend less time learning English" because you can name your variable "über_schwellwert" instead of "exceeds_threshold"?! Come on.
Of course you can keep using ASCII strings for your program but they should be able to name things however they like too.
But that's the point – you'll still need to learn just as much English when you have Unicode identifiers as when you only have ASCII identifiers.
Perl apparently handles this well: you have to declare that a file uses a particular script if you're using non-ASCII identifiers.
So, no calling functions in other files that use other scripts?
A reasonable restriction is to demand that all characters in any one identifier are from the same script, and that purely typographical devices such as ligatures are banned.
I actually am not quite clear on the details: one source made it sound like a limited set of scripts was enabled by default. Perhaps someone who knows Perl better can clarify.
That's insane. Not in my programming language (lawn, driveway, ..).
https://github.com/rust-lang/rust/issues/4928
https://twitter.com/aisamanra/status/923346798093090816
Other languages tend to solve this by stating that version X of the language uses identifiers having those properties in version Y of the Unicode database. This is a problem for C and C++ because of how infrequently those languages update their standards and how glacially slowly implementations adopt new versions of the standards. It's less of a problem for languages like Rust where the language spec and standard toolchain evolve on a much faster (at least one update per year) cadence.
The tweet you link complains about U+2800. That character is not in XID_Start or XID_Continue.
http://www.unicode.org/Public/11.0.0/ucd/DerivedCoreProperti...
So a language adopting XID_Start/XID_Continue would not allow U+2800 in an identifier. And in fact if I try it in Python I get "SyntaxError: invalid character in identifier".
I haven’t fully digested the links to the C working group, but as those Perl docs point out, you have to do some work even after restricting to XID_Start/Continue, even if you don’t have to forbid other characters.
The suggestion it cares most about from UAX#31 is to restrict to scripts that actually are in use (i.e., don't let people name variables using Linear B characters). If you want to layer that on top of a base pool of identifier characters taken from XID_Start/XID_Continue, you can, and that's not the same as "invent your own base pool of identifier characters".
The big thing you get from reading UTR#36 and UTS#39 is learning how to detect or prevent homoglyph attacks (like people registering "paypal.com" but with Cyrillic "a").
And UTS#39 takes the ideas from UAX#31 all the way and gives you an example of defining profiles on top of the base set of identifier characters to deal with specific issues. It's OK to do that!
Finally, the things that, in a programming language's allowed identifier syntax, would be prevented by going to a more restrictive profile on top of XID_Start/XID_Continue, are vanishingly rare and tend to be exploitable only if you're already completely owned. For example, if someone can slip a mixed-script confusable identifier into your source, they can already slip things into your source; you're owned in so many ways at that point that it starts seeming silly to focus with laser intensity on just this one issue. Which means that if you just want an easy-ish to implement baseline recommendation, XID_Start/XID_Continue isn't that bad.
Simply reject any piece of input, such as a user display name, which uses characters from two or more different scripts, or purely typographical devices such as ligatures.
Reject any mixed-script confusable (and for certain types of inputs, like email addresses, validate sub-components separately -- someone might get to choose their local-part but not their domain).
For strings which aren't mixed-script confusable, if you need to use them as identifiers and perform comparisons with them, normalize to NFKC and case fold. Doing so will eliminate any ligatures, stylistic variants, composed versus decomposed forms, or other things that people use to call Unicode "weird".
In Swift all identifiers are Unicode - basic letters are part of Unicode. The entire source file is Unicode all the way through.
> how is a Canadian supposed to easily use it?
Using their keyboard? Can't a Canadian type these letters in the exactly the same way as anyone else would?
> In Swift all identifiers are Unicode - basic letters are part of Unicode.
Yes, and it's an exceptionally bad idea to allow non-basic letters in identifiers, as the linked article demonstrates.
>> how is a Canadian supposed to easily use it?
> Using their keyboard? Can't a Canadian type these letters in the exactly the same way as anyone else would?
No, they can't. ä, ö, ü, ß are part of the German keyboard layout QWERTZ, but they're not on a QWERTY keyboard.
Anyone can type them using a modifier key. I have a UK keyboard and to get ß I hold option and press s. It's no different to using the shift key. If you've got no problem with capital letters you should have no problem with these letters.
Sometimes I boot into a machine where the keyboard layout is set to English. Finding the right keys on my non-English keyboard to even just type "ls -l /" or similar, is a major pain in the ass. It would be the same for a Canadian having to type 'ß' on an irregular basis.
Let people who have to interoperate with multilingual colleagues settle on conventions for doing so. Let people who don't use their own language already.
If you trust some third party enough to download code they wrote and run it, the added risk that you confuse a few identifiers and run code you didn’t intend to run isn’t that big.
If this happens in code you wrote, you fix it when you discover it or learn to live with it, just as when you find the same problems with ASCII identifiers. Those also can be confusing (Il0 vs I1O, color vs colour, misspellings, etc)
Also, inconsistent transliteration to ASCII can introduce problems. Using native strings can prevent that.
Author here, you're absolutely right. When I wrote this, I was thinking of some of the more obvious tricks like the alternate semicolon, parens or curlies. I'm gonna edit the article to qualify that, and add a note about maybe using a SwiftLint exclusion rule containing all these unicode characters.
Thanks for the discussion! :)
In dense scientific code there’s always been a place for special characters. (apl, anyone?) To the right kind of programmer, using the proper mathematical notation for operations and variables in a function can greatly improve readability and clarity.
Unicode for use in theorem proving languages also has a role — agda comes to mind here.
http://perl11.org/blog/unicode-identifiers.html
It's not fun, it's a risk.