Tell HN: I can't buy a PS4 because of my email
Just wanted to share this because I think it's ridiculous.
I happen to have an email with one of the newer TLDs (like .pizza or .website). This somehow makes me unable to check out on the sony website.
See: https://imgur.com/a/gL0u6JY
Of course, the real error is swallowed by their frontend and all I can see is that an error has occurred. I have to dig into the network requests to find out the real reason why they don't want to take my money. Who comes up with this? Why is email validation still a problem in 2022?
How can a normal internet user figure this out? If I wasn't a web developer I would have given up.
137 comments
[ 8.1 ms ] story [ 181 ms ] threadIt reminds me a situation when I could not register a bank account and nobody could help me, even the official support. There was (maybe even still is) an issue when the webpage tells me that my password must contain 4 digits but it did not tell that that digits must be in one chunk (abcd1234 or a1234bcd but not a1b234cd). Several friends of mine had to show me their passwords until I realized that their digits are not sparse.
In my opinion it makes sense to be patient for 3 weeks and get a PS5, which will surely be relevant for 5-7 more years.
Keep in mind, PS4s are also difficult to find right now. They are still selling at the full $300 retail price if you can even find one, and used on eBay they are $250. Why pay $250 for a console that will stop getting new games this year or next year?
Now granted, OP seems to be in a different country so maybe these circumstances are different for them but in the US certainly it does not make sense to buy a PS4, for the same price you can buy an Xbox Series S.
I know, because I did it when the shortage was at its peak. I set up telegram alerts and I kept my personal laptop on standby. If an alert came in, as long as I wasn’t in a work meeting, I opened my laptop and quickly followed through to the link.
It just takes a little bit of patience but it’s not too bad. But hopefully the shortage improves soon and this will no longer be necessary.
> Last Updated Wed Mar 16 07:07:01 2022 UTC
Have fun keeping up-to-date with it.
But also, you can get the tld and match it against the list, then send an email.
Actually, even trying to use regex to see if a email is valid is a fool's errand, but if you have to, the rule above is the only thing you could do. You really need to send a email to address and ask the user to input something from the email in order to actually validate it.
Forward slashes in addresses (e.g. <floor>/<apartment>) cause similar mysterious errors.
It could also be that they see a lot of fraud from people using non-traditional TLDs (basically the gTLDs) so they block those, but serve some generic error that looks like their email is not actually valid.
But then email validation has basically always been an issue. The only real way of validating emails is to send an actual email to specified address and ask the user to input something that that email contains, but that adds a lot of work than some simple regex.
I also used to use a uncommon gTLD, but then we're not "normal internet users", at this point at least, maybe in the future. Best work-around I came up with is having a traditional TLD that I can use when the system doesn't work with the email I usually use. I think this is the best thing you can do moving forward, because Sony is not the only ones fucking this up, there are plenty of services that don't accept your email if you use something like .pizza, .app or any other common gTLD.
Edit: I should clarify. "Rules" is referring to regex rules, for the ones who are engaging in the fruitless endeavour of trying to validate emails with regex, not the "rules" as the specifications for what emails addresses are actually valid. I restate what I said elsewhere: only proper way to validate emails is by sending a email to the address and make the user input some data you sent in said email. Anything else is bound to not work for X% of users.
the single relevant rule is the same ever since:
do an opt in. The spelling of the email address is of zero relevance and must not be nosed at.
Assuming Sony blocks all LEN(TLD)>3, do you think they’ll lose more customers than they’ll lose from them not getting their validation mail because they typoed their mail?
I’d bet that the first group is far smaller than the 2nd.
Now instead of blocking it would be nice if they just asked if you are really sure that’s your email, but it’s probably still better than just accepting everything and trying to send a confirmation, both from a business and from a customer-friendliness perspective.
You need the opt-in and have people confirm or repeat in case.
But QA and business refused to believe RFC rules and instead wanted their own rules. Guess what we end up implementing.
One win we had was, we didn't hardcode any tlds.
And own rules very much sounds like management ever wanted to invent something.
Why do you think so? We didn't have a strict set of TLDs before as far as I know. We just stagnated for a long time. But the list of TLDs has changed many times with countries being created/renamed. If you saw .pizza or .zx, systems should've behaved the same - tried resolving.
Because .pizza wasn't a valid TLD before 2014 when it got delegated? Don't know what to tell you, .pizza haven't always existed, it's a relatively new gTLD that got introduced by the new gTLD program by ICANN.
But I agree with you in general, trying to validate emails with regex will never work in practice, and has lots of issues. Only real way to validate a address is to actually send email, ask user to input data from said email.
Of course it does. It just depends on how you define 'validate emails'. Your statement is ambiguous. The Friedl has an entire appendix on a single regex that validates an email address to the format described in RFC 5322, although of course maybe a proper parser would be better, from a technical perspective. However, an email address being in the right format does not ensure that the user that email address is 'associated' with (for whatever definition of 'associated' that is appropriate to the application we're talking about) can/will actually receive emails send to that address.
You should do both. Do a first 'pass' validation to check the user didn't make a type. This is more for user convenience, as this can give immediate feedback. And then send an actual email to see if email is deliverable there.
The problem here is not that they checked the format. The problem is that they check the wrong way. Although it's not entirely unthinkable that it's sometimes appropriate to add additional constraints to which addresses you want to accept, e.g. banning certain tld's, or even only accepting whitelisted ones. If your fraud stats say that doing that reduces loss, it's not unreasonable to add such a third filter layer.
Blocking an entire TLD because of fraud is insanity. There are plenty of realistic looking domains available for cheap in the dot com TLD, no need to go for dot pizza or dot dev to bypass spam filters.
Wikipedia does this with .xyz links. When I asked about this (I wanted to put a link to finl.xyz on my user page), I was told that this was because of link spamming.
Yes, you can put UX helpers for spelling mistakes on most common domains (gmail…) (helps the majority), but never make it a hard error. It's that simple.
edit: https://datatracker.ietf.org/doc/html/rfc2822#section-3.4.1
Perhaps tech giants could sponsor a new RFC for email addresses.
No they don't. But they are sufficiently complicated that effectively the only way to validate an email address is to try sending a message to it and wait for a reply.
Falsehoods that programmers believe: https://github.com/kdeldycke/awesome-falsehood#emails
As a web developer, I'm sure if you really want it one, you could create a quick throwaway email address using a small array of services.
Let's think about how strange this as far a bit, if you have a smartphone at all, you'll either have an email address provided by Apple or Google. Google. If you choose not to utilize either one of these, that's your choice.
At the same time, Sony has never been good with web technology, the PS4s online play has always been less feature-rich than Microsoft's offering.
There’s definitely a big problem here with email, and I don’t know what the solution is.
I have no idea about any apple email addresses.
Nothing else but the format should be checked and ANY kind of error should be properly communicated.
I had two situations recently that cost me a substantial amount of time and effort, due to f**ed up email handling and non-communication to the user.
Once I tried to sign up to Pinterest with my personal catch-all domain using the emai pinterest@mydomain.tld. Only received random errors no matter what browser and IP I tried, had to contact 'support' (three levels of bots) to eventually find out that 'pinterest' is a reserved word because of which they would not accept my email.
Second situation in a hotel trying to connect to WiFi via a captive portal. Using email hotel@mydomain.tld (which I used to check in and was thus required to use in the portal) the page just refreshed again and again without throwing a user facing error. After significant back and forth at the reception I was educated that I was only allowed to use private / personal emails (meaning GMail). My own domain wasn't personal enough...
Both situations could have been rectified easily if proper errors would have been communicated.
However, the solution is also obvious: get a new email address in a different domain specifically for this purchase. Free, paid, buy a new domain: all of these are options.
>How can a normal internet user figure this out? If I wasn't a web developer I would have given up.
A normal internet user will have a gmail or outlook mail address, not a .pizza one.
If a "is this real" filter is required, they should write a script to check for new TLDs daily.
A normal internet user may also be in a country where ".com" is not the main TLD.
Granted, .pizza is unusual, but there is no valid excuse for not supporting it, and systems that fail to use it likely also fail other completely "normal" email addresses due to other invalid email validation like assuming a TLD has one dot, that an email address has a certain arbitrary max length, or because they failed to enumerate even common TLDs.
The only proper way to validate an email is to send an email to it.
In my experience ccTLDs are always supported (I've used some funky ones over the years like .sh and .dj), while newer general ones (like .online) are sometimes rejected. My "solution" is to keep a ccTLD to use as an email alias.
- .香港 is a valid ccTLD (for Hong Kong)
- .xn--j6w193g is a valid ccTLD (punycode transliteration of the above)
Regardless, the issue is in the validation end here. There's public lists with valid TLD's and any email address validation should ensure valid email addresses are let through.
A normal user does not know about common unsafe assumptions in every aspect of programming and IT integration.
Once the option to get foo.pizza exists, absolutely a ton of normal users take it, just like they do everything else.
From your HN profile:
> fe {at} f19n .dot. com
Are you really commonly using a 42 characters long email? How does that work when you fill out paper forms that asks for emails? I usually use one that is 16 characters in total (including the domain part), and I already sometimes cannot fit it on the papers.
But peoples official names have nothing to do with your personal email addresses, as you usually get to choose whatever you want for those.
Because almost everyone else is, they should be able to too. It's not very difficult to understand really.
Yes, this is precisely why I'm asking, I have a long name but won't use my full name for my email because it is just that, long. Even with my abbreviated email address I'm having a hard time to fit it on forms, why would I make my life even more difficult and use my full name when I can chose whatever I want when creating my email address? It's not very difficult to understand really.
And yeah, life is unfair if you're different, because different things were built for different things. Understanding this is also not very difficult to understand. As someone not-from-the-anglosphere, you learn to try to workaround where your abnormalities don't fit in, otherwise you miss out.
The discussion should end at the conclusion that (those) programmers should write better code, they have written bad code and not that the victim should adapt around the bad code.
I guess are difference is that I'm seeing this as a practical issue that has a pragmatic solution, at least for me, since I care about moving on with my day, while you feel it's better to just give up and not use something if you can't use your email address for it.
The benefit for finding workarounds is that you can still use that thing you were trying to use.
The biggest gripe I have with this situation is that it was almost impossible to figure out what the real problem was. I had to dig into the request to find the error that the frontend refused to show me.
If they allow me to register an account with a weird email, why does having that weird email prevent me from paying?
Often times the merchant or payment processor has different systems entirely (and data is simply scraped from the form and input into their system for you).
Its happened to me before as well. Specifically when donating to my local PBS station...
So the operator doesnt care what your account name is, but the merchant they use to process payments/ensure PCI may care.
They go to site x and it fails in inconsistent and unexplained way. It is correct to stick to that and pretend you don't have special insight into sepeerate building blocks and service boundaries behind the scenes.
My email is <catchall>@david.kitchen and for most shops I will give them <shop>@david.kitchen .
Mostly it works.
But when it doesn't work it really doesn't work. And this is OK. If they don't support one of the recent TLDs then I can just fall back to using a throwaway email address. Or if I really need the item I will shop elsewhere :shrug
The annoying edge cases:
1. When <catchall>@david.kitchen works but <shop>@david.kitchen didn't work... like they're actively trying to fight the use of unique email addresses.
2. When <catchall>@david.kitchen works for registration on a web site, but doesn't work for signing in on their mobile app. Because the app enforces different email validation than the website. This is pretty annoying, especially as some apps will put the "change email" functionality in the app... rendering the account useless.
3. When <catchall>@david.kitchen works for registration on a web site, but doesn't work for the payment provider and checkout section. This is by far the most frustrating as you've got so far and are invested in the purchase and now you're stopped. I have contacted vendors about this, it's brain dead.
When email validation fails it's not because they are using a hard-coded list of domains, it's usually because they have a regexp pattern and only consider a 3 letter suffix valid.
I do tell web sites I have this issue, and sometimes they listen. Mostly I take my business elsewhere. Oh well, my choice to have a "difficult" domain name.
Also: People really don't understand email addresses. "What's your email?", "oh it's <shop>@david.kitchen", "If you work for us you're eligible to the staff discount, do you have your ID card?" - This happens, or some confusion of it, more than I would expect.
My goals are just to identify who was compromised when an email address leaks, and secondarily to prevent entirely unconnected merchants or organisations from trivially joining the dots on a connected identity. In both cases good enough is good enough and I'm not trying to achieve perfect security or privacy with email as that's virtually impossible.
I don't have to set those up on my phone, and I generally found cashiers and other customer service to be understanding (they mostly do not ask twice, except for confirming that "plus" is indeed "+"). Once I was looked at like I didn't understand what an e-mail address was supposed to be, until I told them my custom domain name.
That works fine for the majority of use-cases, but some refuse the "+" character, others (rarely) refuse my custom domain. Once, a shop refused to have its name.
For the edge-cases, I generate a new e-mail address, and sometimes drop them an annoyed e-mail with a technical explanation of what's allowed and how to check for it.
I think most client-side verification is here to avoid typos, but it shouldn't lock out the unexpected. Websites might have more success asking users to double-check unusual e-mail addresses or likely typos (like @gamil.com or smantha.lastname@... -- typo in first name), without outright rejecting them.
It's on a .com domain I've had forever. There are a few inboxes for different people. There is a regex for each and if it matches, it goes in that inbox.
An example rule, it must contain 'zy' in the user and it gets rerouted to inbox 1. If it contains 'yz' it gets rerouted to inbox 2.
so, shopzy@example.com and shopyz@example.com
The problem is that I'd still received a lot of spam. Tried graylisting and other methods, but a hard rule (i.e. name must end in a number) ended up working better.
Another amusing thing about using a personalized catch-all is finding out who had their users' database hacked/leaked ...and, other than the common ones that we all know (the ones in have I been pwned?), I received some others that were kinda big (i.e. a Spanish airline company)
Lesson learned: If a domain is important to you, set up your own reminder to renew.
Interesting aside, I run an online store, something like 90% of customers have an email address from one of a main providers - MS, Google, Yahoo, Apple, and as we are in the UK BT.
We have also seen a significant correlation between customers with a @btinternet.com or @yahoo.com and those that contact support more. You could say they are significantly more likely to be "difficult" customers.
Of course my parents have a @btinternet.com email address...
Finally, Yahoo has by far the worst deliverability, all our email (transactional & support, not mailing list) get through to other providers but with Yahoo it's as much as 50% of email just disappears at times, it's not even in customers spam folders.
Watch out when trying to change it back to your custom domain. I tried that and was locked out of my account. Had to call PlayStation support to change it back.
There are two approaches
1) You can complain that they aren't serving you, and take your money elsewhere. They see the lost revenue and fix the problem, or they decide the cost of supporting you isn't worth the profit
2) You can conform and get a more normal email address like 99% of the world. I have enough issues when I give people my own domain.com address as they are confused it doesnt end in gmail or hotmail
Most of the 1% that have the issues will conform, so you're left with 1% of 1% who draw a line in the sand. And that's great, good luck to you and the 35,000 other Americans that fit in that demographic. Big companies are interesting in the 99.99% that don't though.
(1) is a symptom of the centralising, automation, and reduction in costs of services. If you don't fit in the 99% pattern (or even 90% pattern), you are rejected as you're not worth it, that's the direction the world is travelling in (google etc drop your account automatically rather than spend the money to look at it manually, you can sometimes get redress via complaining to your social network if you're lucky)
This one always makes me laugh. I got a quote from a garden center on some yard work last week. Guy was stunned i used my own TLD, made me verify it a couple times.
Then sends me an email from.....@soandsogardencenter.com.
I was a bit perplexed as to why it was confusing for him. I guess maybe he thought you had to be a company to have your own TLD. He definately didnt understand why i would run my own (which is totally fine, most arent really concerned with the minutia of email).
A TLD is not the same as a domain name, but rather a top-level domain (after the last dot).
It’s worth noting that while my email does use a .com as a backup, it’s primary is a gtld .email
There are too many screwed up websites in the world to get hung up on somebody elses.