> I wonder if pre-emptively moving to a 24 hour trading market has helped reduce load on Robinhood
Not directly, but expanding the product scope may have placed additional selection pressure on engineering choices such that boring and stable options became more likely over time.
It's an interesting topic. I'm sure this is just excess server load and nothing nefarious but could the SEC mandate an SLA? Require disclosure of their DataDog dashboard?
Other institutions, retail whose brokers didn’t fail. Most activity on the markets in terms of volume would generally be, broadly, institution v institution anyway.
I’m not saying it makes it right, but retail is probably better off sitting tight for now. If anything, pros and institutions already had those sell orders ready to fire last night and would get in ahead of retail on market open, at least that’s my guess. So I don’t know what good it would have done me to be at the end of the line to sell my retail stuff.
OTOH, if you desire to go shopping for bargains, yeah, not being able to trade sucks.
Somehow I was able to stay logged into schwab (checked markets at 6am EST) and it's working without a hitch on the other side of login, I was able to make trades at open.
So it seems it is the login servers that are getting destroyed, which is expected, but also annoying that this happens anytime volatility spikes. You'd think they would be prepared by now. Or maybe it's "overlooked" to dampen panic selling by retail...
Both Fidelity and Schwab were fine for me on devices where I was already logged in, but logging into a new device wasn’t working.
This seems like a common issue - even if backend systems can take the load, login systems all seem to have a lower TPS limit and have a lot of trouble during big surge events like this. It reminds me a lot of when a new video game releases where the issue is often the login servers being overloaded.
Revalidating session data could be a smaller surface area than validating a fresh login. But if logging in existing devices works in some cases, and doesn't in others, the failure case is more complex than just fresh logins don't work.
Ah, yes, my original comment was a little unclear. I actually wasn’t logging in on new device vs existing device, it was that I had logged in to one device before 9:30est (presumably before the surge began) and just had an existing already-logged-in-session.
There's a good reason you want to avoid holding significant margin over the weekend, or even over night. Anyways, there will be plenty of good deals going on sale this week. The market is granting opportunity this week. If you're not on margin and looking to be fast then just ignore and keep adding to your investment accounts as you have been.
There goes thinking we had any sort of resiliency in the retail stock market...
I really wonder how all of the major brokers could be having an issue. Is it just one of those "we haven't ever had a real market crash since we started this website in 2015, so let's not do a load test" kind of thing?
For those unaware TD Ameritrade was acquired by Schwab recently and in the year since the acquisition Schwab has had multiple days of outages. I was with TD for years without a single instance I can recall of being unable to access my account.
This is just absolute negligence and neglect of necessary technical investment from management. The only thing that will get their attention is people withdrawing funds and moving to better run brokerages, which is what I intend to do.
I don’t love robinhood as a company but functionally it works really well most of the time.
I think a lot of the newer banks/neobanks are pretty good both because they have less customers and therefore less stampeding herd issues and because they are a lot more oriented towards supporting fast scaling. I use Wealthfront and have enjoyed so far
Robinhood? You mean the company that halted trading of Gamestop stock and lied about why? I don't know, maybe their app isn't "broken" but it certainly isn't reliable.
I’ve been a Schwab user for years, and was a TDA user as well before the merger. TDA had just as many issues like this. Don’t let your recency bias fool you.
Schwab is overall a great bank and brokerage. Fidelity’s user experience quite frankly is much worse, as is E-Trade’s. Vanguard is even worse than those. IBKR looks nice, but the hassle hasn’t been worth it to me. Schwab has been great for me so far.
I'm a bit surprised at the short term memory as well. TDA, and Think Or Swim before that all had outages on Occasion.
My experience was that my ToS desktop app was still logged in and kept an active connection, On my Android phone, the Schwab App was able to work (I only use it as a backup) while TOS (My more daily trading app) required me to wipe the local data to get logging in working again (A fix that was required last time Schwab messed up the log-in system).
Schwab has, in the last year or so, made their login much more complex to work with MFA across all apps. This probably have their log in servers doing a lot more work than they projected.
“Login” is such a misnomer. It sounds like a password check (==read) but includes write activities of various kinds and complex can’t-skip logic.
It’s the step where the system first sees the user and must establish identity, of course. But it’s also an attractive place for system designers to frontload complex logic that result in cache-priming or context-priming which the rest of the system depends upon. And then there’s DDOS, device status, account status, anti-fraud, throttles, etc.
I was planning on opening accounts with Schwab, but after I signed up and went through the login process, I very quickly closed the account and went elsewhere. It was among the worst user experiences I can recall. Everything about it screamed legacy and immutable. I had the same feeling about LastPass a few years ago. Always go with the gut!
It’s the largest trading volume day in years and everyone that isn’t living under a rock is trying to log into their banks to check their portfolio. The banks can’t handle the surge logging in.
It’s not that suspicious. Not everything is a conspiracy.
76 comments
[ 0.24 ms ] story [ 143 ms ] threadI wonder if pre-emptively moving to a 24 hour trading market has helped reduce load on Robinhood
Robinhood down 15% this morning.
Not directly, but expanding the product scope may have placed additional selection pressure on engineering choices such that boring and stable options became more likely over time.
OTOH, if you desire to go shopping for bargains, yeah, not being able to trade sucks.
So it seems it is the login servers that are getting destroyed, which is expected, but also annoying that this happens anytime volatility spikes. You'd think they would be prepared by now. Or maybe it's "overlooked" to dampen panic selling by retail...
This seems like a common issue - even if backend systems can take the load, login systems all seem to have a lower TPS limit and have a lot of trouble during big surge events like this. It reminds me a lot of when a new video game releases where the issue is often the login servers being overloaded.
West Coast retail customers are out of luck.
I really wonder how all of the major brokers could be having an issue. Is it just one of those "we haven't ever had a real market crash since we started this website in 2015, so let's not do a load test" kind of thing?
For those unaware TD Ameritrade was acquired by Schwab recently and in the year since the acquisition Schwab has had multiple days of outages. I was with TD for years without a single instance I can recall of being unable to access my account.
This is just absolute negligence and neglect of necessary technical investment from management. The only thing that will get their attention is people withdrawing funds and moving to better run brokerages, which is what I intend to do.
I think a lot of the newer banks/neobanks are pretty good both because they have less customers and therefore less stampeding herd issues and because they are a lot more oriented towards supporting fast scaling. I use Wealthfront and have enjoyed so far
Schwab is overall a great bank and brokerage. Fidelity’s user experience quite frankly is much worse, as is E-Trade’s. Vanguard is even worse than those. IBKR looks nice, but the hassle hasn’t been worth it to me. Schwab has been great for me so far.
My experience was that my ToS desktop app was still logged in and kept an active connection, On my Android phone, the Schwab App was able to work (I only use it as a backup) while TOS (My more daily trading app) required me to wipe the local data to get logging in working again (A fix that was required last time Schwab messed up the log-in system).
Schwab has, in the last year or so, made their login much more complex to work with MFA across all apps. This probably have their log in servers doing a lot more work than they projected.
I’d hope we’re not talking about them simply not having enough CPU to bcrypt_compare() everyone’s passwords as they try to log in…
In-house service-wide rate limits that are too low?
Two-factor auth steps? (Third party providers may be rate limiting?)
Anti-fraud things? GeoIP lookups? Etc
Writing out logs of logins? Some login audit DB?
(Or is it not actually some login-specific service that is down?)
It’s the step where the system first sees the user and must establish identity, of course. But it’s also an attractive place for system designers to frontload complex logic that result in cache-priming or context-priming which the rest of the system depends upon. And then there’s DDOS, device status, account status, anti-fraud, throttles, etc.
https://www.bbc.com/news/live/c5ykkryglp3t?post=asset%3Ad625...
It’s not that suspicious. Not everything is a conspiracy.