Ask HN: Cause of UK e-gates outage?
A Bank-holiday weekend in the UK, the passport-control system (e-gates) goes down and all passports have to be checked manually, so huge queues etc. I'd assumed an unwise Friday deploy, but the press says the issue is "too sensitive to discuss", eg: https://www.bbc.co.uk/news/uk-65731795
Anyone with inside information they'd like to share?
60 comments
[ 5.6 ms ] story [ 134 ms ] threadhttps://www.youtube.com/watch?v=6Y4PEqvk0Jg
I wonder who has snuck through the chaos. UK border control regularly fails when the systems up, god knows what about when they’re down
Nah, just kidding, but the fact that it's country-wide indicates it's probably a server issue.
Until Tuesday. Monday 29th May is a public holiday in the UK.
Fujitsu's contract runs from 4 September 2021 to 2024 https://www.theregister.com/2021/09/09/fujitsu_border_crossi... https://www.theregister.com/2022/01/14/uk_border_upgrade_con...
Re: Horizon disgrace
Fujitsu has also, so far, escaped financial penalties, whereas the government has been forced to set aside £1bn to cover the costs of compensating victims of the scandal. Meanwhile, Fujitsu is continuing to win significant IT projects with the UK government. https://www.computerweekly.com/news/252526102/Fujitsu-to-fin...
UK Gov & Fujitsu smells bad
As it stands, is yet another example of privatising profits while socialising the costs, just in this case, the costs weren't (entirely) financial.
There was a short period under the coalition where GDS was set up and they created a consistent framework for people to work with and would open a portion of the market up to smaller players.
Now it's mostly large bodyshop middlemen like the prime minister's wife's company taking a 70% cut for the work of a cheap offshore developer.
a legacy of EU membership, but I doubt the government has any will to change it
I don't think that this is the guaranteed outcome for all governments and situations.
In Latvia, the COVID-19 contract tracing application "Apturi Covid" wasn't developed with a bunch of buraucracy beforehand, but as a voluntary community effort between both different orgs and people. As a consequence, it was developed reasonably quickly, actually worked and was helpful (even if the user numbers weren't as great as expected, but that's besides the point): https://lv-m-wikipedia-org.translate.goog/wiki/Apturi_Covid?...
I know this because I was a part of the developer team (the website in particular: https://apturicovid.lv/#en before handing it over to govt. once my participation was concluded) and it was actually encouraging to see how well professionals with a common goal can work together, as opposed to some of the less successful processes I've seen. In particular, our e-health platform was once of those serious government projects with lots of bureaucracy, yet still didn't work after many millions in investments: https://www-lsm-lv.translate.goog/raksts/zinas/zinu-analize/...
(I've mentioned this previously, but the point still stands and the contrast is very apparent)
I don't see why a slightly more streamlined, iterative and goal oriented process couldn't work for commercial software projects, even in the public sector. Of course, if the clients don't know what they want and change their requirements whenever new people come into the office (new management), no particular process is going to save you. Nor will you be successful when people quite frankly don't care about shipping software that works and instead of answering your questions just throw a PDF with 200 pages at you, that doesn't provide the actual answer either.
It's not just UK. They've been terrible everywhere. Maybe just the cheapest bid.
It's "too sensitive to discuss" because it would be inconvenient to admit that the people who have privileged access to this system aren't paid enough to give a shit and are very vulnerable to bribery.
But there are outsourcing companies who put junior developers on difficult projects with almost no support and then overwork them. It’s not racism or colonialism to say such people won’t produce high quality systems. It’s the rich exploiting the poor, rather than one country exploiting another.
I have no idea whether any of this is relevant to the e-gates issue.
So, the usual then?
https://www.theregister.com/2017/06/02/british_airways_data_...
Outbound flights can be delayed too if people can’t disembark from planes because the terminal is over capacity in the arrivals side.
It might be doing facial recognition, but it feels too reliable for the level of facial recognition I expect a consultancy could pull off in a government contract.
The base I think works like this:
- you scan the passport
- you look at the camera
- your passport details & photo are sent to the control room with your live feed
- a guy looks at everything those details plus your immigration history and other info
- same guy decides to let you in or send you to a desk
This was particularly obvious in the old Gibraltar border where you could easily be the only person going through and so the agent had to do the routine just to let you pass.
I do wonder how much is automated. For example, is the gate fully automated in calculating a risk score and then referring you to a border agent if above a threshold? Is a person looking at the details in realtime to decide that, or are they just doing facial recognition, or are they only involved when the gates fail.
I wonder if you're failing due to not recognising your face, or (no offence intended) that you have too high a risk profile in some way due to your passport info, and they decide they want to see you in person.
Passports don't know the current time and thus can't tell whether the presented certificate is within its validity range (as in a malicious attacker could feed an expired certificate as well as a fake "current time" value to make it appear valid), so why are those certificates short-lived?
It's not perfect but if you started you trip in another country with such a system and where a more recent certificate was used, your passport will deny access.
[2] https://assets.publishing.service.gov.uk/government/uploads/...
[1] https://www.vision-box.com/
There was only 1 train an hour leaving, but the trains had 900 seats. You can see the problem.
With airports, I used to come in to Heathrow 50 times a year. There was a great system called Iris, you literally walked up and walked through, barely needed to stop. It never failed on me.
They got rid of it, partly because it wouldn't scale (as it was literally just your Iris pattern, not a pattern + passport), so they only offered it to frequent users.
* Back in the 60s they got a big IBM computer to do some stuff. Then later on they needed to do other stuff. The old computer was too expensive and difficult to replace, so they got a new VAX or something to do the new stuff and talk to the old mainframe. Then some PCs got added to do more stuff, and so on. Today the back end consists of many different systems of different ages all talking to each other using different protocols that were designed against different requirements. Newer systems are forever being patched and updated to cope with new requirements, while the code for old requirements lurks waiting to be accidentally reactivated. Each of these systems has its own specialists for care and feeding, but nobody fully understands the whole thing. When something goes down there are not many people who can diagnose the fault and get it back up.
* Government contracts have lots of rules around them to ensure value for money and prevent corruption (see the UK COVID PPE fiasco for what happens when you try to cut these rules out). But the size and complexity make even bidding for a big contract very expensive and complicated, so it tends to be the preserve of a few big companies who chose to specialise in it. Their core competence is winning these contracts, not delivering on them later.
* These rules mean that everything has to be specified in detail up front, so that everybody knows what is supposed to happen. But this makes the whole thing horribly inflexible. As new requirements emerge from the woodwork there is a continuous process of renegotiation.
* The UK civil service is based around the "cult of the gifted amateur". Senior managers are rotated around departments every few years. So the person who kicks off a project is rarely the person who sees it through. Everybody gets to blame someone else for failure.
* When one of these big contractors fails to deliver, the Government has to chose between suing to get their money back (or some of it) in a few years, or getting the at least part of the system they actually need at a higher price. The government doesn't need the money, it needs the system. So the contractor gets to carry on regardless of failure.
* Humans are very bad at managing small risks with large consequences. Many big disaster stories have at their heart someone who decided that the risk was too small to be bothered with.