222 comments

[ 3.0 ms ] story [ 260 ms ] thread
The piece is censured in the EU ...
Websites have been given a practical choice between annoying popups and blocking Europe.

It makes my life better when they block Europe, and I wish more did. I also wish Europe got rid of the GDPR.

Or they could just not track people...

pretty sad that this is unthinkable.

There is also the the third option of course: not tracking users.
A lot of non-technical organizations deem it safer (and cheaper) to just block EU users than to study the law, become domain experts, and implement lawyer-approved mechanisms to become compliant.

The reality is, however, unless you have a major presence in the EU, or are the size of Google or Facebook... you can freely ignore the GDPR without consequence. EU laws do not apply across the ocean.

Once again, that ignores the third option, simply not embedding javascript tracking bloat in your site, or at least serving that version to the EU (since they've clearly already spent the technical resources of identifying it and serving a different response).

You don't need to be a legal expert to just not collect data you don't need.

It's not that simple for non-technical organizations, I'm afraid. Simply embedding Google Analytics (the de facto website traffic analyzer, for better or worse) is enough to violate GDPR, and embedding Google Analytics is very often a simple checkbox in the web host admin portal.

The "technical resources" involved in identifying EU traffic is nearly always just a simple IP block list, where one can often just checkmark a list of countries in their web host's portal.

I think you grossly overestimate the technical prowess required to operate a website these days, and I think many grossly overestimate the actual real-world impact of GDPR outside the EU borders.

Google Analytics is exactly the kind of tracker bloat I'm pointing out that a website serving static articles doesn't need. It can strip that out and serve a static html page, with no bloat, through Cloudflare, and even get their tracking through Cloudflare as well.

The reason why such "technical prowess" is needed is because businesses have made it more complicated than it needs to be. Keep in mind, the context of this conversation is the arrogance of literally claiming claiming you are being "censored" (because that's what the 451 response code indicates), just because you refuse to remove tracking analytics from your website.

I think this is a forest and trees situation. Using things like Cloudflare, alternative niche "privacy conscience" trackers, adding GDPR popups, etc. are all way above an beyond your average website operator's abilities or understanding.

The dude trying to sell custom T-Shirts has a website because it's a necessity to sell T-Shirts, not because he enjoys operating, optimizing and customizing websites. He just wants to sell more T-Shirts, and some European Union law isn't going to give him pause even for a second if he is physically not in a EU country.

While good intentioned, the GDPR was specifically designed to be a "viral" law and force the entire internet to comply... unfortunately the EU's long arm of the law is fairly short. Further, the EU's ability to enforce the GDPR is limited to businesses physically under it's jurisdiction - ie. the US is not going to extradite some website operator for an alleged GDPR violation, no matter how severe the case may be, and the EU can only fine a business into compliance if the business cares about EU laws (which would imply a physical presence in the EU).

Ignoring the GDPR is the right move for most websites.

If people desire comprehensive, impactful privacy guarantees, then a similar law needs to be passed within the US.

If there's not a tech expert in the room when a content publisher meets with their legal counsel, that is not a possibility anyone even realizes exists.

It is also more expensive to implement than

    if(blocklist.contains(ip)) res.sendStatus(451)
The effective result of GDPR has been less than ideal, but was a step in the right direction. Abolishing it entirely in favor of a vacuum would carry a more weighty cost.
The frustrating part is that the unintended consequences of GDPR were inevitable and clearly foreseeable, yet it was passed into law anyway and now it's incredibly difficult (if not practically impossible) to fix through iterative adjustment. It often seems like politicians and bureaucrats are incapable of seeing the myriad ways their well-intended rules will fail in the real-world. Maybe all legislation should first have to pass an "Unintended Consequences Analysis" review.

Meanwhile, GDPR is yet another disruptive annoyance to users, a burden on overworked, under-funded website teams and isn't delivering the benefits promised.

It doesn't have to be disruptive to users, companies go out of their way to make it disruptive. As others here have said, companies can just stop adding code to track users and stop storing so much information. It's actually way less work.
But why blame the companies who are directly responsible for this annoyance when we can blame the government who is indirectly responsible?
The GDPR covers tracking of users, but is much more broad in scope. Simply not tracking users doesn't stop the GDPR from applying.

The GDPR provides a framework by which bad actors can do their thing legally without too much trouble. To a good actor, who only stores the data they actually need and doesn't track users, the GDPR still applies and is just as big a pain in the ass.

Legitimate interest is excluded from the GDPR, so what kind of "good actors" are you describing that are hobbled by the GDPR? Be specific, please.
Anyone who doesn't want to deal with the hassle of hiring someone in the Union as their Article 27 representative.
>Meanwhile, GDPR is yet another disruptive annoyance to users, a burden on overworked, under-funded website teams and isn't delivering the benefits promised.

Users can now demand we delete their account and data and we have to do it, this is some work for us but it is good for users.

Also do you forget that popups were a thing before GDPR? Modal for you to subscribe to some shit,monthly Terms of Service changes, notifications to see what new feature was added, to see what new stuff some contact posted were a thing, with GDPR you get a one time popup , you make your choice and you should be done.

In summary

Pros for users:

- transparency, you now know about the 100+ partners and you can decide

- you can ask for your data and ask for it to be deleted

- you will get notified if your private data is lost(mioght be adifferent law for security breaches)

Cons:

- Shit websites use dark patterns and annoy you with a popup one time

No Pros for developers, companies: Cons for devs and companies:

- you need to research the law and find ways to screw the user with dark patterns

- you need to update some old project and remove tracking or implement some popup , implement account deletion, data demands etc

- you need to update your ToS documents, privacy policies

- almost never happens, you remove tracking or configure your ad script to don't track and you don't store private data.

From what I see users EU ones) have a lot to win, I assume if you are not in EU seeing the popup is a bug and would be fix .

> a burden on overworked, under-funded website teams

It's only a burden on these people if they choose to be user-hostile.

There are really easy ways to deal with GDPR but these dumb companies insist on avoiding those solutions at all cost.

If companies insist on being dicks, then the only possible government-level solution might be to just ban this kind of tracking outright.

Wouldn't you rather websites just stop invading your privacy? The EU is not to blame for the shitty business practices of these websites.
GDPR has compliance costs even if you are doing nothing that violates privacy. As long as you've got data that can be associated with a user you might have GDPR obligations, including handling inquiries from users about what data you have about them, requests from users to delete their data, and similar.

Just being able to receive such requests has costs, because GDPR requires that you have a contact in the Union that users can contact for such things.

So say you've got an online forum in California and some EU people join and participate. Whether or not that is enough to make you subject to GDPR depends on Article 3(2). There's lots of subjectiveness in Article 3(2) so it is not at all clear where the boundaries are.

If it does, then whether or not you have to have an in-Union representative is covered under Article 27. You do not need one if your processing of covered data is all of the following: (1) occasional, (2) does not fall under some special categories of data, (3) is unlikely to result in a "risk to the rights and freedoms of natural persons".

That's pretty fuzzy. What is occasional processing?

If you aren't sure that GDPR does not apply, or aren't sure that if it does your processing is occasional enough to fly under Article 27's radar, you need an Article 27 representative.

Eventually there will be rulings from EU data regulators that will make the boundaries of Article 3(2) clearer, so that you may be able to stay in "does not apply" without blocking. Or maybe rulings will clarify Article 27 so you will be able to confidently determine that your processing is occasional enough to not need a rep.

There's actually a lot of unclarity in GDPR. Take the data subject's right to have you delete their data. How do you actually implement that? Writing some scripts to delete from your database is probably not too hard.

But what about backups? Do you need to go through all your backup sets and delete their data from those? What about printed records (yes, GDPR covers printed data)? Offsite long term archives?

For a site like a forum, what about data that was in messages they posted that were quoted in messages from other users?

Until all these kind of things are cleared up by EU data regulator rulings it can make a lot of sense for a site that is aimed mainly at a non-EU audience to block EU users.

Correction: they claim that it is. The 451 code is specifically made for that, referencing Fahrenheit 451, so using this specific code implies censorship.

However, it would be totally legal for them to provide the content. They just don't want to take responsibility for their privacy invading practices. So it's not censorship at all.

/me shrugs. Personally, I have yet to find the website I can't just close if it behaves like this.

1) Let my filter out all calls not originating on an US/Canadian landline/mobile carrier

2) Yes, I’m happy to lose all calls that come via VoIP.

3) I don’t answer my phone anymore, so I miss those calls anyway

4) I’m very happy to sacrifice 1% of signal for 100% of noise

Relay the full trace of all IPs and switches involved in sending a call my way and give me a way to filter based on that trace. I want to block an entire source if they don't do a good job of screening their clients.
Similarly, if my phone would just allow me to do some simple filtering based on number or location, it would filter out 99% of all spam calls. I have a Florida number but have lived outside of Florida for about 2 decades. No one who I have any relationship with has a Florida number except for my family who are all in my contacts. Simply filter out my own area code but not in my contacts would eliminate the vast majority of spam calls. Yet, there is no built-in capability to do this on my iPhone as far as I can tell.
^ The fact I haven't lived in my numbers area code for a few decades as well I've noticed the exact same thing. If I don't have a contact for a number from that areacode it is 99.99999% chance of being a scam.
>Yes, I’m happy to lose all calls that come via VoIP

At that point, why not just block everything other than contacts?

Because lots of legitimate callers to me are located in the US, calling from a cell phone or landline, and I don't have them in my contacts.

For instance, contractors coming to do work on the house may be calling from any number of cell phones, or people buying things from Craigslist. For Craigslist, if I were to filter out all the non-US originating calls and texts, would probably result in all the spam being dropped and at no expense to legitimate buyers.

Those are all VoIP callers. Few businesses have enough copper phone wires for everyone in the building.
They’re not VoIP callers. I’m primarily dealing with small to medium sized businesses, which use landlines, or cell plans like Verizon for Small Business, which are basically normal lines on normal cellphones.
A ton of small and medium businesses use voip. Even residential landline phone service can be voip these days.
To reiterate the problem: Legitimate callers don't all use VoIP. I don't have every legitimate caller's phone number in my contact list.

Blocking all numbers not in one's contacts list is therefore not a solution to blocking spam calls while allowing legitimate callers.

Yes, but many legitimate callers are voip so blocking all voip calls also isn't a good solution if you're looking to avoid blocking legitimate callers.
To me it is a good solution. Again, I’m happy to sacrifice some signal, if that loses essentially all noise.
Yes, agreed, blocking VoIP is not a solution, blocking everybody not in your contacts is not a solution - the solution is enforced and widespread STIR/SHAKEN
Seems like maybe a good business would be listing peoples known numbers and making a caller ID app that it outside the carriers’ control that relies on … I dunno a web of trust and crowdsourced data and reputation?
The problem still is with number spoofing, and STIR/SHAKEN seems to not be sufficiently rolled out and enforced

https://en.m.wikipedia.org/wiki/STIR/SHAKEN

Many of the spam calls I receive come from my own area code, with the number spoofed.

> 1) Let my filter out all calls not originating on an US/Canadian landline/mobile carrier

This, plus state/province level filtering as well would completely solve the issue for me.

Do we know if reporting a call as spam in Android gets reported to the FTC? Or do we need to file it as spam and report it ourselves to the FTC?
I'm not sure I'd want it to. On multiple occasions, I've received "returned" calls from people who have received scam calls from my number. I'm sure a lot of money would be wasted on red herrings if those reported numbers end up at the FTC.
Those should be reported too. I want the FTC to take action on falsified caller id! The more of these reports the FTC gets, the more likely to they are to take action. (action probably means talking to congress)
This spring I was applying for jobs so I answered my phone for more unknown numbers than usual. This has lead to a huge increase in spam calls that I receive on a daily basis and Google Phone's spam filter isn't strict enough for my liking (not to mention no longer available since I'm running GrapheneOS).

If you're an Android and Tasker user you can take matters into your own hands and set Tasker as your call screen app. There are two different blocking types, one shows that you missed a call as a notification and one does not. I have set it so that all non-contact calls are rejected but I see a missed call notification. I also have a contact with a growing list of numbers that is rejected and I do not see a notification. Additionally there is a regex filter that rejects several area codes without a notification.

When I start seeing more spam calls from a random area code in Michigan (or wherever), I just add that area code to my regex filter and then just have to worry about clearing my voicemail at the end of the day.

If I'm expecting a call from a doctor or someone who is not a contact in my phone, I just turn off the profile and deal with some spam for a day.

Of course, I would love some strict regulations around this, but I do no expect the legislation to go far enough or be effective (I'm in the US).

Are we at a place where as potential new hire candidates, we request a formal meeting to be scheduled or a quick text to let us know that the next call is legit? Or is it acceptable to tell hiring managers via checkbox on applications to say that all calls will be screened via voice mail, but will be checked with anticipation of it being a call to be returned promptly?
Being high maintenance sounds like a poor job search tactic.
yes, but...

it's not an unusual situation unique to that applicant. it's so rampant, that even the HR/hiring people will be subject to it and totally understand the request. if you want to stand out to me as a potential employer, being proactive and stating we don't have answer unknown calls as alternate arrangements via _____ method to set up a call.

I've tried screening with Google's screening service and recruiters don't use it. So I'm not sure a checkbox will do any better than "Hi the person you are calling is using a Google screening service. Please state why you are calling." They always go to voice mail and then complain about me not picking up.
I dunno how successful this is, but when I get spam called, I do what I can to reach a person and then say, "Put me on the do not call list, please!" and (possibly) as a result, I really don't get as many of these spam calls as it sounds like folks here are getting.
I found that when I started doing this I would just get even more calls, just not the same ones.

I presume this is because these spammers just keep selling other spammers valid phone numbers to add to their own lists.

Agreed, what I have done is just not answer my phone unless they are in my contact list. I also do not share my cellphone number with anyone. Seems like everyone wants it these days and the pandemic made it worse, so I have a Google Voice number for people who insist on having a number.

The only phone number that gets spam calls anymore is that Google Voice number.

For all other numbers, the calls have virtually stopped.

My point is I am not getting "even more" calls, at all.
It does not work. I tried to take the approach of wasting their time by asking for their name and what not, but they are also getting trained to cut such calls off quite fast.
This doesn't work with the car warranty spam. Heck, now that I actually own a car I get letters about it too that happen to know what car we have.
This WILL NOT work. The best thing you can do is sign up for the FCC DNC list. It'll take a month but then after that every legal spam call will stop.

The problem with your method is that spammers call phones just to see if they pick up. This then confirms an active number that they maintain in their lists. This can be done legally because technically calling and hanging up and never calling again (from the same number) isn't spam. So just don't answer.

My point was I have not experienced an uptick in spam calls, and I thought y’all might find that interesting.
> It'll take a month but then after that every legal spam call will stop.

Not the ones from politicians.

That's true. They have an exception for some reason.
I totally expect this will be fairly effective, because there aren't any major forces pushing against.

- It's a simple technical problem to identify potential spammers. There are clear patterns & habits of robocallers to id them.

- It's fairly simple to confirm spam/scams by pretending to be a target.

- Scam spam calls, though profitable, are relatively isolated and any public support will make them easier to trace. So we aren't facing huge lobby pressure to oppose the nearly universal support for spam reduction.

So it's really just a matter of allocating the resources to do something, which is exactly what this is.

I think you're overestimating the ease of detecting spam. A lot of major companies have this issue. You may argue that it's a lack of will, but they have things in place and if it was easy they'd be effective.
90% of those scam calls originate from one city in India- Kolkata. It’s a state that has had communist rule for over 30 years and now a government which also runs on anti-corporate agenda. As a result there are no industries and all jobs have dried up in that state of West Bengal. This is the only industry left and the local government and police turn a blind eye since they get kickbacks and is the only source of income in the state.
This is the sadder but true point that we have too many people and many are stuck doing nonsense and scam work to try to feed their family.
I find your misanthropy surprising since the parent provided a clear explanation of the problem.

Nonetheless, it is a shame that they engage in such behavior and that they live under abusive government structures that funnel them into such things.

I understand the parent point that this is driven by corruption. The point I was trying to convey is that corruption isn't a sufficient reason for there to be literally zero other work that could feed a family. I don't blame the people, they are just trying to get by. But I think the majority of people worldwide find that a major portion of their work life is spent on nonsense work. I think this will only get worse as software and automation continues to eat the world. Of course, this is just my personal opinion.
It's interesting that failed communist regimes often seem to be succeeded by kleptocracies. I wonder if any political scientists have studied this and can explain why.
You need an economist to explain it, not a political "scientist".

Communism does not produce globally competitive industries. So the only thing left to do is either to steal and pillage what is left or work in an industry that has local demand like food or energy. Those tend not to be very attractive jobs.

I'll cite Conquest's 3rd law: The behavior of any bureaucratic organization can best be understood by assuming that it is controlled by a secret cabal of its enemies.

That's not exactly explanatory, though. Any time a position is powerful or enriching well beyond the extent to which it serves others, the people optimizing to seek out and attain those positions are those who want the power and wealth, rather than those who care about the organization's mission. For those people, the original mission is, at best, a distraction, and at worst actively antithetical to their personal goals.

Communism and socialism concentrate power to a single government/state entity and when it fails there is no industrial competition so the most well connected/wealthy people that are able to take control of the industry are able to pretty much run unabated without any natural market forces you'd find in a system/industry that began in a more free enviroment.
(comment deleted)
There is never a shortage of desperate people living under corrupt regimes. That's the wrong problem to focus on
Nothing is true here. The government of West Bengal is BJP. There's tons of industries and jobs there. This is easily Googled.
The communist party was ousted from control of government in West Bengal in 2011[1]. Manufacturing growth is actually down since then[2]. And relevant to the current topic, Indian scam calls have only been an issue in the past decade. A decade of non-communist rule seems to either not improved things much or made things worse. The issue is more complicated than "communism bad".

[1]https://www.aljazeera.com/features/2011/5/13/defeat-rocks-in...

[2]https://www.ideasforindia.in/topics/macroeconomics/west-beng...

(comment deleted)
It's absolutely insane that they are so ineffective. Major phone networks should be held responsible. I've been getting texts like crazy from spammer tries to get me to click their links. The messages are 100% identical and would be trivial to block.
The fact they are identical making it trivial today only.
From other areas where spam is prevalent, I think it's an arms race issue.

Sure, right now, that spam that's identical every time would be easy to detect and block. But if it's blocked, they start adding variation. Block that too and they get more and more creative. Every time a way to block the spam is found, the spammers just get more creative with getting around it. It's an arms race, and is essentially un-beatable because the spammers outnumber those trying to stop them by a magnitude or two.

I just have a hard time believing that's really what's gonna happen. I'm a 34 year old man living in a western country and I have not gotten one robot spam call nor spam text message in my life. I've gotten 5 fake Microsoft support calls, but always from a real human pretending to work for Microsoft.
Yes, but we've gotten quite good at blocking email spam. So much so that it's basically a solved problem for regular users using GMail or Outlook. There's no reason we can't achieve the same with text and phone call blocking. My iPhone is pretty accurate when it classifies incoming calls as "Spam Likely".
you're right. we should just give up, and allow ourselves to be overwhelmed now, instead of wasting all that time pointlessly resisting the inevitable. none of this will ever get better
That's not what I meant. I meant there's no silver bullet that stops all spam forever. There's no easy solutions to the problem. It's an uphill battle and the best you can hope for is to block the majority of it with a lot of continuous effort.
What if they accidentally block an important text and get sued? It's not easy to do at scale.
Enforcing the use of STIR/SHAKEN across networks and providing an easy way (aka in the messaging and dialer app) to report spam messages and calls would likely fix 99% of the problem.

I'm sure Google and Apple are nipping at the bit to add "Report as spam" in their apps, carriers just need to provide a way to ingest that information and have it be actionable by disallowing spoofing.

Being held responsible means the government hands them a huge check and nothing changes.
It is unfortunate that the cellular companies have allowed their core product (voice calls) to be so devalued that they need to be forced to solve this problem instead of just doing it naturally.

The carriers are clearly customer-hostile, as this is really a very solvable problem. Instead, we now are moving to alternate mechanisms to manage audio calls (eg: Signal, Whatsapp, Facebook, etc.). My wide-open Verizon cell-phone number is my least preferred method of contact by far, due to the fact that I cannot trust that unknown numbers are high-value incoming calls.

to the point, I used to refer to the device as a mobile phone or cell phone. now, because I so rarely use the "phone" app, it is just now my "device". 99% of the time my phone app gives me notifications is SPAM LIKELY reaching out to me, but won't take the hint that we broke up years ago and I have zero interest in touching base and catching up. however, if anyone of my known contacts actually uses the phone app to reach me, i know it must be significant to not use a text/chat app like 99.9% of our typical interactions take place.

times have changed for phone calls, but the lack of Anti-Spam from these providers only hastened the change away from their core service

Imagine if nobody knew what a phone was, and then the device manufacturers suddenly introduced the phone app: it would outrage and annoy people!

Here is an app that allows any rando person in the world to cause your device to wake up, interrupt whatever you are doing, display a notification, play a sound, vibrate, and if you push the green button, that rando person (identifiable by a spoofable string of numbers) can hear and talk to you! It would not be well received in 2022 where privacy and digital well-being is increasingly important to people. Yet for historical reasons, it’s pre-installed on every phone.

How is that any different than a notification from any messaging, social media, or any app really with notifications turned on? They all know that the human brain is very susceptible to these notifications.

At least with the old school phone, you could leave the receiver off hook to send the callers a busy signal or just unplugged.

I feel like the legacy POTS system does more harm to society than good. Step back and imagine the web without SSL. Think of the sheer scale of fraud and abuse that would run rampant. That's the service phone carriers are operating right now.
>My wide-open Verizon cell-phone number is my least preferred method of contact by far, due to the fact that I cannot trust that unknown numbers are high-value incoming calls.

I thought it was a legal requirement for phone network operators to leave phone numbers accessible to all, and hence their inability to effectively fight spam.

Otherwise, what would stop the situation from becoming like email, where email is more likely to be delivered if sent from a big player and less likely from a small player.

What does “accessible to all” mean in this context?

Why do I in Europe get almost zero spam calls or text while in the US it seems like a plague?

In Canada and while I do get some it’s fairly rare. Maybe 1 a month.
I get 4-5 a week on Bell, my SO gets 1-2 a day on Rogers.
Have you signed up for the national do not call list?
Yes, of course I have. But the “CRA”, Duct Cleaners, and the women who speaks only Chinese all ignore that list.
I really wonder what the difference is between numbers, I’m on telus and I don’t think they make any attempt to block them? And I’ve had my number for 20 years, with both bell and Rogers in the past. and made no effort to keep it secret. Thou I occasionally answer the calls when I’m bored and waste them time.
Anecdotes do not constitute data. My family in Canada gets tons of robocalls and spam calls, to the point that nobody answers unrecognized numbers unless they're expecting a call...
Things seem to be improving: I’m getting fewer and fewer “neighbourhood” calls.

I suspect the vector is moving away from VoIP systems and more toward buying masses of prepaid plans and stuffing them into various contraptions that hold hundreds of sims to blast out messages/calls.

https://m.aliexpress.com/item/1005004095035108.html

Have they signed up for the national do not call list or renewed it? as it expires iirc. My friends in the same boat often never signed up like I did.
UK here, we get tons of scam robocalls to the point where I leave my phone on airplane mode with Wi-Fi enabled like 95% of the time.
You mean you haven’t been in an accident that wasn’t your fault?
Accessible to all meaning the rules for being part of the public switched telephone network:

https://en.wikipedia.org/wiki/Public_switched_telephone_netw...

Again, I am not sure and very well may be wrong, but I think I recall reading or maybe just assuming that the FCC or maybe a global agreement bars individual network operators from not accepting calls from others within the network.

I imagine this gets very hairy due to international jurisdictions and things like caller ID spoofing.

See section on Legal Considerations:

https://en.wikipedia.org/wiki/Caller_ID_spoofing

An interesting conversation that came up in my research about this topic for why a telco might be barred from blocking against certain numbers:

https://www.techdirt.com/2007/03/16/can-a-telco-block-phone-...

I have no idea why European numbers might not get spam calls with the same frequency as American. Maybe the spammers are looking for English speaking people, or Americans are richer or easier to scam?

Depends on the country. I think it comes down to availability of cheap labour for call-based vectors.

French numbers can get tons of spam calls, because there’s no shortage of impoverished French speakers in the world.

EU may be stricter about enforcing, but I've also experienced it's very person-dependent.

People who need to answer unknown callers for business reasons (small business owners, sales, etc.) get put on lists of "known active numbers" and may get multiple spam calls a day. Folks like me who can just ignore any unknown number & respond only if they leave a real voicemail, get FAR fewer (maybe 1 a month on avg). We tend to complain less about the issue (even if it is annoying & I still treat all unknown calls as spam by default).

Also, if you own a google device, they're very good at IDing spam callers. I recently switched from Google to Samsung and get far more.

Because you have a functioning government.
When you target Americans, you can use a single language to hit 239 million people with their native/first language. You can't do that in Europe.
In the US, one of the loopholes in rules requiring STIR/SHAKEN and other protections was that smaller carriers, usually serving rural or under served markets, were exempt. This became a profit center for them as they became one of the only means for these spam calls, which originated in overseas call centers, to connect to US callers.

Thus the smaller carriers were incentivized by profit to connect the calls.

Now that STIR/SHAKEN is supposedly a thing, is there a way in Android that I can block calls that go through those carriers?
Unfortunately it's carrier to carrier and not data we're privy to. Carriers get paid arbitrage fees for carrying or connecting calls, so they really have no incentive to stop these calls other than legal liability.
Then they really dropped the ball on SHAKEN/STIR. At a minimum they should pass the customer all the hops so we can block shit we don't want (international calls, VOIP calls, known spammers, known bad companies etc.) on our end.
Telecommunications is interesting in that everything strives to be backwards compatible and will accommodate the lowest common denominator to connect a call. It's very much like Microsoft Windows, in that backward compatibility is both a strength and a common source of problems. Government mandates don't usually go beyond borders, so carriers have to accommodate international operators.

And any new features or technologies often introduce or expose problems.

When Caller ID was introduced, there was a huge backlash around privacy. Callers were upset about having the number they were calling from being exposed. If a doctor called a patient it could expose his direct or home number instead of the office number. In the age of phone books, a phone number could be associated with an address. The ACLU sued to block Caller ID until it was possible to disabled it.

Because large call centers operated hundreds of numbers and wanted a way for them to all appear as the same business entity. Carriers later implemented ways for these customers to spoof Caller ID so that the carrier wasn't responsible for maintaining it. Carriers connecting calls had to trust the originating carrier's caller ID info as well.

All these requirements allowed for a bad actor to spoof Caller ID and for it to just be trusted.

Carriers use to have star codes you would use for certain features like disabling Caller ID, disabling Call Waiting, and, famously, calling back a number that called you. These are similar to the AT codes you can issue to the modem in your cellphone via the dialer. Kids today will never appreciate REM's Star 69.

When digital cellphones arrived, they presented some odd challenges too like the fact that call information was transmitted digitally to the carrier and not via analog tones. Because number presses on your cellphone didn't equate to analog tones over the line, cellphone users couldn't interact with IVR menu systems (e.g., "Press 1 for English...").

Character limits in SMS messages come from the fact that they occupied the null space in beacon messages transmitted between cell towers and phones. Before MMS was a thing you couldn't send longer messages or if your phone broke them up into chunks, they might be delivered out of order or not at all.

US has a larger market with a single language, and our regulators demanded calls to mobile phones to be on an equal footing for callers. You never needed to pay more to call a cell phone than a land line from the same rate center; the cell phone user has to pay any additional cost to carry the call over the air. Additionally, inter-carrier rates are regulated and set to approximate costs, which is now pretty much zero. This makes bulk calling and texting inexepensive (less than a penny per message or minute). In (most of?) Europe, callers have to pay for the privilege or calling a cell phone, and the numbers are in different ranges, and it's not cheap, 10 cents or more per minute or message at bulk rates; even though the underlying cost is probably similar to the US, near zero.
Over the last few years the FCC rules on carriers were relaxed so that they can discriminate against calls originating from domestic networks. Previously they could not block calls even if it was obvious it was a scam call. However, it's not always obvious which robo calls are legitimate and which aren't.

The bigger issue though is not about domestic origin calls, it's about foreign originated calls that the FCC has no power over. The domestic carriers had to forward these calls. There is traction gaining to require that calls that originate overseas verify their identity to US based networks. Domestic carriers can reject calls that don't verify (spoofed numbers that originate offshore).

That still may not stop all the spam calls but it will slow it down. The problem here is really that telephony became extremely cheap and accessible to the whole world.

I never, ever get legitimate calls from overseas. Just turn those calls off for my phone.

Well, perhaps callbacks from helpdesks located in other countries, in which case give me time-boxed control over foreign calls, and perhaps a passcode that I can communicate along with my callback number.

There are billions of dollars at stake, of course. This is a solvable problem, and the FCC doesn't even need to get involved. The carriers care more about the money than their customers (so what else is new?)

Can't all of this be done locally on the phone? Why does the phone app not have an option to simply ignore all garbage calls
The phone company doesn't give you the originator information. This is what they need to be forced to do. Then users could block calls and texts from international numbers to politicians to voips to anything from the state of Wisconsin, adblock-style.
I want the ability to block all overseas calls, and I want that ability in a way that is easy enough to opt into that most people choose to. This would resolve the overseas helpdesk problem by forcing domestic companies to use domestic call centers to succeed in reaching people.

I have never received a legitimate phonecall from overseas, and I don't expect I ever will. In every case that I've had a legitimate reason to talk to somebody overseas, we used voip-to-voip, never telephones.

The problem is someone from overseas can claim to be from a US number. The number you see on your phone can be anything the caller wants it to be. A fair number of robocalls are from overseas but looks to you like they come from your country.
That's precisely what the carriers need to solve.

They're not interested.

> However, it's not always obvious which robo calls are legitimate and which aren't.

I can make this real simple. I don't want any robocalls.

I feel you. Still, there are legitimate uses for robocalls. My doctor’s office uses robocalls for lab test results, so that patients can get their lab results as soon as the lab tech puts it in the computer instead of having to wait for a person to make the call. Sure, you can email/text these results for most people, but there are still some older folks who don’t know how to email/text.
That's cool - we can wait two extra days for a call from our doctor if it's warranted. I don't need instantaneous feedback on my bloodwork.
Well, grandma does need her bloodwork results. And she's the one most likely to fall for a robocall scam anyway.

And do you think illicit callers are going to follow the law? They're already committing wire fraud, and the software already exists.

Yea - I think illicit callers are going to follow laws whenever possible. "Criminals always be crime-ing" is an argument that can be used to counter having any laws but it has a fatal flaw - criminals generally don't want unnecessary attention. The more agencies they piss off by violating regulations the more likely it is that their activities will be suspended.

A lot of the crime you see on TV comes from highly irrational and unpredictable people, but scamming is usually a pretty rational decision if you live somewhere where repercussions are unlikely and the expected income is high.

I'm not arguing that we should roll back any legislation against wire fraud or spam calls. I am saying that the evidence demonstrates that those laws were not effective in deterring the overseas criminals who continue to constitute the bulk of these calls. They are simply not in the jurisdiction in which we are passing these laws. If US wire fraud charges aren't enough attention, I'm not sure what marginal increase in attention one could possibly expect from an additional law.
I get an e-mail and they show up on a web UI and it is instantaneous more or less.

And I don't know why I have to get robocalls because gramma can't operate the internet.

And both my 80+ year old parents can navigate the internet well enough to get lab test results off of websites.

We should still take into account the usability of public utilities for those who struggle with internet illiteracy or disabilities, regardless of how well your parents can cope. A full 25% of people 65+ don't use the internet in any capacity at all.
Again, why does this mean that I have to suffer from robocalls, other than cynical justification to defend robocalls in the abstract and continue to inflict them on everyone?
I suppose if you have enough money, you might be able to convince AT&T or your local lawmaker to cater specifically to your desires.

I just doubt that the result of your stated preference is a solution that would be workable enough for the general public. The rest of the world just wants the spam calls to go away but still get calls from their bank/pharmacy/doctor/school/etc. Asking them to choose between the two is not a solution for them, even if it is for you. No law maker is going to propose that. And your carrier already DGAF.

If this was an option available to every consumer I suspect uptake would be closer to 50% or more. It would look a lot more like Apple's privacy features than some bespoke setting that I'm the only person in the world who cares about it. And this is probably what you're actually afraid of for some reason.
I don't doubt that, I'm sure 50% of people will check the option thinking it'll just block spam. The problem is that blocking robocalls entirely would block legitimate calls that they're not aware of until it'll cause pain and frustration down the line.

Most people do want fraud notifications from their bank, notifications from their utilities about maintenance, notifications that their kid's school is closed today, a call back from customer service when they ask for one, etc. But nobody would think or even have the ability to know about this when checking your "block all robocalls" option, and now you have created a new problem.

If you give people a footgun, they will use it, and they will blame whoever came up with the idea. This is why nobody will do what you are asking.

What's the number of people 65+ who don't use a phone in any capacity at all? It's not zero.

There will always be people left to the wayside, and we need to act when that number is manageable. 25% might be way too much to legislate against robocalls in all forms, but we need to weigh the loss against the gains; how many seniors would benefit from less scams?

Of course, there are always exceptions. 25% of a major demographic is not one, though. I know that young people and those working in tech are totally over voice calls, but outside of our bubble, people still use telephones as telephones.
I definitely appreciated getting called quickly the last time i got a covid test
Then you can leave robocalls on.
Imposing that option on people negates its usefulness. Which option would you recommend to your elderly relatives? Accept robocalls from international scammers or receive updates from their doctors office?

You might as well ban robocalls for legitimate uses too, since legitimate users won't be able to rely on it anymore.

I agree it's not a 100% solution for all cases ever. But it's still a strict improvement for many people.
My doctor (well, their hospital system) uses Epic/MyChart as their EMR. I also have the MyChart app on my phone. I get a push notification via the MyChart app when anything new posts to my chart - no phone call, text, or email required. And I don't want the push notifications, I could opt out of that too.

Surely most doctors offices use a vanishingly small amount of EMR providers, and they could support similar very easily.

The people who use the hospital system the most are the ones who are the least likely to be able to use those functions effectively.
Never underestimate how out of touch and incapable of understanding other people HN nerds are.
There is nothing preventing them from providing users with more granular call-rejection controls. Their networks need to be "accessible" on the incoming call side, but that does not mean as a customer I should be forced to have every call intended for my number to be routed through to my phone.

An iptables concept with a user friendly UI would address much of this problem, particularly if we could filter on useful call origination data, or lack thereof.

The major carriers have this but charge for it. Like $5/mo. It's pretty sad.
I thought it was a legal requirement for phone network operators to leave phone numbers accessible to all

Is it a legal requirement to allow spoofed caller ID from bad actors?

Spoofed IP addresses used to be a much bigger problem on the Internet than they are today (many examples but see Smurf Attack).

We collectively implemented security to reduce the threat. Phone companies could do the same for spoofed caller ID for companies that consistently abuse it.

> Spoofed IP addresses used to be a much bigger problem on the Internet than they are today (many examples but see Smurf Attack).

Most of the large DDoS attacks are UDP reflection, the attacker spoofs the victims address and sends to chargen[1] servers or whatever. So spoofing is still alive and well.

[1] yes really, the worst ones are people who install Microsoft's Services for Unix and enable the chargen server to the public... It sends a 64k fragmented response. Thanks Microsoft.

How long have you been on the Internet? I ask because I started an ISP in 1996 and IP address spoofing used to be soooo much worse.

Today’s situation is the product of literally 2.5 decades of encouraging and/or shaming Internet-connected networks into properly configuring their routers to drop spoofed packets egressing from their network with a spoofed IP address not from their network.

Approximately forever, although while I worked at an "ISP" in 1999, it was a two T1 ISP; one for modems, one for upstream; and then we outsourced all the modem stuff to MegaPath when something weird happened to our modem T1, but I was never trained on how that T1 worked, all just magic to me. I kind of sort of get the idea today, but ... I poked at the RADIUS server, and setup webhosting for customers. I never worked for a real DDoS target until 10 years ago (and even then it was mostly people kicking the tires with 90 second DDoS services). I'm sure it's not as bad as it once was, but it's still probably easier to get a big DDoS with UDP reflection than through botnets directly hitting the target.

Back to the topic of telco spam. STIR/Shaken should maybe get us actual caller-ids soonish, I think there's one more deadline for small carriers that service actual phone lines. OTOH, what are you going to do with a real caller id? I'm pretty sure phone companies just don't want to do it anymore, there's no money in voice calling, and it'll fall apart like landlines have.

but it's still probably easier to get a big DDoS with UDP reflection than through botnets directly hitting the target.

We were more sophisticated back then. So picture your netblock is 207.180.10.0/24 and you've directly connected all 200 computers on your Class-C netblock directly to the public Internet without any NAT. 207.180.10.0 is your network ID and 207.180.10.255 is your broadcast address.

Someone comes along and sends a 64 byte ICMP ECHO request to 207.180.10.255. And they get 200 responses. Literally every one of the computers on your entire network responds with an ICMP ECHO reply.

So now they can spoof an IP address from Victim IP address to send 1k ICMP ECHO requests to 207.180.10.255 and multiply their attack 200:1.

But now imagine instead of a tiny Class-C netblock they find a similarly configured broadcast address for a huge Class-A netblock. And their scanner counts 29,000 ICMP ECHO replies for each request they send out. Now their attack is multiplied 29,000:1.

And since they've been scanning for broadcast addresses on the public Internet every day for a month they have compiled a list of hundreds of them so the attack seems to be coming from everywhere.

https://www.fortinet.com/resources/cyberglossary/smurf-attac...

> It is unfortunate that the cellular companies have allowed their core product (voice calls) to be so devalued that they need to be forced to solve this problem instead of just doing it naturally.

And they did this while loudly complaining that they don't want to become a mere dumb pipe! That is the true twist.

It kind of goes both ways, politicians "allowing" themselves to be lobbied by big telecom.
How did they allow this? I think text is just an alternative that most people prefer for large number of use cases.
It's also unfortunate that robocalling isn't bannable across the board because politicians want to continue to have the ability to spam call constituents. It's really unfortunate that all sorts of automatic calls need to continue being permitted just so we can have the delightful luxury of getting spam fundraising calls from politicians.

This has real world consequences, last year a lost hiker ended up declining calls from search and rescue[1] since they assumed they were spam.

There is a really easy and extremely effective solution to this problem - charge calls on both ends. Make the caller pay a few cents for every call placed instead of placing the full charge on the receiver.

1. https://www.npr.org/2021/10/26/1049252333/lost-hiker-mount-e...

Yes, there are lots of solutions if there is a genuine attempt to solve the problem and not just dance around it.

Political calls and texts should absolutely be blockable by users if they so choose.

Cost-based calling may not be practical, IMO. I look at the amount of junk mail I get from the USPS and I am not sure a "sender pays" approach would result in a meaningful reduction with a cost structure that didn't also impact regular users. A rate-limiting approach on call origination could work, but in the end I still think that a simple that gives the users control to block unwanted calls/texts, coupled with an origination source validation would solve most of the problem. There is nothing preventing us from capturing and passing accurate call origination data.

Just an FYI, FTC has a signup for junk mail just like they do for phone calls. I just do the prescreen one and the only junk I get now is for previous tenants. Unfortunately I don't know how to solve that one.

https://consumer.ftc.gov/articles/how-stop-junk-mail

Ugh. That's an awful solution. In order to not be spammed and have your privacy invaded, you have to submit information about yourself to the bad actors in the first place. Plus you have to pay them for the privilege of not harassing you, and on top of it, they only stop for 10 years and then you have to do it all over again. It's fucking ridiculous. This is not a real solution.
It is awful, but they already are buying that information tbh. So I'm not sure there's an information gain through this. I don't do the paid service (which is $2... come on...) and my spam (phone and mail) is significantly reduced. It's not a perfect solution, but it is a real solution.
I've thought this would be a good solution for years. I would like if you could choose the amount to charge people to call you that aren't on your white list. You get 90% of that and the phone company gets 10%. A win-win-win for you the, the phone company, and people trying to talk to you. I would probably set to a dollar or two. Warren Buffet or Elon Musk could set it to $10 million put you would probably need and escrow account for that amount of money.
(comment deleted)
Aren't the robocallers paying for VoIP services now?
Since most robo calls are also illegal scams even if the call was legal all we really need is verified numbers so that we can use phone records to catch the person making the call.

Politicians want you to know who is calling - name recognition is important. "I got a call from X is a big deal".

> The carriers are clearly customer-hostile

It’s actually not the carriers. The carriers actual try to actively block spam.

It’s actual the Twilio’s of the world that originate the bulk of the spam.

The carriers could absolutely implement authenticated caller id as a very first start. Then, you can at least trust the caller ID and screen/block effectively.
Do you mean like Shaken/Stir?

https://en.m.wikipedia.org/wiki/STIR/SHAKEN

OK so I am on T-Mobile, I use an Android device. Is there a way I can automatically display the true origin for all phone calls and texts? I would like to block any calls from VOIPs, international origins, or non-SHAKEN/STIR providers.
It's not at all surprising to me that corporations are happy to redirect people from utilities to products as core means of communicating.
With no evidence to back me up other than the daily volume of scam calls, I feel nevertheless confident concluding my cable company sells my "land line tel number" that "saves me money" on my cable bill to list brokers who service off-shore boiler room ops. One more reason to love the cable monopolists and their K Street influence peddlers.
Some weeks ago I broke my iPhone and had it replaced. During the process I had to get re-authorized with Verizon. They tried to call my partner to get her permission, since she's the account owner. Of course she never picked up the phone, because it was from an unknown caller.

The irony is incredible: the literal phone company cannot use their own product to contact their own customers, because they have let that product become a cesspool of spam.

My wife had to drive all the way back home from the phone store recently because I wouldn't pick up my phone for this exact reason.
All the negative comments are focusing on the old-school providers, but it is worth noting that I get multiple daily spam calls on Google Fi service also.
Could she not leave a message? I have the same setup as someone here mentioning Tasker. I route all calls to voice mail, where I can decide if I need to call back. If they are not willing to leave a voice mail with a return number and reason, the call was not that important.
I have never in my life received or listened to a voice message. To me that's something from 90s movies. Weird how different things can be.
I don't listen to mine—I read them. Built-in feature on my phone, these days. iPhone, but I'd be surprised if Android doesn't have it, too. Quick glance is enough to tell if any of the recent ones were anything other than junk.
ignore all unknown numbers

iOS transcribe feature makes it easy to read the voicemail and then in the 1/20 case where it’s legit, i act accordingly

For a lot of people, not being reachable by voice is not really an option in the case of emergencies, medical offices, business, etc. I do more texting than leaving voice messages but, for example, my dad essentially will not text or email.
I guess it's more that no one here ever leaves a message. A medical office for instance would never call to give out that info over phone, instead I would get an email or sms telling me to log in securely to their portal to see test results etc there.
It's not so much test results. But, for example, just this morning my dentist's office told me that a slot for an appointment I had been waiting for opened up this afternoon. More has switched to texts and portals but there's still a fair bit of calling in my experience in the US.
VM is becoming more of a necessity these days due to robocalls. I have any numbers that aren’t on my contacts go straight to VM, and that’s how I get in touch with non-contacts that I really want calling me.

In my experience, the signal:noise of unknown calls is about 1:20. VMs, on the other hand, are much higher — about 1:2.

I do the same thing. But I also have a habit of letting the voicemails sit around for days if they're not from someone I know. I'm not going to stop what I'm doing and listen to the voicemail, because, 19 times out of 20, it's just another robocall.
Wow, no one from any age group ever listens to voice messages anymore either, at least down here in my country
I also don't listen to voice messages, in my 30s, in the US. My phone doesn't even tell me when I get them and I have no plans to change that (except when dealing with financial or legal bullshit).
The need was synchronous -- I either got authorized right away, or we moved on to plan B. And yeah, as others have said, even listening to VMs this days is outlier behavior :/
It's the modern version of the Problem of Evil.

Are the carriers willing to stop robocalls and spam texts, but not able? The very idea is absurd, as it's their network. As others have pointed out it can't be that hard to identify and interdict abusive behavior on such a massive scale.

Are they able, but not willing? Then they're complicit, and should face wire fraud charges.

Are they both able and willing? Then why don't they?

What's also funny about this is that I've tried using Google's screening service and I've literally never had an important caller answer the screening, they just go to voice mail instead. It's a weird situation because everyone knows that no one picks up calls from unknown numbers but everyone also expects you to pick up for them. It's kinda like when someone texts you "Hi" after exchanging numbers. No, text me "Hi it's so and so from xyz" because I can't differentiate you from spam without that. It's amazing to me that with such a universal problem that people don't adapt to the situation.
Not trolling or being a bad HN citizen but:

Why did this take so long?

There are already laws on the books. New legislation (read: added time) isn't necessary. Everyone suffers from this problem. No one, sans the violators, is in favor of this problem. That is, it's a political win regardless of party, state, etc.

Yet after years of pin-prick torture there's going to be a task force?

I'm confused.

Whenever I got a call not speaking "your car warranty will expire soon" these days I feel like just won some lottery.

Please put those who are doing this to jail for life. I will vote for you in the midterm election.

A couple of weekends ago we got like 20 calls within 2 hours from the same scammer call center. I begged them to stop calling, telling the "agent" it was a waste of their time as they would not be able to scam me and it would be in their best interest to flag my number. I may have finally convinced one of them after talking to him more than once.

BUT... at the same time my wife called AT&T and kind of read them the riot act. We figured if everyone started complaining maybe they'd get their collective act together and start solving this. Overhearing the call, it sounded like the agent recommended moving to VOIP to gain access to tools to help cut this down.

If they can largely solve this on cell (we get maybe a 1/10th of the calls on cell) and voip, why can't they solve this for landlines? I suspect the industry has a vested interest to get us off of landlines/POTS.

I just answer the call, put the phone down, still live, and let them talk away whilst listening to my background noise.

No call center supervisor is going to allow an employee to keep asking 'Hello....hello....can you hear me....?" for very long.

As for robocalls...they just play all the way through. I don't care, as I'm not listening.

I get very few spam calls compared to my friends and colleagues.

the free cruise scam has an INSANELY long script requiring very little interaction from me, the scammee. They read for over a minute uninterrupted. Then I give them a fake CC number from a generator. Then they get mad. Then I tell them to join the Naxalites. But even if I didn't interact with them they'd have a very long time on the line due to telling me all about the cruise.

Haven't gotten one in a year or so. Would love to know why. Does the well dry up a bit for them on that one particular thing?

(comment deleted)
idk I think this is a red herring. You have to ensure the technology itself isn't so susceptible to hacking. Gmail goes a long way to do this and Google is a technology company that helps other companies with their tech. Hmm, I wonder who can help?
I'm actually wondering if I should shut down text messages and phone calls coming from unknown numbers. What would I lose? Basically restaurants calling me for a reservation or something but what else? Everybody I know uses WhatsApp now.
silence unknown callers in iOS is a life saver

i get 2-20 spam call a day

My phone number is a robocall magnet like everyone else's, its the main reason I gave up a land line too. I only need it because so many sites require sending text messages for authentication (esp password resets). Fix authentication I am freed from phones.
I have two contacts on my ATT iPhone to enable and disable cell network calling, based on this that I found somewhere:

"You actually can turn off cellular network calling altogether, if you are willing to do that. Dial (star)#67# (or call 611 if it doesn't show up there) to see what number your voicemail center is. Then dial (star)21(star)1(that number)#. That will automatically forward all calls, at the network level, to your voicemail. To cancel this, dial #21#."

A trick I've used to get certain robocalls to stop: once you get someone on the line, especially after they say the call is recorded. Offer them X amount of bitcoin, in escrow, if they callback to your number at later time and share details about who they work for and for access to their systems.

Nobody has called my bluff, but usually that call center stops trying that scam to your number. I guess that's enough to trigger a do not call by their supervisors or they don't want to get in trouble by even hearing that opportunity.

Unfortunately, I do occasionally get stuck in a system where I get a robocall, but once I get past the sweet robot named Mary asking for final living expense coverages, I now get a 'this number is no longer available' when transferred. After a couple of weeks, I do drop off that list too.

Semi-related project I worked on: https://github.com/evidlo/nanpa_lookup

Simple bash script to lookup what companies own spam numbers. Almost all of my spam calls are from domestic VOIP companies.

    >>> ./lookup.sh -n 907-200-1234
    "GCI COMMUNICATION CORP. DBA GENERAL COMMUNICATION"
    >>> ./lookup.sh -f path/to/numbers.txt
    "GCI COMMUNICATION CORP. DBA GENERAL COMMUNICATION"
    "CELLCO PARTNERSHIP DBA VERIZON WIRELESS - NC"
    "ONVOY, LLC - TN"
That example doesn't work for me. It produces no output for that number. The problem seems to be that the entry in database.csv for that number is this:

  907|200||GCI COMMUNICATION CORP. DBA GE|PCS||6872
Given 907-200-1234 lookup.sh looks it up by grepping for '907\|200\|1'.

Compare to a number that works, 858-598-7654. That ends up grepping for '858\|598\|7' and that matches this line from database.csv:

  858|598|7|T-MOBILE USA, INC.|PCS|tmomail.net|6529
In general lookup.sh looks for the first 3 digits of the phone number in the first field of database.csv, the second 3 digits in the second field, and the next digit in the third field.

Looking up 907200 or 907-200 (but not 907-200-) will match.

I'd guess that the database allows a blank third field to mean that the entry covers all numbers that match the first 6 digits that aren't covered by a more explicit entry.

If that's the case the database is not using that mechanism optimally. For example there is this:

  360|654||ZIPLY FIBER NORTHWEST, LLC DBA|ICO||4324
  360|654|0|ZIPLY FIBER NORTHWEST, LLC DBA|ICO||4324
  360|654|1|ZIPLY FIBER NORTHWEST, LLC DBA|ICO||4324
  360|654|2|ZIPLY FIBER NORTHWEST, LLC DBA|ICO||4324
  360|654|3|ZIPLY FIBER NORTHWEST, LLC DBA|ICO||4324
  360|654|4|LEVEL 3 COMMUNICATIONS, LLC - |CLEC||6121
  360|654|5|AT&T - LOCAL|CLEC||7421
  360|654|6|LEVEL 3 COMMUNICATIONS, LLC - |CLEC||6121
  360|654|7|ONVOY, LLC - WA|CLEC||483E
  360|654|8|ZIPLY FIBER NORTHWEST, LLC DBA|ICO||4324
  360|654|9|ZIPLY FIBER NORTHWEST, LLC DBA|ICO||4324
which could be reduced to:

  360|654||ZIPLY FIBER NORTHWEST, LLC DBA|ICO||4324
  360|654|4|LEVEL 3 COMMUNICATIONS, LLC - |CLEC||6121
  360|654|5|AT&T - LOCAL|CLEC||7421
  360|654|6|LEVEL 3 COMMUNICATIONS, LLC - |CLEC||6121
  360|654|7|ONVOY, LLC - WA|CLEC||483E
There are 92492 different 6 first digit combinations where the database has explicit entries for all 10 possible 7th digits has a wildcard/default entry also. Cleaning up all these will cut the database.csv size to 52% of its current size. Appended is a script to do that.

Anyway, it seems like what lookup.sh should be doing is doing the 7 digit lookup like it does now, but if that doesn't match try a 6 digit lookup.

Here's a script to clean up database.csv. The output is sorted by name not number, but a "sort -n" can fix that.

  #!/usr/bin/perl
  use strict;

  main();

  sub main
  {
    my %db;
    my %wild;

    while (<>) {
        chomp;
        my($f3, $s3, $t1, @rest) = split /\|/;
        my $rest = join '|', @rest;
        my $key = "$f3|$s3|$rest";
        if ($t1 eq '') { $wild{$key} = 1; }
        else { push @{$db{$key}}, $t1; }
    }

    foreach my $key (sort keys %db) {
        my($f3, $s3, @rest) = split /\|/, $key;
        my $rest = join '|', @rest;

        if ($wild{$key}) {
            print "$f3|$s3||$rest\n";
        } else {
            if (10 == scalar(@{$db{$key}})) {
                print "$f3|$s3||$rest\n";
            } else {
                foreach my $t1 (sort @{$db{$key}}) {
                    print "$f3|$s3|$t1|$rest\n";
                }
            }
        }
    }
  }
Thanks, I'll try it out.
My setup seems to work well for me.

I have my mobile phone, which I never published. Only my family has it.

I have a VoIP number tied to my mobile phone, and desktop computer. This allows me to take calls with my headset on, not just on my mobile.

All calls from numbers not in my contacts list go to the VoIP voice mail, which gets auto-transcribed. (If it is English, it works well, but any other languages it fails.)

If someone really wants to talk to me, they will leave a name, reason for the call, and the number I can call them back.

I also use Tasker[0] on my Android mobile. The carrier charges a monthly fee to have the feature to identify spammers and auto-reject them.

   Type
   Incoming
   Number
   !C:ANY
This rejects all calls that are not in the contacts list.

[0] https://tasker.joaoapps.com/

Phone companies are trying to turn this problem into a revenue generation “opportunity”. Not only do they get paid for the calls that traverse their network, but now they also want you to sign up for their “premium” blocking services, at a monthly cost, to stop them. This screams for not only a crackdown on the bad carriers who initiate the calls, but also on the big telcos who are double-dipping.
Only a task force? Not even a blue ribbon committee for the mess that public use comms has become?
(comment deleted)