73 comments

[ 3.8 ms ] story [ 75.6 ms ] thread
We can make them more accurate by leveraging ICE going door to door.
Frankly i see no reason to keep this data private. They should simply publish a full dataset of the census, with no such data coarsening/differential privacy/ etc...

Fundamentally this is public data. If it's to dangerous to make public, it's too dangerous to collect, and people should be aware of exactly what it is.

There are very few things that the state has data on that should not be made public. Census data is simply not one of those things.

publishing should be the default for any data, and to keep it unpublished should require substantially good reasons that impact the country as a whole. Frankly, if it isn't detailed national defence plans, i struggle to see any data that should not be public.

There will be a bunch of people that start off with the premise that this data should be private and make following arguments based on this premise.

So I'll just go ahead and ask, give me good reasons why this data should be private?

My guess is that most of you think we should be counting illegals because they should have representation. And I reject that

The dueling political demands of accuracy and privacy are simply incompatible at some level. After reading this, maybe Hanlon's Razor isn't the right standard. Besides malice and stupidity, there is impossibility. Some problems just aren't solvable under certain constraints. I don't envy the statisticians tasked with finding a politically palatable solution to a math problem.
> Differential privacy makes this trade-off explicit, and thus impossible to ignore. Maybe banning it is a way of pretending that the problem doesn't exist, in the hope that it will go away?

Or it's saying that one of these conflicting goals is more valuable than the other, and so shouldn't be sacrificed for it.

The replies here arguing we should publish it all are wild in the worst kind of first-order thinking way.

It’s a census: it just asks questions.

If you start publishing and weaponizing the data against people with various attributes, they’ll just lie or not answer. And then you are left with worse than nothing: bad data people try to act on.

[flagged]
The term “first-order thinking” just clicked for me. So revealing. One of today’s lucky 10,000
Ban it from the dataset, add it to the analysis. You can choose your own flavor of noise.

I don't know what the political undertones are here, but at some level you need to have actual ground truth, including "this person/household declined".

Publishing raw data though? That seems like shooting yourself in the foot from a national security perspective, not to mention all the other reasons not to do it.

> Ban it from the dataset, add it to the analysis. You can choose your own flavor of noise.

Not sure exactly what you're proposing, but if the noise is added independently to different people, you can just buy multiple copies to reduce it.

There are a lot of ways to do this wrong, which is why so much analysis has gone into differential privacy.

Stalin's demographic researchers kept disappearing until they came up with the numbers he wanted.
The better to sell the data, all your privates are belong to us.
Whatever you do, there is a level of trust that is assumed when census takes place. The trust that this data is then not identified in a way that could be targeted for scams, frauds, and other such evils. But in NY, house sale records are made public but much to the detriment, many mortgage companies fake a bill for payment.

Differential privacy is absolutely necessary, and the social scientists being unable to reconstruct the data at an individual level is intended. A macroscopic description is rather enough for most purposes, and anything more is asking for a surveillance state.

In Ohio (or at least my county) the deed and mortgage are public record. As is a record when the mortgage is paid off. Interestly also property tax charges and payments are, too
Coming from a certain european country, you never know what answer on the census might get you into trouble.

"What is your religious affiliation". Seems perfectly innocuous, but turned out to be retroactively fatal if your answer could be attributed to you by a certain foreign occupier in the 1940s .

They don't ask about religious affiliation on the census.

1. How many people were living or staying in this house, apartment, or mobile home on April 1, 2020?

2. Were there any additional people staying here on April 1, 2020 that you did not include in Question 1?

3. Is this house, apartment, or mobile home?

4. What is your telephone number?

5. What is Person 1’s name?

6. What is Person 1’s sex?

7. What is Person 1’s age and what is Person 1’s date of birth?

8. Is Person 1 of Hispanic, Latino, or Spanish origin?

9. What is Person 1’s race?

Nothing really stops you from lying either.

Can anyone share how other countries handle this?
i have such a hard time reconciling stuff like this:

> The census bureau decided to adopt differential privacy for the 2020 Census

and:

> The consequences will be dire for utility or for privacy, and possibly both. It's hard to understate this point: future statistical releases will either be useless compared to past ones, or they will be incredibly unsafe

so we took the census for centuries before this point, and it was “ok.” and for the last census only we added some privacy items. but if we remove just one of those filters, we are in “dire” circumstances? but there were no privacy features before. so we’re actually still much better off than we were for hundreds of years before this.

this makes it feel like an emotional overblown problem

> but there were no privacy features before. so we’re actually still much better off than we were for hundreds of years before this.

If you are choosing hundreds of years ago, when we had no computers and internet, I wonder how we had worse privacy than the surveillance world today.

> so we took the census for centuries before this point, and it was “ok.”

Yes because we didn't have computers to unearth patterns in the data in a millisecond and politicians could have their career ended for doing the wrong thing, when revealed, instead of being rewarded for it.

> so we took the census for centuries before this point, and it was “ok.”

It wasn't ok - it's been shown that the data released could individually identify people in releases before the 2010 Census.

Pretty sad, in my opinion. In my ideal the state should have visibility into the shape of the people present so that we can make good decisions about our combined organization. I think we’re making a mistake we will come to regret by intentionally damaging our data collection infrastructure.

I think a large amount of the US’s success is the result of good institutions handling granular data. Policies can be adjusted to match outcomes more rapidly than otherwise.

I understand why people decide to diminish all state capacity - they feel that governments are populated by their opponents who will use state capacity against them. But as our relative strength wanes, our ability to overcome these forces of inertia does as well. And then our governments become less capable and eventually life starts getting worse.

We don’t need house-level data immediately (except perhaps in order to place census blocks within their appropriate congressional district etc). But there are aggregation units above which we should be using as good information as we possibly could be.

This does nothing to make government less powerful.

It just makes government stupider so even if they decide they want to do the right thing, now they can’t because they don’t have the information needed to make effective decisions.

> In my ideal the state should have visibility into the shape of the people present so that we can make good decisions about our combined organization.

That ideal became tantamount to enabling genocide when the US government breached the confidentiality of the census in order to prison camp the japanese on the basis of their race.

> I understand why people decide to diminish all state capacity

It's not even just a question of "all". The state should have the absolute minimum capacity to carry out its necessary tasks. Collecting race (just to give one example of many) of any form is not absolutely necessary and so it should not be done.

> they feel that governments are populated by their opponents who will use state capacity against them

Because they may be in the future. -- but even that is too strong, the greatest harm perpetrated by state actors has consistently come from trying to "help" rather than intentionally malicious acts.

The fines for non-compliance are low enough to remain silent.

Do. The American Census Survey (randomly-selected long-form questionairre) is dangerously overinvasive.

Can anyone explain to me the previous state and why it was desirable? I admittedly do not understand why people are getting riled up. I am not being difficult. I really don't understand the original state and the changed state here.
if you want to keep your sanity, I suggest silently adding the phrase

     "...for the next 950 days" 
every time you read some politically spiteful news like this

because the next two years are going to become insanely miserable

I guess this could be implemented externally.

Eg via some app that instructs respondents to enter a specific answer in a pseudorandomly chosen question.

Of course security would be another question.

I think it should be noted that there was a lot of dissatisfaction from users of the census data as far as I know. So it's not been banned just for politicals sake or because they hate privacy... Some people I talked to in the privacy field even called the whole thing a total disaster and weren't shy to put blame on John Abowd who apparently pushed this through despite a lot of internal opposition and concerns. Not sure if that's true, but what is definitely true is that the way the data was released produced serious issues downstream as most researchers and statisticians that ingested the data weren't prepared for receiving noisy data values. Differential privacy was applied in a way such that many invariants that data users cared about weren't preserved, which was expected as it's not possible as you can't preserve all invariants and at the same time add meaningful noise to the data. The thing is, with such a differentially private data release you need to adapt all of the downstream analyses to take into account the exact mechanism the data was altered in. And since the census bureau used a very intricate mechanism that didn't just add Laplace noise to data values but instead relied on a multi-stage process that preserved some invariants but not others it was very difficult to even write routines to account for the changes being made to the data. They essentially asked of every data user to rewrite their whole analysis pipeline based on the exact disclosure mechanism that contained a large number of bespoke choices regarding which data invariants to preserve and basically produced a mix of noisy, synthesized data that was just really hard to reason about. I don't even know if there even would've been a way to do this better, but the fact is that not every small county or school district has top-tier statisticians at hand that can just read a whole monograph on differentially private synthesized census data and then hotpatch their existing analysis systems to work with that data.

I was a big fan of differential privacy but now I think it might be doing more harm than good, as I haven't seen a single case where it was applied successfully in a problem where it actually mattered, and it contributed strongly to discrediting and preventing a lot of work on other anonymization techniques as it was deemed the only way to preserve privacy by the research community, so showing up with enhancements to k-anonymity or any other noise mechanism not rooted in it was a sure way to get ridiculed and ignored. And it's just not a practical mechanism, even when it works for a single disclosure you always end up having to blow up the privacy budget to a ridiculous amount in order to keep disclosing statistics as otherwise you would for almost all real-world data run out of budget after a few publications.

So, for me it's a technique that works in the areas where it doesn't really matter (publishing highly aggregated statistics that pose almost zero privacy risk even without differential privacy) and doesn't work in other areas where it would actually matter (publishing fine-grained data about individuals or small groups). There are some niche use cases but in my view the privacy community has really overblown the importance of differential privacy by portraying it as the only way to reliably anonymize data.

BTW the German census bureau has an interesting approach to anonymization which they use for several decades already and so far I haven't heard of any cases of successful de-anonymization of the data, maybe the US bureau should have a look at that for their own needs.

i think they will use ai as a leverage card to other country to order them
Data shall set you free... or not
I "enumerated" for the last census. Trust in my community was already not high* and I had lots of interesting encounters. I really believed the rather invasive data I was collecting with a friendly face would be used and handled responsibly. I feel for the poor souls that'll sign up to go door to door for 2030 now that the firewalls against weaponizing and monetizing all of our sensitive government data has been torn down, and even more for those that will volunteer information that can hurt them.

The comments that this rather expensive endeavour should just be about getting a head count are also amusing to me. The data collected was such an important baseline of common understanding, and this will not be a good thing for its future quality. I've grown very jaded now seeing all the things taken for granted in this country and lost or degraded recently with a whimper.

*: To be fair, they sent me specifically to places that didn't respond, so I was naturally led to believe that everyone in my region hated the government, ignored bizzarrely threatening fliers, or had recently moved and had no knowledge of the inhabitants (if any) during the census period.

> The comments that this rather expensive endeavour should just be about getting a head count are also amusing to me

Countries conduct censuses so they can understand, in great detail, what is going on with the people who make up the country.

With this accurate information, improvement plans can be made, and life can be improved for everyone.

The comments about just making it a head count give a very interesting window into the mentality of many these days. They don’t want to - it can’t fathom how to - make life better.

It’s sad, really

> The comments that this rather expensive endeavour should just be about getting a head count are also amusing to me. The data collected was such an important baseline of common understanding, and this will not be a good thing for its future quality.

Even without considering the Census data products alone, Census demographic data underlies virtually all extrapolation from other survey research. Everything from national opinion surveys based on tens of thousands of respondents, to small community surveys. A Census product with the most diverse participation pays off almost infinitely for America. It benefits everyone from national newspapers to rural counties.

If the smallest communities lose what little trust remains in the privacy of the Census, they have the most to lose in all of these ways.

That's my thought when I see the Census downplayed -- a massive amount of social sciences research revolves around the ACS. And it gets worse! Census data doesn't just control reapportionment (and/or gerrymandering), it directs how federal dollars trickle down to huge programs like Medicare and Medicaid that are then administered by states. Eroding more detailed household data like income throws programs designed to help people more out of whack, likely harming civic engagement even more.
The real decline started after Edward Snowden and all the information that came out about the NSA. It really sparked distrust in the government. Trying to get people to respond to surveys was already hard, why would those general people believe the Census Bureau is actually keeping their data safe? Doesn’t matter when it comes to laws and the constitution, if you work for an Agency. You are the government. Response rates keep going down, now we have attacks from the President on statistics about the economy. I’m a little cynical and I just assume they will continue to shrink the statistical agencies and make the statistics more useless (which is what this recent policy change does), and they will shift to the private industry. Even though the private industry cannot do the work in the Field that the government does.
I respect that you recognize that the sample you were given for investigation may not be representative of your community in all respects. Does the Census Bureau publish statistics on the people they have to send interviewers for? Just asking, maybe I'll have to go check their site.

Other comments here about Germany and whatnot falls flat, and short. This is America, and part of it is being ok with your neighbors (unless they're truly insane and release plagues of rabbits or burn themselves out of their homes, but I digress), try to find some common ground even if it's awkward and you have to look the other way about some slights and trespasses.

I don't know how much trouble they went to to "fit" you to the cohort they tasked you with, or how much of a natural fit there was. Did you grow up / do you live in the area? There's an element of "liking" (Cialdini) each other which colors participants' perception of tasks. OTOH nothing like talking to perfect strangers to flex that muscle or find out you really don't have it.