Hey there! A big part of my project is to make FOIA a bit more accessible for larger scale requests, where $5 per-request would be a bit out of the question. I realize that FOIA machine can do a lot of that, but there's a pretty large discrepancy between their list of contacts and Muckrock's, so FOIA Machine isn't really a viable option, either. Muckrock's wasn't as full as I was hoping, either (though, still incredibly impressive) so I'm working at expanding the contacts that Muckrock's collected. Some web crawlers were kicked off recently that once polished, should be able to improve on what you guys already have.
When and why did you create a faxing API? I know that may seem like a snarky question but I really mean it seriously. I'm someone who hasn't sent a fax in at least 10 years and so I just assumed it was a nearly dead form of communication.
The terms are a bit restrictive but not incredibly so.
FBI eFOIPA: Terms of Service:
Please read before continuing...
Not all requests can be fulfilled using the eFOIPA system. You will be notified if your request cannot be serviced through the eFOIPA system.
A valid e-mail address will be required for authentication.
Requests for fee waivers will require additional documentation.
Submissions are limited to events, organizations, first party requests (Privacy Act requests), and deceased individuals.
You will be required to upload proof of death for requests for records responsive to deceased individuals who are younger than 100 years of age. Acceptable formats include .pdf or .doc. Other formats will not be accepted.
If you are making a request on an event, organization, or deceased individual, the primary form of correspondence that you will receive from the FBI will be e-mail. If you are making a request on a first party (Privacy Act request), the primary form of correspondence that you will receive from the FBI will be through standard mail.
If you are making a request on a living third party, your request cannot currently be serviced using the eFOIPA system.
The combined file size of all attachments may not exceed 30 megabytes.
You are limited to making one request per day and one request per submission.
It is recommended that you visit http://www.justice.gov/oip/doj-foia-regulations if you have any additional questions or concerns prior to submitting your Freedom of Information Act or Privacy Act (FOIPA) request to the FBI.
An FBI Criminal History Summary Check—often referred to as a criminal background check, criminal history record, police background clearance, police/good conduct certificate, or “rap sheet”—is a listing of certain information taken from fingerprint submissions retained by the FBI in connection with arrests and, in some instances, federal employment, naturalization, or military service. It can also be used to satisfy a requirement to live, work, or travel in a foreign country; for employment or licensing; or for adopting a child. To obtain a copy of your FBI Criminal History Summary Check, please contact the FBI’s Criminal Justice Services Division (CJIS) in Clarksburg, West Virginia by writing to Federal Bureau of Investigation, CJIS Division, Attention: Record Request, 1000 Custer Hollow Road, Clarksburg, WV 26306.
I think the bigger change here is that you can only submit one request per day via the web portal. That makes it almost worthless. Most news organizations that file FOIA requests have to file them for multiple pieces of information from different sources. This would limit them to only making one request for a source online and then require faxing or mail for all the others. As the current FOIA system sometimes takes months for records to be released, I can only imagine how bad this will get for them going forward. :-/
Someone pointed out that there's technically a web portal that requesters can use. That web portal, however, goes beyond the law to add a bunch of restrictions including:
* Limiting you to one request per day.
* Not allowing you to request internal memos and a variety of other classes of documents.
* The website doesn't work on weekends.
More detailed look how bad the website they launched to replace email is:
Websites that don't work on weekends (of which I've seen more than a few) are really quite possibly the stupidest thing I've ever seen. I shudder to think that somewhere there was a requirements document that details what days the site should and shouldn't work...
The UK National Lottery website is non-functional overnight, presumably to prevent drunken gambling. That's the one good reason for implementing that sort of restriction I've seen.
Even support chats it annoys me - support open Mon-Fri 9-5 - fine, but can you not just take my message and let's do it over email, or send me a reminder in opening hours with a link that automatically starts a chat with the request I type out now?
truly sounds as if they don't staff the receiving end with anyone who has authority on the weekend to either approve or or even accept the requests. part of me wants to think its just work place rules but the cynical side keeps thinking they do this on purpose to make it annoying
I had to file some forms on irs.gov about a year ago, and the web portal was only open on weekdays between 9am and 5pm eastern time.
My fantasy explanation is that the form feeds directly into a printer, and they have a guy who takes the printouts, staples them, and puts them in a box. If the website is open 24/7 then when Stapleman shows up to work Monday morning the backlog might be discouraging. Not to mention if the printer jams during off hours and nobody's there to notice, filings may be lost. Or maybe I'm overthinking this.
I'm not sure if the IRS is trending towards a paperless office or not (is that even still a thing?), but triggering a print job for every form submitted via a web portal sounds very wasteful. I'd hope they have have some sort of workflow system in place instead.
Another theory I have involves concessions to webserver unions and/or acknowledging the need for time off to take care of the child processes. These fantasies are the only place I can find closure and sanity when my carefully filled out submission is rejected by the website because it is now 4:01 pm and that's Miller Time on the east coast.
The IRS hasn't had a budget in many years now. Congress passes emergency reauthorizations every year that's just enough to keep the lights on, but does not allow for any kind of long-term planning or upgrades. I'd imagine that, if they do have crazy systems like this, they'd love to get rid of them but can't.
It sounds good, but I'm going to wait for the budget to actually pass before popping the champagne. Considering everything we've seen so far, my confidence that Trump's appointees will actually live up to their smash-the-system claims, and not just be more crony capitalists, is quite low.
The IRS has 79,890 employees. $12B annual budget. So...
edit: Another comment linked to a good explanation as to why the site isn't up 24/7. TL;DR legacy COBOL that everyone is terrified of touching. Just in case the Secretary of Treasury is reading this: I, for one, would gladly have my tax dollars go to modernizing irs.gov. Oh wait, I just remembered healthcare.gov. Nevermind.
It was a hugely complex site that had to manage and connect to a large number of existing systems for each insurer that was part of the market place. It also had to be done in an extremely short time frame because the details of the site and it's operation weren't solid until the law was signed.
Far more has been spent on Facebook and Twitter (for example), and they don't do anything nearly as important as providing health care coverage to people.
A little more eco-friendly but no less insane explanation: the software processing forms is running on someone's regular work machine, which they turn on each morning, and then turn off when they go home...
The power options are controlled by group policy, and as yet nobody's been able to take a week off from their work to track down "the administrator" and convince them to add an exception.
A while ago I had the pleasure of visiting a government office that indeed had such a box with the writing "internet" on its side, with printouts of all requests that arrived by email. But as far as I recall, the email server and printing were asynchronous, so I had that going for me...
In Japan, quite a few ATMs would not work during weekends and evenings, I've always speculated that they hired people to sit in the atm and hand over the money. It also would explain why credit card fees are higher when withdrawing money during weekends (in the few ATMs that allows you to do that)...
The help desk for users outside the Service is only up during those hours weekdays and is constantly having its scope reduced. On the weekends the mainframe is processing several million transactions that went on during the week. Even though it is a Web 1.0-looking interface, those web-based bits are a peripheral to a fairly ancient mainframe running COBOL. That mainframe still updates several millions of individual accounts and cranks through millions upon millions of account updates at night and especially on the weekend.
Your fantasy explanation is way, way off the mark. Nothing is feeding into a hopper from a printer. It is just a huge mainframe that does many, many things for hundreds of millions of individual taxpayers as well as millions upon millions of businesses and other entity types.
B&H, an electronics store in NYC, won't take orders by web from Friday night to Saturday night but the website itself works otherwise. It is owned by Hasidic Jews (Satmar) and apparently a Rabbi must have issued a responsa that allows them to keep the site on but not to take orders.
All those agreeable office workers and higher class people can access the site whenever (including work). All the icky people working in factories, services or places you sweat don't need govt. services anyways.
My alma mater had one like that. Worse, it had operating hours in general: it was closed at night, too.
It was the Student Information System that we used to register for classes, pay tuition, update our contact information, etc. So it was a pretty important system.
One day, I asked someone who would know. I was told that it was because SIS ran on an old mainframe that couldn't read and write the disk at the same time. So it made all changes in memory during the operating hours, and then it shut down and wrote out the disk at night.
(also, our username for SIS was our full SSN... and the password they assigned to us was the last four digits of our SSN)
This subsidy is plausibly a good one, to avoid discouraging requests from private citizens, but it can also probably justify why FOIA requests would be rate-limited. (In some situations, you might be able to avoid ever paying by breaking you request up into a bunch of sub-requests that can individually be completed in less than 2 hours.)
Are there any sorts of outside investigations that are substantially hampered by the rate limiting? It seems like the turn-around time is going to be more than a day anyways, so you might as well bundle all your requests into one and submit daily. Or is there an aspect of this I'm missing?
Incidentally, are there estimates on how much the FBI (or other agencies) spend to fulfill FOIA requests? This report from 1980 says ~$50M, but that that might be a substantial underestimate since there were large ancillary costs it doesn't cover.
Maybe worth pointing out that following media interest/inquiries today, the FBI now claim (according to The Daily Dot) that the website terms and conditions will be changed on March 1st to allow 24/7 access and unlimited requests from individuals.
Also: "The FBI eFOIA portal has been under development and testing for two years." That's a long time to spend developing a website that doesn't work on weekends....
It would be interesting to fax in a FOIA request to find out the reasoning behind the decision to not allow email FOIA requests. And how the restrictions for the web portal came to be since they are not required by law.
Nice. My first thought was that somebody should make such a service. My second thought was that somebody probably already had. Do you get upticks in traffic when things show up in the news?
Definitely. Trick is keeping things running smoothly during news cycles and off, but there's been a lot more interest the past few months around a number of topics.
The USDA announced that public access to that information would now be mediated through Freedom of Information Act requests.
There was a dog food poisoning reported [earlier this week](http://patch.com/illinois/hydepark/s/g0ven/illinois-company-...) The "supplier" of the poison dog food is probably pretty happy the public can't figure out which other pet food manufacturers (Besides Evanger's) they vendor.
This move was likely designed to hide facts from the public and to make investigative journalism a lot more difficult.
Public health and safety data needs to be public and squelching it is akin to public endangerment.
Not that wikipedia is a definitive source [1], but I think you could make the case by its definition that data gathered by the government on our behalf should be provided to us as easily as possible.
Sure it is! You know that, due to the sort of thing you're doing, you ought to try to be careful. But you make the decision not to.
Think about driving. I can easily imagine making the conscious decision to simply not drive very carefully--whether for fun, because there's an emergency, because I want to hurt people, etc.
You might think of it as analogous to willful ignorance. You know that there is something you ought to know, but you make a decision to either not seek the knowledge out, or to insulate yourself from it.
It's not going to be easy or cheap and there will be a significant discovery problem for others. To be effective, we need to abandon the nonprofit silo model and switch towards an open unitary model, a wiki-web built on relationships rather than centered on ownership or exclusive curation content.
I've also noticed that USCIS has been making materials that were previously publicly available inaccessible, like immigration guidance documents. The links are still there on their homepage, but now they just point back to the host page.
This is another troubling action meant to chip away at what government transparency we have left. I'm sure that the argument could be made that government agencies are "inundated" with requests that they can't keep up with and all this is too "costly."
If I don't see a privately operated website providing a workaround within a week, I'll be embarrassed. The Internet views censorship as damage and routes around it.
Corporations have been profiting from your contact information for years, now you can donate your contact information for the public good.
99 comments
[ 3.2 ms ] story [ 177 ms ] threadWe're also open source: github.com/MuckRock/muckrock/
Anywho, I'll shoot you an email pretty soon!
(edit: clarity and a bit more info)
I don't personally send lots of faxes but businesses send millions of them and it's a crucial part of their workflow.
There is a web portal https://efoia.fbi.gov
The terms are a bit restrictive but not incredibly so.
FBI eFOIPA: Terms of Service: Please read before continuing...
Not all requests can be fulfilled using the eFOIPA system. You will be notified if your request cannot be serviced through the eFOIPA system.
A valid e-mail address will be required for authentication. Requests for fee waivers will require additional documentation.
Submissions are limited to events, organizations, first party requests (Privacy Act requests), and deceased individuals. You will be required to upload proof of death for requests for records responsive to deceased individuals who are younger than 100 years of age. Acceptable formats include .pdf or .doc. Other formats will not be accepted.
If you are making a request on an event, organization, or deceased individual, the primary form of correspondence that you will receive from the FBI will be e-mail. If you are making a request on a first party (Privacy Act request), the primary form of correspondence that you will receive from the FBI will be through standard mail.
If you are making a request on a living third party, your request cannot currently be serviced using the eFOIPA system. The combined file size of all attachments may not exceed 30 megabytes.
You are limited to making one request per day and one request per submission.
It is recommended that you visit http://www.justice.gov/oip/doj-foia-regulations if you have any additional questions or concerns prior to submitting your Freedom of Information Act or Privacy Act (FOIPA) request to the FBI.
An FBI Criminal History Summary Check—often referred to as a criminal background check, criminal history record, police background clearance, police/good conduct certificate, or “rap sheet”—is a listing of certain information taken from fingerprint submissions retained by the FBI in connection with arrests and, in some instances, federal employment, naturalization, or military service. It can also be used to satisfy a requirement to live, work, or travel in a foreign country; for employment or licensing; or for adopting a child. To obtain a copy of your FBI Criminal History Summary Check, please contact the FBI’s Criminal Justice Services Division (CJIS) in Clarksburg, West Virginia by writing to Federal Bureau of Investigation, CJIS Division, Attention: Record Request, 1000 Custer Hollow Road, Clarksburg, WV 26306.
The linked article points all of this out in the first few sentences. FUD?
Call a spade a spade.
https://www.muckrock.com/news/archives/2017/feb/07/fbi-foia-...
Even support chats it annoys me - support open Mon-Fri 9-5 - fine, but can you not just take my message and let's do it over email, or send me a reminder in opening hours with a link that automatically starts a chat with the request I type out now?
That's how it goes for banks, anyways...
My fantasy explanation is that the form feeds directly into a printer, and they have a guy who takes the printouts, staples them, and puts them in a box. If the website is open 24/7 then when Stapleman shows up to work Monday morning the backlog might be discouraging. Not to mention if the printer jams during off hours and nobody's there to notice, filings may be lost. Or maybe I'm overthinking this.
I'm not sure if the IRS is trending towards a paperless office or not (is that even still a thing?), but triggering a print job for every form submitted via a web portal sounds very wasteful. I'd hope they have have some sort of workflow system in place instead.
It's a government worker union thing. They don't have a contract for being called at 4am when the website breaks, so you better shut it off at 5pm.
If you think this is the stupidest thing in government you should ensure you never have to work in that environment over your lifetime :)
Most likely: they don't have anyone on call outside those hours for breakages, so they only turn the system on during working hours.
edit: Another comment linked to a good explanation as to why the site isn't up 24/7. TL;DR legacy COBOL that everyone is terrified of touching. Just in case the Secretary of Treasury is reading this: I, for one, would gladly have my tax dollars go to modernizing irs.gov. Oh wait, I just remembered healthcare.gov. Nevermind.
Source: https://www.washingtonpost.com/news/fact-checker/wp/2013/10/...
Far more has been spent on Facebook and Twitter (for example), and they don't do anything nearly as important as providing health care coverage to people.
Your fantasy explanation is way, way off the mark. Nothing is feeding into a hopper from a printer. It is just a huge mainframe that does many, many things for hundreds of millions of individual taxpayers as well as millions upon millions of businesses and other entity types.
[Is break over yet?]
All those agreeable office workers and higher class people can access the site whenever (including work). All the icky people working in factories, services or places you sweat don't need govt. services anyways.
It was the Student Information System that we used to register for classes, pay tuition, update our contact information, etc. So it was a pretty important system.
One day, I asked someone who would know. I was told that it was because SIS ran on an old mainframe that couldn't read and write the disk at the same time. So it made all changes in memory during the operating hours, and then it shut down and wrote out the disk at night.
(also, our username for SIS was our full SSN... and the password they assigned to us was the last four digits of our SSN)
https://www.foia.gov/faq.html#cost
This subsidy is plausibly a good one, to avoid discouraging requests from private citizens, but it can also probably justify why FOIA requests would be rate-limited. (In some situations, you might be able to avoid ever paying by breaking you request up into a bunch of sub-requests that can individually be completed in less than 2 hours.)
Are there any sorts of outside investigations that are substantially hampered by the rate limiting? It seems like the turn-around time is going to be more than a day anyways, so you might as well bundle all your requests into one and submit daily. Or is there an aspect of this I'm missing?
Incidentally, are there estimates on how much the FBI (or other agencies) spend to fulfill FOIA requests? This report from 1980 says ~$50M, but that that might be a substantial underestimate since there were large ancillary costs it doesn't cover.
https://www.justice.gov/oip/blog/foia-update-costs-and-benef...
I don't think FOIAs should be free nor anonymous. I imagine foreign governments would exploit that to try to probe for information.
Also: "The FBI eFOIA portal has been under development and testing for two years." That's a long time to spend developing a website that doesn't work on weekends....
http://www.dailydot.com/layer8/fbi-foia-records-requests-ema...
Well, when you're building websites in COBOL...
https://www.muckrock.com/foi/united-states-of-america-10/foi...
The page gets updated as documents come back.
https://www.muckrock.com/foi/create/
You can also browse almost 30,000 other requests filed by users:
https://www.muckrock.com/foi/
The USDA announced that public access to that information would now be mediated through Freedom of Information Act requests.
There was a dog food poisoning reported [earlier this week](http://patch.com/illinois/hydepark/s/g0ven/illinois-company-...) The "supplier" of the poison dog food is probably pretty happy the public can't figure out which other pet food manufacturers (Besides Evanger's) they vendor.
This move was likely designed to hide facts from the public and to make investigative journalism a lot more difficult. Public health and safety data needs to be public and squelching it is akin to public endangerment.
[edit: thx for grammar correction.]
I don't think you know what public good means
[1] https://en.wikipedia.org/wiki/Public_good
Think about driving. I can easily imagine making the conscious decision to simply not drive very carefully--whether for fun, because there's an emergency, because I want to hurt people, etc.
You might think of it as analogous to willful ignorance. You know that there is something you ought to know, but you make a decision to either not seek the knowledge out, or to insulate yourself from it.
Does anyone know of a place to post such things?
Edit: To be clear, I think there is no good reason for this to be hidden behind a FOIA wall. But, assuming that we're stuck with the FOIA wall...
[0] https://www.muckrock.com/
[1] https://www.youtube.com/watch?v=M5rp3nJ62Nc
This is worrisome! Will the feds start charging fees to view electronic docs like the State of Wyoming? http://www.wyomingnews.com/news/wyoming-court-ok-to-charge-f...
Corporations have been profiting from your contact information for years, now you can donate your contact information for the public good.
See also https://www.muckrock.com/