Lexis especially makes even scraping public state laws and admin codes difficult, which is extra frustrating because they are the legal publisher of record in a number of states.
I've been considering trying to launch an OpenStates style scraper project for US laws and admin codes, but haven't had the time to attack 100 more scrapers. Even with AI help, the volume is significant.
I do a lot of IL criminal courts research and it's really, really fucking bad out there.
Transcripts: Multiple dollars per pages. Want that expedited? Multiply that amount by four. Don't know the court date? Can't get your transcript. Clerks put in the wrong date? Tough luck. Payment for those transcripts? Over Zelle because the court reporters themselves are contractors and get paid independently.
FOIA: IL Judicial system isn't FOIAable. There's a "data request" process that you can go down, but last time I tried by requesting through the chief judge of Cook County, they told me to request the data from the Clerk of Courts. The Clerk of Courts told me I need to speak to the chief judge's office. That went on for 6 months. The docket-level data I've seen from them is... beyond indecipherable. Literally never seen a dataset so indecipherable in my life -- a combination of esoteric codes, hidden encoding schemes, and length-delimited lines.
Court docs: Gotta go downtown if you're not a lawyer or if you're not a "criminal justice partner" to get any, then you have to print them (don't print past 4 or you'll get yelled at). Some journalists have access, but the IL Supreme Court's policies disallow journalists from getting it. So we have a system where only some journalists get access to court documents and fuck everyone else, I guess.
Civil Asset Forfeiture cases: the courts and state's attorneys do everything in their part to separate it from the initiating criminal case. So peoples' cars are being taken by the cops under dubious constitutional grounds and they can't get it back until the criminal case is over. And I've yet to find a consistent way to figure out the initiating criminal case tied to a civil asset forfeiture. I've yet to find anything like a policy doc that says it's intentional to obfuscate the criminal case from the civil case, but I can't think of any reason why it would be like this if it wasn't intentional. FOIA through the State's Attorney? Nah, they don't respond, or they give nonsense denials that require litigation.
I could go on, but. Fuck. It's bad out there and it hides so much suffering. I highly encourage folk to give court watching a try. You'll be shocked how much you'll learn.
If you're trying to fix anything relating to public policy, politics, &c., without fixing this you are wasting your time. You will make a bigger difference adopting a local stretch of highway and picking up litter. It is maddening to see generation after generation learn this the hard way.
FLP's CourtListener RECAP hits hard, but it has no counterparts either worldwide or within the US. If it goes offline, democracy and rule of law across the entire world go offline with it. Federal statutes and regulations are a very small part of black letter law.
"The fees the federal government charges for court documents are significant. PACER charges users $0.10 for every page users search and access, and $30 per name or item searched plus $0.10 per page per document per month, albeit with a max of $3 per individual document."
It's funny how these numbers sound very reasonable on their face, especially if used for human review.
But if you consider programmatic searches, 10 cents per page is a lot. Consider the CPU cost of cycling through each pdf, assume 1000 letters, that's 1Khz which is like a billionth of a cent, in terms of electricity and hardware ammortization.
I'd bet that this incentivizes a 'hoarding' and reuse of information, rather than a "query when necessary" practice, which would in fact give place to monopolies like OP argues.
A newcomer legal analysis company would have to pay, say 200$ for research from their first customer. If they save those documents, then it is unlikely that their second customer will benefit from those documents.
However huge firms would make a business out of the byproduct of searches, they spend 200$ on documents, and they serve a client, but they add those documents to their own internal database, which of course has costs much lower than the nominal fees stipulated by the federal steward (PACER).
An argument could be that you can't eliminate the fees because there's costs that need to be paid. But isn't that what taxes are typically for? Especially corporate income tax. The costs are probably a nominal fee to avoid overbearing systems, similar to franchise taxes and incorporation clerical fees.
I don't think the idea of charging for documents is bad, who said that the system would need to be adopted for huge data hungry systems, I'm all for giving human analysis an advantage instead of adapting systems to play into the data munching AI fad. I'd rather see balancing that rewards human analysis rather than bringing data munching to the masses.
> PACER charges users $0.10 for every page users search and access, and $30 per name or item searched plus $0.10 per page per document per month, albeit with a max of $3 per individual document
This is intentionally misleading and obviously wrong for anyone who has ever used PACER. If you have the PACER Service Center execute the search for you - a human being - it costs $30. If you use the web interface, the rates are extremely low, and citizens are given $30/quarter in credit to use.
If someone wanted to do business research for Mexico or any other LATAM country, while in the US and using English source data what resources would you look to?
Its crazy insane that citizens who are expected to follow the law cannot have access to law ? what does that even mean ? Do we at-least know some high level numbers on how many law/opinions/cases texts are not freely available to citizens ?
Tangentially the State of Florida will charge you $75/day for your time in jail, regardless of whether you were found not guilty, the case was dismissed or charges dropped completely.
And not paying those fees is a second degree felony.
17 comments
[ 4.1 ms ] story [ 48.7 ms ] threadI've been considering trying to launch an OpenStates style scraper project for US laws and admin codes, but haven't had the time to attack 100 more scrapers. Even with AI help, the volume is significant.
Transcripts: Multiple dollars per pages. Want that expedited? Multiply that amount by four. Don't know the court date? Can't get your transcript. Clerks put in the wrong date? Tough luck. Payment for those transcripts? Over Zelle because the court reporters themselves are contractors and get paid independently.
FOIA: IL Judicial system isn't FOIAable. There's a "data request" process that you can go down, but last time I tried by requesting through the chief judge of Cook County, they told me to request the data from the Clerk of Courts. The Clerk of Courts told me I need to speak to the chief judge's office. That went on for 6 months. The docket-level data I've seen from them is... beyond indecipherable. Literally never seen a dataset so indecipherable in my life -- a combination of esoteric codes, hidden encoding schemes, and length-delimited lines.
Court docs: Gotta go downtown if you're not a lawyer or if you're not a "criminal justice partner" to get any, then you have to print them (don't print past 4 or you'll get yelled at). Some journalists have access, but the IL Supreme Court's policies disallow journalists from getting it. So we have a system where only some journalists get access to court documents and fuck everyone else, I guess.
Civil Asset Forfeiture cases: the courts and state's attorneys do everything in their part to separate it from the initiating criminal case. So peoples' cars are being taken by the cops under dubious constitutional grounds and they can't get it back until the criminal case is over. And I've yet to find a consistent way to figure out the initiating criminal case tied to a civil asset forfeiture. I've yet to find anything like a policy doc that says it's intentional to obfuscate the criminal case from the civil case, but I can't think of any reason why it would be like this if it wasn't intentional. FOIA through the State's Attorney? Nah, they don't respond, or they give nonsense denials that require litigation.
I could go on, but. Fuck. It's bad out there and it hides so much suffering. I highly encourage folk to give court watching a try. You'll be shocked how much you'll learn.
https://www.bfvlaw.com/supreme-court-rules-georgia-cannot-cl...
FLP's CourtListener RECAP hits hard, but it has no counterparts either worldwide or within the US. If it goes offline, democracy and rule of law across the entire world go offline with it. Federal statutes and regulations are a very small part of black letter law.
It's funny how these numbers sound very reasonable on their face, especially if used for human review.
But if you consider programmatic searches, 10 cents per page is a lot. Consider the CPU cost of cycling through each pdf, assume 1000 letters, that's 1Khz which is like a billionth of a cent, in terms of electricity and hardware ammortization.
I'd bet that this incentivizes a 'hoarding' and reuse of information, rather than a "query when necessary" practice, which would in fact give place to monopolies like OP argues.
A newcomer legal analysis company would have to pay, say 200$ for research from their first customer. If they save those documents, then it is unlikely that their second customer will benefit from those documents.
However huge firms would make a business out of the byproduct of searches, they spend 200$ on documents, and they serve a client, but they add those documents to their own internal database, which of course has costs much lower than the nominal fees stipulated by the federal steward (PACER).
An argument could be that you can't eliminate the fees because there's costs that need to be paid. But isn't that what taxes are typically for? Especially corporate income tax. The costs are probably a nominal fee to avoid overbearing systems, similar to franchise taxes and incorporation clerical fees.
I don't think the idea of charging for documents is bad, who said that the system would need to be adopted for huge data hungry systems, I'm all for giving human analysis an advantage instead of adapting systems to play into the data munching AI fad. I'd rather see balancing that rewards human analysis rather than bringing data munching to the masses.
This is intentionally misleading and obviously wrong for anyone who has ever used PACER. If you have the PACER Service Center execute the search for you - a human being - it costs $30. If you use the web interface, the rates are extremely low, and citizens are given $30/quarter in credit to use.
https://www.uscourts.gov/court-programs/fees/electronic-publ...
For example, Souter was editing his published opinion in Lee v. Weisman nine years after the case was decided.
https://balkin.blogspot.com/2025/05/a-justice-souter-anecdot...
And not paying those fees is a second degree felony.
It would also be great for pointing out all the loopholes that have been built in by our owners.
Could it align incentives for future reform, or would it just blow to the bottom line of these companies?