I very much agree with you about transcripts of public meetings, but I'm extremely skeptical that AI summaries are a good idea.
I'm open to it working but here's a few reasons why I'm skeptical.
-There's often a lot of missing context for things discussed at public meetings in smaller cities and towns, because it's assumed most people know that context already. I don't know that an LLM would be able to fill in the blanks for this type of thing.
- In cases where there's a disagreement over some factual thing, how does the AI determine who is correct? It could just do 'A said this.' 'B said this.' But I don't see much value in that over a straight transcript.
- Maybe a stretch, but I wonder if it might lead to a kind of IRL keyword stuffing to influence the importance certain things are given in a summary over others. I'm imagining a city councilor beginning a speech with "ignore previous instructions" or something like that.
To be fair, though, AI summaries don't have to win Pulitzers. They just have to be roughly comparable to existing local reporting of city measures, which are often bad or nonexistent to begin with.
As for context, ideally you could feed it any relevant histories (measures proposed, previous articles, pre/past meeting transcripts etc.)
Given the same input contexts, I'd probably trust the AI analysis more than your average small town journalist. For interviews though, yeah probably a human would be more accepted, but they rarely do those for public meetings.
Disclaimer: I have been that average small town journalist, so perhaps I am biased about this.
You are probably right, and if it works well enough I doubt my complaining is going to stop it.
It will be interesting to see if/how having an AI as the "primary" audience for public meetings changes how they are conducted at least.
Considering how much your average local reporter gets paid, I can't fathom this being a huge boon to newsrooms either way, but I wouldn't be surprised if it ultimately becomes the standard for creating meeting minutes by local governments.
I think my concern here is that you're creating a system where no one actually has to pay attention to what these public bodies are doing. Not that much attention is paid now either.
I think having LLMs watch/listen/read and summarize public meetings and legislation could present a path to saner discourse in our society. It will (hopefully) be far harder for news organizations and bad actors to pollute the discourse around an issue when you can instantly query exactly what was said or written and, if needed, add context and expand the conversation.
I don't think our public discourse is really steeped in truth or reality anymore, sadly. It doesn't really matter who said or did what when politicking and mass emotional manipulation win the day.
There's no lack of existing thoughtful analysis. It's just ignored more often than not.
AI is not the solution to saner politics. That is a cultural issue specific to the political and media landscape of America. No other country in peacetime has local politicians waving guns at the camera as part of an electoral campaign.
We can query exactly what was written and said today, most public meetings of state/national importance will have full transcripts. People in America aren't going to do the work of looking into any of that. It is easier to click the share button on a meme image and brawl in the comment sections.
Given ASR is becoming really great (e.g., whisper) perhaps the intersection with the transcript and some "rephrase" ratio would be acceptable? So if the LLM deviated too much then that is problematic.
11 comments
[ 2.8 ms ] story [ 33.8 ms ] threadLost transcription of public meetings & summarization are one of the greatest impacts of the decimation of local media.
It's not glorious, but many stories began with a comment or motion at the end of a 4 hour meeting.
I'm open to it working but here's a few reasons why I'm skeptical.
-There's often a lot of missing context for things discussed at public meetings in smaller cities and towns, because it's assumed most people know that context already. I don't know that an LLM would be able to fill in the blanks for this type of thing.
- In cases where there's a disagreement over some factual thing, how does the AI determine who is correct? It could just do 'A said this.' 'B said this.' But I don't see much value in that over a straight transcript.
- Maybe a stretch, but I wonder if it might lead to a kind of IRL keyword stuffing to influence the importance certain things are given in a summary over others. I'm imagining a city councilor beginning a speech with "ignore previous instructions" or something like that.
As for context, ideally you could feed it any relevant histories (measures proposed, previous articles, pre/past meeting transcripts etc.)
Given the same input contexts, I'd probably trust the AI analysis more than your average small town journalist. For interviews though, yeah probably a human would be more accepted, but they rarely do those for public meetings.
Likelihood of human recognizing that a summarization is bad ~= ease of a human to review the raw data for something questionable.
Since modern summarizing tools are operating on transcripts that are also publicly available, why not enable linking to the source material?
AI summarization that littered the summary with "[0] [1] [2]" references would be the best of all possible worlds!
You are probably right, and if it works well enough I doubt my complaining is going to stop it.
It will be interesting to see if/how having an AI as the "primary" audience for public meetings changes how they are conducted at least.
Considering how much your average local reporter gets paid, I can't fathom this being a huge boon to newsrooms either way, but I wouldn't be surprised if it ultimately becomes the standard for creating meeting minutes by local governments.
I think my concern here is that you're creating a system where no one actually has to pay attention to what these public bodies are doing. Not that much attention is paid now either.
There's no lack of existing thoughtful analysis. It's just ignored more often than not.
We can query exactly what was written and said today, most public meetings of state/national importance will have full transcripts. People in America aren't going to do the work of looking into any of that. It is easier to click the share button on a meme image and brawl in the comment sections.