Science Research doesn't happen for its own sake. Every effort needs to be a part of the pipeline of demand and supply. Otherwise it's just a tune that you sing in the shower.
There is an enormous gulf between research in general and the people who should be reading it from a professional point of view. Science communication is really broken and what makes the trade press or press generally is largely about whether a papers authors manage to write a good press release and effectively writes an article themselves.
We need more New Scientist type magazine like things that do decent round ups of scientific findings for various fields that do a good job of shuffling through the thousands of papers a month and finding the highest impact papers. The pipeline from research to use in professions can drastically be improved. At the moment you end up having a hosepipe of abstracts and its a lot of time to review that daily.
> Thanks to software research, we know that most code comprehensibility metrics do not, in practice, reflect what they are supposed to measure.
Linked research doesn't really agree. But if it did, so?
If comprehensibility is not a simple metric then who's got a magic wand to do the fancy feedback? Sounds like it'd take a human/AGI which is useless, that's why we have metrics.
Are any real programmers who produce things for the world using comprehensibility metrics or is it all the university fakers and their virtual world they have created?
Shameless plug. I run LeetArxiv. It's a successor to papers with code built on the thesis "Every research paper should be a brief blog post with relevant code".
So far, the best reference for software engineering research appears to be R. Glass et al.'s 2002 work, Facts and Fallacies of Software Engineering. I haven't found a better or more comprehensive reference.
The only others who compare are his contemporaries, Steve McConnell, Timothy Lister, Tom DeMarco, and Barry Boehm.
Unfortunately, they're all basically retired. It feels like this kind of interest in software development, at least the publishing, ended around the mid-2000s.
My guess is the shift to blogs from books, adoption of Agile (in whatever form), and a shift in industry focus to getting rich rather than getting good ended the efforts to come up with resources like Glass put together.
For the audience here, the opposite side of the coin is more relevant: Why don't you read software research?
Based on this and other articles (and on experience), it's an especially underutilized resource. By reading it, you would gain an advantage over competition. Why aren't you using this advantage that is there for the taking?
The audience of software research is other software researchers.
The expectation that a practicing CS graduate, even with a master's degree, should be able to read, understand, and apply in their work research articles published in academic journals is not very meaningful.
Not because they are not capable people, but because research articles these days are highly specialized, building upon specialized fields, language, etc.
We don't expect mechanical engineers read latest research on fluid mechanics, say, making use of Navier-Stokes equations. I am a mechanical engineer with a graduate degree in another field and I would be immediately lost if I tried to read such an article. So why do we expect this from software engineers?
> active science communication has been sparse in the area of software research, and those who have tried often find their efforts unrewarded or unsuccessful.
The authors suggest:
> Identify your target audience to tailor your message! Use diverse communication channels beyond papers, and actively engage with practitioners to foster dialogue rather than broadcasting information!
What I would emphasize is that many researchers just don't know how to do it. It isn't as simple as just thinking up a target audience and churning out a blog post. If you are the median researcher, ~0 people will read that post!
I think people underestimate:
- How hard it is to find the right target audience
- How hard it is to understand the target audience's language
- How hard it is to persuade the target reader that this work you've done should matter even a little to their work, even when you designed it specifically for them
- How few people in the audience will ever understand your work well
- How narrow your target audience should be
I also think many researchers want to be able to, if not as a primary career goal then at least as a fulfilling, public service type activity. Currently testing this out a bit (more: https://griffens.net).
This assumes a wide audience, but that tends not to be the case. Say you have a paper on some sort of database optimization. How many people are genuinely working on database optimizers in industry? Even a quite successful social media post has low odds of reaching them.
15 comments
[ 3.2 ms ] story [ 38.0 ms ] threadWe need more New Scientist type magazine like things that do decent round ups of scientific findings for various fields that do a good job of shuffling through the thousands of papers a month and finding the highest impact papers. The pipeline from research to use in professions can drastically be improved. At the moment you end up having a hosepipe of abstracts and its a lot of time to review that daily.
> Thanks to software research, we know that most code comprehensibility metrics do not, in practice, reflect what they are supposed to measure.
Linked research doesn't really agree. But if it did, so?
If comprehensibility is not a simple metric then who's got a magic wand to do the fancy feedback? Sounds like it'd take a human/AGI which is useless, that's why we have metrics.
Are any real programmers who produce things for the world using comprehensibility metrics or is it all the university fakers and their virtual world they have created?
If this is their 'one example' it sucks.
Till today I still share with my coworkers this 15yo article from Microsoft Research:
https://www.microsoft.com/en-us/research/blog/exploding-soft...
https://youtube.com/@twominutepapers?si=hyvCvW4UwS0QBbrZ
Here's our "Sora's Annotated Diffusion Transformer" writeup (code+paper side-by-side) Link: https://leetarxiv.substack.com/p/the-annotated-diffusion-tra...
It would be great to see an updated edition.
Do you know a better source of information?
The only others who compare are his contemporaries, Steve McConnell, Timothy Lister, Tom DeMarco, and Barry Boehm.
Unfortunately, they're all basically retired. It feels like this kind of interest in software development, at least the publishing, ended around the mid-2000s.
My guess is the shift to blogs from books, adoption of Agile (in whatever form), and a shift in industry focus to getting rich rather than getting good ended the efforts to come up with resources like Glass put together.
Based on this and other articles (and on experience), it's an especially underutilized resource. By reading it, you would gain an advantage over competition. Why aren't you using this advantage that is there for the taking?
And why don't we see papers posted to HN?
The expectation that a practicing CS graduate, even with a master's degree, should be able to read, understand, and apply in their work research articles published in academic journals is not very meaningful.
Not because they are not capable people, but because research articles these days are highly specialized, building upon specialized fields, language, etc.
We don't expect mechanical engineers read latest research on fluid mechanics, say, making use of Navier-Stokes equations. I am a mechanical engineer with a graduate degree in another field and I would be immediately lost if I tried to read such an article. So why do we expect this from software engineers?
The authors suggest:
> Identify your target audience to tailor your message! Use diverse communication channels beyond papers, and actively engage with practitioners to foster dialogue rather than broadcasting information!
What I would emphasize is that many researchers just don't know how to do it. It isn't as simple as just thinking up a target audience and churning out a blog post. If you are the median researcher, ~0 people will read that post!
I think people underestimate:
- How hard it is to find the right target audience - How hard it is to understand the target audience's language - How hard it is to persuade the target reader that this work you've done should matter even a little to their work, even when you designed it specifically for them - How few people in the audience will ever understand your work well - How narrow your target audience should be
I also think many researchers want to be able to, if not as a primary career goal then at least as a fulfilling, public service type activity. Currently testing this out a bit (more: https://griffens.net).
How to Have Real-World Impact: Five “Easy” Pieces - https://emeryberger.medium.com/how-to-have-real-world-impact...