Is "proof of vulnerability" a marketing term, or do you actually claim that XBOW has a 0% false positive rate? (i.e. "all" reports come with a PoV, and this PoV "proves" there is a vulnerability?)
This doesn't say anything about many false positives they actually have. Yes, you can write other programs (that might even invoke another LLM!) to "check" the findings. That's a very obvious and reasonable thing to do.…
I agree that peer review can be a strong filter, but it's a filter for claims and evidence that sound true. CS papers can and do hide important details in the code (details which, I argue, would get a paper rejected if…
I don't understand why took the time to leave (three?) personal attacks, rather than just provide your perspective? I'm willing to acknowledge that my opinion has limitations (I think it mainly applies to CS-adjacent…
I agree with you. But what part of the blog post, or the peer review process in general, do you think ensures that only true ideas get in front of eyeballs? I can write anything I want in the paper, but at the end of…
It's not too difficult to state any idea, even a surprising one. But often, papers with surprising ideas (or maybe the right thing to say is surprising results?) turn out to be wrong! I think it's still the case that…
Academics seem to have this fixation on "ideas": > And it’s not just a pace thing, there’s a threshold of clarity that divides learned nothing from got at least one new idea. But these days, ideas are quite cheap: in my…
>The real test of a paper is whether it is cited and built upon by other scientists after publication. Many papers are published and then forgotten, or found to be flawed and not used any more. This does seem true, but…
> No set of a bit different results or missing exact hyperparameter settings really invalidates the value of the aforementioned research. If this is the case, the paper should not include a performance evaluation at…
> by-and-large catches the most glaring faults. I did not dispute that peer review acts as a filter. But reviewers are not reviewing the science, they are reviewing the paper. Authors are taking advantage of this…
Peer review is not designed for science. Many papers are not rejected because of an issue with the science -- in fact, reviewers seldom have the time to actually check the science! As a CS-centric example: you'll almost…
They needed to be close to a large city center. Chicago is too expensive -- so the option you are proposing is something like Indianapolis?
Even large, damaging tornadoes have quite localized impacts (max of maybe a mile in path width) -- and you don't generally do much more than stay up to building code in order to prepare for one. In contrast, earthquakes…
Is "proof of vulnerability" a marketing term, or do you actually claim that XBOW has a 0% false positive rate? (i.e. "all" reports come with a PoV, and this PoV "proves" there is a vulnerability?)
This doesn't say anything about many false positives they actually have. Yes, you can write other programs (that might even invoke another LLM!) to "check" the findings. That's a very obvious and reasonable thing to do.…
I agree that peer review can be a strong filter, but it's a filter for claims and evidence that sound true. CS papers can and do hide important details in the code (details which, I argue, would get a paper rejected if…
I don't understand why took the time to leave (three?) personal attacks, rather than just provide your perspective? I'm willing to acknowledge that my opinion has limitations (I think it mainly applies to CS-adjacent…
I agree with you. But what part of the blog post, or the peer review process in general, do you think ensures that only true ideas get in front of eyeballs? I can write anything I want in the paper, but at the end of…
It's not too difficult to state any idea, even a surprising one. But often, papers with surprising ideas (or maybe the right thing to say is surprising results?) turn out to be wrong! I think it's still the case that…
Academics seem to have this fixation on "ideas": > And it’s not just a pace thing, there’s a threshold of clarity that divides learned nothing from got at least one new idea. But these days, ideas are quite cheap: in my…
>The real test of a paper is whether it is cited and built upon by other scientists after publication. Many papers are published and then forgotten, or found to be flawed and not used any more. This does seem true, but…
> No set of a bit different results or missing exact hyperparameter settings really invalidates the value of the aforementioned research. If this is the case, the paper should not include a performance evaluation at…
> by-and-large catches the most glaring faults. I did not dispute that peer review acts as a filter. But reviewers are not reviewing the science, they are reviewing the paper. Authors are taking advantage of this…
Peer review is not designed for science. Many papers are not rejected because of an issue with the science -- in fact, reviewers seldom have the time to actually check the science! As a CS-centric example: you'll almost…
They needed to be close to a large city center. Chicago is too expensive -- so the option you are proposing is something like Indianapolis?
Even large, damaging tornadoes have quite localized impacts (max of maybe a mile in path width) -- and you don't generally do much more than stay up to building code in order to prepare for one. In contrast, earthquakes…