That's some doggedly determined back tracing to uncover an unexpected heisenbug (loose meaning).
So a total of 46% of the crashes were due to this rogue force-unload of a DLL. This is a case of bucket spray, where a single underlying cause generates a large number of different types of crashes.
I see posts like this, this deep dive into the call stacks and am always humbled and reminded of the limits of my knowledge about computers and programs.
What MSFT support policy do you need to have the legendary Raymond Chen take a look at it?
I say this because we've reported a bunch of Windows bugs (mainly running Windows under virtualization) and getting them to pay attention at all is an up-hill battle.
>I asked for the 100 most recent crashes in that third party program and put them into a pivot table so I could see the distribution.
Always wondered if crash reporting is some kind of shady business. It's good to know it does, at minimum, do what it promises and give valuable crash data to MS.
The fact that Raymond Chen is debugging these kind of issues, tells me Microsoft is short on staff that has his particular set of skills, handing him the hairiest issues from the annals of Windows. The new hires are probably all about .NET and JavaScript and what have you -- whatever Microsoft is about these days. I doubt it's C/C++. Chen is probably on standby and is paid handsomely as a de-facto VIP consultant. He is a legend, but he's becoming somewhat of a vintage developer.
I do wonder how Microsoft will manage the transition of the NT generation. Raymond Chen has been doing this kind of work for thirty years. He probably has, what, another ten years, max? Who are the next generation of Windows gurus that will take up the mantle?
Hanselman is good on the blog part, but not in Chen's class re Windows domain expertise. And Windows is not, despite all that is said, going away anytime soon. I think this could be a real problem.
Hopefully there is a set of 25 year old developers in the Windows team who have deep and growing skills in Winternals, and Microsoft have the good sense to encourage them in their career.
These were my thoughts more or less, but how many Chens does the world's largest software company need to at least maintain the now "legacy" code before they find a way not to need them any longer. Maybe they're betting on large language models having trained on the cumulative output of experts like Chen, but I think that's fit for contingency situations -- maintaining existing code and making sure it doesn't break the bottom-line -- but for all the praise of AI I have my doubts Microsoft are crazy enough to just leave it to AI to develop Windows containing hundreds of thousands of SLOC of the kind only Chen can effectively debug.
As for 25 year olds -- I am sure there will always be those around, people have been pretty consistently varied in their pursuits, but it going to be so niche they'll be like COBOL/FORTRAN developers hired by the banks today -- far and in between and paid so handsomely they can pick and choose consultancy any day of the week anywhere in the world. So I guess good for them, but again -- can Microsoft depend on that form of provision of labour they still need (and will need for a few years ahead at least)?
13 comments
[ 4.6 ms ] story [ 33.3 ms ] threadThe story of software development through the ages.
https://devblogs.microsoft.com/oldnewthing/20260626-00/?p=11...
I say this because we've reported a bunch of Windows bugs (mainly running Windows under virtualization) and getting them to pay attention at all is an up-hill battle.
Always wondered if crash reporting is some kind of shady business. It's good to know it does, at minimum, do what it promises and give valuable crash data to MS.
Hanselman is good on the blog part, but not in Chen's class re Windows domain expertise. And Windows is not, despite all that is said, going away anytime soon. I think this could be a real problem.
Hopefully there is a set of 25 year old developers in the Windows team who have deep and growing skills in Winternals, and Microsoft have the good sense to encourage them in their career.
As for 25 year olds -- I am sure there will always be those around, people have been pretty consistently varied in their pursuits, but it going to be so niche they'll be like COBOL/FORTRAN developers hired by the banks today -- far and in between and paid so handsomely they can pick and choose consultancy any day of the week anywhere in the world. So I guess good for them, but again -- can Microsoft depend on that form of provision of labour they still need (and will need for a few years ahead at least)?