Might not be my actual longest (and certainly isn't anywhere near the stories that I'm sure others here have to tell), but my favorite debugging problem was one I worked on a couple years ago. I had a Python service that was doing a bunch of number crunching, so I used Cython to speed up the math-intensive portions of the code. Added type definitions, referenced several C math functions from math.h, and it was working great on Windows. Set it up using the side-by-side file approach, which lets you leave your original Python files alone, and add all the type info in a ".pxd" file that has to match the exact name of the original ".py" file. So, if I had "FileA.py", I'd also have "FileA.pxd".
Unfortunately, when I went to run this on Linux, it compiled fine, but started throwing some weird exceptions buried deep down in the code. I hadn't actually written the math portion of this myself, but traced it pretty far down. Finally concluded it wasn't finding "cos" somehow, which didn't make any sense, because I was explicitly importing the C function from math.h if using Cython, and importing the normal Python function from the "math" module otherwise.
Took me about 6 hours of tracing, investigating, and steadily growing confusion before I finally figured out what was going on: there was a single letter casing mismatch between a .py file and its corresponding .pxd file. One had a capital 'O', the other had a lower-case 'o'. Worked fine on case-insensitive Windows, but under case-sensitive Linux, it never actually found the proper .pxd file, didn't get the right imports, and thus stuff eventually exploded.
Definitely a pretty high hours-of-work to number-of-characters-changed ratio :)
Longest so far has been two weeks spend single stepping assembly code and viewing memory output. Turns out in the latest OS release memory was not initialized to 0 which caused junk to get copied allowing the program to execute with out crashing but not give the expected results.
2 - 3 months where I was primarily "focused" on the issue which was ongoing for the place I worked at the time (ongoing being years and multiple people).
It was an embedded system problem and I didn't fix it and I don't know if it ever did get fixed - sometimes I wonder if it ever was figured out.
10 comments
[ 2.7 ms ] story [ 37.3 ms ] threadUnfortunately, when I went to run this on Linux, it compiled fine, but started throwing some weird exceptions buried deep down in the code. I hadn't actually written the math portion of this myself, but traced it pretty far down. Finally concluded it wasn't finding "cos" somehow, which didn't make any sense, because I was explicitly importing the C function from math.h if using Cython, and importing the normal Python function from the "math" module otherwise.
Took me about 6 hours of tracing, investigating, and steadily growing confusion before I finally figured out what was going on: there was a single letter casing mismatch between a .py file and its corresponding .pxd file. One had a capital 'O', the other had a lower-case 'o'. Worked fine on case-insensitive Windows, but under case-sensitive Linux, it never actually found the proper .pxd file, didn't get the right imports, and thus stuff eventually exploded.
Definitely a pretty high hours-of-work to number-of-characters-changed ratio :)
I still cannot display video on my Chromecast while continuing to play audio on my phone while using any given Chromecast app.
It was an embedded system problem and I didn't fix it and I don't know if it ever did get fixed - sometimes I wonder if it ever was figured out.