I want to learn/build low level software with other people but I don't know any
I have a PhD in applied physics. All I used during grad school was Python and MATLAB (yes, MATLAB).
I barely know enough Linux to scrape by at my current job. I don't know why data sometimes has to be aligned to cache lines or what cache lines even are.
I try to learn these things in my spare time but it's a real struggle. It's so easy to just watch TV or scroll reddit instead of reading a 300 page book on Linux.
It was so much easier in school when I was surrounded by amazing people who all wanted to learn. I don't know where to find such people. I've tried going to a lot of networking events to meet CS grads but I haven't found anyone who's interested in the nitty gritty details of Linux. Or in optimizing some C code as much as possible.
I don't know how to meet those people. Any ideas?
7 comments
[ 2.7 ms ] story [ 28.7 ms ] threadin any case, what about going to a conference? or joining a local group on meetup.com?
Sharing what I've learned or helping someone else or working together is a strong motivator for me. I am interested in the problems but it's easy for me to get distracted and say 'no one cares' if I'm doing it alone.
> I don't know why data sometimes has to be aligned to cache lines or what cache lines even are.
> I try to learn these things in my spare time but it's a real struggle.
Well the good news is you don't need to learn this in your spare time, because it's your job, so you can learn it on the clock?
Take a look at Inside the Machine [1] to get an in-depth, but accessible, understanding of how a somewhat dated Intel processor works, then read the optimization manual for the processor you use, and you'll have a much better foundation for your work.
Figure out how to use Intel vTune or similar, and you'll be able to get a good idea of what your system is doing. Read posts about Netflix's CDN machines, which are very high performance networking machines; you'll absorb a lot of information and terminology, hopefully.
Sorry, that doesn't help you meet people, I'm a curmudgeon and don't have advice there. Cultivate your network of people you'ld like to bounce ideas off of? But keep in mind, lots of people want to leave work at work.
[1] https://www.amazon.com/Inside-Machine-Introduction-Microproc...
I also need to deliver code on the clock. With layoffs all around, it's not great to be learning on the job..
Thank you for these resources!
Search online for "software optimisation techniques" and you will get lot of tutorials on how to optimise software such as data structure packing, locality, cache optimization, loop unrolling etc.
For self-study, as others have mentioned, keep the manuals of CPU as well as network devices you are using handy. It will help you understand the low level code in the kernel as well as the related drivers.
For getting started with Linux kernel development, start at these places:
https://kernelnewbies.org
https://www.linuxfromscratch.org
https://lwn.net/Kernel/LDD3 (outdated but still helps build your frame of mind navigating the device driver code, in your case network drivers)
https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/lin...
https://beej.us/guide/bgnet
https://man7.org/training/download/Linux_System_Programming_...
https://igm.univ-mlv.fr/~yahya/progsys/linux.pdf
https://users.cs.cf.ac.uk/Dave.Marshall/C
https://www.dpdk.org
For some fun way of understanding about computer architecture in general, have a look at:
https://www.coursera.org/learn/build-a-computer
https://www.coursera.org/learn/nand2tetris2