Great book by a great teacher. I read this during my masters and was my route into machine learning.
David did many great things but sadly passed away too young in 2016, surely would have continued to have a great positive impact in the world had he lived longer. His other projects included the sustainable energy book https://www.withouthotair.com/ and information theory inspired keyboard software for people with severe disabilities https://www.bltt.org/software/dasher/
I had the honour of being a PhD student in Cambridge during his last year of life, and saw him give talks a few times in our lab. It's truly a remarkable experience to see someone who is a passionate explainer in action. Like how I imagine it must have been to see Feynman in the flesh.
(Writing this post feels wrong because he's had such a positive impact on my life and those of many others, and this post definitely can't do it justice)
His books are fantastic and his writing is captivating. It's such a shame he passed -- cancer is a bitch, but the blog post about the radiators in the hospice that he died in are one of the most heart rending pieces of writing I've come across from a mathematician.
As a follow on from Sustaianble Energy he also drove the original version of the my2050 game, which communicates the tradeoffs we need to make in decarbonizing to the general public very well. More recent update here https://my2050.beis.gov.uk/?levers=433233211331111
I've taken this as inspiration for my own transport models, though I have a model with some sliders to drag around I don't have a flashy game yet!
Tried Googling "mackay site:news.ycombinator.com" and found a lot of times where this book has been mentioned; I didn't see anything that clearly said "I am David Mackay" but that might be a good start.
(You'll also find the occasional reference to Charles Mackay's book Extraordinary Popular Delusions and the Madness of Crowds.)
I did some searches but couldn't find anything. If someone else does and the thread is closed for replying, let me know at hn@ycombinator.com and I'll reopen it.
one of my favorites (even though the pdf is free i keep a copy of the book). very clear introductory chapters in probability leading into a really nice blend of applied math and signal processing, finishing out with neural networks.
his father published was something of a figure in the fields of perception and cognition as well.
I attended an amazing talk by David circa 2006 at Princeton where he demoed his probabilistic text entry system called Dasher [1,3]. I think it was made into an iOS app later.
I also fondly recall his videos on Gaussian processes - best explanation on GPs I ever came across. [2]
Cybernetics is alive the same way punk is alive: people say it's dead, but then there are punk bands and punk records and just like a truckload of punks. This book, like those written by his dad, is cybernetic theory in everything but name. I absolutely adore it.
His books have taught me soooo much; in addition to information theory, I have learnt many tricks of LaTeX typesetting (the info theory book's LaTeX source is open) and how to estimate (from his other book on sustainable energy)
David was a fantastic ultimate frisbee player. He taught me how to properly throw forehand and overhand.
"My work has depended on the generosity of free software authors. I wrote
the book in LATEX 2ε. Three cheers for Donald Knuth and Leslie Lamport!
Our computers run the GNU/Linux operating system. I use emacs, perl, and
gnuplot every day. Thank you Richard Stallman, thank you Linus Torvalds,
thank you everyone"
31 comments
[ 2.6 ms ] story [ 81.7 ms ] threadDavid did many great things but sadly passed away too young in 2016, surely would have continued to have a great positive impact in the world had he lived longer. His other projects included the sustainable energy book https://www.withouthotair.com/ and information theory inspired keyboard software for people with severe disabilities https://www.bltt.org/software/dasher/
I had the honour of being a PhD student in Cambridge during his last year of life, and saw him give talks a few times in our lab. It's truly a remarkable experience to see someone who is a passionate explainer in action. Like how I imagine it must have been to see Feynman in the flesh.
This is a great short video to get a feel for his style of explaining: https://youtu.be/UR8wRSp2IXs
(Writing this post feels wrong because he's had such a positive impact on my life and those of many others, and this post definitely can't do it justice)
I've taken this as inspiration for my own transport models, though I have a model with some sliders to drag around I don't have a flashy game yet!
It's been a privilege to see him lecturing at the Cavendish in 2002.
Here's his last video interview: https://www.youtube.com/watch?v=sCyidsxIDtQ
I will also note the author made available the (non compilable) latex source [1] of the text for easy searching!
[1]http://www.inference.org.uk/mackay/itprnn/booktext.txt
Information Theory, Inference, and Learning Algorithms (2003) [pdf] - https://news.ycombinator.com/item?id=11289488 - March 2016 (9 comments)
Information Theory, Inference, and Learning Algorithms (free ebook edition) - https://news.ycombinator.com/item?id=852562 - Sept 2009 (21 comments)
(You'll also find the occasional reference to Charles Mackay's book Extraordinary Popular Delusions and the Madness of Crowds.)
“Marginalization is a macho activity, appropriate only for those fluent in definite integration”
http://www.inference.org.uk/itprnn/book.pdf#page=331
his father published was something of a figure in the fields of perception and cognition as well.
http://videolectures.net/course_information_theory_pattern_r...
...Okay?
I found the videos on youtube as well: https://www.youtube.com/watch?v=BCiZc0n6COY&list=PLruBu5BI5n...
> Ok, so, we have an answer of 1000 plus or minus 30 thanks to "Mr. Binomial", the inventor of the distribution that bears his name.
Lol, this is great.
[1]: https://www.youtube.com/watch?v=BCiZc0n6COY&t=971s
I also fondly recall his videos on Gaussian processes - best explanation on GPs I ever came across. [2]
[1] https://www.inference.org.uk/dasher/
[2] https://www.youtube.com/watch?v=NegVuuHwa8Q
[3] https://dasher.acecentre.net/about/
EDIT: Fixed the youtube link in [2], added [3].
David was a fantastic ultimate frisbee player. He taught me how to properly throw forehand and overhand.
"My work has depended on the generosity of free software authors. I wrote the book in LATEX 2ε. Three cheers for Donald Knuth and Leslie Lamport! Our computers run the GNU/Linux operating system. I use emacs, perl, and gnuplot every day. Thank you Richard Stallman, thank you Linus Torvalds, thank you everyone"