Heres a review from one of the other books by pakt and this author:
If you are an inexperienced programmer or new to Python/Jupyter/Anaconda DO NOT BUY THIS OR ANY OTHER PACKT Publishing book as the code contains errors that are difficult to rectify. Packt Publishing DOES NOT verify code like the CRC Press - for instance, Statistical Rethinking by CRC press - i.e. Bayesian Analysis in R - has code AND excellent content that is helpful for both the academic and practitioner. Also, the Packt Publishing description/outline of probability is weak, at best, and is confusing to many of my students.
If correcting Python code is not a big deal for you then Packt books are a nice intro. But why buy something that you have to fix before you can start working with it?
This is a December 2018 update - DO NOT BUY THIS OR ANY OTHER PACKT PUBLISHING BOOK UNLESS YOU CAN VERIFY THERE IS AN ERRATA FILE TO ACCOMPANY IT. PACKT PUBLISHING DOES NOT PRODUCE RELIABLE TEXTS. MY STUDENTS HAVE HAD A TERRIBLE TIME WITH THIS BOOK – THE COMPANY HAS NOT RESPONDED TO ANY QUESTIONS/REQUESTS - THE UNIVERSITY WHERE I TEACH IS NOW LOOKING CLOSELY AT THE VALIDITY OF THESE PUBLICATIONS.
I want to second the recommendation for Statistical Rethinking from that review, it’s one of the best books ever written on computational Bayesian inference.
However, it is R based, and therefore arguably not an alternative to the book we are discussing here.
(I think the author of the book discussed above, Osvaldo Martin, is the primary or sole contributor for the Rethinking implementations, in fact -- he had a full implementation in his own repo (https://github.com/aloctavodia/Statistical-Rethinking-with-P...) before deprecating it in favor of the above-linked one.)
Agreed - that book is phenomenal at communicating the theory and mindset of Bayesian analysis. My favorite quote from the lectures - "Statistics should be subbordinate to Science." A very healthy emphasis on logic and proper model design - there will be no shoving data into black boxes!
R is a good language for this but he uses a library of convenience functions that are not on CRAN and are effectively just for educational purposes. So even if you want to stick with R, you'll need to translate your code into production-ready libraries anyway. There are several nearly-complete translations based on other R packages as well as ported to Julia and Python.
Or you could just put his code into production (source: I worked at a super successful gaming startup that had this code scattered all over their repo).
They had a bunch of other code problems too, but that was definitely the weirdest thing I saw in my (very short) stay there.
You are cheating yourself if you only read the book. His lecture series covering the material on Youtube is phenomenal.
I went through the book + season 1 videos, and had a glance at some of the season two videos. The season two has some visuals that made some intuition click for me.
Agree about Pakt, but not about $50 self publish per se (nor the tenured prof remark). I am in the process of writing a code book, but it is compiled via quarto so at least you know the code works as expected.
The author was also a lead author on a CRC red-series book, Bayesian Modeling and Computation in Python Learning, published a few years ago (short review here):
I think that price point is fine. Self publishing has a lot of variance of course, but I would not categorically say that 50 dollars is too high. We as potential readers just need to do our due diligence, checking reviews etc.
A book not worth 50 USD is not worth 30 or even 20 in my opinion - it is the wanted time spent reading that is the main issue.
Both actively pursue potential authors. I know this because I’ve been contacted by both. Made me realize I have neither the will nor the patience to write a book - being an author sounds great on paper (pun intended).
Manning seems fine to me. I've read a few books from them, from quite good to blah.
I've read one book from Packt that was quite good; they've also published literal plagiarism (which they withdrew after the original author, a friend, tweeted about it -- this wouldn't have come to my attention otherwise). Can't recall seeing another that interested me, flipping through the pages.
Think Bayes by Allen Downey is an amazing and free course teaching Bayesian statistics in Python. Save yourself the $50 and try it first:
https://allendowney.github.io/ThinkBayes2/
Thanks for posting this as it seems very interesting. Did you go through the whole book/course? the preface claims that “You don’t need to know calculus or linear algebra. You don’t need any prior knowledge of statistics”. I am always a bit skeptical when i read these claims as the texts tend ultimately to be a bit superficial; it would be nice to have an informed opinion about it before starting it out :-) thanks in advance.
Yeah I have completed it, and I really like his teaching style. I’d say it might be a little tough if you have absolutely no knowledge of statistics. But as someone who wanted to do a deeper dive into Bayesian stats specifically I found it a good resource. The reason I ended up doing it was precisely because I felt that the section on bayes in most stats courses is too superficial
I worked through large chunks of the book a long time ago (perhaps an earlier version). I agree with the other responder-- if you've had zero stats it may be more difficult, but the author does an excellent job of discretizing almost everything, which means that integrals and derivatives are replaced with adding and subtracting. For this book I think the most advanced math you need is multiplication and division. It's a clever trick that relies on the fact that computers are really good at multiplying 10,000 things by 10,000 other things and adding them up (which is what discretization implies here).
Before computers that was much harder to do so you needed clever math tricks to "do it all at once," which gets you to all the analytical methods -- calculus, etc. Still all very useful! Just hard to teach to a broad audience. Downey really leans into the technical advantage to computers provide and thus can cover a lot of material before getting into complicated math.
> the preface claims that “You don’t need to know calculus or linear algebra. You don’t need any prior knowledge of statistics”.
Downey makes this statement because his premise is that if you know Python, you can use that knowledge to learn the concepts without the pure math approach.
For example, he will do iteration to calculate integrals rather than teaching integration. And he will plot statistical distributions using a Python library to explore and teach the statistical concepts.
He also has a book called “Think Statistics” that is excellent in this regard.
It is a challenging book. But it is an introductory level material on Bayesian statistics. One can teach an undergrad course from the book. IMO, doing computation along the way makes the process enjoyable. Stan / julia-turing / pymcmc all good tools.
In my experience, PyMC leads to models that are orders of magnitudes slower than equivalent models written in JAGS. Profiling is also extremely tedious, and there is no section in the PyMC docs that touches upon model performance.
I really like PyMC's API, but as soon as you move towards bigger datasets JAGS or Stan seem to be the only practical options.
PyMC has a pretty active community where we help people with questions like these. Its hard to come up with one doc as folks have different hardware, datasets, models etc. I hope though you'll the devs and community to be friendly though!
For models with >100 parameters, there are theoretical reasons for why JAGS can fail badly. It has to do with the mixing time of Gibbs samplers versus Hamiltonian Monte Carlo.
I recently found the Statistical Rethinking course on YouTube. I found it super refreshing in its very applied focus, direct advice and thorough pedagogy. It assumed that one has a little bit of stats and Bayesian thinking from before, but I found it accessible with just basic introduction to stats in engineering and ML.
Relatedly, I think for introductory Bayesian analysis, Statistical Rethinking is the best book. It uses R code but you will find GitHub repositories for translated Python code.
In the R ecosystem, brms is currently the best (in my opinion) package for people who want to learn Bayesian analysis. Here is an online free book that translated all the original R code to brms code: https://bookdown.org/content/4857/
It's a great prequel to Gelman et al. Bayesian Data Analysis (BDA). I have a formal background in Bayesian inference, I do statistics for a living and I still learned many things from ROS. Yet, it is totally approachable for beginners.
Statistical Rethinking is something you can read after ROS, before BDA. Personally, I think it is a great book, but it's not a good first book and I think it would benefit from some editing. For example, chapter titles are funny, but totally uninformative if you are browsing the book. Nonetheless, I still think it's a great read.
While I'm mostly happy with this book. Yet there's also many improvements we can make. We're compiling suggestions for how to make a second edition better so if you have any please share! We also recently updated the code to the newest version of PyMC.
I also use(d) Bayesian stats extensively in practice at SpaceX, sweetgreen, now Google. It's powerful stuff, it was absolutely crucial in many analyses I did. It's my work at SpaceX that really made me appreciate Bayesian statistics.
To respond to some comments below.
* Osvaldo is a tenured professor, not an amateur.
* PyMC can compiles to Jax and rust backends now speeding things up. Stan, Numpyro and other PPLs are fantastic too pick what works for you
* There are many ways to learn Bayesian stats these days. Statistical Rethinking and BDA3 are great. We felt like there was a gap which is why we wrote another book. CRC agreed which is why they published it. Pick what works best for you.
I did not write this Packt book with Osvaldo but I do think the material itself is great and it fits a certain audience.
If you have questions about learning or using Bayesian stats, I'm happy to answer them!
Typically people have chosen one book, enjoyed it to some extent, and praise it, but are not well posed to comment on comparative advantages of each book; at most they have skimmed the rest.
You on the other hand have made a sizable time and effort (and possibly money?) investment on writing a book, and you stated that you made that decision after identifying the gap, so I assume that you have a very grounded opinion on the differences / points of view / strengths and weaknesses of each of those books?
It would be great if you could expand a little bit more about each of them.
For recommendations here's my suggestions split by persona
* I want a PHD in statistics or to write novel research - Go read BDA3 cover to cover. Gelman etal are amazing, and amazingly brilliant. The book is dense though, even after years of experience I need to now read chapters 2 or 3 times and write out proofs
* I want have a smooth on ramp into Bayesian stats with lots of code and beautiful writing. I also like video lectures and undergraduate college courses - Statistical Rethinking. As noted below Richard does a wonderful job explaining these concepts with metaphors like golems, interspersing it with this experience as an anthropologist, and using his teaching experience to write well structured introduction into Bayesian stats. This book does assume you understand basic statistics and probability theory.
* I'm a programmer type person that likes hands on "build from scratch" using code - Allen Downey's think bayes builds up bayes theorem from Numpy arrays. He's also a brilliant instructors
* Im an applied practitioner that is focusing more on my specific problem and I need to use the latest PPLs and code to get it done - This is my book. I had to make estimations in SpaceX supply chain with some quick deadlines and I didn't have time to take an undergrad course. I also needed my code to be robust, testable, and scalable. I didn't find that other books provided this so that's why I wrote this with CRC and Osvaldo. Osvaldo and I are heavy contributors to PyMC, ArviZ, Preliz and other libraries so naturally we take a code first approach.
https://bayesiancomputationbook.com/welcome.html
* (Shameless self promotion) Im a professional, I need to learn fast, and my company will pay for training - For this specific niche me and other Bayesian colleagues created an online course designed specifically for professionals. Yes it's expensive so let me plainly state no one at any point needs to spend any money to learn Bayesian stats. That being said hundreds of people have purchased this course and the feedback we've gotten on this course has been quite positive. So I want to underscore before Hacker news rips me apart. No one is being forced to buy this, if you want this style course here it is, if you don't there's many many ways to learn Bayesian stats.
rramadass↗
One thing I would love to read would be an archeological exposition of the internals of PyMC over its various versions and backends. This is not in anyway to indicate that its old and dated, just that its evolution interests me.
46 comments
[ 3.1 ms ] story [ 87.5 ms ] threadThats tenured professor textbook money.
Pakt publishing books can be written by anyone with even a surface level understanding of a topic.
If you are an inexperienced programmer or new to Python/Jupyter/Anaconda DO NOT BUY THIS OR ANY OTHER PACKT Publishing book as the code contains errors that are difficult to rectify. Packt Publishing DOES NOT verify code like the CRC Press - for instance, Statistical Rethinking by CRC press - i.e. Bayesian Analysis in R - has code AND excellent content that is helpful for both the academic and practitioner. Also, the Packt Publishing description/outline of probability is weak, at best, and is confusing to many of my students.
If correcting Python code is not a big deal for you then Packt books are a nice intro. But why buy something that you have to fix before you can start working with it?
This is a December 2018 update - DO NOT BUY THIS OR ANY OTHER PACKT PUBLISHING BOOK UNLESS YOU CAN VERIFY THERE IS AN ERRATA FILE TO ACCOMPANY IT. PACKT PUBLISHING DOES NOT PRODUCE RELIABLE TEXTS. MY STUDENTS HAVE HAD A TERRIBLE TIME WITH THIS BOOK – THE COMPANY HAS NOT RESPONDED TO ANY QUESTIONS/REQUESTS - THE UNIVERSITY WHERE I TEACH IS NOW LOOKING CLOSELY AT THE VALIDITY OF THESE PUBLICATIONS.
However, it is R based, and therefore arguably not an alternative to the book we are discussing here.
https://github.com/pymc-devs/pymc-resources
(I think the author of the book discussed above, Osvaldo Martin, is the primary or sole contributor for the Rethinking implementations, in fact -- he had a full implementation in his own repo (https://github.com/aloctavodia/Statistical-Rethinking-with-P...) before deprecating it in favor of the above-linked one.)
R is a good language for this but he uses a library of convenience functions that are not on CRAN and are effectively just for educational purposes. So even if you want to stick with R, you'll need to translate your code into production-ready libraries anyway. There are several nearly-complete translations based on other R packages as well as ported to Julia and Python.
They had a bunch of other code problems too, but that was definitely the weirdest thing I saw in my (very short) stay there.
I agree, it works beautifully and performance is ok for desktop use.
I went through the book + season 1 videos, and had a glance at some of the season two videos. The season two has some visuals that made some intuition click for me.
Now I see he has a 2023 playlist, which may yield even further improvements: https://www.youtube.com/playlist?list=PLDcUM9US4XdPz-KxHM4XH...
This is my favorite helpful advice for pricing, https://pubsonline.informs.org/doi/abs/10.1287/mnsc.2020.360.... Estimate max someone will buy, then divide by two (assuming no marginal cost, like e-book).
The author was also a lead author on a CRC red-series book, Bayesian Modeling and Computation in Python Learning, published a few years ago (short review here):
https://academic.oup.com/jrsssa/article/185/Supplement_2/S76...
It's a shame because there is good stuff in their catalogue, but it's littered with trash.
Both actively pursue potential authors. I know this because I’ve been contacted by both. Made me realize I have neither the will nor the patience to write a book - being an author sounds great on paper (pun intended).
I've read one book from Packt that was quite good; they've also published literal plagiarism (which they withdrew after the original author, a friend, tweeted about it -- this wouldn't have come to my attention otherwise). Can't recall seeing another that interested me, flipping through the pages.
Before computers that was much harder to do so you needed clever math tricks to "do it all at once," which gets you to all the analytical methods -- calculus, etc. Still all very useful! Just hard to teach to a broad audience. Downey really leans into the technical advantage to computers provide and thus can cover a lot of material before getting into complicated math.
Downey makes this statement because his premise is that if you know Python, you can use that knowledge to learn the concepts without the pure math approach.
For example, he will do iteration to calculate integrals rather than teaching integration. And he will plot statistical distributions using a Python library to explore and teach the statistical concepts.
He also has a book called “Think Statistics” that is excellent in this regard.
http://www.stat.columbia.edu/~gelman/book/
https://github.com/dbonattoj/cookiecutter-pystan
https://github.com/pyro-ppl/numpyro
PyMC now supports jax and numba as well, and can use the numpyro samplers. https://www.pymc.io/projects/examples/en/latest/samplers/fas...
Between all these tools we hope the modern bayesian can find the one that works best for them
I really like PyMC's API, but as soon as you move towards bigger datasets JAGS or Stan seem to be the only practical options.
https://discourse.pymc.io/t/how-to-increase-sampling-speed-w...
This is still my favorite statistics book. MacKay is a staunch Bayesian, but his writing is relaxed and witty.
http://www.inference.org.uk/itila/book.html
https://youtube.com/playlist?list=PLDcUM9US4XdPz-KxHM4XHt7uU...
In the R ecosystem, brms is currently the best (in my opinion) package for people who want to learn Bayesian analysis. Here is an online free book that translated all the original R code to brms code: https://bookdown.org/content/4857/
It's a great prequel to Gelman et al. Bayesian Data Analysis (BDA). I have a formal background in Bayesian inference, I do statistics for a living and I still learned many things from ROS. Yet, it is totally approachable for beginners.
Statistical Rethinking is something you can read after ROS, before BDA. Personally, I think it is a great book, but it's not a good first book and I think it would benefit from some editing. For example, chapter titles are funny, but totally uninformative if you are browsing the book. Nonetheless, I still think it's a great read.
While I'm mostly happy with this book. Yet there's also many improvements we can make. We're compiling suggestions for how to make a second edition better so if you have any please share! We also recently updated the code to the newest version of PyMC.
I also use(d) Bayesian stats extensively in practice at SpaceX, sweetgreen, now Google. It's powerful stuff, it was absolutely crucial in many analyses I did. It's my work at SpaceX that really made me appreciate Bayesian statistics.
To respond to some comments below.
* Osvaldo is a tenured professor, not an amateur.
* PyMC can compiles to Jax and rust backends now speeding things up. Stan, Numpyro and other PPLs are fantastic too pick what works for you
* There are many ways to learn Bayesian stats these days. Statistical Rethinking and BDA3 are great. We felt like there was a gap which is why we wrote another book. CRC agreed which is why they published it. Pick what works best for you.
I did not write this Packt book with Osvaldo but I do think the material itself is great and it fits a certain audience.
If you have questions about learning or using Bayesian stats, I'm happy to answer them!
Typically people have chosen one book, enjoyed it to some extent, and praise it, but are not well posed to comment on comparative advantages of each book; at most they have skimmed the rest.
You on the other hand have made a sizable time and effort (and possibly money?) investment on writing a book, and you stated that you made that decision after identifying the gap, so I assume that you have a very grounded opinion on the differences / points of view / strengths and weaknesses of each of those books?
It would be great if you could expand a little bit more about each of them.
https://ravinkumar.com/
and here's a video for the most common question I get about bayes vs frequentist
https://www.youtube.com/watch?v=foSPfzYs4yY
For recommendations here's my suggestions split by persona
* I want a PHD in statistics or to write novel research - Go read BDA3 cover to cover. Gelman etal are amazing, and amazingly brilliant. The book is dense though, even after years of experience I need to now read chapters 2 or 3 times and write out proofs
* I want have a smooth on ramp into Bayesian stats with lots of code and beautiful writing. I also like video lectures and undergraduate college courses - Statistical Rethinking. As noted below Richard does a wonderful job explaining these concepts with metaphors like golems, interspersing it with this experience as an anthropologist, and using his teaching experience to write well structured introduction into Bayesian stats. This book does assume you understand basic statistics and probability theory.
* I want a great comparison of Bayesian vs Frequentist stats- This book covers both topics well, compares them fairly, and has all the proofs to back things up https://www.routledge.com/Understanding-Advanced-Statistical...
* I'm a programmer type person that likes hands on "build from scratch" using code - Allen Downey's think bayes builds up bayes theorem from Numpy arrays. He's also a brilliant instructors
* I want to read about the history and people and politics - Bernoulli's fallacy, The Theory That Would Not Die:, and Probably Overthinking it are all "non mathy" great armchair readings https://cup.columbia.edu/book/bernoullis-fallacy/97802311999... https://www.amazon.com/Theory-That-Would-Not-Die/dp/03001882... https://www.allendowney.com/blog/
* Im an applied practitioner that is focusing more on my specific problem and I need to use the latest PPLs and code to get it done - This is my book. I had to make estimations in SpaceX supply chain with some quick deadlines and I didn't have time to take an undergrad course. I also needed my code to be robust, testable, and scalable. I didn't find that other books provided this so that's why I wrote this with CRC and Osvaldo. Osvaldo and I are heavy contributors to PyMC, ArviZ, Preliz and other libraries so naturally we take a code first approach. https://bayesiancomputationbook.com/welcome.html
* (Shameless self promotion) Im a professional, I need to learn fast, and my company will pay for training - For this specific niche me and other Bayesian colleagues created an online course designed specifically for professionals. Yes it's expensive so let me plainly state no one at any point needs to spend any money to learn Bayesian stats. That being said hundreds of people have purchased this course and the feedback we've gotten on this course has been quite positive. So I want to underscore before Hacker news rips me apart. No one is being forced to buy this, if you want this style course here it is, if you don't there's many many ways to learn Bayesian stats. rramadass ↗ Thank you very much!