HN Explorer

8 comments

[ 3.6 ms ] story [ 26.5 ms ] thread
minimax
Cool project. Can you say how much performance improvement you are getting with dSFMT versus a naive C loop?

Another fast monte carlo anecdote: at Bloomberg they are doing this sort of thing with GPUs.

http://www.wallstreetandtech.com/it-infrastructure/bloomberg...

chrisconley
I'm trying to parse my HipChat conversation with our Architect (who did most of the speed improvements) and it looks like going from Python to naive C loop was a 200x improvement (though that's not a direct comparison of course). Going from naive C to dSFMT we went from one particular simulation taking 14:30 down to 2:30 (minutes).
marmaduke
You could have stayed in Python and used PyOpenCL or Numba or weave inline or f2py or...

What is the real reason to use C here?

[deleted]
(comment deleted)
chrisconley
Interesting, I'll have to take a look at those.

The real reason is that it was just an idea that we came up with and decided to try it. It worked well, so we stuck with it. :)

mrcactu5
Is the random number generator the bottleneck for significance-testing experiments like this?

Monte-Carlo is more meaningful than using some sort of chi-squared. But why is it faster?

sanskritabelt
From: "If we see that 1000 of the 10,000 random iterations had a difference of more than $1.50, we can say that there is a 10% chance that our $1.50 observed difference was due to randomness."

I'm wondering if maybe you're doing way too many simulations in the calibration, do you really need more than a few hundred to a thousand or so? The 0.1 quantile is reasonably well separated from 0, and I would have expected you'd get "good enough" convergence pretty quickly.

Also: "we can compare our computed p-value of 11% to our simulated 10% result to determine whether or not the model is accurate enough." you're getting a full pdf out of the simulations, are you also comparing to the full distribution of your test statistics?