It's complicated. The Army Corps of Engineers has had a civilian mandate to support flood control prevention since 1917 [1]. Beyond that, they are also involved in large public works projects such as the building of…
Ilan Schnell is not "some guy". He's the original primary author of the Anaconda distribution. One of the main reasons that so many data scientists use Python.
NumPy is a library that provides typed multidimensional arrays and functions that run atop them. It does provide a built-in LAPACK/BLAS or can link externally to LAPACK/BLAS, but that's a side effect of providing typed…
Except that Docker containers play terribly with virtualization solutions. Still, some sort of configuration/infrastructure-as-code would go a long way.
This is a really interesting article and I'm glad this is getting attention. It's especially refreshing to see a theoretical treatment bridging algorithms with more "modern" hardware implementations. One opportunity I…
I don't think this would work for a number of reasons. If it's a database that you're modifying, you can see that a lot of operations (increment, delete, etc...) will do the wrong thing if they're called twice. If the…
Awesome, I'm a huge fan of new and innovative tools that help improve the process of refactoring and improving existing code. This looks like a really promising tool for Ruby developers, and I'm always grateful when…
Shiny, and I love the interface/layout! This looks a lot like the Jupyter/IPython Notebook, which is a free and open source "scientist's notebook". If you're interested in mixing LaTeX, Markdown, and code from almost…
Have you looked at conda and http://anaconda.org? We spent a lot of time curating the most important Python packages for data science into the Anaconda distribution, and conda packages are a great format for…
Thanks, I clearly wasn't being careful. I'll update my Gist... edit: On reviewing, I think the intent of the original blog post was to modify images in place (or at least to do it as quickly as possible with in-place…
Agreed, for some reason when I was looking at this last night I thought I couldn't use broadcasting, I've added the example to the gist: https://gist.github.com/ahmadia/c1f8be119f3cb2d2b8e5 Would you believe that Numba…
Great post, William. I really appreciated your exposition on both the challenges you folks were facing and your solution to the problem. As a few others have pointed out, sending a lambda function through NumPy is…
Yes, I'm thinking of discretizations of elliptic 2D/3D PDEs. They don't scale in the weak or strong sense, and they can't hold O(n log n) asymptotic complexity due to fill-in from Cholesky/LU-style factorizations.
That's because direct solvers can't scale. If you want to solve a large (distributed over hundreds of nodes) sparse linear algebra problem as fast as possible, decades of research have been poured into efficient…
Hi Niels. I have contacts within the DoD HPCMP and have run Python 2.7 on several of the heavy machines available on the Army side of the fence. I work for Continuum, which provides both a free and a commercial Python…
Let's hope this is not just bluster :(
I believe "conda install seaborn" will do what you need :)
Yes, but it's not ideal. Since everything is done through Javascript, we have to first identify all the math, then hide it from the Markdown parser, then parse the Markdown, then restore the math, then call MathJax on…
I agree, this is the first thing that jumped out at me when I was looking at the landing page. I don't know of quantitative evidence that this sort of thing causes harm, but I don't see any harm in mixing it a bit.
Yeah, Kyle is a rock star!
Yup, thanks for the comment. I think FPGA has interesting potential for sequencing (and we need more sequencing compute), I'm not sure what's ahead for the simulation machines.
It's worth pointing out that this is what most people think of as the traditional "Linux Operating System" but sans the Linux kernel. Also, there are a few components on these supercomputers that you won't find on a…
Unfortunately, NumPy uses deep knowledge of the CPython API in quite a few places, which is one of the reasons implementing NumPyPy has been so challenging.
edit: Jack Poulson's Elemental, and Andreas Waechter's IPOPT are also written in C++, and are both very important libraries within scientific computing and optimization, to add a few more examples of the usefulness of…
If you want/need maximum performance, why are you programming in a "high-level language"? Many of the highest-performance scientific computing libraries are written in assembly or use inline assembly instructions. The…
It's complicated. The Army Corps of Engineers has had a civilian mandate to support flood control prevention since 1917 [1]. Beyond that, they are also involved in large public works projects such as the building of…
Ilan Schnell is not "some guy". He's the original primary author of the Anaconda distribution. One of the main reasons that so many data scientists use Python.
NumPy is a library that provides typed multidimensional arrays and functions that run atop them. It does provide a built-in LAPACK/BLAS or can link externally to LAPACK/BLAS, but that's a side effect of providing typed…
Except that Docker containers play terribly with virtualization solutions. Still, some sort of configuration/infrastructure-as-code would go a long way.
This is a really interesting article and I'm glad this is getting attention. It's especially refreshing to see a theoretical treatment bridging algorithms with more "modern" hardware implementations. One opportunity I…
I don't think this would work for a number of reasons. If it's a database that you're modifying, you can see that a lot of operations (increment, delete, etc...) will do the wrong thing if they're called twice. If the…
Awesome, I'm a huge fan of new and innovative tools that help improve the process of refactoring and improving existing code. This looks like a really promising tool for Ruby developers, and I'm always grateful when…
Shiny, and I love the interface/layout! This looks a lot like the Jupyter/IPython Notebook, which is a free and open source "scientist's notebook". If you're interested in mixing LaTeX, Markdown, and code from almost…
Have you looked at conda and http://anaconda.org? We spent a lot of time curating the most important Python packages for data science into the Anaconda distribution, and conda packages are a great format for…
Thanks, I clearly wasn't being careful. I'll update my Gist... edit: On reviewing, I think the intent of the original blog post was to modify images in place (or at least to do it as quickly as possible with in-place…
Agreed, for some reason when I was looking at this last night I thought I couldn't use broadcasting, I've added the example to the gist: https://gist.github.com/ahmadia/c1f8be119f3cb2d2b8e5 Would you believe that Numba…
Great post, William. I really appreciated your exposition on both the challenges you folks were facing and your solution to the problem. As a few others have pointed out, sending a lambda function through NumPy is…
Yes, I'm thinking of discretizations of elliptic 2D/3D PDEs. They don't scale in the weak or strong sense, and they can't hold O(n log n) asymptotic complexity due to fill-in from Cholesky/LU-style factorizations.
That's because direct solvers can't scale. If you want to solve a large (distributed over hundreds of nodes) sparse linear algebra problem as fast as possible, decades of research have been poured into efficient…
Hi Niels. I have contacts within the DoD HPCMP and have run Python 2.7 on several of the heavy machines available on the Army side of the fence. I work for Continuum, which provides both a free and a commercial Python…
Let's hope this is not just bluster :(
I believe "conda install seaborn" will do what you need :)
Yes, but it's not ideal. Since everything is done through Javascript, we have to first identify all the math, then hide it from the Markdown parser, then parse the Markdown, then restore the math, then call MathJax on…
I agree, this is the first thing that jumped out at me when I was looking at the landing page. I don't know of quantitative evidence that this sort of thing causes harm, but I don't see any harm in mixing it a bit.
Yeah, Kyle is a rock star!
Yup, thanks for the comment. I think FPGA has interesting potential for sequencing (and we need more sequencing compute), I'm not sure what's ahead for the simulation machines.
It's worth pointing out that this is what most people think of as the traditional "Linux Operating System" but sans the Linux kernel. Also, there are a few components on these supercomputers that you won't find on a…
Unfortunately, NumPy uses deep knowledge of the CPython API in quite a few places, which is one of the reasons implementing NumPyPy has been so challenging.
edit: Jack Poulson's Elemental, and Andreas Waechter's IPOPT are also written in C++, and are both very important libraries within scientific computing and optimization, to add a few more examples of the usefulness of…
If you want/need maximum performance, why are you programming in a "high-level language"? Many of the highest-performance scientific computing libraries are written in assembly or use inline assembly instructions. The…