So a minor correction here. We did explore placing ftz on various instructions inside the matmul ops, but it turns out you don't need anything more than what is already baked into tf by default. All tf gpu primitives…
Actually if you read section 4.6 of that paper you'll see that denormals are the default on sm_20 and above. But you can see in that same section this this can easily be disabled with the ftz flag. I had to give Jakob…
I was aware of this deal before leaving. I made out pretty well with my vested shares but was more interested in working on the cutting edge of research than in continuing on with pure hardware optimization and design.…
So a minor correction here. We did explore placing ftz on various instructions inside the matmul ops, but it turns out you don't need anything more than what is already baked into tf by default. All tf gpu primitives…
Actually if you read section 4.6 of that paper you'll see that denormals are the default on sm_20 and above. But you can see in that same section this this can easily be disabled with the ftz flag. I had to give Jakob…
I was aware of this deal before leaving. I made out pretty well with my vested shares but was more interested in working on the cutting edge of research than in continuing on with pure hardware optimization and design.…