To me, it's less that an autoencoder approximates an SVD, and more than an SVD is the OG autoencoder.
I made a demo of the SVD part. https://jmalicki.github.io/svd-grad/ The devil was in the details, and that escalated quickly from a simple idea to actually getting it to work sucked me into a ton of random deep corners.
I also forgot that you have to constrain U and V to be orthonormal - otherwise you end up with the same low rank approximation D', but exactly which numbers go in U, S, or V can shift as it's underdetermined without…
Yes, that too, in addition to the other differences I pointed out. But it's just an SVD with a few more bells and whistles in my view.
I didn't find a direct proof earlier, just assertions - I've only seen it proved in textbooks that aren't linkable. Theorem 1, section 1.3, page 2 shows that the expected variance of the gradient of the loss function…
That's on the submitter of the PR to do. Don't put that burden on the reviewer.
The second moments of the gradient and the Hessian are absolutely related! See the Fisher Information, and the Cramer-Rao Lower Bound (an inequality on how much the inverse covariance matrix and the Hessian can differ).…
If you're getting code without tests to review in a PR, that should be an instant reject without even looking at the code.
Some fun stuff about SVDs: If you want to take a low rank approximation to a matrix D, let's call our approximation D'. The approximation that minimizes mean square error of the reconstructed matrix vs. the original…
ADAM is related if your second derivative matrix happens to be diagonal. Of course, it takes about 5 minutes to show that any DNN is going to have very very high magnitude off-diagonal terms by the way it's constructed,…
LACFD, and they encouraged the patients to call ubers, didn't call them themselves (I couldn't find the cite). https://www.cbsnews.com/losangeles/news/la-county-changes-am...
Unlike an ambulance, an uber driver won't be able to keep a hemodynamically unstable patient alive for a 30 minute transport. LAFD has already called Ubers for non-emergency transport to hospitals.
Tylenol is one of the things on the shelf most likely to kill you from liver failure. It is way more dangerous than many prescription drugs.
Memantine https://medlineplus.gov/druginfo/meds/a604006.html
It is entirely reasonable to say that it should be RX-only to be monitored by a physician for these reasons.
Acetaminophen (Tylenol) is probably the OTC drug that is at the top of the list to be made RX-only due to its dangers.
An interesting new drug is Auvelity, where Dextromethorphan is proposed to help stimulate neurotropic growth factor to help the brain repair itself, and similar related drugs like dextromethorphan and ketamine and other…
For break-the-bank options, Stanford and Columbia have good programs too. If you're looking at the potential returns to a graduate degree in a high paying field, $60k spread over a few years isn't an insane investment,…
Thank you! The link had some of the details, but not the intelligent thought linking them into a narrative, much appreciated! This is something super important I would have been foot-gunned on at some point.
> hides more fine-grained transaction control In what way does having autocommit=False hide more fine-grained transaction control? autocommit=False gives full control to the programmer to do whatever they want.
> The regular implicit transactions there plus explicit where needed aren’t supported in any python mode Specific examples would be extremely useful. You've done some work learning and deducing this stuff, others could…
> changes how sqlite's transactions work What specifically are you referring to? The apswutils website also does not explain.
Then that leads to the question... is it proper for the farmer to take the deduction for the costs of growing crops he is not legally allowed to sell?
They'll become cattle projects you don't know the names of. Like an LLM listening to you curse at software, fixing it/creating extensions etc., installing them, and you're not even aware.
If he donates these to a charity, what are the rules on taking a tax deduction for this unusual situation? Can it have value for the purposes of a donation if you can't sell it? Would taking a tax deduction trigger a…
To me, it's less that an autoencoder approximates an SVD, and more than an SVD is the OG autoencoder.
I made a demo of the SVD part. https://jmalicki.github.io/svd-grad/ The devil was in the details, and that escalated quickly from a simple idea to actually getting it to work sucked me into a ton of random deep corners.
I also forgot that you have to constrain U and V to be orthonormal - otherwise you end up with the same low rank approximation D', but exactly which numbers go in U, S, or V can shift as it's underdetermined without…
Yes, that too, in addition to the other differences I pointed out. But it's just an SVD with a few more bells and whistles in my view.
I didn't find a direct proof earlier, just assertions - I've only seen it proved in textbooks that aren't linkable. Theorem 1, section 1.3, page 2 shows that the expected variance of the gradient of the loss function…
That's on the submitter of the PR to do. Don't put that burden on the reviewer.
The second moments of the gradient and the Hessian are absolutely related! See the Fisher Information, and the Cramer-Rao Lower Bound (an inequality on how much the inverse covariance matrix and the Hessian can differ).…
If you're getting code without tests to review in a PR, that should be an instant reject without even looking at the code.
Some fun stuff about SVDs: If you want to take a low rank approximation to a matrix D, let's call our approximation D'. The approximation that minimizes mean square error of the reconstructed matrix vs. the original…
ADAM is related if your second derivative matrix happens to be diagonal. Of course, it takes about 5 minutes to show that any DNN is going to have very very high magnitude off-diagonal terms by the way it's constructed,…
LACFD, and they encouraged the patients to call ubers, didn't call them themselves (I couldn't find the cite). https://www.cbsnews.com/losangeles/news/la-county-changes-am...
Unlike an ambulance, an uber driver won't be able to keep a hemodynamically unstable patient alive for a 30 minute transport. LAFD has already called Ubers for non-emergency transport to hospitals.
Tylenol is one of the things on the shelf most likely to kill you from liver failure. It is way more dangerous than many prescription drugs.
Memantine https://medlineplus.gov/druginfo/meds/a604006.html
It is entirely reasonable to say that it should be RX-only to be monitored by a physician for these reasons.
Acetaminophen (Tylenol) is probably the OTC drug that is at the top of the list to be made RX-only due to its dangers.
An interesting new drug is Auvelity, where Dextromethorphan is proposed to help stimulate neurotropic growth factor to help the brain repair itself, and similar related drugs like dextromethorphan and ketamine and other…
For break-the-bank options, Stanford and Columbia have good programs too. If you're looking at the potential returns to a graduate degree in a high paying field, $60k spread over a few years isn't an insane investment,…
Thank you! The link had some of the details, but not the intelligent thought linking them into a narrative, much appreciated! This is something super important I would have been foot-gunned on at some point.
> hides more fine-grained transaction control In what way does having autocommit=False hide more fine-grained transaction control? autocommit=False gives full control to the programmer to do whatever they want.
> The regular implicit transactions there plus explicit where needed aren’t supported in any python mode Specific examples would be extremely useful. You've done some work learning and deducing this stuff, others could…
> changes how sqlite's transactions work What specifically are you referring to? The apswutils website also does not explain.
Then that leads to the question... is it proper for the farmer to take the deduction for the costs of growing crops he is not legally allowed to sell?
They'll become cattle projects you don't know the names of. Like an LLM listening to you curse at software, fixing it/creating extensions etc., installing them, and you're not even aware.
If he donates these to a charity, what are the rules on taking a tax deduction for this unusual situation? Can it have value for the purposes of a donation if you can't sell it? Would taking a tax deduction trigger a…