Recently, I’ve been trying to better learn all the math that powers machine learning. Most recently, I’ve been learning about optimization (so a lot of multivariable calculus). Today I was learning about Lagrange multipliers. Optimization being the primary focus, most of it was about the gradients, not much about \(\lambda\), the Lagrange multiplier, itself. That is until Khan Academy dropped this bomb:
”\(\lambda^{*}\) tells us how much more money we can make by changing our budget”
In other words, the Lagrange multiplier is the derivative of profit with respect to your constraint (e.g. budget). So much in microeconomics boils down to maximizing profit with respect some input (e.g. price, quantity, etc.), so this is a super relevant and interesting nugget.
I don’t think I’ve mentioned it anywhere on this site yet, but I was an Economics major in college, so this was a pleasant and unexpected bit of mathematical application and intuition for me. This real life connection means I’ll have a much better chance to actually remember this in the future! (This blog post will also help hopefully.)
(I may have actually learned this back in college, but honestly that was so long ago now if I did I totally forgot; hence, why I’m trying to properly learn / relearn all the math behind machine learning.)