AI should earn its keep: Introducing the AI Productivity Guarantee (cognition.ai)

Seems like a wild move from the Devin team. Very curious about the metrics going into their "estimator" and also that they compared it to human estimates as a measure of success.