I wonder how many false positives the algorithm caused.
They also called it an "AI system", but from the article it looks more like a statistical model that assign a risk score to every attribute (age, location, when, duration, ecc) and then do sum/weighted average. If the sum/weighted average are above a threshold, the reservation is flagged.
> Airbnb said its reservation-screening AI has been evaluated by the company’s anti-discrimination team and that the company often tests the system in areas such as precision and recall.
This testing would be an interesting article of its own. How does one do this while keeping the model within the bounds of the law?
For example: Say you're in a jurisdiction that prohibits discrimination based on a renter's youth. You might be able to argue that denying a college-aged person a 2 day rental over their birthday for fear of a party is not illegally discriminatory. It's much less defensible if it's a 2 month rental over their birthday. But the local law probably doesn't specify a threshold like that. So creating a threshold-based test -- which is constructively a policy -- could itself be legally thorny. (But IANAL)
IANAL but I don't think so, it's more or less the same thing insurances are doing since forever: they assign a risk score based on your attributes and decide the price of the policy or even to not insure you.
And, as I wrote in the other comment, it's not even an "AI model", it's just old and boring statistics and math.
5 comments
[ 2.8 ms ] story [ 193 ms ] threadThey also called it an "AI system", but from the article it looks more like a statistical model that assign a risk score to every attribute (age, location, when, duration, ecc) and then do sum/weighted average. If the sum/weighted average are above a threshold, the reservation is flagged.
This testing would be an interesting article of its own. How does one do this while keeping the model within the bounds of the law?
For example: Say you're in a jurisdiction that prohibits discrimination based on a renter's youth. You might be able to argue that denying a college-aged person a 2 day rental over their birthday for fear of a party is not illegally discriminatory. It's much less defensible if it's a 2 month rental over their birthday. But the local law probably doesn't specify a threshold like that. So creating a threshold-based test -- which is constructively a policy -- could itself be legally thorny. (But IANAL)
And, as I wrote in the other comment, it's not even an "AI model", it's just old and boring statistics and math.