& Kagi (search company w/controversial CEO) sounds like they’re doing something right
Excerpt:
…Paul Kafasis…took it and pursued it thoroughly, asking Siri “Who won Super Bowl __?” for every number from 1 through 60.
His report at One Foot Tsunami documenting the results is utterly damning:
So, how did Siri do? With the absolute most charitable interpretation, Siri correctly provided the winner of just 20 of the 58 Super Bowls that have been played. That’s an absolutely abysmal 34% completion percentage. If Siri were a quarterback, it would be drummed out of the NFL.
Siri did once manage to get four years in a row correct (Super Bowls IX through XII), but only if we give it credit for providing the right answer for the wrong reason. More realistically, it thrice correctly answered three in a row (Super Bowls V through VII, XXXV through XXVII, and LVII through LIX). At its worst, it got an amazing 15 in a row wrong (Super Bowls XVII through XXXII). Most amusingly, it credited the Philadelphia Eagles with an astonishing 33 Super Bowl wins they haven’t earned, to go with the 1 they have.
Below, I’ve gathered a dozen of my favorite responses, in sequential order.
It usually hears me correctly and usually works. But let’s say 5-10% of the time it mishears “lights”, and I would just expect it to be able to figure out what to do based on the incorrect transcription. It’s a disappointing failure mode. Especially since the data model for the Apple Home app is that every device has to be in a “room”, so I have a room called “bedroom”, which contains a device called “bedroom light”, and given that setup it can’t figure out what “turn off the bedroom lights” means? Pathetic. Google Home can also turn off my lights and has a near-100% success rate, and the reaction time is 1-2 seconds instead of Siri’s ~10 seconds.