AI is like a magic trick: amazing until it goes wrong, then revealed as a cheap and brittle effect

AI is like a magic trick: amazing until it goes wrong, then revealed as a cheap and brittle effect

Pay no attention to the algorithm behind the curtain. "Writing in Forbes, Kalev Leetaru compares today's machine learning systems to magic tricks, and boy is the comparison apt: "Under perfect circumstances and fed ideal input data that closely matches its original training data, the resulting solutions are nothing short of magic, allowing their users to suspend disbelief and imagine for a moment that an intelligent silicon being is behind their results. Yet the slightest change of even a single pixel can throw it all into chaos, resulting in absolute gibberish or even life-threatening outcomes." And just like magicians, the companies and agencies that use machine learning systems won't let you look behind the scenes or examine the props: Facebook won't reveal its false positive rates or allow external auditors for its machine learning system, which is why Instagram's anti-bullying AI is going to be a fucking catastrophe. Another important parallel: magic tricks depend on the deliberate cultivation of a misapprehension of what's going on. A magician convinces you that they're doing the same trick three times in a row, while really it's three different tricks, so the hypothesis you develop the first time is invalidated when you see the trick "again" a second time. Meanwhile, terms like "artificial intelligence" and "machine learning" and "understanding" deliberately mislead the users of these systems, who anthropomorphize relatively straightforward statistical tools and believe that there's some cognition going on, which "leads us to be less cautious in thinking about the limitations of our code, subconsciously assuming that someone it will 'learn' its way around those limitations on its own without us needing to curate its training data or tweak its algorithms ourselves.""