I remember reading an article or blog post years ago that persuasively argued that the danger of AI is not going to be that it ends up doing things better than humans, but that it causes a lot of harm when entrusted with tasks it actually isn’t good at. I think that thesis seems much more plausible now, watching people respond to clearly flawed AI systems.
That reminds me of a fairly recent article about research around visualisation systems to aid with interpretable or explainable AI systems (XAI). The idea was that if we can make AI systems that explain their reasonings, then they can be a useful tool, especially in the hands of domain experts.
Turns out that actually, the fancy visualisations that made it easier to understand how the model had come to a conclusion actually made subject matter experts less accurate in catching errors. This surprised researchers and when they later tried to make sense of it, they realised that they had inadvertently dialled up people’s likelihood to trust the model because it looked legit.
One of my favourite aphorisms is “all models are wrong, some are useful.” Seems that the tricky part is figuring out how wrong and how useful.
I remember reading an article or blog post years ago that persuasively argued that the danger of AI is not going to be that it ends up doing things better than humans, but that it causes a lot of harm when entrusted with tasks it actually isn’t good at. I think that thesis seems much more plausible now, watching people respond to clearly flawed AI systems.
That reminds me of a fairly recent article about research around visualisation systems to aid with interpretable or explainable AI systems (XAI). The idea was that if we can make AI systems that explain their reasonings, then they can be a useful tool, especially in the hands of domain experts.
Turns out that actually, the fancy visualisations that made it easier to understand how the model had come to a conclusion actually made subject matter experts less accurate in catching errors. This surprised researchers and when they later tried to make sense of it, they realised that they had inadvertently dialled up people’s likelihood to trust the model because it looked legit.
One of my favourite aphorisms is “all models are wrong, some are useful.” Seems that the tricky part is figuring out how wrong and how useful.