How will we know?

Our world is being transformed by data processing. Algorithms can now look at our data and make better decisions on our behalf than we ever could.

But as the algorithms get smarter, some results will be counterintuitive to us. When we see a decision that seems intuitively to be suboptimal, that might be for any of these reasons:

  1. The algorithm is smarter than us.
  2. The algorithm is optimising for something other than our individual benefit.
  3. There is a bug. ☹

Which is it? And how will we know?