Humans v AI: We found out who’s better at making money
Who’s cashing in?
AI’s financial forecasts
We analyzed 27 peer-reviewed studies by academic researchers published between 2000 and 2018. These describedifferent kinds of stock market forecasting experimentsusing machine learning algorithms. We wanted to determine whether these forecasting techniques could be replicated in the real world.
Our immediate observation was that most of the experiments ran multiple versions (in extreme cases, up to hundreds) of their investment model in parallel. In almost all the cases, the authors presented their highest-performing model as the primary product of their experiment – meaning the best result was cherry-picked and all the sub-optimal results were ignored. This approach would not work in real-world investment management, where any given strategy can be executed only once, and its result is unambiguous profit or loss – there is no undoing of results.
Running multiple variants, and then presenting the most successful one as representative, would be misleading in the finance sector and possibly regarded as illegal. For example, if we run three variants of the same strategy, with one losing -40%, the other one losing -20%, and the third one gaining 20%, and then only showcase the 20% gain, clearly this single result misrepresents the performance of the fund. Just one version of an algorithm should be tested, which would be representative of a real-world investment setup and therefore more realistic.
Models in the papers we reviewed achieved a very high level of accuracy, about 95% – a mark of tremendous success in many areas of life. But in market forecasting, if an algorithm is wrong 5% of the time, it could still be a real problem. It may be catastrophically wrong rather than marginally wrong – not only wiping out the profit but the entire underlying capital.
We also noted that most AI algorithms appeared to be “black boxes”, with no transparency on how they worked. In the real world, this isn’t likely to inspire investors’ confidence. It is also likely to be an issue from a regulatory perspective. What’s more, most experiments did not account for trading costs. Though these have been decreasing for years, they’re not zero, and could make the difference between profit and loss.
None of the experiments we looked at gave any consideration to current financial regulations, such as the EU legal directiveMIFID IIor business ethics. The experiments themselves did not engage in any unethical activities – they did not seek to manipulate the market – but they lacked a design feature explicitly ensuring that they were ethical. In our view, machine learning and AI algorithms in investment decision-making should observe two sets of ethical standards: making the AI ethical per se, and making investment decision-making ethical, factoring in environmental, social, and governance considerations. This would stop the AI from investing in companies that may harm society, for example.
All this means that the AIs described in the academic experiments were unfeasible in the real world of the financial industry.
Are humans better?
We also wanted to compare the AI’s achievements with those of human investment professionals. If AI could invest as well as or better than humans, then that could herald a huge reduction in jobs.
We discovered that the handful of AI-powered funds whose performance data were disclosed on publicly available market data sources generally underperformed in the market. As such, we concluded that there is currentlya very strong casein favor of human analysts and managers. Despite all their imperfections, empirical evidence strongly suggests humans are currently ahead of AI. This may be partly because ofthe efficient mental shortcuts humans takewhen we have to make rapid decisions under uncertainty.
In the future, this may change, but we still need evidence before switching to AI. And in the immediate future, we believe that, instead of pinning humans against AI, we should combine the two. This would mean embedding AI in decision-support and analytical tools but leaving the ultimate investment decision to a human team.
Article byBarbara Jacquelyn Sahakian, Professor of Clinical Neuropsychology,University of Cambridge;Fabio Cuzzolin, Professor of Artificial Intelligence,Oxford Brookes University, andWojtek Buczynski, PhD candidate / consultant,University of Cambridge
This article is republished fromThe Conversationunder a Creative Commons license. Read theoriginal article.
Story byThe Conversation
An independent news and commentary website produced by academics and journalists.An independent news and commentary website produced by academics and journalists.
Get the TNW newsletter
Get the most important tech news in your inbox each week.
