GPT-4 Outperforms Human Analysts in Financial Statement Analysis, Claims Research

In the paper, GPT-4 achieved 60.31 percent accuracy in correctly analysing the financial statements of a company.

Written by Akash Dutta, Edited by Siddharth Suvarna | Updated: 27 May 2024 17:15 IST

Photo Credit: Pexels/Shantanu Kumar

The researchers believe GPT and human analysts are complementary to each other

Highlights

Researchers found GPT-4 performs best with ‘chain-of-thought’ prompts
Gemini AI model performed worse than GPT-4 in financial analysis
In short term, GPT-4 performed similarly to automatic neural network

GPT-4 can outperform human analysts when it comes to predicting the future on the basis of financial statement analysis, claimed a new research paper. The paper, which has been published in a preprint journal found in its tests that GPT-4 gave superior results compared to human counterparts in the short-term period (ranging between one month to six months). It achieved 60.31 percent accuracy in its predictions compared to 56.7 percent of human analysts. However, the paper did not suggest that the AI model could replace humans.

Research paper's objective

Published in the preprint journal Social Science Research Network (SSRN), the 54-page paper titled “Financial Statement Analysis with Large Language Models” attempted to find out the role conventional artificial intelligence (AI) models can play in analysing the financial statements of an organisation and predicting its performance in the stock market in the near future.

Such analysis has always been understood to be very complicated as a wide range of factors can influence the performance of companies. Even as some financial firms use artificial neural networks (ANN) to assist individuals in their analysis, large language models (LLMs) have not been used for this. The researchers wanted to see if a state-of-the-art (SOTA) LLM such as GPT-4 can be a valuable addition to this or not.

What did the GPT-4 research paper find?

Researchers fed GPT-4 anonymised and standardised corporate financial statements (to prevent biases emerging from mentioning the company's name). Next, the researchers used two methods to test the capabilities of the LLM. The first was designing a simple prompt that directed the chatbot to analyse the statements and predict future earnings. The second was to use a “chain-of-thought' (CoT) prompt that taught the AI model to mimic financial analysts.

The CoT method asked GPT-4 to identify notable trends, compute key financial ratios, and to form expectations about future earnings. While the simple prompt did not fetch noteworthy results, the CoT prompts achieved 60.31 percent which was higher than the average human analyst's performance.

GPT-4 vs human analyst
Photo Credit: Research paper: Financial Statement Analysis with Large Language Models

“We find that an LLM excels in a quantitative task that requires intuition and human-like reasoning. The ability to perform tasks across domains points towards the emergence of Artificial General Intelligence,” the paper stated.

However, the researchers were quick to point out that GPT and human analysts are complementary instead of the former replacing the latter. Specifically, the paper claimed that LLMs have an advantage in areas where humans tend to show bias and disagreement. Humans, similarly, add value when the analysis requires additional contextual information that is not likely to be available within the financial data.

Is the Samsung Galaxy Z Flip 5 the best foldable phone you can buy in India right now? We discuss the company's new clamshell-style foldable handset on the latest episode of Orbital, the Gadgets 360 podcast. Orbital is available on Spotify, Gaana, JioSaavn, Google Podcasts, Apple Podcasts, Amazon Music and wherever you get your podcasts.

Affiliate links may be automatically generated - see our ethics statement for details.

Comments

For the latest tech news and reviews, follow Gadgets 360 on X, Facebook, WhatsApp, Threads and Google News. For the latest videos on gadgets and tech, subscribe to our YouTube channel. If you want to know everything about top influencers, follow our in-house Who'sThat360 on Instagram and YouTube.

Further reading: AI, GPT, OpenAI, Artificial Intelligence, chatbots

Akash Dutta Email Akash Dutta

Akash Dutta is a Senior Sub Editor at Gadgets 360. He is particularly interested in the social impact of technological developments and loves reading about emerging fields such as AI, metaverse, and fediverse. In his free time, he can be seen supporting his favourite football club - Chelsea, watching movies and anime, and sharing passionate opinions on food. More