ChatGPT Of Finance is here, Bloomberg combines AI and Fintech
Bloomberg brings to finance what GPT and ChatGPT brought to everyday chatbots for general use.
The article released by Bloomberg reveals the great technical depth of the BloombergGPT machine learning model, applying the type of AI techniques that GPT applies to financial datasets. Bloomberg’s terminal has been the trading and financial world’s premier resource for financial market data for over four decades. As a result, Bloomberg has acquired or developed a large number of proprietary and curated datasets. In many ways, this data is Bloomberg’s crown jewel, and in this version of BloombergGPT, this proprietary data is used to build an unparalleled financial research and analysis tool.
The major language models that drive such AI experiments are syntactic and semantic, and are used to predict a new outcome based on existing relationships within and across source texts.
Machine learning algorithms learn from source data and produce a model, a process known as “training”. Training the BloombergGPT model required approximately 53 days of computations run on 64 servers, each containing 8 NVIDIANVDADIA 40GB A100 GPUs. In comparison, when we use ChatGPT, we give a model (or formula) an input, known as the prompt, and the model then produces an output, much like giving an input to a formula and observing the output. Generating these models requires enormous amounts of computing power and Bloomberg therefore collaborated with NVIDIA and Amazon Web Services in the production of the BloombergGPT model.
Since each GPU costs tens of thousands of dollars, if purchased new, and is only used for a short relative duration for model generation, the BloombergGPT team chose to use AWS cloud services to run the computation. Since the cost per server instance is $33 per hour (which is currently publicly announced), we can do a back-of-the-napkin cost estimate of more than $2.7 million to produce the model alone.
Part of feeding content to a machine learning model involves fragmenting the content into chunks or tokens. One way to think about tokens is as ways we can break down an essay, into words being the most obvious, although there may be other strategies for tokenizing or fragmenting an essay, such as breaking it up into sentences or paragraphs. A tokenizer algorithm decides at what granularity to fragment, because, for example, fragmenting an essay into letters may result in the loss of some context or meaning. The fragmentation would be too granular to be of practical use. BloombergGPT fragments its financial data source into 363 billion tokens using a Unigram model, which offers certain efficiencies and benefits. To play with a tokenizer, try the GPT tokenizer here.
The Bloomberg team used PyTorch, a popular free and open source Python-based deep learning package, to train the BloombergGPT model.
In the case of BloombergGPT, source datasets include some weighted shares of financial news, corporate financial filings, press releases and Bloomberg News content, all collected and curated by Bloomberg over decades. On top of these finance-specific sources, BloombergGPT integrates into some general and common datasets such as The Pile, The Colossal Clean Crawled Corpus or C4 and Wikipedia. Combined, BloombergGPT can offer a whole new way of doing financial research.
As for the Bloomberg data used for training, spanning March 1, 2007, to July 31, 2022, Bloomberg refers to this financial collection of data as FINPILE. FINPILE consists of five main sources of financial content, namely:
- Financial Web. General web content (such as websites and documents), but limited to specific websites that can be categorized as financial, is used. Even within this category, BloomberGPT only crawls what it considers reputable, high-quality sites.
- Financial news. Although the web crawls sites that are financial in nature, news sites that generate news information require special attention. While the web can contain a multitude of content types, from PDFs to images, news sites require stricter curation.
- Company registrations. Anyone conducting research on a public company must consider studying the company’s records. In the United States, the SEC’s EDGAR database is typically the repository used to search and retrieve records.
- Press releases. A company’s formal public communications can often contain financial information, and this was included as a source in BloombergGPT.
- Bloomberg content. Given that Bloomberg is also a media company, the news content was used and fed to BloombergGPT. This includes opinion and analysis pieces.
While it remains to be seen how BloombergGPT will impact the fintech industry, some of the potential uses of BloombergGPT could include:
- Generates a first draft of a Securities and Exchange Commission filing. Given a large amount of filing data and much like how ChatGPT can produce a preliminary patent filing or custom programming code, it may be entirely possible to generate an SEC file, potentially reducing the cost of filing.
- The BloombergGPT newspaper provides an example of summarizing a text containing economic content in a headline. If, for example, the text is the following:
The US housing market shrank in value by $2.3 trillion, or 4.9%, in the second half of 2022, according to Redn. It is the biggest drop in percentage since the housing crisis in 2008, when values fell 5.8% in the same period.
BloombergGPT will produce the following output:
“House prices are seeing the biggest drop in 15 years.”
- Provide a company map of an organization and links between a person and several companies. Because company names and names of executives are fed into the BloombergGPT model, it is entirely possible that at least the organization’s executive level structure can be queried.
- Automating the generation of drafts of routine market reports and summaries for clients
- Retrieval of specific elements in the accounts for specific periods via a single message
BloombergGPT represents a significant leap forward for the financial and AI communities. Currently, the model is not publicly available, and there is no API, much less a chat interface, to access it. It is unclear when or if public access will be available, or even the current incarnation of BloombergGPT will still see further revisions. The BloombergGPT team concludes in their article that “we are wrong and following the practice of other LLM developers by not releasing our model” and will not make the model available to the public.
With OpenAI’s valuation over $20 billion, who can blame them?
Follow me on Twitter or LinkedIn. check out my website.
