How data and digitisation are transforming commodities trading
Commodity trading has always been about gathering and utilising information.
Historically, traders relied on local knowledge about weather patterns, crop yields, and supply and demand.
Those with the best information could make a favourable deal at the right price, and protecting that data was everything. As recently as a decade ago, traders would still make decisions based on that exclusive access to information.
Either holding the physical assets or using their hard-won network of global sources to provide the data they needed to inform a trade. When the moment came to execute, a select group of traders were the only people in the world with access to the relevant information at the right time.
But a 2,000-year-old model was always ripe for disruption, and when it came, it came rapidly. From agriculture to energy, shipping to critical minerals, access to data and digital technology has transformed how commodities are traded. Cloud computing means the immense processing power and storage needed to analyse large datasets or run complex models is now available without excessive costs.
Our traders no longer rely on extensive traditional information networks to gain advantages from proprietary data. We’ve invested in dedicated teams that use predictive analytics to draw valuable proprietary insights from common data sources. This helps us to trade better and manage risk more effectively.
The data revolution
For most of the history of commodities trading, market intelligence from a handful of experienced operators was the currency. But in just ten years, in the blink of an eye in relative terms, access to information has become universal and democratic.
Data is omnipresent and available to everyone.
As an industry, we’ve seen trading margins erode and competition increase significantly in parallel to this explosion of data. The transparency and visibility of information have greatly improved the efficiency of commodities markets but reduced arbitrage opportunities. We’ve also seen the explosion of data enabling the rise of algorithmic trading, operating on thin margins and high volume.
In this new world, where we can build an accurate digital model of reality comparable to the physical, the traders can harness that information, filter out the noise and detect the right signals that are finding the greatest success.
Seeing this direction of travel shaped our decision in 2019 to build out a footprint of data science teams, weather scientists, and fundamental analysts to reorientate the company away from physical markets towards a business building genuinely cutting-edge IP around this research-based approach to commodity trading.
You can read more about this pivot from our CEO, Huw Jenkins, in our last edition of ‘The Commodity Perspective’.
How data is shaping commodity markets
Across all commodities, there’s been an extraordinary increase in the availability of real-time data. Cheap sensors, ubiquitous internet connections, real-time monitoring, and even social media tracking make it possible to accurately quantify many of the commodities we trade.
We’re particularly interested in four areas – satellite imagery, weather data, the real-time positioning of shipping through the Automatic Identification System (AIS) and data from the news media.
In agricultural commodities, for example, our quant analysts can process weather model output data and prevailing teleconnections to assess the likely weather outturns. This can be coupled with remote sensing imagery of current crop states and other variables to predict crop yields and, coupled with demand modelling, be used to predict price outturns.
For oil traders, it was once the case that the amount of oil held in U.S. storage hubs was published once or twice a week by the Energy Information Administration (EIA). The lack of richer data than just the ‘weekly Cushing’ made it extremely difficult for analysts to get enough information to make accurate real-time models of oil flows. But now, thanks to sensors and infrared cameras monitoring storage levels, such data is abundant, and traders can make decisions with greater speed and confidence.
We’ve also seen data play an integral role in reshaping the global gas markets since the Russian invasion of Ukraine. The markets for natural gas were traditionally localised, but LNG and the explosion of European demand have globalised the trade.
Almost overnight, traders needed vast datasets, from maritime tracking, satellite imagery and terminal sensors, to build an accurate market picture.
For all the progress made in the democratisation of data and the transparency it’s brought to commodities markets, there are still challenges.
Mining, for example, has been slow to integrate technology like blockchain that could revolutionise the integrity of the metals and minerals supply chain, which is crucial as the energy transition gathers pace.
Downstream customers face increasing reputational risks unless they know exactly where their raw materials originated and how they’ve been mined.
Data analysis has allowed traders to understand the world in a much more granular way. Being able to utilise this knowledge effectively will become increasingly critical in the coming years.
As technology allows trading to be more automated, every second matters. The challenge is no longer about getting access to information; it’s how you assimilate, analyse, and integrate that information at pace that gives a genuine advantage.
The Covid Catalyst
Trading commodities has always relied on vast paper trails to execute, authenticate, and process each transaction. Indeed, the commodities industry has resisted digitisation for most of the digital age. It was viewed with suspicion across fragmented supply chains that relied on personal relationships and closely guarded information.
But then the Covid-19 pandemic forced the hand of many industry players. Decades of digitisation happened in just a few months.
The speed of the virus’s spread, leading to global lockdowns and radical supply-chain disruption, took everyone by surprise, underlining the need for real-time data in commodities markets to better manage risk and exposure.
And now we’re seeing that digital transformation become self-perpetuating: the more digitisation there is, the more data we have, and the greater the utilisation of digital tools.
The pandemic also accelerated the adoption of cloud computing, making it possible to run these complex algorithms and utilise AI and machine learning without prohibitive overheads.
‘When we look back on the history of the cloud, we’ll see the pandemic accelerated adoption of this technology by years’
Andy Jassy CEO, Amazon
What does the future look like?
We’re no longer just living in the digital age; history will also judge this as the beginning of the data age.
Although the commodities industry has been relatively slow to digitise and harness the power of data, the acceleration is well underway. Commodity traders now rely on data from multiple sources to build highly accurate digital models of the markets in which we’re trading.
So, if the beginning of commodities’ digital and data age is characterised by the proliferation and democratisation of data sources, then the next chapter must be led by the question of how best to integrate this vast volume of data into trading strategies.
There’s little doubt that automation, AI and machine learning will have significant roles to play in this next stage. Access to huge datasets makes it easier to train and deploy AI to analyse historical market data, identify patterns and look for price signals.
Tools like virtual and augmented reality could also be used to increase the visualisation capability of analysts utilising these complex datasets.
Data security will be an important trend in the coming years. There’s evidence that cyberattacks are increasing against high-value commodities targets, like energy infrastructure, and those risks will only increase as the commodities complex becomes more digital.
Industry players will also need a greater assurance of a commodity’s providence as ESG becomes an even greater priority. That’s where blockchain and NFT technology still have so much unrealised potential.
There’s no longer any competitive advantage from the data itself.
Traders are now competing on how well they can integrate the data into their models and harness the technology to generate insights to build a successful trading strategy. AI, ever-more sophisticated analytics tools, and increased automation will continue to push our industry forward.
But this digital revolution doesn’t mean machines will replace human traders and analysts. Instead, we see the technology and data analysis augmenting the capabilities of our traditional trading desks. The machines can mine the data for intelligence, and our traders can act on it.
Commodities trading has been transformed in the last decade – but it’s becoming clear that this is just the beginning.