Skip to content
AI, Machine Learning and Big Data Analytics – The new strategic imperative to drive business transformation in FX
Nicholas Pratt examines how artificial intelligence (AI), machine learning (ML) and big data analytics are being incorporated into the FX market, what will drive its adoption and what benefits it will bring participants.
First Published: e-Forex Magazine 80 / Special Report / March, 2018
Excerpt:
Compliance
Brandon Daniels is president of Exiger Tech and global head of its analytics practice which uses data-driven strategies for its regulatory and financial crime compliance services. The size and liquidity of the FX market made it a breeding ground for arbitrage-driven and high frequency trading and, consequently, became one of the first places that people recognised the potentially beneficial and detrimental impacts of automated decision-making, says Daniels.
He says there are currently two levels of maturity when it comes to the use of AI and ML software for compliance purposes. “There are sophisticated participants using predictive technology to assess complex jargon or code for manipulative or collusive practices. Or, to identify conversations with inherent control risk, such as the exchange of proprietary data across institutions; essentially, using the systems as a de facto compliance analyst. But, most banks are still grappling with legacy data issues and getting applications onto a single platform so that these models can find any potential misconduct.”
However, in two to three years, compliance will be completely transformed in terms of tracking both good and bad conduct, says Daniels. “If I’ve transparently communicated all of the details I’ve shared with the market –– and also with compliance –– the technology can track that good behaviour and learn from it. If I evidenced all of that good conduct, it is just as useful as evidencing bad conduct. So, the industry is still grappling with understanding the technology and how best to use it whilst getting the technology to understand the nuances of human behaviour.”
There are some major challenges for the likes of Exiger and IBM in their quest to use AI and ML as a compliance and surveillance tool – the data that they are using does not lend itself to analytics in the same way as pure and structured data. It is more dirty and incomplete. It consequently takes time to come up with truly representative examples of good behaviour and it also takes time to identify future issues and predict what bad behaviour will be in the future.
However, given that FX is not just fighting a reputation for price fixing and collusion, but also as a conduit for money laundering, sanctions-busting and fraud, there is enormous motivation to invest in the technology, says Daniels. “There are hundreds of millions of dollars at risk, so there has been a massive increase in investment across the board.”
Brandon Daniels
“..the industry is still grappling with understanding the technology and how best to use it whilst
getting the technology to understand the nuances of human behaviour.”
Full Article
The FX market is seemingly a fertile breeding ground for new services employing artificial intelligence (AI) and machine learning (ML) software. Data is critical to the application of cognitive computing and as the most liquid of capital markets, FX has data in abundance. Indeed, it is a virtuous circle – the more electronic trading there is, the more data is available and the more participants will access the market.
Furthermore, there are more venues reporting their trades and more participants able to interpret the data, which is a good starting point for developing ML services, says Brad Bailey, research director with consultant Celent and its capital markets division.
There are busier asset classes, such as equities, that are further ahead in their use of AI and ML software but the FX market is catching up, says Bailey. “The first step is to use the technology to replicate what the best traders are doing, how they react to market movements, small changes in liquidity and how they construct the order book,” he says.
For a long time, the FX market has been consuming machine readable news which allows for faster consumption of digitally produced news from both structured sources like central banks, as well as social media. Applying AI and ML to the trading process itself as well as the post-trade and compliance functions may still need more time to develop, says Bailey, but he anticipates some massive improvements in the next five years.
“I really expect the FX market to change more in the next five years than it has in the last 25, despite the fact that it has changed so much already,” says Bailey. “It still has very high back and middle office costs because there are still so many things that are done manually. For example, voice will continue to be used for trading but it will become a smaller part of the total. But if you can get these manual processes into your data systems, it allows you to use more AI and ML software.”
Adoption and development
Perhaps the greatest validation of the FX market’s potential as a breeding ground for AI and ML services is the number of AI specialists that are moving into the asset class. One such firm is Metafused, a data analytics firm that uses AI to enhance decision making. The team have worked in data, treasury and partner with Google. Their CTO and Chief Data scientist, Matthew Yeager is a graduate of MIT and previously headed R&D Engg at EMC. The CEO, Madhuban Kumar has worked in three continents, was a venture capitalist, worked in treasury, headed teams that built products like the Barclays-Oyster card and bought IP data assets before founding Metafused. The company has received funding from angels who have founded and sold fx businesses as well as senior management from Apple.
Madhuban Kumar believes that there are several factors that could hasten the development of AI and ML in the FX market. “With new toolkits and storage costs being negligible, there is no excuse for not analysing all your data to reveal new patterns, strategies and trading opportunities. However, the big elephant in the room is the ballooning cost of regulations, which is increasing operational costs by 15-20% compared to 8-10% a few years ago, along with the scarcity of data scientists.”
By 2020 the number of Data Science and Analytics job listings is projected to grow by nearly 364,000 listings to approximately 2,720,000 with 59% from financial services at an average of $115,000 per annum. “Demand will far outstrip supply,” says Kumar. “And AI has a huge role in not just streamlining and automating processes, but also extracting intelligence from repeatable tasks and using that for new patterns recognition and trends.”
Metafused is building what Kumar claims is an ‘end-to- end reproducible workflow’ giving team of 4-5 equivalent output of 15-20 data scientists without anywhere near the cost base.“Today’s challenge is that data scientists are typically working with fragmented data sets and one-off models so 80% of their time is spent collecting and cleaning data that degrades rapidly over time,” says Kumar. “Furthermore, business teams have very little visibility into the process and this often results in a mismatch of expectation.”
The company aims to use AI to deliver an initial data audit in hours that will then take seconds thereafter, says Kumar. “Once the data scientist has a clean data set, the key challenge is to have multiple models running simultaneously with varied data sets and with different trading strategies that are based on liquidity, risk, time and jurisdiction, and to allow traders to make informed decision in real-time based on all of these factors.”
Kumar’s team has developed what she calls MetaBricks, reusable modules with proprietary data sets covering natural language, simulation and other disciplines designed to make for efficient processing of information.
“The key benefit is to replace the historic overnight rules-based analysis to ongoing continuous analysis and allowing business and quants to collaborate like never before. New trading algorithms based on ML may be less predictable than current rule-based applications and may interact in unexpected ways,” says Kumar. “To the extent that these algo trading firms using AI or ML techniques can generate higher returns or lower trading costs, it is likely that incentives for adoption will increase.”
Liquidity
AI and machine learning could also increase liquidity in financial markets by making trading speedier and more efficient and it could enhance risk management by detecting excessive risks and designing more effective hedges, says Kumar. “More specifically, the technology may allow for much tighter liquidity buffers, higher leverage, and faster maturity transformation than in cases where AI and ML had not been used.”
A greater use of AI may also have some unintended consequences says Kumar. “The ability to make use of big data from new sources may lead to greater dependencies on previously unrelated macroeconomic variables and financial market prices, including from various non-financial corporate sectors such as e-commerce and the ‘sharing economy’. As institutions find algorithms that generate uncorrelated profits or returns, there is a risk these will be exploited on a sufficiently wide scale that correlations actually increase. But these unforeseen connections will only become clear as the technologies are actually adopted,” she says.
On the whole, the likely impacts will be beneficial, particularly so if they can allow for more sophisticated insights into larger data sets, says Kumar. “A bank’s trading platforms may generate as much as 50GB of tic data daily yet many banks still struggle to get a single view of this data because of the siloed nature of their operations.”
There is a need to capture trader actions, strategy, liquidity and market context in a symphony with time of day and currency pairs on an ongoing context and that is hard, given the need to traditionally look at pricing and execution very closely, but elements that drive and effect it as secondary and run with historical benchmarks.
“We need to take all the data and to build a much more holistic picture than what we do currently and AI gives us that window. However, the key is how we view scale and whether firms decide to spend millions on internal builds that take more than two years or choose instead to partner with third parties who understand the importance of a non-siloed approach and a rapid deployment,” says Kumar.
The benefits that can be accrued from AI and machine learning are really down to banks’ and brokers’ different priorities, says Kumar. New capital allocation rules from MiFID II and Basel III are likely to increase the cost of trading for certain risky transactions including some FX trades, so using AI to build systems that can look risk and tighten spreads may help reduce those costs.
New trade reporting rules from MiFID II require more granular transaction analysis, using data inputs to look at the investment lifecycle – not just the price at execution but how you arrived there – and AI can help provide the real-time analysis needed, says Kumar. Similarly, applying AI to transaction cost analysis can improve trading efficiency by identifying the more expensive trades, she says.
However, transparent system architecture will be key in realising many of these benefits says Kumar “The real opportunities lie not with the AI black boxes but with the open systems where there is transparency and collaboration bringing the best of machine computation and augmenting it with trader knowledge.”
Standards and regulations
One issue that is yet to be addressed by the AI industry is that of international standards or regulations. The International Organization of Securities Commissions reported on the impact of new technologies including algorithmic trading on market surveillance, and made recommendations. Meanwhile the Senior Supervisors’ Group, a forum for senior representatives of supervisory authorities from around the world, issued principles for supervisors to consider when assessing practices and key controls over algorithmic trading activities at banks.
International regulators note that, from a supervisory perspective, firms developing algorithmic models based on AI and machine learning should have a robust development process in place,” says Kumar. “They need to ensure that possible risks are considered at every stage of the development process. This is particularly important in order to avoid market abuse and prevent the strategy from contributing to, or causing, disorderly market behaviour.”
Many current providers of AI and machine learning tools in financial services may fall outside the regulatory perimeter or may not be familiar with applicable law and regulation, adds Kumar, especially where financial institutions rely on these third-party providers of AI and ML services for critical functions but rules on outsourcing may not be in place or not be understood. “The lack of interpretability or ‘auditability’ of AI and machine learning methods has the potential to contribute to macro-level risk if not appropriately supervised by microprudential supervisors,” she adds, suggesting that it is an area that will be addressed as the technology develops and its adoption increases.
When it comes to implementing AI and machine learning into a firm’s strategic business plan, Kumar says that it is best to start small, considering the scarcity of talent. “This could involve doing short proof of concepts with a well-defined use case and a success criteria that is well understood at the outset. At Metafused we work on 10:10:10 – delivering a PoC in 10 weeks, use case outcomes in 10 days and a defined delivery in 10 hours based on the data. This approach helps banks to introduce AI into as many as 30 to 40 areas outside of pricing without the frightening and often expected cost associated with the technology.”
Trading perspectives
AI and ML technology is being increasingly explored across both institutional and retail FX trading markets. Forex Artilect, for example, is an AI-driven fund founded by David Lopez Onate who believes that human capital is one important factor in the future adoption of AI for FX trading.
“There’s a lot of research to do and a lot of skill is required in order to accomplish some profitable advancement in FX trading,” he says. “For each path you take, there could be many dead ends. There are plenty of resources, algorithms and computer processing capacity already to start an AI research project, however it is the expertise and required resilience that are lacking.”
AI technology can enhance many of the human limitations in the trading process, says Lopez Onate, such as fear, anxiety and optimism. “Because it takes an objective and quantitative approach to data, an AI trading model can extrapolate non-evident patterns and try to anticipate what is coming. This results in a better adjustment of strategy and risk management.”
Although this improvement is important, it is not an entirely automatic process, says Lopez Onate. “There will be some fundamental views on which the system is based and that need to be fine-tuned because of the dynamic nature of the markets,” he says.
As a developer with a background in AI technology rather than financial trading, Lopez Onate is not aware of any current regulation that limits the use of AI in FX and believes that “it should stay that way if it is meant to progress or advance”.
The first priority for FX firms, he reiterates, should be human capital and the recruitment of data scientists, software engineers and ML developers. “The next phase is the planning and actual development which will differ greatly according to the targets of the research project and the team itself. The most important phase is the last one – validation and testing. Here you check if the models actually work and are useful in the real world. And it is here where you will spend the most time and energy.”
Data is key
Data is key for the development of AI and ML-based trading services, says Serge Kassibrakis, Head of Quantitative Asset Management at Swissquote Group which provides online trading services to a largely retail and corporate client base. “In the FX industry, data is the new gold. Market data is of great interest. Prices, high frequency offers and order books all give an insight into price formation. But behavioral data is also important because it will help brokers to provide their clients with the right information deduced from the market data.”
For both market and behavioral data, the quality and quantity are mandatory and the spectrum must also be broad, says Kassibrakis. “The key with behavioural data is the presumption that no two traders will react in the same way, depending on risk aversion, history and other factors. And when it comes to developing sentiment analysis, we can also consider text and other forms of unstructured data on samples of thousands of traders and different trader ‘types’.”
More sophisticated real-time insights into FX trading on larger data sets, such as real-time databases of user behaviour, should also be possible in the future, says Kassibrakis. “We are working with Swiss university Ecole Polytechnique Federale de Lausanne to reconstruct some hidden information just by using available data but it’s not done yet.”
User-behaviour data is highly sensitive and companies are not keen to share them even if they are anonymous, says Kassibrakis, but it will come and it will be useful. “It will help brokers with their risk management and the information they provide to their clients.”
Applying AI and ML to market and behaviour data will also help firms to manage their inventory by predicting any global trading imbalances within a one hour time interval, says Kassibrakis. “If we know with high confidence that our client will buy a given currency and if at the same time we are close to the hedging barrier, we can avoid going to the market to offload the risk and decrease the inventory themselves and avoid a negative P&L. AI will also help develop new investment strategies by adapting trading parameters in real-time.”
Hardware will also play a role in the development of AI and ML-based trading but not to the same extent as the data, says Kassibrakis. “New hardware and greater computational speed and power will help generate more signals which will lead to more and more trading decisions from traders and their algorithms.”
And while larger and wealthier market participants may have access to more sophisticated hardware, computational power is increasingly accessible to all and the difference between the elite and the mainstream in terms of this access is continually reducing. Furthermore, quantum computing is a technology predicted to accelerate this reduction, in which case, CPU will not be a significant issue any more, says Kassibrakis. “Data will become the only ‘gold’ along with the brains needed to build the algorithms.”
As with Lopez Onate, Kassibrakis does not believe that new regulations are needed for AI and ML, although he does concede that it is difficult to know how different AI-driven algorithms will interact with each other. “This could be a major issue as was the case with the traditional algorithms. But in terms of regulations, I do not think we’re in the same situation as the car industry and driverless cars. Rules-based algorithms have been used to manage portfolios and trading strategies for some time and I think the regulations are already well adapted even if there may need to be some additional adjustments in the future,” says Kassibrakis.
So what steps should firms take to bring AI and ML technologies into their strategic business plans? “More and more use case shows that AI & ML are not magical engines in which you just have to put as much as data as you can and wait for the answer. The quality of the result will depend on the questions you ask and your knowledge of the field.”
Surveillance
The FX market’s issues with collusion, price-fixing and manipulation have been well-chronicled in recent years so it is not surprising that efforts are being made to apply AI and ML technology into the surveillance area. “I think AI can improve the surveillance capabilities,” says Keith Bear, vice president, Financial Markets, IBM. “It can enable you to take in multiple sources of data, both structured and unstructured such as emails and chatroom transcripts. That enables you to spot potential risks of collusion.”
IBM is one of the pioneers in artificial intelligence thanks to the development of its AI platform Watson, first developed in 2011 and deployed commercially in 2013. In 2016, IBM then acquired Promontory, a US-based risk management and compliance consulting firm. Since then, it has focused on combining the industry experience of Promontory’s former regulators with the cognitive capabilities of Watson and applying it to voice and text communications between traders.
Proof of concepts are going into production and while the technology is proven, there are other aspects to consider, says Bear, and it may take some months to fully develop and implement the systems. There is also a huge amount of information out there, some of which is highly specific such as on the trading room floor.
Another limiting factor is that banks are only able to surveil their own employees so for any communications between banks, they may only get one side of the conversation. Consequently, IBM hopes that by adding the industry know-how of the Promontory Group, it will give an insight into how best to implement the technology.
“It is about understanding patterns that have occurred,” says Bear. “Traders that are communicating, the language that is being used and what constitutes an abnormal or unconventional discussion. For example, there are various linguistic signs to look for in potential collusion, although these signs vary according to geography. In the UK, traders start to swear when they are about to collude. But in the US, they tend to clean up their language. Repetitive language and unusual meeting requests are also typical signs.”
For FX trading firms and banks, the significant financial penalties for wrong-doing are a strong motivation for implementing AI and ML technology for surveillance. In May 2015, six banks were fined a total of $5.6bn for their rigging of the FX markets. “The size of these fines are way in excess of the cost of the technology so it is a major motivation,” says Bear. “It is also a very positive step in terms of the shareholders’ perspective.”
The same technology applies in a regulatory context, says Bear. “There is currently a huge amount of regulation. It is predicted that by 2020 there could be up to 300 million pages of financial services regulation and up to 40,000 regulatory changes per year. AI and ML can be applied to digest the information and spot the requirement and changes that need to take place.”
IBM’s AI work is not limited to surveillance though, it is looking at the trading process and making Watson available through APIs and developing emotion and tone analysis for both structured and unstructured data. It is also working with CLS, the financial institution that provides settlement services to its members in the FX market, on database services and FX forecast data.
“It is an interesting illustration of utilities that are taking the extensive data that they hold, currency data and using machine learning to make predictive forecasts on their core data, says Bear. “Roughly 30% of exchanges’ revenue streams come from data-related services. For clearing and settlement services, they have found new ways to monetise the data, not just the raw data but the predictive data.”
Compliance
Brandon Daniels is president of Exiger Tech and global head of its analytics practice which uses data-driven strategies for its regulatory and financial crime compliance services. The size and liquidity of the FX market made it a breeding ground for arbitrage-driven and high frequency trading and, consequently, became one of the first places that people recognised the potentially beneficial and detrimental impacts of automated decision-making, says Daniels.
He says there are currently two levels of maturity when it comes to the use of AI and ML software for compliance purposes. “There are sophisticated participants using predictive technology to assess complex jargon or code for manipulative or collusive practices. Or, to identify conversations with inherent control risk, such as the exchange of proprietary data across institutions; essentially, using the systems as a de facto compliance analyst. But, most banks are still grappling with legacy data issues and getting applications onto a single platform so that these models can find any potential misconduct.”
However, in two to three years, compliance will be completely transformed in terms of tracking both good and bad conduct, says Daniels. “If I’ve transparently communicated all of the details I’ve shared with the market –– and also with compliance –– the technology can track that good behaviour and learn from it. If I evidenced all of that good conduct, it is just as useful as evidencing bad conduct. So, the industry is still grappling with understanding the technology and how best to use it whilst getting the technology to understand the nuances of human behaviour.”
There are some major challenges for the likes of Exiger and IBM in their quest to use AI and ML as a compliance and surveillance tool – the data that they are using does not lend itself to analytics in the same way as pure and structured data. It is more dirty and incomplete. It consequently takes time to come up with truly representative examples of good behaviour and it also takes time to identify future issues and predict what bad behaviour will be in the future.
However, given that FX is not just fighting a reputation for price fixing and collusion, but also as a conduit for money laundering, sanctions-busting and fraud, there is enormous motivation to invest in the technology, says Daniels. “There are hundreds of millions of dollars at risk, so there has been a massive increase in investment across the board.”
Conclusion
AI promises to bring many benefits to FX on both the cost and the revenue side, states Kumar. “It really comes down to what use cases are priorities for banks and brokers and what’s the low hanging fruit. The real opportunities in the future will lie with open systems and not the current AI blackboxes, where there is more transparency and collaboration bringing the best of machine computation and augmenting it with trader knowledge.”
____________________________________________________
Learn more about Exiger’s AI-powered compliance solutions: DDIQ & Exiger Insight 3PM.
Download a PDF of Brandon‘s interview below .
