Fighting corruption—especially grand corruption—requires effective anti-money laundering (AML) systems capable of efficiently and correctly flagging suspicious transactions. The financial institutions responsible for identifying and reporting suspicious transactions employ automated systems that identify transactions that involve certain red flags—characteristics like transaction amount, location, or deviation from a customer’s typical activity; when the automated system flags a transaction, this triggers further review. But—given the ever-increasing volume and complexity of financial transactions that occur each day, as well as the increasing sophistication of kleptocrats, criminal groups, and others in disguising their illicit activities to avoid the usual red flags—picking out the genuinely suspicious transactions can be extraordinarily difficult. Even the cleverest compliance system designer couldn’t hope to incorporate every potential red flag into the automated system.

The need to stay one step ahead of the bad actors has fueled greater interest in how new advances in data processing technology may help make automated suspicious transaction detection systems more effective. Techno-enthusiasts are particularly interested in deploying deep learning artificial intelligence (AI), as well as classic algorithms that fall under the machine learning (ML) umbrella, in the AML context. ML and AI systems extract patterns from training datasets, and “learn” (by induction) what data patterns are associated with particular identifiable categorizations. Email spam filters provide a simple example. A spam filter, which can be created to conduct a process known as classification, sorts input variables into two categories: “spam” and “not spam.” It makes its categorization based on individual characteristics of the emails (such as the sender, body text, etc.). In the AML context, the idea would be to train an algorithm with data on financial transactions, so that the system “learns” to identify suspicious transactions even in cases that might lack the usual red flags that a human designer would program into an automated system. Advocates hope that ML/AI systems could be used both to filter out the false positives (transactions which are flagged as suspicious but turn out, on review, not to raise any concerns—an estimated 99% of all flagged transactions), while also identifying unusual, potentially fraudulent behavior that may be overlooked by human regulators (false negatives). Indeed, industry experts are understandably enthusiastic about AI systems that will cut costs while improving accuracy, and proponents claim that “AI holds the keys to a more efficient and transparent AML stance[,]” urging that “[b]anks must take hold of this new [AML] weapon[.]”

To the extent that AI tools can improve upon the admittedly-clunky automated systems currently in use, it could be a step forward. But ML/AI systems have a less than stellar track record in other contexts, and a model targeted at AML compliance presents some unique challenges.

• First, ML models require at least some data in their training set with known truth values. In the spam filter example, a supervised ML model can be trained on data that is labeled “spam” and “not spam” so that the machine can learn which characteristics are associated with spam and non-spam emails, respectively. Even if a model were created using unsupervised learning – where the machine finds patterns within unlabeled data, and could group highly anomalous transactions together for further human investigation – researchers would need to be able to determine whether the “suspicious” transactions identified by the model actually captured instances of money laundering. In short, to train a model to identify likely money laundering, the model must either be trained with large sets of transactions labeled “money laundering” or “not money laundering” or researchers must be able to determine the truth values for transactions that the model groups together, to test its validity. Without knowing the correct labels for the data, researchers could struggle to train a supervised learning model well, or find themselves unable to verify an unsupervised model’s results. Unfortunately, there may not be many transactions known to have involved illegal activity, and researchers may be unable to identify whether the flagged transactions are actually suspicious. (Financial institutions rarely receive confirmation from regulators regarding which of their suspicious activity reports actually involved money laundering, in part because regulators have limited investigatory resources and can only pursue the most suspicious transactions.) An alternative approach would be for the regulated institution to train the algorithm using the institution’s preexisting risk scores for individual transactions, but this approach assumes the accuracy of those assessments, which is clearly problematic. To the extent that insufficient or inaccurately labeled data is used to train a model, the model will replicate those flaws.
• Second, and relatedly, ML and AI models have greater difficulty correctly identifying rare events. And money laundering, while far too common, is (so far as we know) quite rare when considered as a percentage of overall financial activity. The most pernicious forms of illicit financial activity—such as kleptocrats hiding their stolen loot—are even rarer. As a result, even if financial institutions were able to confidently identify those prior transactions that were verifiably-illicit (or verifiably-suspicious), there may simply not be enough of them to train certain types of ML and AI systems to accurately identify suspicious transactions in the future. Building a dataset that draws on information from multiple sources, as opposed to just one bank’s customer base, would help alleviate this problem, but AML data is difficult to collect and, for the most part, isn’t publicly available. This means that most institutions would probably have to rely on their internal transaction information, and there might not be sufficient verified money laundering incidents in that data to train some types of models (particularly those that employ supervised learning) adequately.
• Third, and related to the above problems, it can be very difficult to assess whether an ML system is relying on reasonable indicia of potential wrongdoing, or if the algorithm—perhaps due to inadequacies of the training data—has “learned” to associate money laundering with irrelevant transaction characteristics (for instance, the names of account holders), even when those characteristics have no actual relationship with suspicious activity. This problem is compounded by the opacity of certain types of ML models, particularly those that employ supervised deep learning. Because of the complexity of these systems, it can be extremely difficult to determine how these models make their decisions. Researchers have developed certain techniques to test whether an ML system is relying on spurious or irrelevant patterns, but these techniques generally require that the system engineers be able to make educated guesses about what types of information the model may erroneously be treating as significant.

For these reasons, it is not yet clear whether ML and AI models will deliver their promised error-reduction and efficiency gains. If these tools are not built with correctly labeled data sets, they will not deliver quality results (calling to mind the old adage, “garbage in, garbage out”). Moreover, there’s a risk that the enthusiasm for AI could foster a false sense of security within an institution, leading to overreliance on a highly imperfect tool. To be sure, some of the above concerns can be mitigated through appropriate measures (the use of synthetic data, or specialized models that predict rare incidences), and ML/AI systems may still represent an incremental improvement over existing automated review systems. But the challenges inherent in developing a robust model in the AML context suggest the need to proceed with greater caution.