Since 1999, I have been heavily involved in the field of computer-aided audit tools and techniques (CAATT). Consulting and publishing in this area has been, so to speak, my ‘day job’. All the books published to date by Ekaros Analytical have related to CAATTs. This fairly specialized work has overlapped very conveniently with my broader personal research interest in the maximum entropy principle (MEP).
The basic premise of CAATTs is that large accounting data sets contain an inherent order, a set of typical patterns. Error and fraud tend to show up as deviations from some pattern, and can be efficiently detected by using the pattern as our ‘prior expectation’. This much is commonly understood by auditors and fraud examiners everywhere. What I have discovered over the years is that the typical patterns studied by CAATTs are generally attributable to maximum entropy considerations. To understand the ultimate cause of this inherent order, and to exploit it fully, we need the MEP.
For example, while employed by ACL in their Global Audit Publications unit in 2000, I edited and published Mark Nigrini’s book Digital Analysis Using Benford’s Law. Benford’s Law starts with an empirical observation, that the first digits of data from a variety of fields are not evenly distributed in the range 1 through 9. Instead, a first-digit ‘1’ appears about 30.1 percent of the time, first-digit ‘2’ appears 17.8 percent of the time, and so on down to first-digit ‘9’ which appears just 4.5 percent of the time. The two-digit and three-digit combinations are likewise very orderly, with ’11’ appearing more often than ’12’ or ’13,’ and ‘242’ appearing more often than ‘278,’ and on.
If a data set deviates from Benford’s Law, the amount and direction of the deviation can point to items in need of investigation. For example, if there are more invoices from suppliers that begin with ‘179’ than expected, possibly the reason is that a large number of similar items were needed in a short time period and the purchasing was not well-organized. Perhaps there were dozens of separate purchases of $179 boots, which should have been made as a group and paid with a single check. This would be a finding worth reporting to management. Policies and procedures could be tightened up for next time and significant cost savings achieved.
Although there have been occasional papers on this subject since the 1930’s, Dr. Nigrini’s work was the first full-length book ever published about the first-digit-frequencies law. Digital analysis using Benford’s Law has become an accepted standard technique in the field, and Dr. Nigrini has become well-known.
My own work seeks to extend and deepen our knowledge of how and why nature shapes so many different data sets in this consistent way. One question we still need to answer is why Benford’s Law works the way it does. This is not so much an accounting question as a scientific one. The Decline Effect deals with this question in detail.
More pressing in practical terms is how to make the tools work better. Unfortunately, digital analysis and other standard CAATTs tests like searching for duplicate transactions still tend to generate large numbers of ‘false positives’. From the perspective of a busy auditor, this means CAATTs have a serious drawback.
Rather than look at all the transactions in a data set, perhaps 250,000 per year, the auditor has to perform sampling. CAATTs tests will pick out suspicious transactions that have a higher probability of containing error or fraud, but they are only so efficient. It would not be unusual for a CAATT report to contain 10,000 items flagged as suspicious from an original pool of 250,000. Clearly that is still far too many to handle manually.
For the past several years I have worked closely with Richard Lanza of Cash Recovery Partners to build next-generation CAATTs tools, such as adaptive sampling. Much of this research remains proprietary, shared only with our consulting clients, but much is available for general readers as well.
In 2001-02 I published a series of newsletters about my research in this area, which the reader is welcome to sample.