With data analytics within reach of almost any team, the sample has started to look like an admission of weakness. If everything can be tested, testing twenty-five items sounds like not having done the homework. The Global Internal Audit Standards do not say that.
What the Standards say
The considerations for Standard 14.1 put the decision plainly. When gathering information, internal auditors "should consider whether to test a complete data population or a representative sample." And if they choose a sample, they should apply methods to make it as representative of the entire population as possible.
Standard 14.2 adds that internal auditors should understand and use technologies such as software applications "that enable testing of an entire population rather than just a sample." It is a clear invitation to use the full population when possible. It is not a ban on sampling. In fact, the considerations for Standard 13.6 state that when sampling is used, the work program should include the sampling methodology, the population, the sample size and whether the results can be projected to the population.
What the Standards ask for, then, is that the choice be made on purpose and written down.
Three cases where the sample is the right call
- Incomplete or unreconciled data. If the extract does not agree with the accounting total or the corporate directory, testing a hundred per cent of that extract is testing the wrong thing with precision. The first finding is the reconciliation gap. While it is being resolved, a sample drawn from the verified source can say more than the whole of a file nobody can trust.
- Unstructured information. Contracts, minutes, supplier reports or review evidence held in loose documents. Technology can locate and classify them, but judging whether a clause protects the organisation or whether a set of minutes records a real decision requires reading and judgement.
- Tests that call for human inspection. Standard 14.1 lists among its procedures directly observing a process and inspecting physical evidence. Checking that a review really happened and was not just a signature, or that an inventory exists, is not settled with a query.
In many engagements the answer is mixed. The full population for whatever can be tested with data, such as every payment having an approver different from the requester, and a sample for whatever needs judgement, such as whether that approval actually looked at anything.
What a sample has to disclose
- It is disclosed. In the working papers and in the report. Among the basic elements of a working paper, Standard 14.6 lists a description of the population evaluated, including the sample size and the selection method. A conclusion that does not say what it rests on cannot be reviewed.
- It is chosen by risk. High amounts, out-of-hours transactions, new suppliers, privileged users. A targeted selection is legitimate because it points at where the risk is, but it is not representative.
- It is not projected beyond what it supports. A targeted sample that finds errors shows those errors exist. It does not show what share of the population is wrong. Only a statistical method designed for that purpose allows projection, and the program has to have said so before testing.
A well-disclosed sample is evidence. A sample presented as if it were the population is a risk for whoever signs.
The useful question is not whether to sample. It is which part of the risk can be tested in full with the data available and which part needs someone looking. With access, for example, the user census is cross-checked in full against the corporate directory, which is exactly where testing the entire population changes fieldwork. The quality of a recertification, on the other hand, can only be judged by reading actual decisions.
