Criteria, not common practice
The yardstick in performance audits and compliance audits within government internal oversight
Summary
An audit finding arises from comparing the existing state with criteria. In practice, that yardstick sometimes shifts, unnoticed, from the applicable criteria to common practice. The shift usually surfaces in a familiar line of defence: that everyone does it this way anyway.
This note argues that the shift is not caused by an absence of criteria. It happens because knowledge of the criteria is unevenly spread, and because auditors themselves may work in agencies that follow the same habits. What is lost is not only the accuracy of findings, but also anyone's reason to propose a better way of working.
The note also sets limits on its own thesis. Procedures updated on the basis of field practice remain legitimate, and are even necessary, as long as that practice is tested against the higher-level requirements that govern it. What is rejected is not practice, but practice that has not been tested and is then used as a yardstick.
1. A sentence that sounds like a defence
In performance audits and compliance audits, and now and then in evaluations too, one sentence comes up often when a condition is called into question. It goes roughly the same way: other agencies do it this way too.
The sentence does not dispute the condition. It does not claim that the state of affairs in question in fact complies with the rules. It only moves the yardstick, from what ought to be to what commonly happens.
The move is easy to accept because it feels fair. Holding one party to a requirement that others do not meet looks like unequal treatment. Yet fairness of treatment and accuracy of the yardstick are two different things, and this note is about the second.
Indonesia's regulation on the government internal control system divides audits into performance audits and special-purpose audits (audit dengan tujuan tertentu). Special-purpose audits cover audits that are not performance audits (Pemerintah Republik Indonesia, 2008, Article 50(1) and (3)). In this note, compliance audits fall under special-purpose audits.
2. Criteria according to the standards
Indonesia's government internal audit standards state that adequate criteria are needed to evaluate governance, risk management, and internal control. Auditors must ascertain whether management has established adequate criteria, use those criteria where they are adequate, and develop other criteria together with management where they are not (Asosiasi Auditor Intern Pemerintah Indonesia, 2021, paragraph 2210.A3). The interpretation of that provision names three types of criteria: internal criteria, such as the organisation's policies and procedures; external criteria, such as regulations; and leading practices, for example industry and professional guidance.
The Global Internal Audit Standards give a broader list of examples. It includes authoritative practices, such as frameworks, standards, guidance, and benchmarks specific to a particular industry or profession, followed by established organisational practices, and procedures that may not be formally documented (The Institute of Internal Auditors, 2024, Standard 13.4). Read in context, the organisational practices meant here are those of the organisation under audit, and the same standard asks auditors first to assess whether such criteria are adequate.
That standard also advises auditors to compare management's criteria with those used by other organisations. Under that advice, it is the criteria that are compared, to judge their adequacy, not the condition, to justify the existing state. Elsewhere, the Global Internal Audit Standards list comparison with information from similar organisations as one example of an analysis, and analyses are used to establish whether the condition differs from the criteria set under Standard 13.4 (The Institute of Internal Auditors, 2024, Standard 14.2). Information from other organisations does not replace those criteria.
Two things, then, are not found in either standard. First, practice that is common among other agencies is not named as a type of criteria, and leading practice is not the same as the most widespread practice. Second, neither standard names practice as a substitute for the provisions in force. Both ask for the adequacy of criteria to be assessed first.
3. Why common practice takes the place of criteria
The criteria exist, but knowledge of them is uneven. Regulations and technical guidelines are available to anyone, but the only people who read them are those who actually want to know. Many of the staff who carry out the work in audited agencies do not read them, and not everyone in the auditing agencies reads them either. When the requirements are not known, all that is left to go by are the habits people can see.
Auditors may follow the same habits in their own agencies. Auditors do not come from a place free of habits. If the way of working being questioned is also followed in their own office, they tend to regard it as normal before the assignment has even begun. The yardstick has shifted before a single condition has been examined.
The two causes reinforce each other. Those being audited do not know the criteria, those doing the audit are used to the common practice, and the claim that every agency does it this way is accepted by both sides without anyone feeling that they are setting the rules aside.
4. What is lost
The accuracy of findings. A condition that departs from the requirements goes unrecorded as a finding because it matches the condition elsewhere. The deviation is still there; it is simply no longer visible.
A reason to propose improvement. If the yardstick is common practice, an agency whose way of working is already the same as that of most agencies has no reason to change. The auditor has no basis for recommending a better way, and the audited agency has no incentive to look for one. Both lose their reason at once.
A footing for the principles of economy, efficiency, and effectiveness. All three rest on the question of whether resources could be used better. For an agency whose way of working is already the same as that of most agencies, common practice offers no better yardstick, so gradual improvement has nothing to aim at.
Procedures that are no longer a design. Operating procedures drawn up with common practice as their reference only record what happens to be in place already. They no longer state how the work ought to be done.
5. Practice is not the enemy
This note does not argue that procedures must always be drawn up from the rules alone, without looking at the field. Procedures that are continually updated in line with conditions and practice in the field are in fact the most likely to be right, because the work changes faster than the documents that govern it.
Feldman and Pentland (2003) show that the performance of a routine does not merely repeat how it is formulated; it is also a source of adaptation and change.
There is one limit. Practice is legitimate material for updating procedures as long as it does not breach business processes or higher-level requirements, such as technical guidelines and laws and regulations. Practice that has been tested against those requirements is material for improvement. Practice that has not been tested, and is then used to assess a condition, is common practice disguised as criteria.
6. The working rules
Rule 1. The applicable criteria are looked for first. Before a condition is compared with any practice, the auditor establishes whether a requirement governing it exists. As long as that requirement exists, common practice is not the yardstick.
Rule 2. Other agencies' practice is used to assess the adequacy of criteria, not to justify a condition. Comparing an agency's criteria with the criteria used by other agencies is a recommended approach. What is not legitimate is using a condition that is common in other agencies as a reason to accept a condition that departs from the requirements. The number of parties doing it is not a reason.
Rule 3. The source of the criteria is named in the working papers and in the report. A yardstick whose source is not named cannot be told apart from the auditor's own habits.
Rule 4. Criteria are communicated in a form the auditee can read. Because one of the causes of the shifting yardstick is uneven knowledge, stating the requirements clearly is part of the audit work, not an add-on to the report.
Rule 5. Habits in the auditor's own agency are measured against the same criteria. An auditor examining a way of working first checks whether the same way of working is followed in their own office, so that they know where the sense that it is normal comes from.
Rule 6. Procedures updated on the basis of practice are tested against the higher-level requirements before they are formally adopted. Field practice may be a source of updates, but without that test it does not become the basis for adoption.
7. How to tell whether the rules work
- Does every finding name its criteria with a clear source, whether a requirement (a regulation, a contract, or an adopted policy) or other criteria whose adequacy has been assessed, rather than common practice among other agencies? A condition that cannot yet be matched against criteria of that kind remains a working-paper note, not a finding.
- Do the working papers record that the applicable requirements were looked for before other agencies' practice was used as a yardstick?
- When the auditee answers that other agencies do the same, is that answer recorded, and does it change the conclusion?
- How many times a year are operating procedures updated, and does each update record the requirements used to test it?
8. Limits of the claims
This note states working rules, not test results, and does not yet report any measurement.
Deviations that keep recurring gradually come to be accepted as normal. The phenomenon is not an idea this note introduces; in general terms it has long been described, by Vaughan (1996) among others. The obligation to use adequate criteria is likewise already written into the standards. The contribution this note claims is limited to two causes of the shifting yardstick in government internal audit, namely uneven knowledge of criteria and the prevailing habits that may also be followed in the auditor's own agency, together with their effect on the reason to propose improvement. This note does not claim that the problem has never been discussed in the performance audit literature.
Reviews are not discussed. Their scope is narrower, and the problem is different.
The rules above are the author's proposals, not an official interpretation of any standard. This note does not discuss any audited entity, contains no audit results, and does not assess any particular work unit.
References
- Asosiasi Auditor Intern Pemerintah Indonesia. (2021). Peraturan Nomor PER-01/AAIPI/DPN/2021 tentang Standar Audit Intern Pemerintah Indonesia [Regulation No. PER-01/AAIPI/DPN/2021 on the Indonesian Government Internal Audit Standards]. https://itjen.pertanian.go.id/storage/master/file/knno0IEe_saipi2021.pdf
- Feldman, M. S., & Pentland, B. T. (2003). Reconceptualizing organizational routines as a source of flexibility and change. Administrative Science Quarterly, 48(1), 94-118. https://doi.org/10.2307/3556620
- The Institute of Internal Auditors. (2024). Global internal audit standards. https://www.theiia.org/globalassets/site/standards/globalinternalauditstandards_2024january9.pdf
- Pemerintah Republik Indonesia. (2008). Peraturan Pemerintah Nomor 60 Tahun 2008 tentang Sistem Pengendalian Intern Pemerintah [Government Regulation No. 60 of 2008 on the Government Internal Control System] (Lembaran Negara Republik Indonesia Tahun 2008 Nomor 127).
- Vaughan, D. (1996). The Challenger launch decision: Risky technology, culture, and deviance at NASA. University of Chicago Press.
About the author
Faizul Abrori is an auditor at Inspektorat Aceh, the internal audit office of the Aceh Provincial Government in Indonesia. The work is mostly reviews and audits of regional government finances. The rest goes into designing and building, personally, the tools used in that work, and into writing about oversight practice from personal experience and views. This note is one of them.
Email halo@fairi.id. Other writings and a full profile at https://fairi.id.
The views expressed in this note are the author's own and do not represent the official position of Inspektorat Aceh.