Monitoring as Measurement
- Posted by David Gray
- On August 27, 2020
- Accountability, Any Reasonable Interpretation, Monitoring
I’ve had occasion in recent times to observe a number of boards undertaking monitoring activities. A couple of takeaways have stuck in my mind as a result of these experiences, one of which I’ll describe below and the other I’ll describe in a subsequent blog.
One characteristic common to each of the monitoring processes I observed was that they took the form of a description by the CEO of the organisation’s operations. Some of these descriptions were very enlightening and gave great insights into what was happening within the organisations. Furthermore, board members were clearly interested in hearing this information and usually gave the CEO plenty of time to speak.
However, none of the descriptions—or narratives, or stories—comprised an adequate monitoring process.
Now, I know from having sat on many boards myself over the years, that monitoring generally isn’t the most exciting or interesting part of a board’s agenda (especially financial monitoring, which is guaranteed to induce glazed eyes in many board members). However, disciplined monitoring is crucial to effective organisational performance, so it’s worth reiterating what a good monitoring process should comprise.
Fundamentally, monitoring is a measuring process. The board is measuring the extent to which the CEO is complying with its policies, whether Ends policies or Executive Limitation policies.
Under Policy Governance, the CEO is delegated authority to make any reasonable interpretation of the board’s policies. When it comes time for the board to assess the CEO’s interpretations, therefore, there are some key things for the board to look for.
First, the board must ensure that the CEO’s interpretation comprises an operational definition of the policy—a clear, concise, detailed definition of a measurement procedure in accordance with the CEO’s interpretation. This operational definition must be such that anyone could repeat the measurement process independently if necessary.
Second, the board must consider whether the interpretation—i.e. the operational definition—is reasonable. For an interpretation to be reasonable, it must include a defensible measure or standard of measurement against which achievement of the policy can be assessed. A defensible measure is one that is externally verifiable by virtue of research or testing or some other credible confirmation process.
Third, the measure must be specific enough so that anyone looking at it would know exactly what to measure—say, for example, an external auditor who was asked to undertake some third-party investigation and reporting.
Fourth, the measure must include a level of achievement on the measure which shows compliance. Is achieving 10% of the measure adequate to show compliance? 50%? 95%? The interpretation must include this information.
Finally, the interpretation must include a rationale justifying both the choice of the measure and the choice of the level of achievement on that measure. I’ll have more to say about rationale in the next blog.
For a monitoring process to be effective, it must include each of these components. Most of the CEO narratives I’ve listened to in recent times haven’t included all—or even most—of these components, and have therefore failed to meet the boards’ need for appropriate monitoring information.
(By the way, there’s nothing wrong with the CEO telling the board a story about how the organisation is performing, or about other aspects of the organisation’s operations—just not as part of the monitoring process. Such information, in my experience, is better kept for the information section of the board’s agenda, or for other forums such as board orientation or education, background briefings for policy development purposes, and so on.)
In the next blog, I’ll take a closer look at the role of the rationale in the monitoring process.