The Problem with Questionnaires on GRC Departments’ Functioning

There is an interesting discussion ongoing in The Internal Auditors Institute on LinkedIn regarding Key Performance Indicators. One of the respondents has given a link to a questionnaire ( ) for conducting an Internal Audit Effects and Effectiveness Study.

It is an interesting questionnaire as it asks many relevant points. However, my one issue with questionnaires is that internal audit function has many subjective aspects, so can it really be captured in an objective questionnaire. Look at the questionnaire since I have decided to play a devil’s advocate out here. I am first going to give you an exemplary IA function and then cross question the points raised in the questionnaire.

 Business Case Study

A multinational organization operating in India has a global independent team of internal auditors who periodically conduct audits for the Indian arm and other operations in the group. The Indian business arm additionally has risk managers in different risk departments namely internal controls and compliance, information security, fraud risk, physical security and disaster recovery. The heads of the risk management departments report to different CXOs, however there is a risk management committee where all meet regularly. Since the organization is geographically spread, there are local risk managers who report to the risk head and local head.

The risk management departments regular functioning is as follows. The global internal audit department and internal controls and compliance department issue an early audit calendar. The other risk departments prepare an annual plan and additionally keep time for contingencies. For example, in case of fraud department, there is a fraud prevention plan and time for investigations, as and when fraud occurs. Risk management department budgets are approved locally on an annual basis. Risk management departments issue reports to India CXOs and their respective global risk managers.     

Now doesn’t this case study sound good? If we start ticking the questionnaire, we would rate most of them somewhat to totally agree.

The Questionnaire

1.      Basic Details

The first section is regarding respondent demographics requesting for details of staff strength, revenue, number and qualification of auditors.

My question is what is the right ratio for staff strength to auditors or revenue to auditors? How does one measure whether the risk management departments are adequately staffed? A banking corporation has higher risks than manufacturing corporation, so how does one determine the right number?

The next question is that if let us say organization “A” has 10 highly trained and experienced audit staff and “B” has 20 somewhat experienced audit staff, which is better?

In the business case mentioned above, if operations team members are put as auditors, though they are highly skilled operations people, can they be assumed equally effective as auditors? Will they not be requiring much more training than an experienced audit staff? Will having number of heads in various risk management department serve much purpose? An untrained staff is unlikely to identify the real problems and may give pleasant reports stating that the organization is risk free.

2.      Reporting of Internal Audit Head & Independence

The reporting of Internal Audit head, and objectives and deliverables are considered critical to measure the independence of the internal audit department.

Now let us take the business case mentioned above. Looking at the departments’ structure, one would say that it is quite reasonable though not perfect. The good aspect is that internal auditors are reporting to global head in a different division. Although, the internal controls and compliance head is reporting to the CFO, it is not all that bad. Other risk management departments’ heads are reporting to different CXOs, so there is some level of independence.

Let me put a twist to the case. In the past one year, internal controls and compliance departments were merged. The compliance departments previously had highly trained senior expats from UK and USA, working in India and they returned to their respective countries. Their positions were not filled with replacements, but the juniors’ team was merged with the internal controls department. The key seniors in fraud and IT security were fired during the year; and replaced with cheaper and less experienced resources. The story was that all the risk managers who put their foot down were either fired or moved out of their respective departments. The strategy adopted by senior management was strike the shepherd and sheep will scatter. Hence, the surviving risk managers started complying with senior management orders without raising any questions whatsoever.

In such a case, how does one measure effectiveness and deliverables of the risk management departments? Do the questionnaires really help in identifying the real problems or are they simply superficial? In such a scenario, will the operations team actually respect the risk manager’s advice? Will the internal auditors sitting in India actually report the politics to the global team? Even if global internal audit head receives the report, will that report be useful? What will actually get reported to the local and global board? My opinion is that in this setup the senior managers will ensure that risk managers only write what they wish. There is no independence in the functioning of risk management departments.

3.     Performance of Internal Audit Department

As you will see in the section 5 of the questionnaire there are over 34 questions to judge the performance of the internal audit department. They relate to ethics, trustworthiness, capability, value addition etc.

Normally, an ethical internal audit department will be one that operates independently, reports accurately and maintains confidentiality and integrity. The internal audit department will be considered effective if recommendations for critical observations reported are implemented, it has access to board and audit committee, participates in strategy development and influences senior management positively.

Now if I take the business case with the details mentioned in point 1 &2 above, my question is on what basis does one measure effectiveness of internal audit department? Should the appraisal or feedback of senior managers be considered into account? In my view, in such a scenario the senior managers will provide positive feedback to the risk managers who show least resistance and comply with their orders. They may consider trustworthy as one who can be relied upon to issue non-controversial reports. In this case, senior management lacks commitment for risk management initiatives.

My argument may sound harsh (I think there are no maybe’s, it is harsh) but that is the reality in some organizations where the culture is aggressive and/or deviant. Therefore, the point I am making is, that if organization culture is not taken into account, most of results of these questionnaires will be misleading. The subjective factors of the functioning need to be taken into account for even objective questionnaires.

One comment on “The Problem with Questionnaires on GRC Departments’ Functioning

  1. Pingback: Whistle Blowing Survey- Reasons Why Surveys are Unreliable « Sonia Jaspal's RiskBoard

Comments are closed.