Gaining Insights of Manual and Automated Tests of Effectiveness

The Control Test of Effectiveness involves periodic testing to ensure that controls are functioning effectively at a specific point in time within an organization.

Both manual and automated controls can validate a test of effectiveness. For a detailed understanding of the distinctions between these types, please refer to the previous unit on "Identifying the Key Capability: Perform and Monitor."

This lesson focuses specifically on manual controls and their evaluation.

The Workflow of Test of Effectiveness consists of eight steps that outline the complete process, starting from defining a manual test plan to resolving any identified issues.

Now, let's delve into the workflow of a manual test of effectiveness in detail

1. Define a Manual Test Plan: A manual test plan contains a sequence of steps/tests that the control tester should perform to test the operating effectiveness of the control. Within the manual test plan central library, you can define all the test procedures and then tag them to the applicable controls (this will be further explained in the following section) It is possible to do it for controls that are created centrally or already local mapped to an organization.
2. Map Test Plans to Controls: Once the test plan is defined, it needs to be allocated to a control.
3. Schedule the Planner: In the "Planner" application the GRC or internal control manager (depends on the role assignment in the organization) can create the plan activity "Test of Effectiveness". This step refers to Unit "Identifying the Key Capability: "Plan".
4. Complete the Test Plan: The control tester receives the workflow in their Work-Inbox and must complete the test plan. There are two potential outcomes during the testing phase: the testing result can either pass or fail. Let's explore both scenarios.
   1. Testing Result: Pass

      If the control's operating effectiveness is adequate, the control tester rates it as "Pass" after completing the test plan.

      The control tester:

      Completes each step of the test plan.

      Submits results and additional information in the Comments section.

      Concludes all required tests and marks the overall testing rating as Pass.

   2. Testing Result: Fail

      If the control is not operating effectively (steps 6-8), the control tester rates it as "Fail" after completing the test plan. In this scenario, remediation steps must be taken, involving the control tester, issue owner, and remediation owner.

5. Review Testing: Depends on the outcome during the testing phase the lead of the internal controls team (mostly internal control manager) which is responsible for reviewing the testing can either approving or rejecting the results. If the testing result is passed than the test plan can be completed. But if the testing result is failed then the yellow steps (6-8) will be performed. Please note, that this step "review testing" is optional and can be enabled during the customizing phase.

6/7/8 Issue Remediation, Implementation of Remediation Plan, and Close Issue: If step 4 is failed the issue needs to be validated if a remediation plan is necessary or not. This will be explained in more detail in the following lesson "Exploring the Concept of Issue and Remediation" of this Unit Identifying the Key Capability: Evaluate.

Creating and Editing Manual Test Plans is possible via accessing the APP Manual Test Plan. A list of test plans and their associated controls appear. Select Create (1.) to define a new test plan or Open to change an existing plan.

On the General (2.) tab, you can enter or change general data like Test Name, Description and a date range for the validity of the test plan. At the Test Steps (3.) pane, select Add to add new steps or, to delete an existing step, select the step and then select Remove. In the Step or Test dropdown menu, select either Step or Test to indicate if this step is for manual controls or is a test for automated controls. In the Required dropdown menu, select Yes or No to indicate whether or not this step is required. The Fail Ends Test dropdown menu indicates whether or not to end the test if this step fails. In the Initial Sample field, enter a description for the initial sample and in the Sampling Method dropdown menu, select the desired sampling method.

Optionally, select the Attachments and Links (4.) tab to attach files or links to your test plan.

In General Accounting a business unit of CRG International, Inc., the role of GRC Admin is held by Internal Control Manager Ian Robb. He activates the plan "Test of Effectiveness" in the planner application (Steps 1-3) for June 2024. The details for the manual control to be tested are as follows:

• Control Name: Consolidate Financials & Disclosures Reviewed.
• Control Description: Consolidated financial statements and disclosures are timely reviewed and validated with any other departments involved (Legal department for tax, Financial controller for P&L information, Treasurer for cash flow and debt/equity) before the final sign-off.
• Operation Frequency: Monthly.
• Control Group: Financial Reporting and Disclosure.

After Ian activates the plan, Jenny , as the control tester, receives an item in her Work Inbox. She proceeds to complete the test plan (Step 4) by reviewing the consolidated financial statements and disclosures. She ensures that the review and validation process was completed before the final sign-off, collecting and examining evidence such as emails, meeting minutes, or signed validation forms from each involved department, including the Legal department, Financial controller, and Treasurer.

Test Execution Process

Jenny reviews the documentation, checks timestamps and sign-off dates for timeliness, and collects necessary evidence from the involved departments. She marks the "Perform" step as failed because the test procedures, as performed, did not meet the requirements outlined in the attached guidance.

Test Results Process

The test result is marked as failed because the Financial controller's validation was completed two days after the final sign-off, indicating a failure to review timely. Additionally, the Treasurer's validation for cash flow and debt/equity was missing for the month tested.

Issue Creation Process

Jenny creates an issue summarizing that the control "Consolidate Financials & Disclosures Reviewed" failed the effectiveness test due to the lack of validation from the Legal department. This failure poses a potential risk of financial misstatements due to incompleteness.

The issue is assigned to the issue owner for issue remediation (Steps 6-8), which will be explained in the next Lesson, "Exploring the Concept of Issue and Remediation".