When you think about digital trust, you often think of how effective your security program is or how well data within your enterprise are protected. One often overlooked but extremely important aspect of digital trust, however, is the appropriate skilling of staff who perform those operations as well as staff at large—and whether they are poised to succeed in an ever-changing technology landscape.
This column will discuss this topic using an example of a framework—in this case the beta version of ISACA’s Digital Trust Ecosystem Framework (DTEF)1—to demonstrate how a framework can aid enterprises in guiding their workforce and prioritizing training of their staff to ensure and enhance digital trust.
My first exposure to a CMM was the Software-Capability Maturity Model (SW-CMM) which has now evolved to and integrated into the Capability Maturity Model Integration (CMMI).2 The DTEF is a framework built, in part, on the CMMI which is based on the previous Software-CMM.3 In this old Software CMM, in order for an organization to reach Maturity Level 3 (ML3), that organization had to have a well-defined organizational training plan.4 I was part of implementation team for the US Air Force organization, serving as one of the training officers at the time we were attempting a CMMI-Development ML3 certification.
The Old Software CMM said this about the need for a training program: “The purpose of the Training Program key process area is to develop the skills and knowledge of individuals so they can perform their roles effectively and efficiently.”5 When my organization first attempted to receive recognition as an ML3 organization, it did not have a properly documented training program. This deficiency was one of the ones cited as to why we did not receive an ML3 rating on the initial review. The team spent a significant effort reviewing how our training program ensured that we met the expectations for a successful training program as defined by the SW-CMM. As a result, the second time around we achieved our target rating as an ML3 organization.
Does the DTEF Care About Skills and Training?
The obvious answer is, “yes.” Thinking about how a capability maturity model works, getting to a level 3 status means having an organizational wide capability at whatever is being measured. An organization does not maintain this level of capability without proper training and retraining of its people. Given that the DTEF is focused on trust, not only in whether the organization is going to properly handle matters of data, privacy, and security, but also whether it can deliver a consistently acceptable (or exceptional) level of performance for goods and services, it should not surprise us that training and skills are emphasized in the DTEF.
Think about any time you’ve heard something akin to, “Your call may be monitored and recorded for quality and training purposes.” That speaks to the fact that there is a certain expectation level of performance and training is going to be required to meet that level of performance.
So where in the DTEF do we see reference to skills training? We find it in the Culture domain (figure 1).
In CU.02 Create and Manage the Digital Trust Cultural Environment, we have a practice to determine what skills and competencies digital trust requires. And then in CU.03 Manage Skills and Competencies, we have two practices around resources and training. The practices are further broken down into activities with defined outcomes that should be achieved in order to meet the requirements of the practice. The activities of CU.03.02 Conduct Regular Training include:
- CU.03.02.1 Conduct information and cybersecurity training.
- CU.03.02.2 Conduct privacy and data protection training.
- CU.03.02.3 Conduct ethics training.
- CU.03.02.4 Conduct continuity plan (back-up) training.
- CU.03.02.5 Maintain records of successfully completed employee training programs and credentials earned.
The first four activities comprise a significant amount of training. One could argue that most organizations require annual security awareness training, annual data privacy training for employees who handle sensitive data and annual code of conduct/ethics training. Some organizations conduct business continuity or disaster recovery tests, and this serves as a form of training, too. Therefore, when we look at those four activities, we are not seeing anything out of the ordinary. That is the point: the DTEF codifies what organizations should be doing, whether they are adhering to the DTEF or not.
That is the point: the DTEF codifies what organizations should be doing, whether they are adhering to the DTEF or not.
The one activity that stands out for some organizations is in the area of maintaining training records. Peter Drucker is quoted as saying two things that apply here. The first is, “You can’t manage what you can’t measure” (which he didn’t actually say; it is a simplified version of one aspect of Drucker’s nuanced view on the importance of measuring success).6 In context, what he meant is if you can’t define what is successful in a way that can be measured, you are going to have a hard time knowing if you’re truly successful. The second saying, which clarifies the first is: “If you can’t measure it, you can’t improve it.” Again, this isn’t something Drucker directly said or wrote, but he believed in measuring what can be measured, but that measurement alone wasn’t the only mechanism for determining success.7 That’s why the training records activity is crucial for any CMM-based framework. Without data to look at, it’s typically not possible to know how successful the organization is with regards to that activity. This is an area that can be measured and, therefore, should be. And, from that data, you can work to improve the effectiveness of the training which enhances the trustworthiness of your organization.
Digging Deeper into the DTEF’s Expectations
When we look at any particular activity within the DTEF, we should see three things: the outcome, key performance indicators (KPIs), and key risk indicators (KRIs). Breaking down the activity for information and cybersecurity training, we find that the outcome expected is: “cybersecurity training is created, delivered and measured for effectiveness.” The outcome is not just that the organization offered a class and people could sign up. There is an expectation that the organization is doing something to measure the effectiveness of whatever training is provided. This is where the KPIs and KRIs come in.
The KPIs for CU.03.02.01 include the fact that cybersecurity training exists, the number of sessions offered, what percentage of the organization has not completed the training, the number of feedback results and results from tests around the training’s effectiveness. One might wonder, “How can we test effectiveness?” The tests at the end of training should validate that those who have attended the training can and do apply what they have learned.
For instance, if the cybersecurity training covered phishing emails, an effectiveness test could be a simulated phishing campaign. The test could measure who clicked on the phishing link in the email, who reported it as a possible phishing attempt and who did not do either. We would expect personnel to be more aware immediately after training, but that is not always the case. For example, if the training was not clear or did not give those attending the training actionable steps to spot the majority of phishing emails, you can infer that the training was not effective. Measuring the effectiveness of training is important so that the organization is not lulled into a false sense of security.
A degradation in quality and performance will lead to a degradation of trust.
While we do have a good level of detail as to what is required to meet the requirements in the DTEF, keep in mind that the DTEF gives the “what” and not the “how.” For instance, the DTEF says to have a cybersecurity training program which is measured for effectiveness. It does not tell an organization how to do this as a singular approach will not work in every organization.
None of This Should Be New
A solid framework should lay out what an organization should be doing, regardless of their adherence or application of said framework. Understanding this concept, we should not be surprised that the DTEF is codifying what should be a reasonable expectation for any organization. However, the reason frameworks are built and best practices are codified is because there are plenty of organizations that either are not doing the reasonably expected things, or they are not doing them to an acceptable level. When we think about training, there is a saying that when an organization experiences financial stress, training is one of the first budgets to be cut. This happens all too often. However, cutting training results in the degradation of skills within the organization. That degradation of skills will directly lead to a degradation of both quality and performance. And from a digital trust perspective, a degradation in quality and performance will lead to a degradation of trust.
Appropriate and timely training is crucial for a solid and consistently performing organization. Measuring the effectiveness of training is necessary for the organization to have confidence in its ability to execute at an expected level of performance. If an organization wants a high level of trust in the digital world, just as in the physical world, it needs to ensure that its people are properly equipped to meet expectations.
Endnotes
1 ISACA®, Digital Trust Ecosystem Framework (DTEF), USA, 2022. The DTEF is currently in limited release and can be accessed at gm0h.caminal-equip.com/dtef-ebook
2 White, S. K.; “What Is CMMI? A Model for Optimizing Development Processes,” CIO Magazine, 1 June 2021, http://www.cio.com/article/274530/process-improvement-capability-maturity-model-integration-cmmi-definition-and-solutions.html
3 Ibid.
4 Paulk, M. C.; C. V. Weber; S. M. Garcia; M. B. Chrissis; M. Bush; Key Practices of the Capability Maturity Model, Version 1.1., Software Engineering Institute, Pennsylvania, USA, February 1993, p. 1-15
5 Ibid. p. L3-25
6 Zak, P.; “Measurement Myopia,” Drucker Institute, 4 July 2013, http://drucker.institute/thedx/measurement-myopia/
7 Ibid.
K. BRIAN KELLEY | CISA, CDPSE, CSPO, MCSE, SECURITY+
Is an author and columnist focusing primarily on Microsoft SQL Server and Windows security. He currently serves as a data architect and an independent infrastructure/security architect concentrating on Active Directory, SQL Server and Windows Server. He has served in a myriad of other positions, including senior database administrator, data warehouse architect, web developer, incident response team lead and project manager. Kelley has spoken at 24 Hours of PASS, IT/Dev Connections, SQLConnections, the TechnoSecurity and Forensics Investigation Conference, the IT GRC Forum, SyntaxCon, and at various SQL Saturdays, Code Camps and user groups.