Security Magazine logo
search
cart
facebook twitter linkedin youtube
  • Sign In
  • Create Account
  • Sign Out
  • My Account
Security Magazine logo
  • NEWS
    • Security Newswire
    • Technologies & Solutions
  • MANAGEMENT
    • Leadership Management
    • Enterprise Services
    • Security Education & Training
    • Logical Security
    • Security & Business Resilience
    • Profiles in Excellence
  • PHYSICAL
    • Access Management
    • Fire & Life Safety
    • Identity Management
    • Physical Security
    • Video Surveillance
    • Case Studies (Physical)
  • CYBER
    • Cybersecurity News
    • More
  • BLOG
  • COLUMNS
    • Cyber Tactics
    • Leadership & Management
    • Security Talk
    • Career Intelligence
    • Leader to Leader
    • Cybersecurity Education & Training
  • EXCLUSIVES
    • Annual Guarding Report
    • Most Influential People in Security
    • The Security Benchmark Report
    • The Security Leadership Issue
    • Top Guard and Security Officer Companies
    • Top Cybersecurity Leaders
    • Women in Security
  • SECTORS
    • Arenas / Stadiums / Leagues / Entertainment
    • Banking/Finance/Insurance
    • Construction, Real Estate, Property Management
    • Education: K-12
    • Education: University
    • Government: Federal, State and Local
    • Hospitality & Casinos
    • Hospitals & Medical Centers
    • Infrastructure:Electric,Gas & Water
    • Ports: Sea, Land, & Air
    • Retail/Restaurants/Convenience
    • Transportation/Logistics/Supply Chain/Distribution/ Warehousing
  • EVENTS
    • Industry Events
    • Webinars
    • Solutions by Sector
    • Security 500 Conference
  • MEDIA
    • Videos
      • Cybersecurity & Geopolitical Discussion
      • Ask Me Anything (AMA) Series
    • Podcasts
    • Polls
    • Photo Galleries
  • MORE
    • Call for Entries
    • Classifieds & Job Listings
    • Continuing Education
    • Newsletter
    • Sponsor Insights
    • Store
    • White Papers
  • EMAG
    • eMagazine
    • This Month's Content
    • Advertise
  • SIGN UP!
CybersecurityManagementSecurity NewswireSecurity Enterprise ServicesSecurity Leadership and ManagementLogical SecuritySecurity & Business ResilienceCybersecurity News

Technical competency gaps in 151,000 IT auditors in the audit industry

By Dr. Blake Curtis
cybersecurity-audit-fp1170x658v.jpg

Image via Freepik

November 1, 2022

IT audits have a bad rap for being a “tick-box” activity that only focus on surface-level risks and abstract controls. A recent international study, “The Next Generation Cybersecurity Auditor,” found a significant knowledge gap in 151,000 Big Four IT Auditors’ theoretical knowledge and practical skill. 


The study discovered that IT auditors’ lack of hands-on skill in information technology influences data breach likelihood and technical evidence interpretation for critical infrastructure (power, water, communication, and banking). This study distinguished what the auditors know (declarative knowledge) from what the auditor can do (procedural knowledge) by creating declarative and procedural knowledge assessment inquiries.


For instance, this assessment expounded on common concepts like least privilege and separation of duties via task-based activities. This strategy required the respondent to test their knowledge against specific technologies like Microsoft Server, Amazon Web Services (AWS), Palo Alto firewalls, Kubernetes containers, and Microsoft Azure.


The study’s population consisted of IT auditors, IT professionals, and cybersecurity practitioners. The primary target, IT auditors, either currently or previously, worked in first and second-tier audit firms like Deloitte, PwC, Ernst & Young, KPMG, RSM Tenon, BDO, Grant Thorton, and Smith & Williamson. The study collected survey/assessment responses from 108 IT auditors and 108 subject matter experts (SMEs) and can be generalized to 151,000 IT Auditors in the industry.


The study leveraged the National Institute of Standard and Technology (NIST) ’s National Initiative for Cybersecurity Education (NICE) framework and the Skills Framework for the Information Age (SFIA) to measure IT auditor and SME competence. Specifically, this study utilized NICE’s Security Control Assessor & IT Program Auditor work roles, tasks, and knowledge, skills, and abilities (KSAs).


Let’s take a look at the most significant findings from the study. 


IT Auditors Have Sufficient Book Knowledge, But Inadequate Practical Skills

The results indicate that IT auditors and subject matter experts (SMEs) have sufficient declarative knowledge (book knowledge). IT auditors had an average declarative knowledge score of 25.56 (Level 3 – Apply) on the Skills Framework for Information Age (SFIA) scoring model. The SMEs had an average declarative knowledge score of 33.61 (Level 4 – Ensure).


Table

Description automatically generated

The Skills Framework for Information Age (SFIA) levels for Autonomy and Knowledge are used to grade technical competency.

 

 

Timeline

Description automatically generated

SME and IT Auditor SFIA Theoretical Knowledge Scores


Conversely, the IT auditors had inadequate levels of procedural knowledge. For example, IT auditors had an average procedural knowledge score of 19.35 (Level 2 – Assist). In contrast, the SMEs performed better on procedural knowledge questions than IT auditors and achieved 25.19 (Level 3 – Apply). These findings suggest that the current education models in IT certification exams, college curricula, certification boot camps, and training seminars do not provide task-based skills to help implementers and assessors improve their procedural knowledge (demonstrable skills).


Timeline

Description automatically generated with medium confidence

SME and IT Auditor SFIA Practical Knowledge Scores

IT Auditors' Proficiency Influences Audit Quality & Data Breach Likelihood

Each declarative and procedural knowledge question was associated with missing safeguards reported in notable breaches like the 2019 Capital One data breach and the 2017 Equifax data breach. Therefore, failure to identify the appropriate safeguards or interpret technical evidence would result in unidentified technical risks and increase the data breach likelihood. These results suggest that the cybersecurity, IT, and audit professions should emphasize virtualized and scenario-based training to equip the next generation of cybersecurity professionals. This strategy will prepare implementers and assessors for emerging technologies such as Industry 4.0 and reduce the likelihood of data breaches.


Significant Gap Between IT Auditors' and SMEs' Declarative and Procedural Knowledge

There was a 17% decline from the IT auditors’ declarative to procedural knowledge. In other words, IT auditors had sufficient theoretical knowledge (Level 3 – Apply), but their ability to apply the same concepts procedurally decreased significantly (Level 2 – Assist). These findings suggest a significant knowledge gap exists between IT auditors’ declarative and procedural knowledge in first and second-tier accounting/audit firms.

Graphical user interface

Description automatically generated with medium confidence

IT Auditor’s Knowledge Decline from Theoretical Knowledge to Practical Skill

Likewise, there was a 26% decline from the SMEs’ declarative to procedural knowledge. In other words, SMEs had high levels of theoretical knowledge (Level 4 – Enable), but their ability to apply the same concepts procedurally significantly decreased to Level 3 – Apply. These findings suggest a significant knowledge gap between SMEs’ declarative and procedural knowledge.

 

Graphical user interface

Description automatically generated

SME’s Knowledge Decline From Theoretical Knowledge to Practical Skill


Confidence Levels vs. Actual Performance (The Dunning-Kreuger Effect)

 

IT Auditors' Confidence Did not Align With Their Performance

The IT auditors possessed high confidence levels in interpreting technical evidence to form conclusions on modern information technology systems’ overall security/risk. Conversely, the auditors’ procedural knowledge score (19.35) did not align with their self-perception (confidence) rating (3.98). These results are congruent with previous findings, demonstrating how auditors had overconfidence in audit tasks, even when those activities fell outside their knowledge, skills, and abilities.


SMEs had above-average confidence levels regarding their ability to interpret technical evidence to form conclusions on modern information technology systems’ overall security/risk. The SMEs had a procedural knowledge score of 25.19 (Level 3 – Apply) and a confidence level of 3.50 (Above-average). Although SMEs’ confidence was more aligned with their ability to audit and interpret evidence than IT auditors, there was no significant alignment between their self-perception (confidence) and procedural knowledge.

 

Age Influences Over-Confidence

This study discovered that age might negatively affect confidence for both IT auditors and SMEs. For example, both SMEs’ and IT auditors’ confidence increased with age. However, no data suggested that higher confidence levels translated to higher performance.

 

Certain Certifications Influence Over-Confidence

This study discovered that certifications like the Certified Information System Auditor (CISA) and Certified Information Systems Security Professional (CISSP) were associated with 28 IT auditors who rated their self-perception (confidence) as Very Confident. Conversely, IT auditors that rated themselves as Very Confident were novice performers with an overall score of 40.48 (Level 2 – Apply). In addition, this study also discovered that the CISSP certification was associated with 17 SMEs who rated their self-perception (confidence) as Very Confident. Similarly, SMEs who rated themselves as Very Confident were average performers with an overall score of 57.06 (Level 3 –Apply).


The figure below compares the IT auditors and SMEs’ self-perception (confidence levels) to their theoretical knowledge (declarative) and practical skill (procedural). Proficiency for overall performance occurs at SFIA Responsibility Level 3 – Apply (50 to 70). However, declarative and procedural knowledge proficiency occurs at SFIA Responsibility Level 3 – Apply (21 to 30).

 

Chart

Description automatically generated

 

Academic Degrees Do Not Significantly Influence IT Auditors' Performance

Higher academic degrees did not significantly influence IT auditors’ overall performance. Although there were no Level 5 (Ensure) IT auditors in overall performance, four of the nine Level 4 (Enable) auditors held bachelor’s degrees, and the remaining five held master’s degrees. The study also had five IT auditors with doctoral degrees. On average, the doctoral participants achieved Level 2 (Assist). These findings suggest that solely using degrees for IT auditors is a flawed way to measure performance. In contrast to mature fields like healthcare, no state and federal boards require students to obtain licensure after graduation. Most importantly, obtaining degrees does not illuminate the student’s actual grades, and many college courses may curve the class’s scores.

 

Academic Degrees Do Not Significantly Influence SMEs’ Performance

There were six Level 5 (Ensure) respondents from the SME perspective. Four respondents had master’s degrees, and two held bachelor’s degrees. There were 15 Level 4 (Enable) SMEs. One had no degree with some college credit, three participants had bachelor’s degrees, and the remaining 11 had master’s degrees. Lastly, one SME had a doctorate and achieved Level 3 (Assist). These results indicate that academic degrees do not substantially improve performance for IT auditors or SMEs. This disparity could be explained by the curriculum, tasks, and benchmarks utilized in college courses.


Top Performers Had a Combination of Implementing and Auditing Experience

Individual certifications and licenses did not substantially influence performance. However, certain vendor-specific, cyber, and IT audit certifications potentially improved performance in professionals with hands-on experience implementing and auditing technology. For example, the Level 4 (Enable) IT auditors had both implementing and IT auditing experience. In addition, all Level 4 IT Auditors also held the Certified Information Systems Auditor (CISA) certification and credentials by (ISC)2, CompTIA, EC Council, GIAC, and The Open Group (e.g., TOGAF).


Chart, radar chart

Description automatically generated

Chart

Description automatically generated


Additional Years of Exposure (Experience) Did Not Improve Performance

Simply utilizing years of experience (exposure) did not yield increased declarative and procedural knowledge performance. The study discovered that using years of experience was too broad and did not consider the hours spent performing job tasks in an eight-hour workday.


This study found that the average employee only spends 2 to 4 hours completing the activities listed in their job description. These findings suggest that although someone may have more years of exposure, they could have less task-based experience than other professionals with fewer years of exposure.


Previous research indicates that reviewing additional years of experience or seniority is a poor proxy for auditing or implementing expertise. Moreover, the world’s top expert on expertise, Anders Ericsson, says that acquiring 10,000 hours or numerous years of exposure in a particular role does not make one an expert.


In fact, the study debunked the years of experience fallacy. The research found that neither age nor additional years of experience influenced declarative or procedural knowledge. However, the study found that the number of times a professional performed a task along with the task quality, capacity, and speed was a more objective way to measure experience versus using time spent on earth or time spent in a certain role. Conversely, employers who utilize years of exposure to hire and promote candidates are not employing objective measures. Instead, enterprises that subjectively review resumes and prioritize years of exposure may hire individuals with novice task performance.

 

 

Next-Gen Auditors Can Protect the Nation's Critical Infrastructure

The role of the IT auditor is too broad to be effective in the 33 specialty areas (subdisciplines) of cybersecurity. The rapid introduction of emerging technologies, such as Industry 4.0, will require competent assessors to identify technical risks and recommend appropriate safeguards.

Today, the role of the IT auditor is analogous to a general healthcare provider. For example, the physician performs your routine checkup, and if they discover anything awry, they refer you to a specialist like a surgeon, neurologist, or other types of physicians. In contrast, the industry requires the IT auditor to play the role of the general healthcare provider and numerous specialists.

Most importantly, this study discovered that the lack of vendor-specific certifications and hands-on experience implementing technology potentially influenced the auditor’s evidence interpretation and audit quality.

 

Creating Specialty Cybersecurity Auditor Roles

The future of the IT auditing profession should create specialty positions and hybrid roles to improve technical competency and audit quality. For example, auditors required to audit artificial intelligence, machine learning, operating systems, networks, and firewalls should have hands-on training and implementation skills in the same technologies. These competencies enable the auditor to identify material misstatements, identify technical risks, communicate with SMEs, and recommend safeguards that reduce data breach likelihood.

KEYWORDS: auditing cyber security data breach NIST cyber security framework risk management

Share This Story

Looking for a reprint of this article?
From high-res PDFs to custom plaques, order your copy today!

Blake curtis

Dr. Blake Curtis, Sc.D, CISA, CRISC, CISM, CGEIT, CDPSE, COBIT, has a proven track record of creating global information assurance programs for government, commercial, international, and healthcare sectors. He leads teams that assess various aspects of risk and ensures compliance with applicable state, federal, and regulatory requirements. In addition, he manages large initiatives that leverage a combination of governance and security frameworks to develop tailored programs for enterprises. Dr. Curtis also helped debunk the 10,000-hour rule. Most importantly, Dr. Blake Curtis is the first scientist to scientifically debunk the "years of experience" fallacy. His study proved that task-based experience is more objective than time-based experience. Blake is also the author of "How to Complete Your Master's Degree in One Semester," which has over 15,000 views and has helped over 150 students complete their master's degrees in record-setting times.

Recommended Content

JOIN TODAY
To unlock your recommendations.

Already have an account? Sign In

  • Security's Top Cybersecurity Leaders 2024

    Security's Top Cybersecurity Leaders 2024

    Security magazine's Top Cybersecurity Leaders 2024 award...
    Security Leadership and Management
    By: Security Staff
  • cyber brain

    The intersection of cybersecurity and artificial intelligence

    Artificial intelligence (AI) is a valuable cybersecurity...
    Logical Security
    By: Pam Nigro
  • artificial intelligence AI graphic

    Assessing the pros and cons of AI for cybersecurity

    Artificial intelligence (AI) has significant implications...
    Technologies & Solutions
    By: Charles Denyer
Subscribe For Free!
  • Security eNewsletter & Other eNews Alerts
  • eMagazine Subscriptions
  • Manage My Preferences
  • Online Registration
  • Mobile App
  • Subscription Customer Service

More Videos

Sponsored Content

Sponsored Content is a special paid section where industry companies provide high quality, objective, non-commercial content around topics of interest to the Security audience. All Sponsored Content is supplied by the advertising company and any opinions expressed in this article are those of the author and not necessarily reflect the views of Security or its parent company, BNP Media. Interested in participating in our Sponsored Content section? Contact your local rep!

close
  • Crisis Response Team
    Sponsored byEverbridge

    Automate or Fall Behind – Crisis Response at the Speed of Risk

  • Perimeter security
    Sponsored byAMAROK

    Why Property Security is the New Competitive Advantage

  • Duty of Care
    Sponsored byAMAROK

    Integrating Technology and Physical Security to Advance Duty of Care

Popular Stories

Red laptop

Cybersecurity leaders discuss Oracle’s second recent hack

Pills spilled

More than 20,000 sensitive medical records exposed

Coding on screen

Research reveals mass scanning and exploitation campaigns

Laptop in darkness

Verizon 2025 Data Breach Investigations Report shows rise in cyberattacks

Computer with binary code hovering nearby

Cyberattacks Targeting US Increased by 136%

2025 Security Benchmark banner

Events

May 22, 2025

Proactive Crisis Communication

Crisis doesn't wait for the right time - it strikes when least expected. Is your team prepared to communicate clearly and effectively when it matters most?

November 17, 2025

SECURITY 500 Conference

This event is designed to provide security executives, government officials and leaders of industry with vital information on how to elevate their programs while allowing attendees to share their strategies and solutions with other security industry executives.

View All Submit An Event

Products

Security Culture: A How-to Guide for Improving Security Culture and Dealing with People Risk in Your Organisation

Security Culture: A How-to Guide for Improving Security Culture and Dealing with People Risk in Your Organisation

See More Products

Related Articles

  • cyber security freepik

    Addressing cybersecurity and it gaps in an ever-changing Workplace: 4 keys to staying safe in your new digital office space

    See More
  • insurance-freepik

    The beginning of a beautiful friendship: How the insurance industry can partner with IT to create true digital transformation

    See More
  • video management

    Video technologies help retailers deliver a safe in-store shopping experience in the age of COVID-19

    See More

Related Products

See More Products
  • physical security.webp

    Physical Security Assessment Handbook An Insider’s Guide to Securing a Business

  • databasehacker

    The Database Hacker's Handboo

See More Products
×

Sign-up to receive top management & result-driven techniques in the industry.

Join over 20,000+ industry leaders who receive our premium content.

SIGN UP TODAY!
  • RESOURCES
    • Advertise
    • Contact Us
    • Store
    • Want More
  • SIGN UP TODAY
    • Create Account
    • eMagazine
    • eNewsletter
    • Customer Service
    • Manage Preferences
  • SERVICES
    • Marketing Services
    • Reprints
    • Market Research
    • List Rental
    • Survey/Respondent Access
  • STAY CONNECTED
    • LinkedIn
    • Facebook
    • YouTube
    • X (Twitter)
  • PRIVACY
    • PRIVACY POLICY
    • TERMS & CONDITIONS
    • DO NOT SELL MY PERSONAL INFORMATION
    • PRIVACY REQUEST
    • ACCESSIBILITY

Copyright ©2025. All Rights Reserved BNP Media.

Design, CMS, Hosting & Web Development :: ePublishing