Compliance frameworks
Do you need penetration testing for compliance?

Do you need penetration testing for compliance?

This article is authored by Software Secured, a penetration testing provider and Vanta partner.

A lack of compliance is increasingly becoming a major barrier for sales, forcing security directors to be more in tune with their organization’s revenue and growth goals than ever before. To help ease this pressure, companies are seeking to fulfill compliance requirements faster.

In this article, we’re deep diving into the two most common security testing options that companies employ for their compliance initiatives: penetration testing and vulnerability scanning. We’ll also do a comparison on which one is most valuable to your project.

What is penetration testing?

When you’re beginning to work on earning your compliance, you’ll likely be informed by your auditor or compliance team that you should consider doing a penetration test. Some will say you need a penetration test. A penetration test is a comprehensive security assessment done by ethical hackers to measure the security defenses that you currently have in your systems.

Typically, penetration testing involves testing all your security controls such as authentication, authorization, integrity, and confidentiality against industry standards. A good penetration test should also test your application’s business logic and ensure there are no bypasses that can lead to serious issues.

Penetration testing can be conducted on all types of systems, including web and mobile applications, IoT devices, infrastructures, and networks. There are various types of penetration tests such as black box, gray box, and white box testing that approach the test in different ways, but all leveraging a human hacker to do the testing.

What is vulnerability scanning?

When a company is debating on the type of security test that they want to conduct, the choice is usually made between penetration testing or vulnerability scanning. Using a vulnerability scanner means employing an automated tool that identifies high-level vulnerabilities in your application. This is good to know and to keep in mind as you learn about the nature of each compliance framework below.

The difference between a prescriptive and descriptive compliance framework

All compliance frameworks cover different areas and have different requirements. As such, the way that penetration testing is suggested or required also varies across each framework. To better understand this, it’s important to know that compliance frameworks can either be prescriptive or descriptive in how they approach the security testing requirement.

Prescriptive compliance frameworks

Prescriptive frameworks are extremely helpful because they outline exactly what you need to do. There is no beating-around-the-bush. There are clear outlines for what constitutes a pass or a fail on your compliance. This makes it easy to know if you should get a penetration test, vulnerability scan, or neither.

This includes:

Descriptive compliance frameworks

Descriptive frameworks, on the other hand, are much more vague. They often outline a recommendation to complete a form of security testing, but they don’t clarify the type of test that is needed or on which areas of your system(s) you need to have tested.

This includes:

  • SOC 2
  • ISO 27001

What are the security requirements for each compliance framework?

Unsure if you need penetration testing or vulnerability scanning for your upcoming compliance audit? Customers who turn to Software Secured are often concerned about what that entails for each different compliance framework. Let’s dive into the most common compliance frameworks that our customers ask us about, including PCI DSS, HIPAA, SOC 2, and ISO 27001.


PCI DSS stands for the Payment Card Industry Data Security Standard. It is a standard for any company (digital or non-digital) that manages or stores cardholder data from any payment card provider (including Mastercard, VISA, American Express, Discover, and JCB). This framework is very prescriptive in nature and is very actionable for developing your security program. There are four compliance reference levels built into this framework:

  • Level 1 is the highest level. It is for any merchant who processes over six million transactions annually OR any company that has suffered a data breach resulting in the capture of cardholder data. This level requires yearly penetration testing and quarterly vulnerability scans, among other security requirements.
  • Level 2 is for any merchant who processes between one to six million transactions annually. At this level, merchants must complete quarterly security scans and a yearly self-assessment questionnaire.
  • Level 3 is for any merchant who processes between 20,000 to one million transactions annually. Security testing requirements are the same as level 2.
  • Level 4 is for any merchant who processes less than 20,000 transactions annually. There is no security testing requirement, but it is highly recommended to conduct the testing required in levels 2 and 3.

Even at levels that don’t require penetration testing, it is still recommended. No matter how many transactions your business processes each year, if you suffer a data breach that exposes customer cardholder data, you are automatically assigned to Level 1 which includes very strict and specific security requirements. This also may include a forensic investigation and other possible consequences that arise from your breach, such as legal fees, reputation risk, fines from card processing companies, and possible loss of card processing privileges (for companies that have experienced a breach multiple times).


The Health Insurance Portability and Accountability Act (HIPAA) is a major compliance framework for any company handling sensitive Protected Health Information (PHI) about their users. It is relevant to all covered entities (ie. doctors, nurses, insurance companies) as well as business associates (ie. lawyers, accountants, IT personnel in the healthcare industry) that may have access to PHI. HIPAA has descriptive privacy and security rules, which identify the following requirements relevant to security testing:

Under the Security Rule

  • There is a “General Rule” to maintain reasonable and appropriate administrative, technical, and physical safeguards for protecting e-PHI (PHI data available digitally). Within this rule is a specific requirement to “Identify and protect against reasonably anticipated threats to the security or integrity of the information.There is no specific method that is recommended for identifying anticipated threats, though the framework does ask businesses to opt for a solution that considers the company’s size, complexity, capabilities, infrastructure, costs, and impact of potential risks to e-PHI. Businesses need to consistently review their security practices to ensure it is up to date with the changing technology, risk, and business environments.
  • Under the “Risk Analysis and Management Rule,” a requirement asks companies to build a risk analysis process that includes a minimum of four activities around the evaluation, implementation, documentation, and maintenance controls for risk management. Like the General Rule, there is also no specific method recommended under this rule.

Both a penetration test and vulnerability scan can help meet both rules.

Under the Privacy Rule

  • Among many other requirements, there is an Administrative Requirement in the Privacy Rule to ensure Data Safeguards are in place. This includes “maintain[ing] reasonable and appropriate administrative, technical, and physical safeguards to prevent intentional or unintentional use or disclosure of protected health information…” A regular penetration test or vulnerability scan may be able to prove that proper technical safeguards are in place.

You can learn more about the HIPAA privacy rules here and security rules here.


SOC stands for System and Organization Controls, and is relevant for service organizations. It was developed by the American Institute of CPAs (AICPA) to measure if an organization’s practices are sufficient for safeguarding any customer data that they may access, store, or transmit. This framework is descriptive in nature.

  • Reports on controls within a service organization that are relevant to the five “Trust Services Principles” including security, availability, processing integrity, confidentiality, and privacy of customer data.
  • The framework looks at your internal controls, including your control environment, risk assessment, control activities, information and communications, and monitoring. Both penetration testing and vulnerability scanning can support the risk assessment control, in addition to a multitude of other criteria.

SOC 2 is offered with two types of reports:

1. A Type 1 report is a “snapshot in time” look at your organization’s controls. A one-time penetration test or vulnerability scan will suffice for a Type 1 report.

2. A Type 2 report continuously monitors your organization’s information and privacy controls, for at least 3 months, typically once a year for 12 month cycles. If you are practicing more frequent testing for a Type 2 report, you may consider Penetration Testing as a Service (PTaaS) for quarterly penetration testing, or you can continue purchasing a one-time pentest or vulnerability scan each year.

ISO 27001

ISO 27001 was published by the International Organization for Standardization (ISO) to bring a benchmark for information security practices in enterprises. Companies that strive for ISO 27001 compliance need to renew this standard every three years at minimum. It is a descriptive framework.

There are many control requirements which can be tested and proven through various security testing methods in Annex A. For example, there are control requirements regarding:

  • Handling digital and physical assets
  • Appropriate limiting of employee access to data
  • Proper encryption of sensitive data
  • Logging and monitoring of software and known vulnerabilities
  • Technical vulnerability management
  • Network security management
  • Security of information as it shared within and outside of the organization

ISO 27001 is incredibly thorough in the number of aspects that it looks to seek protection on, but it doesn’t recommend one specific solution for testing the implementation of those protections. Vulnerability scanning can cover most security requirements but organizations should opt for a penetration test as it can cover all requirements.

The security testing requirements under each compliance framework

There's a lot of information out there and security compliance is complex. Building a security program is an iterative process, here are some quick points to make this decision a little easier.

Pros of getting a penetration test

  • Comprehensive penetration tests usually produce a higher number of true positives and no false positives
  • Support with remediation following report delivery
  • Will definitely meet the requirements for all compliance frameworks
  • Can upgrade anytime to Penetration Testing as a Service (PTaaS), which runs a penetration test quarterly
  • Support on completing self-assessment questionnaires and vendor security questionnaires
  • Assists in addressing other areas of your program to improve for compliance (i.e. logging and monitoring, authentication and authorization, etc).
  • Supports global sales expansion (ie. organizations in Europe can have more stringent requirements around data security and privacy)

Cons of getting a penetration test

  • High-quality penetration tests can get expensive
  • As any manual test, penetration tests take more time to schedule and execute

Is a vulnerability scan sufficient for compliance?

In very rare cases, a vulnerability scan is enough. For example, if your client base isn't concerned with how their data or PII is secured. Or, if your company is focused solely on compliance and not investing in strong security measures for other business needs.

Pros of getting a vulnerability scan

  • Much more affordable for smaller businesses.
  • Little to no onboarding time after the tool is downloaded.
  • Can be done in-house with a DAST tool.
  • Faster results, usually within 24 hours.

Cons of getting a vulnerability scan

  • May not be suitable for all types of compliance
  • Only finds high-level vulnerabilities so potentially leaves deeper, unidentified security gaps
  • Often produces false positives, which adds administrative burden to your technical team

Things that both will provide:

  • A report outlining the vulnerabilities, often both in an online dashboard and a downloadable report file (ie. .docx, html, pdf, etc.).
  • Both will work with almost any programming language or framework.

The ultimate answer

If you have the budget and time, getting a penetration test is your safest bet. Not only will you find more vulnerabilities, but you will also receive support for remediating these security gaps before your compliance audit. You will have much higher confidence in the software you are delivering and you will prove your commitment to security to your enterprise clients. Having a better score on your compliance certification will help you close more sales - so think of it as an investment for your future revenue.

About Software Secured

Software Secured is a leading provider of penetration testing services. Based in Canada, they are specialized at helping fast-growing SaaS companies conduct penetration tests for compliance, closing enterprise deals, secure code deployment, and securing investment rounds. If you’re interested in learning more about their penetration testing services, contact them to book a meeting here.

Written by
No items found.
Access Review Stage Content / Functionality
Across all stages
  • Easily create and save a new access review at a point in time
  • View detailed audit evidence of historical access reviews
Setup access review procedures
  • Define a global access review procedure that stakeholders can follow, ensuring consistency and mitigation of human error in reviews
  • Set your access review frequency (monthly, quarterly, etc.) and working period/deadlines
Consolidate account access data from systems
  • Integrate systems using dozens of pre-built integrations, or “connectors”. System account and HRIS data is pulled into Vanta.
  • Upcoming integrations include Zoom and Intercom (account access), and Personio (HRIS)
  • Upload access files from non-integrated systems
  • View and select systems in-scope for the review
Review, approve, and deny user access
  • Select the appropriate systems reviewer and due date
  • Get automatic notifications and reminders to systems reviewer of deadlines
  • Automatic flagging of “risky” employee accounts that have been terminated or switched departments
  • Intuitive interface to see all accounts with access, account accept/deny buttons, and notes section
  • Track progress of individual systems access reviews and see accounts that need to be removed or have access modified
  • Bulk sort, filter, and alter accounts based on account roles and employee title
Assign remediation tasks to system owners
  • Built-in remediation workflow for reviewers to request access changes and for admin to view and manage requests
  • Optional task tracker integration to create tickets for any access changes and provide visibility to the status of tickets and remediation
Verify changes to access
  • Focused view of accounts flagged for access changes for easy tracking and management
  • Automated evidence of remediation completion displayed for integrated systems
  • Manual evidence of remediation can be uploaded for non-integrated systems
Report and re-evaluate results
  • Auditor can log into Vanta to see history of all completed access reviews
  • Internals can see status of reviews in progress and also historical review detail

PCI Compliance Selection Guide

Determine Your PCI Compliance Level

If your organization processes, stores, or transmits cardholder data, you must comply with the Payment Card Industry Data Security Standard (PCI DSS), a global mandate created by major credit card companies. Compliance is mandatory for any business that accepts credit card payments.

When establishing strategies for implementing and maintaining PCI compliance, your organization needs to understand what constitutes a Merchant or Service Provider, and whether a Self Assessment Questionnaire (SAQ) or Report on Compliance (ROC) is most applicable to your business.

Answer a few short questions and we’ll help identify your compliance level.


Does your business offer services to customers who are interested in your level of PCI compliance?


Identify your PCI SAQ or ROC level

The PCI Security Standards Council has established the below criteria for Merchant and Service Provider validation. Use these descriptions to help determine the SAQ or ROC that best applies to your organization.

Good news! Vanta supports all of the following compliance levels:


A SAQ A is required for Merchants that do not require the physical presence of a credit card (like an eCommerce, mail, or telephone purchase). This means that the Merchant’s business has fully outsourced all cardholder data processing to PCI DSS compliant third party Service Providers, with no electronic storage, processing, or transmission of any cardholder data on the Merchant’s system or premises.

Get PCI DSS certified


A SAQ A-EP is similar to a SAQ A, but is a requirement for Merchants that don't receive cardholder data, but control how cardholder data is redirected to a PCI DSS validated third-party payment processor.

Learn more about eCommerce PCI

for service providers

A SAQ D includes over 200 requirements and covers the entirety of PCI DSS compliance. If you are a Service Provider, a SAQ D is the only SAQ you’re eligible to complete.

Use our PCI checklist

Level 1 for service providers

A Report on Compliance (ROC) is an annual assessment that determines your organization’s ability to protect cardholder data. If you’re a Merchant that processes over six million transactions annually or a Service Provider that processes more than 300,000 transactions annually, your organization is responsible for both a ROC and an Attestation of Compliance (AOC).

Automate your ROC and AOC

Download this checklist for easy reference


Learn more about how Vanta can help. You can also find information on PCI compliance levels at the PCI Security Standards Council website or by contacting your payment processing partner.

The compliance news you need. Delivered securely to your inbox.

Subject to Vanta's Privacy Policy, you agree to allow Vanta to contact you via the email provided for marketing and other purposes