By: Shane Fast
Imagine you’re sitting across the table from a technology provider you’re considering. You ask:
“Who can access your administrative portal?”
They answer confidently: “Only authorized administrators.”
You nod.
Then you ask: “Can you show me?”
That is where vendors get uncomfortable.
Not because the first answer was wrong, but because the evidence doesn’t exist (or buried and a pain to dig up).
Being secure and being able to prove it are two different things.
As a buyer running risk assessments, questionnaires, and renewal reviews, that gap is where you find out who actually stands behind their claims, and who’s hoping you don’t ask twice.
Hold Technology Vendors Accountable to Demonstrating They’re Secure
Most vendors do a lot of things right, and increasingly not just out of good intentions but out of necessity!
A breach doesn’t just cost them some effort to patch things up, it costs them trust, contracts, and revenue.
Multi-factor authentication, offboarding procedures, patch management, access reviews, these controls are table stakes and are the default more and more, because the business case for skipping them keeps getting worse.
The problem is that they often live as informal practices rather than demonstrable ones.
Take a common answer you’ll hear: “We remove access when employees leave.”
It sounds reassuring, and maybe it’s true. But as a customer, “sounds true” isn’t something you can put in a risk assessment. Your job is to keep going:
- Can you show when it happened?
- Who approved the removal?
- How were exceptions handled?
- What prevents an account from slipping through?
Suddenly you’re not talking about a process. You’re talking about evidence. And that’s where a lot of vendors reveal their controls exist, but their evidence doesn’t.
The Gap Between a Statement and Evidence
One of the clearest ways to pressure-test a vendor’s claims is to put their common statements next to the evidence you should actually ask them to produce:
| Statement | What Evidence Actually Looks Like |
| We remove access when employees leave. | Offboarding tickets, account disablement logs, access reviews |
| We patch critical vulnerabilities. | Scan results, remediation tickets, exception records, closure dates |
| We retain data appropriately. | Retention policies, lifecycle configurations, deletion reports |
| We review our vendors. | Assessment records, approvals, documented risk decisions |
| We back up critical systems. | Backup reports, monitoring alerts, restoration test results |
| Only authorized admins can access the portal. | VPN records, access group membership, infrastructure definitions, access logs |
The gap is definitely worth reflecting on because confidence and intent isn’t evidence. Even a well-written policy isn’t evidence.
Evidence is the observable output of a process actually being executed, and a vendor who can’t produce it on request is telling you their control is a policy, not a practice.
From Control to Evidence: A Practical Framework
Here’s a useful lens, drawn from real work. When we built Protected B (a Canadian government security classification) compliance into one of our own admin portals, the hard part wasn’t implementation. It was the demonstration.
The questions came quickly: How is access restricted? Who qualifies as an authorized administrator? How is access removed? How do you know the restriction is still in place?
These are the same questions worth asking any vendor, broken into four layers:
Control → What outcome is required? Administrative access shall be restricted to authorized personnel.
Process → How do they achieve that outcome? Access requires approved VPN connectivity and authorized group membership.
Task → What specific actions occur? Configure network restrictions, manage VPN access, approve administrators, perform periodic access reviews.
Evidence → What proves those actions happened? Infrastructure code, VPN configuration records, group membership logs, review records, access logs.
None of that evidence comes from the control statement itself, it comes from the proof of work.
A vendor who can walk you through all four layers, and back each one up, has a program that’s actually operating. One who stalls at “process” and can’t get to evidence has a policy, not a practice.
Evidence Should Be a Byproduct of Normal Operations
Watch for this pattern: some vendors treat evidence as something they scramble to assemble when an audit or questionnaire lands on their desk.
The vendors worth trusting generate it continuously, as a natural output of how they work every day.
Approval workflows create records.
Identity systems create logs.
Ticketing systems create history.
Infrastructure-as-Code creates version-controlled change records.
When a vendor’s evidence is baked into daily operations instead of reconstructed after the fact, that’s a strong signal they are taking security seriously. Their answers to you aren’t a performance, rather a quiet byproduct of a system that was already running that way.
In the admin portal example above, this distinction is the difference between a vendor saying:
“A manually configured firewall rule satisfies today’s requirement,”
and one saying:
“Infrastructure managed through code creates a repeatable, reviewable, versionable source of truth, here’s how we know it’s still configured. Check it anytime.”
That’s the difference that separates vendors who can answer your follow-up question from those who can’t.
Listen for it.
The Evidence Ladder
Not all evidence a vendor hands you is built the same. Use this ladder to grade what you’re being shown.
Consider the claim: “The administrative portal is not publicly accessible.”
| Level | Evidence | Confidence |
| We think it’s configured correctly | Someone says it is restricted | Low |
| We documented the configuration | It’s written down in a knowledge base or policy | Moderate |
| We captured a screenshot | Screenshot of a firewall rule | Moderate |
| We defined it in code | Infrastructure-as-Code definition | High |
| We define it in code and continuously validate it | Infrastructure-as-Code with change history and validation | Very High |
If a vendor’s answer tops out at “someone told us,” you haven’t bought security, you’ve bought a rumor.
Why This Is a Leadership Issue
For CIOs, IT directors, and security leaders vetting vendors, this isn’t just a due-diligence checkbox, it’s an essential business decision.
Push harder during procurement. Expect continuous demonstration, not point-in-time snapshots. Look for evidence of organizational maturity, not reassurances and statements.
The vendors worth partnering with aren’t the ones with the thickest policy binders. They’re the ones who can trace a line from a requirement to a process, a process to a task, and a task to evidence, and show you each link when you ask.
Security is measured by what didn’t happen but rather its the discipline of being able to explain why it didn’t happen. As a customer, your job is to demand they show their work.
Most vendors don’t fail a review because their controls are weak. They fail because they can’t find the email that proves the controls are working.
