Twelve questions to ask a GenAI security vendor

1. Which AI apps, AI features inside SaaS apps, agents and MCP servers does the product recognise, and how is the list kept current? 2. Which traffic does it see: browser, desktop apps, devices off the network, server workloads? 3. How does it treat AI on unmanaged or personal devices?

4. Does it inspect prompts and responses, including uploaded files and images? 5. Which responses can a policy take: watch, coach, redact, block? 6. How does it decide on an agent's tool call, and can it stop the call before it runs? 7. What happens when the inspection service is slow or unavailable: fail open or fail closed?

8. Is AI policy written in the same console and engine as the rest of your security policy, or in a separate product? 9. Can enforcement run on infrastructure you host, and which features are cloud-only? 10. Where are prompts, verdicts and logs stored, for how long, and who at the vendor can see them?

11. Which attestations and authorizations cover the AI features specifically, not only the company? A trust page that lists FedRAMP or SOC 2 may not say which products are in scope. 12. What is published about how detection models are tested and updated?

Ask every shortlisted vendor the same questions in writing, and compare the answers next to what their pages state. Where a page is silent, as recorded under Not in our sources on our profiles, the answer in writing is the only evidence you have.

Vague ones. If a vendor cannot say which traffic its control sees, assume it sees less than you need. If the answer to question 7 is that it depends, ask for the default setting and how to change it. If the attestation in question 11 covers the company but not the AI product, note that the AI features may sit outside the audited scope. And if logs of prompts are kept by default for a long time, treat that as a data protection question for your privacy team, not only a security one. A good answer names the component, the setting and the document where it is written down.