Compliance is not safety. TVM demands falsifiable, renewable proofs: audits, guardrails, and drift checks, so AI serves people without laundering bias at scale.
Thanks for writing this, it clarifies a lot. This piece is brilliant, you realy hit the nail on the head. How do we actually push for these proofs, not just vague promises, in AI safety?
Great question! Tough and very long answer! The gap between "we should demand proofs" and "here's how we actually get them" is where most good ideas die. On the upside, we have already written the methodology; on the downside, uptake is zero. But I am nothing if not relentless. ;-)
Detailed response below - it is the first time I have received the "Please type a shorter comment." from Substack - LOL
Here are concrete strategies at different levels of leverage:
1. For Professionals Working in AI/Tech
Make proof requirements non-negotiable in your work:
-- If you're in engineering: Refuse to ship without documented safety proofs. Make "where's the audit trail?" your default question in design reviews.
-- If you're in product: Build proof requirements into your specs from day one. "How will we prove this is safe?" should be part of every product brief.
-- If you're in data science: Demand distributional testing across subgroups before any model goes to production. Push back when teams want to skip fairness metrics "to move faster."
Create internal accountability:
-- Document everything. When leadership asks you to skip safety validation, get it in writing. Forward it to yourself. BCC your personal email.
-- Form "safety guilds" within your company, informal groups that review each other's work and hold each other accountable to proof standards.
-- Use the Altman defense: When asked to cut corners, say, "If something goes wrong, who's accountable? Will you put your name on this?"
Use your leverage at hiring time:
-- In interviews, ask: "Walk me through your last safety incident. What proofs failed? What changed?"
-- Ask to see their model spec, their audit process, and their drift monitoring. If they can't or won't show you, that's data.
-- Negotiate proof requirements into your employment contract: "I require access to safety metrics and the right to escalate concerns without retaliation."
2. For Investors and Board Members
Demand proof infrastructure as a condition of funding:
-- Make proof capabilities part of due diligence. Don't just ask "Do you have an ethics policy?" Ask: "Show me your last three safety audits. Who conducted them? What changed afterward?"
-- Require proof renewal cycles in term sheets. Build in mandatory reassessment triggers: before Series B, before any pivot, before expanding to new markets.
-- Establish proof-linked tranches: "You get the next $10M when you demonstrate X, Y, Z proofs validated by independent auditors."
Create market incentives:
-- Form investor coalitions that only fund companies meeting proof standards. If 10-20 major VCs coordinate, they reshape the market.
-- Publish an annual "AI Proof Index" ranking of portfolio companies. Make it reputational suicide to score poorly.
-- Price risk appropriately: Companies without proofs should get worse valuations, period. Make this explicit in your investment theses.
Use your board seat:
-- Move the motion: "This company will not deploy system X until proofs Y and Z are validated by an independent audit." Force a vote. Make the no-votes own the risk.
-- Require quarterly proof review as a standing agenda item, not buried in committee.
-- Establish personal liability: Board members who approve deployment without adequate proofs should face professional consequences.
3. For Policymakers and Regulators
Draft proof-mandate legislation:
-- Model it on financial auditing: Just as public companies must have audited financials, AI companies above certain thresholds must have audited safety proofs.
-- Make it sector-specific initially: Start with high-risk domains (healthcare, criminal justice, credit) where the case is clearest.
-- Build in sunset provisions: "If industry self-regulation produces validated proofs within 18 months, this regulation can be revisited." Give them a deadline and an out.
Establish proof registries:
-- Create a public database of AI systems and their proof status (analogous to FDA drug approval database).
-- Require quarterly filing: Any AI system making more than X decisions per day must file proof attestations.
-- Make it searchable: Journalists, researchers, activists should be able to look up any major AI system and see its proof status.
Leverage procurement power:
-- Federal/state governments are huge AI customers. Pass rules: "No agency may procure AI systems without proof of certification."
-- This forces the market. If you can't sell to the government without proof, you'll get proof.
-- Start with the DOD/intelligence community; they already have strict vendor requirements. Expand from there.
Create enforcement teeth:
-- Establish meaningful penalties: Not $100K fines for billion-dollar companies, but 2-5% of global revenue per violation.
-- Enable private right of action: Let individuals harmed by AI systems without adequate proofs sue for statutory damages.
-- Whistleblower protection + bounties: Pay people to expose proof theater. Make it financially attractive for companies to reveal when they are lying about safety.
-- Develop standardized proof protocols that are actually implementable; not just theoretical frameworks but "here's the code, here's the test suite, here's the dashboard."
-- Open-source everything: Make it impossible for companies to claim that proofs are "too hard" when the tools are freely available.
-- Publish "proof validations": Audit major AI systems and publish findings. Make it routine for academic labs to assess whether GPT-X or Claude-Y actually has the proofs it claims to have.
Train the auditors:
-- Create graduate programs in "AI Safety Auditing," not AI ethics philosophy, but practical auditing skills.
-- Develop certification programs: "Certified AI Safety Auditor" should become a credential like CPA or PE.
-- Build the talent pipeline: We need thousands of people who can actually conduct these audits. Start training them now.
Establish independent testing:
-- Form academic consortia that provide independent proof validation services—like academic IRBs but for AI safety.
-- Compete with industry red teams: When OpenAI claims their internal red team validated safety, universities should be able to say "we also tested it, and here's what we found."
-- Make results public: Academia's advantage is that we can publish. Use it.
5. For Civil Society and Advocacy Groups
Make proof transparency a campaign issue:
-- Target specific companies: "Open AI is Not Open: They Won't Show Us Their Safety Proofs"
-- Name and shame: Maintain a "Wall of Shame" for companies with the worst proof records.
-- Credit where due: Also highlight companies doing it right. Create competitive pressure from both sides.
Build coalitions:
-- Unite labor unions (worried about job displacement), civil rights groups (worried about bias), privacy advocates (worried about surveillance), and consumer protection groups (worried about manipulation).
-- One demand: "Show us the proofs or stop deployment." Harder to ignore when it's 50 organizations speaking with one voice.
Use litigation strategically:
-- Sue under existing laws: Consumer protection, discrimination, negligence. Force discovery. Make companies show their proofs (or lack thereof) under oath.
-- Establish precedent: Even if you lose, you've forced the question into the legal record. Future cases build on this.
-- Target vulnerable points: Find the clearest cases of harm from systems without adequate proofs. Make them poster children.
Engage the public:
-- Translate technical proof requirements into stories people understand: "Would you trust a bridge built by someone who said 'I'm thinking on the spot about whether it'll collapse'?"
-- Create simple asks: "Before AI decides whether you get a loan/job/medical treatment, demand proof it's been tested fairly."
-- Make it visual: Infographics showing company proof scores, interactive tools where people can look up AI systems they interact with.
6. For Users and Citizens
Vote with your feet:
-- Choose products from companies with strong proof records when you have alternatives.
-- Check the proof status: Just like you might check privacy policies, make checking proof status habitual.
-- Reward transparency: Write reviews, post on social media, tell companies when their commitment to proofs influenced your choice.
Demand transparency from systems you interact with:
-- When AI makes a decision about you, ask: "What proofs validate this system's safety and fairness? Can I see the most recent audit?"
-- Most companies won't have good answers. That's the point. Force the question into the open.
-- File complaints: With FTC, state AGs, industry regulators. Create a paper trail.
Support organizations doing the work:
-- Donate to groups like Center for AI Safety, AI Now Institute, Algorithmic Justice League, and organizations working on accountability.
-- Volunteer skills: These groups need developers, lawyers, communicators, and organizers.
-- Amplify their work: Share their reports, attend their events, cite their research.
7. The Coordination Play
Here's the key: None of these tactics work in isolation. The power comes from coordination:
The Pincer Movement:
-- Investors demand proof from the top (conditioning funding)
-- Regulators mandate proofs from the side (legislation and enforcement)
-- Workers demand proofs from the inside (refusing to ship without them)
-- Users demand proofs from the bottom (refusing to use products without them)
-- Researchers provide proofs as infrastructure (making it possible)
-- Advocates create reputational pressure (making it necessary)
When all six move together, companies can't ignore it. They're surrounded.
Start Local, Scale Fast:
-- Don't try to change everything at once. Pick one high-visibility company or one specific AI system.
-- Build a coalition. Get 3-5 organizations representing different stakeholder groups to commit.
-- Make one specific demand: "System X must demonstrate proofs Y and Z by date D, validated by independent audit, or face consequence C."
-- Execute the campaign. Use all the tactics above in coordination.
-- When you win, replicate: "We got Company A to commit. Now let's get Company B."
Will this be easy? No. Companies will fight back. They'll claim proofs are "impossible" or "anti-innovation." They'll produce proof theater, fake audits designed to look real. They'll try to water down standards until they're meaningless.
But here's what we have going for us:
-- The Altman Interview: Leaders keep saying the quiet part loud. They admit they're making it up as they go. Use their own words against them.
-- Incident Rate: AI-related harms are rising sharply. Every new scandal is ammunition. "This is what happens without proofs."
-- Regulatory Momentum: Governments are already moving. EU AI Act, state-level US laws, and international frameworks. The window is open.
-- Market Fear: Companies know the risk. They just need a push to act. One major lawsuit or one significant regulatory action could shift everything overnight.
-- Proof is Possible: We're not asking for the impossible. Financial audits work. Safety engineering works. Medical device approval works. We know how to do this.
The question isn't whether we can demand proofs. It's whether we will. And the answer to that depends on whether people like you actually organize to push for them.
The framework exists. The research exists. The tools exist. What's missing is the coordinated political will to force adoption.
Thanks for writing this, it clarifies a lot. This piece is brilliant, you realy hit the nail on the head. How do we actually push for these proofs, not just vague promises, in AI safety?
Great question! Tough and very long answer! The gap between "we should demand proofs" and "here's how we actually get them" is where most good ideas die. On the upside, we have already written the methodology; on the downside, uptake is zero. But I am nothing if not relentless. ;-)
Detailed response below - it is the first time I have received the "Please type a shorter comment." from Substack - LOL
Here are concrete strategies at different levels of leverage:
1. For Professionals Working in AI/Tech
Make proof requirements non-negotiable in your work:
-- If you're in engineering: Refuse to ship without documented safety proofs. Make "where's the audit trail?" your default question in design reviews.
-- If you're in product: Build proof requirements into your specs from day one. "How will we prove this is safe?" should be part of every product brief.
-- If you're in data science: Demand distributional testing across subgroups before any model goes to production. Push back when teams want to skip fairness metrics "to move faster."
Create internal accountability:
-- Document everything. When leadership asks you to skip safety validation, get it in writing. Forward it to yourself. BCC your personal email.
-- Form "safety guilds" within your company, informal groups that review each other's work and hold each other accountable to proof standards.
-- Use the Altman defense: When asked to cut corners, say, "If something goes wrong, who's accountable? Will you put your name on this?"
Use your leverage at hiring time:
-- In interviews, ask: "Walk me through your last safety incident. What proofs failed? What changed?"
-- Ask to see their model spec, their audit process, and their drift monitoring. If they can't or won't show you, that's data.
-- Negotiate proof requirements into your employment contract: "I require access to safety metrics and the right to escalate concerns without retaliation."
2. For Investors and Board Members
Demand proof infrastructure as a condition of funding:
-- Make proof capabilities part of due diligence. Don't just ask "Do you have an ethics policy?" Ask: "Show me your last three safety audits. Who conducted them? What changed afterward?"
-- Require proof renewal cycles in term sheets. Build in mandatory reassessment triggers: before Series B, before any pivot, before expanding to new markets.
-- Establish proof-linked tranches: "You get the next $10M when you demonstrate X, Y, Z proofs validated by independent auditors."
Create market incentives:
-- Form investor coalitions that only fund companies meeting proof standards. If 10-20 major VCs coordinate, they reshape the market.
-- Publish an annual "AI Proof Index" ranking of portfolio companies. Make it reputational suicide to score poorly.
-- Price risk appropriately: Companies without proofs should get worse valuations, period. Make this explicit in your investment theses.
Use your board seat:
-- Move the motion: "This company will not deploy system X until proofs Y and Z are validated by an independent audit." Force a vote. Make the no-votes own the risk.
-- Require quarterly proof review as a standing agenda item, not buried in committee.
-- Establish personal liability: Board members who approve deployment without adequate proofs should face professional consequences.
3. For Policymakers and Regulators
Draft proof-mandate legislation:
-- Model it on financial auditing: Just as public companies must have audited financials, AI companies above certain thresholds must have audited safety proofs.
-- Make it sector-specific initially: Start with high-risk domains (healthcare, criminal justice, credit) where the case is clearest.
-- Build in sunset provisions: "If industry self-regulation produces validated proofs within 18 months, this regulation can be revisited." Give them a deadline and an out.
Establish proof registries:
-- Create a public database of AI systems and their proof status (analogous to FDA drug approval database).
-- Require quarterly filing: Any AI system making more than X decisions per day must file proof attestations.
-- Make it searchable: Journalists, researchers, activists should be able to look up any major AI system and see its proof status.
Leverage procurement power:
-- Federal/state governments are huge AI customers. Pass rules: "No agency may procure AI systems without proof of certification."
-- This forces the market. If you can't sell to the government without proof, you'll get proof.
-- Start with the DOD/intelligence community; they already have strict vendor requirements. Expand from there.
Create enforcement teeth:
-- Establish meaningful penalties: Not $100K fines for billion-dollar companies, but 2-5% of global revenue per violation.
-- Enable private right of action: Let individuals harmed by AI systems without adequate proofs sue for statutory damages.
-- Whistleblower protection + bounties: Pay people to expose proof theater. Make it financially attractive for companies to reveal when they are lying about safety.
4. For Researchers and Academics
Build the proof infrastructure:
-- Develop standardized proof protocols that are actually implementable; not just theoretical frameworks but "here's the code, here's the test suite, here's the dashboard."
-- Open-source everything: Make it impossible for companies to claim that proofs are "too hard" when the tools are freely available.
-- Publish "proof validations": Audit major AI systems and publish findings. Make it routine for academic labs to assess whether GPT-X or Claude-Y actually has the proofs it claims to have.
Train the auditors:
-- Create graduate programs in "AI Safety Auditing," not AI ethics philosophy, but practical auditing skills.
-- Develop certification programs: "Certified AI Safety Auditor" should become a credential like CPA or PE.
-- Build the talent pipeline: We need thousands of people who can actually conduct these audits. Start training them now.
Establish independent testing:
-- Form academic consortia that provide independent proof validation services—like academic IRBs but for AI safety.
-- Compete with industry red teams: When OpenAI claims their internal red team validated safety, universities should be able to say "we also tested it, and here's what we found."
-- Make results public: Academia's advantage is that we can publish. Use it.
5. For Civil Society and Advocacy Groups
Make proof transparency a campaign issue:
-- Target specific companies: "Open AI is Not Open: They Won't Show Us Their Safety Proofs"
-- Name and shame: Maintain a "Wall of Shame" for companies with the worst proof records.
-- Credit where due: Also highlight companies doing it right. Create competitive pressure from both sides.
Build coalitions:
-- Unite labor unions (worried about job displacement), civil rights groups (worried about bias), privacy advocates (worried about surveillance), and consumer protection groups (worried about manipulation).
-- One demand: "Show us the proofs or stop deployment." Harder to ignore when it's 50 organizations speaking with one voice.
Use litigation strategically:
-- Sue under existing laws: Consumer protection, discrimination, negligence. Force discovery. Make companies show their proofs (or lack thereof) under oath.
-- Establish precedent: Even if you lose, you've forced the question into the legal record. Future cases build on this.
-- Target vulnerable points: Find the clearest cases of harm from systems without adequate proofs. Make them poster children.
Engage the public:
-- Translate technical proof requirements into stories people understand: "Would you trust a bridge built by someone who said 'I'm thinking on the spot about whether it'll collapse'?"
-- Create simple asks: "Before AI decides whether you get a loan/job/medical treatment, demand proof it's been tested fairly."
-- Make it visual: Infographics showing company proof scores, interactive tools where people can look up AI systems they interact with.
6. For Users and Citizens
Vote with your feet:
-- Choose products from companies with strong proof records when you have alternatives.
-- Check the proof status: Just like you might check privacy policies, make checking proof status habitual.
-- Reward transparency: Write reviews, post on social media, tell companies when their commitment to proofs influenced your choice.
Demand transparency from systems you interact with:
-- When AI makes a decision about you, ask: "What proofs validate this system's safety and fairness? Can I see the most recent audit?"
-- Most companies won't have good answers. That's the point. Force the question into the open.
-- File complaints: With FTC, state AGs, industry regulators. Create a paper trail.
Support organizations doing the work:
-- Donate to groups like Center for AI Safety, AI Now Institute, Algorithmic Justice League, and organizations working on accountability.
-- Volunteer skills: These groups need developers, lawyers, communicators, and organizers.
-- Amplify their work: Share their reports, attend their events, cite their research.
7. The Coordination Play
Here's the key: None of these tactics work in isolation. The power comes from coordination:
The Pincer Movement:
-- Investors demand proof from the top (conditioning funding)
-- Regulators mandate proofs from the side (legislation and enforcement)
-- Workers demand proofs from the inside (refusing to ship without them)
-- Users demand proofs from the bottom (refusing to use products without them)
-- Researchers provide proofs as infrastructure (making it possible)
-- Advocates create reputational pressure (making it necessary)
When all six move together, companies can't ignore it. They're surrounded.
Start Local, Scale Fast:
-- Don't try to change everything at once. Pick one high-visibility company or one specific AI system.
-- Build a coalition. Get 3-5 organizations representing different stakeholder groups to commit.
-- Make one specific demand: "System X must demonstrate proofs Y and Z by date D, validated by independent audit, or face consequence C."
-- Execute the campaign. Use all the tactics above in coordination.
-- When you win, replicate: "We got Company A to commit. Now let's get Company B."
Will this be easy? No. Companies will fight back. They'll claim proofs are "impossible" or "anti-innovation." They'll produce proof theater, fake audits designed to look real. They'll try to water down standards until they're meaningless.
But here's what we have going for us:
-- The Altman Interview: Leaders keep saying the quiet part loud. They admit they're making it up as they go. Use their own words against them.
-- Incident Rate: AI-related harms are rising sharply. Every new scandal is ammunition. "This is what happens without proofs."
-- Regulatory Momentum: Governments are already moving. EU AI Act, state-level US laws, and international frameworks. The window is open.
-- Market Fear: Companies know the risk. They just need a push to act. One major lawsuit or one significant regulatory action could shift everything overnight.
-- Proof is Possible: We're not asking for the impossible. Financial audits work. Safety engineering works. Medical device approval works. We know how to do this.
The question isn't whether we can demand proofs. It's whether we will. And the answer to that depends on whether people like you actually organize to push for them.
The framework exists. The research exists. The tools exist. What's missing is the coordinated political will to force adoption.
That's the work. Sounds easy, right?