Security Design and AI Governance in the 'Assume Breach' Era: Lessons from 2025 Incidents

Introduction: The Current State of Business Management and Secure Systems

I recently delivered a lecture at Meiji University titled “Business Management and Secure Systems — The Importance and Practice of Information Security.” This article uses the 2025 domestic incident cases covered in the lecture as a starting point, organizing from a practitioner’s perspective the structural challenge of “why systems fail even at large enterprises with sufficient countermeasures,” and the integrated design of “security, privacy, and governance” demanded in the rapidly advancing AI era.

1. Security Is “Mixed Martial Arts” — Single-Point Measures Can’t Win

In the lecture, I compared security measures to “Mixed Martial Arts (MMA)” — because no single technology or one-off initiative can handle today’s compounding threats in practice. Beyond programming, networking, and security products, one must consider relevant legislation, public relations and crisis response, physical security, and human vulnerabilities.

2. 2025 Cases: Reading “Structure” from Public Disclosures

The lecture used several ransomware cases published in 2025 as case studies. The important focus is not “who got hit” but rather the scope of what stopped, the difficulty of recovery, and where design weaknesses manifested.

Major domestic manufacturer A (September 2025): Public disclosures described unauthorized access and ransomware encryption damage, with potential impact on approximately 1.9 million personal records (aggregating multiple categories). Beyond system downtime business losses, the situation raised questions of social responsibility for privacy violations.

Major domestic retail/logistics company B (October 2025): Ransomware infection caused e-commerce site shutdowns affecting orders and shipments. Recovery required phased service restoration with partial functionality restrictions and manual operations.

These cases demonstrate that once breached, recovery extends long-term with serious impacts on both business continuity and privacy protection.

3. Why “Even Large Enterprises Can’t Prevent It”: The Swiss Cheese Model

Major incidents rarely result from “a single mistake” — they occur when small holes across multiple defense layers accidentally align, a phenomenon known as the Swiss Cheese Model. This underscores the growing importance of concepts like the cyber kill chain and zero trust as design and operational fundamentals.

Three representative layers that become fatal when they align:

  • Entry defense limitations: Email or VPN appliance vulnerabilities allow the first wall to be breached
  • Internal defense gaps: Insufficient network segmentation or privileged ID management enables lateral movement
  • Recovery design misalignment: Backup data itself is compromised, or recovery procedures don’t align with BCP (Business Continuity Plan), prolonging recovery

4. Implementing “Assume Breach” in Practice

Security design going forward requires more than “building higher walls.” The recommendation is to design detection, containment, and recovery holistically, assuming breach will occur.

  • Read the attack chain: Anticipate attacker behavior (reconnaissance → intrusion → expansion) and design where to break the chain
  • Segment lateral movement: Thoroughly implement network segmentation and privilege design, verifying that breach in one area doesn’t cascade
  • Operate endpoint behavior monitoring: Deploy EDR not as “install and forget” but paired with monitoring, analysis, and response operations (SOC structure, external MDR, etc.)
  • Pre-agree on recovery and alternatives: Align RTO (Recovery Time Objective) and RPO (Recovery Point Objective) with management and business units in advance; prepare recovery procedures and alternatives from network-isolated states
  • Supply chain assumption: Design with the premise that subcontractors and trading partners can become breach entry points

5. “Defense” in the AI Era: Converging Security, Privacy, and Governance

The latter half of the lecture explored issues accompanying rapid AI adoption. In the AI era, organizations must integrate three perspectives beyond traditional IT security: AI security, privacy, and AI governance.

(1) AI Security and Emerging Threats

AI models themselves are now attack targets. The lecture covered adversarial examples that deceive image recognition and the OWASP Top 10 for LLMs — new threats targeting large language models. Prompt injection and data poisoning attacks in particular cannot be fully blocked by traditional firewalls.

(2) Privacy and Regulatory Compliance (AI Governance)

As AI processes massive amounts of personal data, privacy protection is paramount. International rule-making continues with the EU AI Act, ISO/IEC 42001, and NIST AI RMF, while Japan’s AI Business Operator Guidelines also require compliance. Governance ensuring not just usability but legal and ethical safety is essential.

(3) Distinguishing Transparency, Explainability, and Accountability

  • Transparency: Disclosing system mechanisms — data sources, algorithm overviews — in understandable form
  • Explainability: Being able to explain technically and legally why a specific output was produced
  • Accountability: Taking organizational responsibility for development and operational outcomes, ensuring auditability and continuous monitoring

6. Student Q&A (Selected)

Q: I’m considering a career in IT. What preparation is needed?

A: Security, which is essential across IT, is “mixed martial arts” — narrow expertise alone won’t suffice. First, build fundamental fitness in IT basics (networking, OS, programming). Then try building a complete application yourself — a three-tier web application plus something using AI fundamentals and popular libraries. Cultivate the “persistence” to stay current with technology and the “communication skills” to discuss technology in business context. Certifications serve as good milestones for systematic learning.

Q: If ransomware actually hits, what should the frontline do?

A: The most critical initial response is containment — preventing damage spread. Isolating the infected terminal from the network (unplugging the LAN cable, etc.) is step one. Hasty reboots can destroy memory-resident evidence, so careful judgment is required. Regarding ransom payment, most governments recommend against it from counter-terrorism financing perspectives. If self-recovery proves difficult, consulting experts or police cybercrime divisions are options.

Conclusion: The Expert’s Value Lies in “Comprehensive Capability” and “Practical Effectiveness”

There is no silver bullet in security. What future experts need is comprehensive capability spanning technology, legal, management, and privacy considerations, and practical effectiveness that reliably protects the business in emergencies.

  • Security: Design that is “hard to stop, and quick to restore when stopped”
  • Privacy: Design that “minimizes individual impact and fulfills accountability”
  • AI Governance: Design that “addresses new threats while ensuring auditability of decision processes”

Optimizing these three separately causes rework in field operations — designing them integrated from the start is the practical answer demanded by AI-era security.

Disclaimer: This article represents the personal views of the author based on information available as of December 2025, and does not represent the views of any affiliated or related organization.