Attacking the model itself

Every organisation deploying a model has created an attack surface that did not exist two years ago. Prompt injection, training data poisoning, model extraction, and agent hijacking are real techniques with almost no installed base of defences against them.

Historically security tooling defended networks, endpoints, and applications. A model that takes instructions from untrusted text and then acts on systems does not fit any of those categories cleanly.

The structural driver is agent deployment. A chatbot that answers questions is low risk. An agent with tool access that reads an attacker controlled document and then executes actions is a genuinely new problem.

The technology layer spans input and output filtering, model firewalls, agent permission systems that constrain what a model may do, red teaming services, model provenance and integrity checks, and monitoring that spots a model behaving unlike itself.

Adoption economics are pre incident for most buyers, which is the hard part. Security budgets move after a breach, and few organisations have experienced a public model compromise yet.

The beneficiaries include AI security specialists, established security vendors adding model coverage, red teaming consultancies, and the identity firms extending permissioning to non human actors.

The value chain runs from model providers through deployment platforms to the enterprise. Whoever controls the agent permission layer holds the most defensible position.

The overlooked layer includes non human identity management, agent audit logging, model integrity verification, and the specialist consultancies doing adversarial testing.

Competitive dynamics are early and crowded, with many small vendors and the large security platforms deciding whether to build or buy. Consolidation is likely and quick.

Risks: the category is pre budget in most organisations, large platforms may absorb it as a feature, threat models are shifting faster than products, and a quiet year would delay urgency considerably.

What to watch: publicly disclosed agent compromises, regulatory requirements for AI system security, non human identity adoption, and security platform acquisitions in this space.