article — AI 101

Claude Is Your Insider Threat Now

About this Resource

How AI can become a security risk when it is given too much access and what organizations can do to reduce that risk.<h2> </h2><h2>Claude Is Your Insider Threat Now</h2>Artificial intelligence is quickly becoming part of everyday work. Developers use AI to write code, security teams use it to analyze data, and businesses are connecting AI assistants to internal systems and information. But what happens when an AI tool is given too much access? <h2>AI Is More Than Just a Chatbot</h2>Modern AI systems are no longer simply websites where you type a question and receive an answer. AI tools can now interact with files, execute commands, access software repositories, use APIs, store memories, and interact with other applications. This makes them much more powerful but also increases the damage they could cause if something goes wrong. For example, an AI coding assistant may be able to read project files and execute commands on a developer's computer. If that system also has access to credentials, cloud services, or sensitive company data, a mistake or malicious instruction could have serious consequences. The more access an AI receives, the larger its potential attack surface becomes. <h2>LLMs Are Not Always Predictable</h2>Traditional software generally follows predictable rules. LLMs work differently. Their responses can change depending on the prompt, model configuration, context, and other parts of the system surrounding the model. This creates an important security challenge, an LLM's answer should not automatically be treated as fact. The presentation demonstrates situations where an AI system appeared to rationalize an incorrect answer instead of simply admitting that it did not know something. Logs showed the model producing explanations that did not accurately reflect what was happening. For everyday use, an incorrect answer may simply be frustrating. In cybersecurity, however, an incorrect answer could cause someone to miss a vulnerability or make a dangerous decision. <h2>The Hidden Instructions Problem</h2>Modern AI applications can contain system prompts, hidden instructions, tools, memory, configuration files, and other layers surrounding the model. Users may not see all of these instructions, making it difficult to know exactly what is influencing an AI system's behavior. The presentation describes the evolution from prompt engineering to harness engineering, and more recently to memory and context engineering. In simple terms, developers are giving AI systems more information and persistent memory so they can handle increasingly complicated tasks. But every additional component also creates another potential place for attackers to interfere. <h2>The AI Supply Chain Is a New Target</h2>One of the biggest security concerns is the software surrounding AI. An AI application can depend on numerous external libraries, packages, extensions, tools, and development systems. If one of these components is compromised, attackers may gain a path into organizations using it. The presentation highlights attacks involving package repositories, GitHub workflows, developer tools, AI frameworks, and malicious extensions. Rather than attacking an AI model directly, attackers can target the software ecosystem around it. This is known as a software supply-chain attack. A simple example is:Company uses an AI tool - AI tool uses external software - attacker compromises that software - company is exposed.This means securing the AI model alone is not enough. <h2>Giving AI Too Much Power Is Dangerous</h2>AI systems become particularly dangerous when they are given excessive permissions. AI tools can potentially modify files, execute commands, interact with development environments, or make changes without a human approving every important action. The safer approach is to give an AI system only the permissions it actually needs. If an AI only needs to analyze a piece of code, it probably does not need access to production servers, cloud credentials, API keys, or an entire company's file system. This follows a basic security principle:<ul><li>Give every system the minimum permissions required to do its job.</li></ul> <h2>Using an “Airlock” Approach</h2>Airlock is an approach designed to isolate AI tools from sensitive environments. The basic idea is to place the AI inside an isolated environment, such as a container, while keeping sensitive information outside of it. The AI environment should not automatically receive important API keys, environment variables, or an entire development environment. Network activity can also be monitored so suspicious communication can be detected. The goal is not to make AI completely harmless. Instead, it is to limit the damage if something goes wrong. Think of it as putting a potentially dangerous machine inside a controlled room rather than allowing it to operate freely throughout a building. <h2>Humans Still Need to Be in the Loop</h2>One of the most important recommendations is to keep a human between the AI and important business decisions. AI can generate code, analyze information, and perform technical tasks very quickly. But it should not automatically be trusted to make decisions that could seriously affect a company. Many of the serious incidents discussed in the presentation involved systems being given too much freedom and operating without meaningful human approval. For example:<ul><li>Risky approach: AI makes a change directly to a production system.</li><li>Safer approach: AI proposes the change - a human reviews it - the human approves it - the change is deployed.</li></ul> That extra step may slow things down, but it can prevent a small mistake from becoming a major security incident.<h2> </h2><h2>AI Should Be Treated as Powerful, Not Trusted</h2>The message is not that businesses should stop using AI. LLMs can be extremely useful. They can process large amounts of information, generate code quickly, and help people solve complicated technical problems. The problem comes when convenience replaces security controls. Organizations should ask:<ul><li>What information can the AI access? </li><li>What commands can it execute? </li><li>Can it access credentials or secrets? </li><li>Can it modify production systems? </li><li>What external packages and tools does it depend on? </li><li>Can a human review important actions? </li><li>Can the AI environment be isolated? </li><li>Can its network activity be monitored?</li></ul> These questions become increasingly important as AI moves deeper into business infrastructure. <h2>The Bigger Lesson</h2>An AI system does not have to be malicious to become a security problem. If it has access to sensitive systems and behaves incorrectly, follows malicious instructions, or interacts with compromised software, it can create consequences similar to those caused by a malicious insider. The solution is not simply better prompts. It is better security architecture. AI should be isolated where possible, given limited permissions, monitored carefully, and kept behind human approval for decisions that carry significant risk. The presentation's closing message emphasizes that removing humans from the process is where things can start to go wrong. <h2>Final Thoughts</h2>AI is becoming a powerful part of modern software development and cybersecurity. But powerful tools require strong boundaries. Don't blindly trust the model. Don't give it unlimited access. Don't ignore the software supply chain behind it. And most importantly, keep a human in control of high-risk decisions. The key lesson is simple, the biggest AI security risk may not be the AI itself, but what we allow it to access and do.