Skip to main content

Agentic AI Safety: Will We Fear Autonomous Systems, or Are They the Ones Who Protect Us?

Ilustrační obrázek pro jarvis-ai.cz
Agentic AI represents a shift from models that merely generate text to systems that perform real tasks. While chatbots wait for commands, agents plan, use software, and communicate with other systems. This shift brings enormous potential for efficiency, but at the same time opens up entirely new dimensions of cyber risks. Could it be that the security of agentic AI will become a "non-existent problem" because we will build it directly into the core of autonomy?

Listen to this article:

In recent months, we have witnessed a massive increase in the adoption of systems we call agentic AI. According to a report by Cloudera, 57% of organizations have already implemented AI agents and 96% plan to expand their use in the coming year. For the Czech market and the European space, this means we must prepare for an era when software will not just be a tool, but an active collaborator.

What exactly does "agentic AI" mean?

To understand the problem, it is necessary to distinguish between ordinary generative AI (like standard ChatGPT) and agentic AI. While a regular model works on the principle of "input – output", an agentic system operates in a cycle of planning – execution – verification. The agent has tools at its disposal: it can send an email, edit a file in Excel, run code in a terminal, or make a purchase in an e-shop.

This process requires a high degree of autonomy. If you tell it: "Plan me a trip to Paris and book a hotel", the agent will not just write an itinerary, but will actually log into an account, compare prices, and carry out a transaction. This is where the security gap appears: What happens when the agent makes a mistake or is manipulated?

A new era of cyber threats

Traditional cybersecurity focuses on protecting data and access. With agentic AI, however, we must ask about decision-making integrity. There are three main areas of risk that experts such as Andrew Palmer identify as critical:

  • Prompt Injection: An attacker can insert a hidden command through external data (for example, an email the agent reads) that forces the agent to unwanted action, such as sending sensitive data to an external server.
  • Uncontrolled tool usage: If the agent has access to your terminal or cloud infrastructure, one misinterpretation can lead to data deletion or server shutdown.
  • Loss of human oversight: As the speed of automation increases, people become "real-time testers", which is dangerous because the human ability to respond to an AI error is slower than the agent's own process.

For Czech companies, which are increasingly integrating into global digital chains, this topic is critical. EU regulation, specifically the AI Act, already defines strict rules for high-risk systems. Autonomous agents that make decisions about important processes will likely be subject to very strict oversight within the European legal framework.

Can security become a "non-existent problem"?

The question in the title is provocative, but it contains logic. There is a scenario where the security of agentic AI becomes a marginal issue. This would happen if security stops being perceived as a "barrier built around AI" and becomes part of the agent's architecture itself.

Modern approaches discussed by experts from companies such as SentinelOne suggest that AI can be its own best defender. Instead of humans manually checking every step of the agent, we can deploy a "supervisory agent". This type of system has the sole task of monitoring the logic and security protocols of the main agent. If the main agent begins to exhibit anomalous behavior (e.g., an attempt at an unusual money transfer), the supervisory agent immediately stops it.

This concept of "AI vs. AI" can transform cybersecurity. As Steve Stone of SentinelOne states, we don't use AI to replace humans, but to eliminate routine, boring tasks, which allows analysts to focus on complex threats. In this context, security becomes an automated process running in milliseconds.

Technology comparison and availability

When choosing models for agentic workflows, it is important to monitor the reasoning capability. Here is a brief comparison of current leaders in the context of agentic tasks:

Model / Tool Reasoning capability (Agentic) Price (approximate) Czech availability
GPT-4o (OpenAI) High (very stable tools) Free tier / $20 month Yes (excellent)
Claude 3.5 Sonnet (Anthropic) Extremely high (top in coding) Free tier / $20 month Yes (very good)
Gemini 1.5 Pro (Google) High (huge context window) Free tier / $20 month Yes

For the Czech user, it is important to know that most of these models handle the Czech language at a very high level, which allows creating agents that can communicate with Czech customers or work with Czech documents. However, most developer frameworks for agents (such as LangChain or AutoGPT) are primarily documented in English.

Practical impact for companies and individuals

What does this mean for you? If you are a small business owner in the Czech Republic or an IT manager, you cannot adopt agentic AI without clear rules. "Guardrails" must be set before the first launch.

Recommendations for implementation:

  1. Principle of least privilege: An agent should never have access to more data or tools than is absolutely necessary for its task.
  2. Human-in-the-loop: For critical operations (payments, data deletion), human confirmation must always be required.
  3. Audit trail: Every step the agent takes must be logged and immediately traceable.

Agentic AI can give us back thousands of hours of work, but only if we have confidence that its autonomy is in our hands, not beyond them. Security here is not an obstacle to innovation, but its essential cornerstone.

Can agentic AI accidentally drain my bank account or credit card?

If you provide the agent with direct access to payment details without intermediate human control, yes, it is technically possible. That is why it is recommended to use virtual cards with limits or systems that require manual approval of each transaction.

Is agentic AI compliant with the European AI Act?

It depends on the area in which you use it. If the agent makes decisions about loans, in medicine, or in critical infrastructure, it will be classified as a high-risk system and must meet strict requirements for transparency, security, and human oversight.

How do I know my agent is "safe"?

A safe agent is one that has clearly defined boundaries (sandbox), logs all its steps, and its decision-making process is transparent. It should never have the ability to perform actions that are not explicitly part of its assignment.

X

Don't miss out!

Subscribe for the latest news and updates.