AI Security Risks

The development of AI has also brought new forms of security threats, which are often difficult to distinguish from legitimate communication. This module explains how these threats arise and how to recognize them in practice before they cause damage.
Participants will learn about personalized phishing and social engineering, which today, thanks to AI, appear much more credible than before. The module explains how synthetic voice, image and video are created, and how deepfakes abuse the trust in what we see or hear. Attention is also paid to topics specific to AI systems themselves - prompt injection, misuse of instructions or data leakage through inappropriately formulated assignments.
The module also shows how output manipulation and false authority of a model can occur, when a convincing answer is mistakenly considered trustworthy. Participants will adopt basic protective habits for ordinary users and learn how to proceed if abuse or fraud is suspected - i.e. who to contact and what steps to take before damage occurs.

• employees and managers who may be the target of personalized phishing or deepfake fraud,
• companies that want to increase their team's security awareness regarding AI threats,
• people who want to understand new types of attacks and know how to respond to them,
• participants following previous AI Academy modules.

• Personalized Phishing and Social Engineering
• Synthetic Voice, Image, and Video
• Deepfakes and Trust Manipulation
• Prompt Injection and Instruction Abuse
• Data Leakage via Inappropriate Input
• Output Manipulation and False Model Authority
• Basic Security Habits for Regular Users
• How to Respond to Suspected Abuse or Fraud

• Recognize personalized phishing and social engineering techniques.
• Understand how synthetic voice, image, and video are created and used.
• Understand the principles of deepfakes and trust manipulation.
• Be familiar with threats specific to AI systems, such as prompt injection.
• Know how to respond to suspected abuse or fraud.

What is personalized phishing and why does AI make it more dangerous?
It is a targeted fraudulent attack that AI allows to be tailored to a specific person or situation, making it appear much more believable than generic fraudulent messages.

What is a deepfake?
A synthetic image, voice, or video created by AI to look or sound like a real person, often with the aim of deceiving or manipulating.

What does deepfake mean? prompt injection?
A technique where an attacker misuses input or content to cause an AI system to execute an unintended instruction or release sensitive information.

How to protect yourself from these types of threats?
It is important to verify unusual requests through other channels, not to trust a representative solely on the basis of their persuasiveness, and to know the internal procedure for reporting suspected abuse.

What to do if you suspect abuse or a scam?
Do not react immediately, verify the situation through another channel and contact the responsible person or security department in the organization.