Projects, Research, and Courses
This page is primarily intended for Technion students interested in exploring GenAI safety and security, or advanced architecture and training methods in foundation models. Here, you'll find basic information about our lab courses and the available research projects.
Alignment, Security, and Adversarial Methods for AI Models
Advanced Topics in Deep Learning Models 236207
This course explores adversarial and safety methodologies in deep learning and cybersecurity issues, all on up-to-date models from Vision to LLMs. We begin with the foundations of adversarial attacks in computer vision, introducing the attacks FGSM, PGD, and universal perturbations, followed by adversarial training.
From there, we transition to transformers and LLMs, examining their architecture, pre-training objectives, and alignment. Continue to latent space vulnerabilities, including the Refusal and Unlearning Directions. We then examine jailbreak attacks and the mechanisms behind instruction manipulation and alignment evasion.
We expand to gradient-based attacks in non-LLM domains such as audio, robotics, recommendation systems, and reasoning models. We also study optimization and interpretability techniques ranging from activation-based defenses to adversarial training.
Finally, we address real-world AI security case studies. These include attack surfaces in multi-agent protocols (e.g., Google’s A2A), LLM hijacking, DLP vulnerabilities in AI agent integrations, insecure code generation, and attacks on AI-based search engines. Guest lectures by domain experts will offer additional applied perspectives. You will be graded according to homework assignments and your final project, which will be in groups of 2.
Open Research Proposals
Registration and Information:
These research projects begin as 1–2 semester tracks, typically worth 3 credit points per semester. The work is expected to take approximately 12–18 hours per week, after those 1-2 semesters, the project work may lead to joining the lab for deeper work for outstanding students who deliver strong results.
For these projects, you should have completed a Deep Learning course, and by the first two weeks of the project, you’ll need to learn about Transformers, NLP, and other advanced methods in AI safety.
For more information, please send an email to Amit at Amitlevi@campus.technion.ac.il, including your grade sheet, CV, and a short personal note (maximum half a page) where you can share your motivation and any relevant experience.
After your email is received, you’ll be given a short time to read 3–4 papers and learn 1–2 topics, which you’ll then present and discuss for 30 minutes. You’re free to use any tools during the learning process and presentation, nothing will be considered cheating.
Investigate how and when (detection) hallucinations emerge during multi-step reasoning processes.
Hallucinations in Reasoning
(Self-learning, MSc Advantage)
Combine conformal prediction with adversarial defenses to achieve minimal latency and high-precision protection in LLMs.
Mixture Of Defenses
Implement memory mechanisms to compress and retain history across long contexts.
Long-Term Memory for LLMs
(Self-learning, MSc Advantage)
We explore how LLMs can appear safe during reasoning, while masking deeper safety failures. This project questions whether current methods truly align reasoning processes — or just create the illusion of it.
Natural Safety Understanding
(MSc Advantage)
Break down vision models into segment-wise reasoning/classification pipelines, and VLLMs.
Per-Segment Classification
(Vision, MSc Advantage)
We will design a benchmark to probe decision-level behavior and latent representations, showing that while debiasing may reduce biased outputs, it often fails to alter the underlying bias. Our goal is to expose the gap between implicit and explicit debiasing and motivate more robust, interpretable interventions.
Evaluating Limits of LLM Debiasing
Reframe LLMs as based classifiers through interpretable embedding-space trajectories.
Flashing LLMs as Judge
Building on recent findings about failures in safety alignment (e.g., Jailbroken: How Does Alignment Fail on Unsafe Prompts?), we extend this failure to unlearning. Our preliminary results show that deleted knowledge can still be recovered, we will explain the structural reasons why unlearning in LLMs remains fundamentally brittle.
Where Unlearning Fails
(MSc Advantage)
The students will work with transformer-based large language models and use internal shared representations to perform greedy probing, aiming to uncover how information was implicitly represented in the training data. This approach will help analyze the encoding structure of factual knowledge and improve interpretability of LLMs.
Activation Directions in Training Data Traces(MSc Advantage)