Student Project Proposals

Privacy & Anonymization

The word privacy is derived from the Latin word "privatus" which means set apart from what is public, personal and belonging to oneself, and not to the state. There are multiple angles of privacy and multiple techniques to improve them to varying extent. Students can work on the following topics:

• (De-)Anonymization of Medical Data: ECG (Electrocardiogram) and CTG (Cardiotocography), diagnostic images (MRI, X-ray), are very sensitive datasets containing the medical records of individuals.The task is to (de-)anonymize such datasets (or some aggregates computed over such data) for data sharing with strong, preferably provable privacy guarantees which are also GDPR compliant.
  (Contact: Gergely Ács)
• Poisoning Differential Privacy: Differential Privacy is the de facto privacy model used to anonymize datasets (see US-Census data). Small noise is added to the data which hides the participation of any single individual in the dataset, but not the general statistics of the population as a whole. The noise is calibrated to the influence of any record. However, if the data is coming from untrusted sources, the attacker can inject fake records into the dataset in order to increase the added noise that eventually degrades the utility of the anonymized data. The task is to design and implement such an attack.
  (Contact: Gergely Ács)
• Differential Privacy Amplification: Nowadays, the standard privacy-preserving mechanism is Differential Privacy. It aims to hide the presence or the absence of a data point in the final result by adding noise to the original query, making the two outcomes (one with and one without a single data point) statistically indistinguishable (up to the privacy- parameter). For example, the average salary of BME last year's graduates are published with added noise, so even if an adversary knows all alums' salaries except its target, it cannot deduce that with certainty.
  Besides the size of the added noise, privacy protection can further be increased by so-called amplification techniques, such as sampling from the data (instead of utilizing all). For instance, only half of the alums are considered for this statistic.
  The student's task is to learn and experiment with these amplification techniques and to find the optimal setting (amplification mechanisms and its parameters) to obtain a desirable trade-off between the provided privacy protection and the obtained accuracy.
  (Contact: Balázs Pejó)
• Own idea: If you have any own project idea related to data privacy, and we find it interesting, you can work on that under our guidance.
  (Contact: Gergely Ács or Balázs Pejó)

Required skills: none
Preferred skills: basic programming skills (e.g., python)

Machine Learning & Security & Privacy

Machine Learning (Artificial Intelligence) has become undisputedly popular in recent years. The number of security critical applications of machine learning has been steadily increasing over the years (self-driving cars, user authentication, decision support, profiling, risk assessment, etc.). However, there are still many open security problems of machine learning. Students can work on the following topics:

• Security of Machine learning based Malware Detection:   Adversarial examples are maliciously modified program code where the modification is hard to detect yet the prediction of the model on this slightly modified code is very different compared to the unmodified code. For example, the malware developer modifies a few bytes in the malware binary which causes the malware detector to misclassify the malware as benign. A potential task can be to develop solutions to detect adversarial examples, develop robust training algorithms for malware detection, or design backdoor and membership attacks.
  (Contact: Gergely Ács)
• Robustness of Large Language Models:   Large Language Models (LLMs) are a new class of machine learning models that are trained on large text corpora. They are capable of generating text that is indistinguishable from human-written text. The increasing reliance on Large Language Models (LLMs) across academia and industry necessitates a comprehensive understanding of their robustness to prompts. The task is to study and test different adversarial prompts against LLMs (such as adversarial attacks, or prompt injection, or any other adversarial prompts). (Contact: Gergely Ács)
• Application of Large Language Models to solve Security/Privacy Problems:   It has been shown that Large Language Models (LLMs) can also solve simple classification problems with few shot learning. The task is to study the application of Large Language Models (LLMs) to solve security/privacy problems, such as anomaly detection, malware detection, or anonymization.
  (Contact: Gergely Ács)
• Testing Data Inference: In Machine Learning, the underlying data is separated into training and testing, where the model is trained on the former to make an accurate prediction. Yet, besides general patterns, the model could learn explicit information corresponding to specific training data points, which could lead to privacy leakage. For instance, it is possible to determine whether a particular sample was used for training via a Membership Inference Attack. It takes advantage of the confidence values of the prediction: a model predicts more confidently on samples it was trained than on samples it has never seen before.  
  The student's task is to learn about state-of-the-art privacy attacks, which single-mindedly aim at the training data. Inspired by them, it is desired to adopt or create novel techniques to infer details about the dataset used for testing (if possible). 
  (Contact: Balázs Pejó)
• Meta Learning: In online media, there is legit news as well as fake news. While the former is usually of higher quality, the latter is often associated with low-quality writing. Several machine learning models focus on classifying these two in the scientific literature. Moreover, in the scientific literature itself, there are lower and higher-quality publications as well. 
  The student's task is to get familiar with these models and experiment with their applicability to differentiate non-peer-reviewed scientific papers (e.g., on ArXiv) from articles that appeared in well-established venues (such as S&P, CCS, etc.). 
  (Contact: Balázs Pejó)
• Improving Machine Learning by Preclassification: In a classical Machine Learning scenario, data (e.g., traffic signs) is fed in random order into the model (e.g., traffic sign predictor) for every training iteration. While the model would eventually learn the difference between each data class (e.g., traffic sign type), randomly feeding the data into the model might slow the learning process: one would think it would be faster to differentiate between the very distinct data classes (e.g., the Stop sign has a unique shape) first (i.e., early training rounds) and latter (i.e., later training rounds) focus on the more similar classes (e.g., the Speed Limit 100 and 120).
  On the other hand, the model does not know which classes are similar or distinct before training. In fact, this is what it aims to learn in the first place.
  The student's task is to review the previous research efforts toward solving this chicken-egg problem and to propose and implement a novel speed-up solution.
  (Contact: Balázs Pejó)
• Own idea: If you have any own project idea related to the security/privacy of machine learning, and we find it interesting, you can work on that under our guidance.
  (Contact: Gergely Ács or Balázs Pejó)

Required skills: none
Preferred skills: basic programming skills (e.g., python), machine learning (not required)

Federated Learning - Security & Privacy & Contribution Scores

Federated learning enables multiple actors to build a common, robust machine learning model without sharing data, thus allowing to address critical issues such as data privacy, data security, data access rights and access to heterogeneous data. Its applications are spread over a number of industries including defense, telecommunications, IoT, and pharmaceutics. Students can work on the following topics:

• Security and Privacy of Federated Learning: Federated learning allows multiple parties to collaborate in order to train a common model, by only sharing model updates instead of their training data (e.g., mobile devices train a common model for input text prediction, or hospitals train a better model for tumor classification). Even if this architecture seems more privacy-preserving at first sight, recent works have highlighted numerous privacy and security attacks to infer private and sensitive information. The task is to develop privacy and/or security attacks against federated learning (data poisoning, backdoors, reconstruction attacks), and/or mitigate these attacks.
  (Contact: Gergely Ács)
• Federated Learning Framework for Medical Data: Federated learning is going to be adopted in health care, where different organizations want to train a common model for different purposes (tumor/disease classification, prediction of survival time, finding an explainable pattern of Covid on whole slide images of livers, etc.) but organizations lack of sufficient training data individually. The task is to develop federated learning framework for such tasks.
  (Contact: Gergely Ács)
• Robust Federated Learning: In a classical Machine Learning scenario, data (e.g., textual) is fed into the model (e.g., next-word predictor) for training. The larger this dataset, the greater the prediction power of the trained model. However, in many real-world applications, no central dataset is available for training, but smaller pieces are scattered among various entities (e.g., on mobile phones). A model trained on small local datasets might be biased or have low accuracy. Hence, it is desired to train a joint model collaboratively.
  Due to various privacy regulations, directly sharing sensitive data (e.g., text messages) is not feasible. Federated Learning tackles such a distributed setup where multiple entities (e.g., mobile devices) train a single machine learning model together (e.g., next-word predictor) based on their overall dataset, which is never shared with other participants. 
  On the other hand, the participants could be malicious. Their goal could vary from performance degradation to increasing/decreasing the occurrence of desired strings (e.g., Adidas). There are several techniques to mitigate the effect of the malicious participants in Federated Learning; some work in the client selection phase, while others work in the aggregation phase. 
  The student's task is to explore these techniques, experiment with them and propose an optimal solution that combines several.
  (Contact: Balázs Pejó)
• Free RIder Detection using Attacks (FRIDA): In Machine Learning, the model is designed to learn patterns from a dataset based on which it should make its prediction. However, besides general patterns, the model could learn explicit information corresponding to specific data points, which could lead to privacy leakage. Such leakage could be revealed with Membership Inference Attack, which determines whether a particular data sample was used for training.
  In Federated Learning, multiple individuals train a single model together in a privacy-friendly way, i.e., their underlying datasets remain hidden from the other participants. As a consequence of this distributed setup, dishonest participants might behave maliciously by free-riding (enjoying the commonly trained model while not contributing to it).
  The student's interdisciplinary task is to read about the Membership Inference Attacks and the free-riding problem in Federated Learning. Furthermore, to propose a framework that connects the two, i.e., use Membership Inference Attack to determine whether the participant used actual data or just random noise during training.
  (Contact: Balázs Pejó)
• Capture the Fairness/Robustness/Accuracy/Privacy Trade-Off (FRAP): The chief goal of today's Machine Learning models is Accuracy: the higher this value, the better the model. This is a vast overlook, as there are several other objectives a Machine Learning model should be optimized for, such as Robustness, Fairness, and Privacy.
  Robustness captures how resistant the model is towards hostile behavior, e.g., injecting malicious samples in the training set. For instance, misclassifying 1% of the Stop signs as Give Way in autonomous driving is more desirable than only misclassifying 0.1% as Speed Limit 100.
  Fairness considers the model's behavior on different subgroups of the population. For example, a face recognition software with 99% accuracy over the whole population could be more desirable than another with 99.9% accuracy overall but with 50% on a small ethnic group.
  Finally, Privacy deals with undesired information leakage: a model with superior performance might leak a severe portion of its training data. In contrast, another with decent performance only reveals a negligible amount.
  As highlighted above, there are clear connections between Accuracy and the three mentioned aspects. Additionally, there are initial research papers on any triad with Accuracy. Yet, other combinations are unexplored. The student's task is to select two (from Robustness, Privacy, and Fairness) and measure their effect on each other. A Game-theoretic model is desired to determine the optimal setting based on some predefined incentives.
  (Contact: Balázs Pejó ot (Contact: Gergely Biczók)
• Fairness of Shapley Approximations: In any distributed setting with a single common product, such as in Federated Learning (where multiple participants train a Machine Learning model together in a privacy-friendly way), the contribution of the individuals is a crucial question. For instance, when several pharmaceutical companies train a model together, which leads to a huge breakthrough, how should they split the pay-off corresponding to the model?
  Equal distribution is as unfair as the one based on the dataset sizes, as neither considers the data quality. What does fair mean in the first place? Shapley defined four fundamental fairness properties and proved that his reward allocation scheme is the only one that satisfies all. On the other hand, it is exponentially hard to compute, so it is standard practice to approximate it in real life.
  The student's task is to study existing approximation methods and verify (theoretically or empirically) to what extent these methods respect the four desired properties.
  (Contact: Balázs Pejó or Gergely Biczók)
• Own idea: If you have any own project idea related to the security/privacy of federated learning, and we find it interesting, you can work on that under our guidance.
  (Contact: Gergely Ács or Balázs Pejó or Gergely Biczók)

Required skills: none
Preferred skills: basic programming skills (e.g., python), machine learning (not required)

Economics of (cyber)security and (data)privacy

As evidenced in the last 10-15 years, cybersecurity is not a purely technical discipline. Decision-makers, whether sitting at security providers (IT companies), security demanders (everyone using IT) or the security industry, are mostly driven by economic incentives. Understanding these incentives are vital for designing systems that are secure in real-life scenarios. Parallel to this, data privacy has also shown the same characteristics: proper economic incentives and controls are needed to design systems where sharing data is beneficial to both data subject and data controller. An extreme example to a flawed attempt at such a design is the Cambridge Analytica case.
The prospective student will identify a cybersecurity or data privacy economics problem, and use elements of game theory and other domain-specific techniques and software tools to transform the problem into a model and propose a solution. Potential topics include:

• CPSFlipIt: attacker-defender dynamics in cyber-physical systems
• Risk management for cyber-physical/OT systems
• Incentives in secure software development: why should programmers have proper security training?
• Interdependent privacy: modeling inference with probabilistic graphical models
• BYOT: Bring Your Own Topic!

Required skills: model thinking, good command of English
Preferred skills: basic knowledge of game theory, basic programming skills (e.g., python, matlab, NetLogo)