In this page you can find some proposal for master and bachelor thesis. Unless specified, the proposal are for master thesis. The page is organised in the five broad research areas we cover in the group.
Explainable and Neuro-Symbolic AI
- Large Language Models for Logic Embeddings: this thesis aims at integrating symbolic knowledge and large deep learning models, in the field of neural symbolic computing. The goal is to solve requirement mining and optimization tasks by using large language models like [1] to map logic formulae in a semantic-preserving continuous latent space. Moreover, such an approach could be applied in the context of XAI for complex systems.
- Active Learning for Kernel Methods: starting from the kernel defined in [2] the goal of this thesis is to devise an active learning strategy to select a small number of instances used to compute the kernel itself, providing guarantees on the quality of the coverage obtained. Results could be applied in the context of anomaly detection and model checking of IoT and cyber-physical systems.
- XAI for medical time-series (and images): investigate the state of the art of XAI techniques for time series, apply and test the stability of some of them applied to medical data (electrocardiogram signals). Validate the appropriate metrics to be used in order to objectively describe the stability of a XAI method. Potentially it can be extended to a reasoning on the efficacy of an ensemble approach to produce better results. Extensions to medical images are also possible.
- Combining Reasoning and Large Language Models: investigate in restricted knowledge domain how map responses of LLM and activation patterns of their attention matrices to concepts of such domain, leveraging reasoning tools to validate correctness. An example: if the LLM is outputting text about mathematical concepts and mathematical reasoning, verify if such reasoning is correct using tools such as proof verifiers or symbolic computations.
[1] Y. Li, et al., “Competition-level code generation with alphacode”, Science 378 (2022).
[2] L. Bortolussi, et al., “Learning model checking and the kernel trick for signal temporal logic on stochastic processes”, TACAS 2022.
Simulation Intelligence and Reinforcement Learning
- Symbolic Regression for Ordinary Differential Equations: symbolic regression is a regression task which aims to find the best mathematical expression that fits a given dataset. In this thesis different techniques (genetic, programming, transformers) will be explored to fit data from dynamical systems governed by ordinary differential equations. Comparisons with black-box techniques for learning the dynamics of a system can also be possible.
- Surrogate modeling of continuous time dynamical systems: surrogate models are simplified approximations of more complex and possibly high-order models. The aim of this thesis is to explore data-driven AI techniques for approximating dynamical systems governed by differential equations.
- Hybrid diffusion-based generative models: Investigate diffusion/score-based models for data that has both continuous and categorical dimensions (for example integrating score-based approaches with Continuous-time Markov chain ones), and elaborate a hybrid diffusion model that can be applied to real world datasets (for example tabular data) [1, 2, 3].
- Multi-Task Learning for Combinatorial Optimization: recent studies have shown that machine learning can improve exact combinatorial optimization solvers by learning variable selection and branching strategies [4]. Yet, these methods fail to generalize to problems not seen during training. The aim of this thesis is to explore multi-task learning approaches [5] [6] to study their effect on generalization to new problems.
- Deep Symbolic Regression with Reinforcement Learning: symbolic regression [7] is a widely studied problem to recover mathematical expressions from data. In [8], gradient-based optimization was combined with geometric semantic genetic programming (GSGP) [9] to improve its performance. The aim of this thesis is to leverage deep neural networks and reinforcement learning to learn evolutionary operators to extend the work in [8] and further improve the performance of GSGP.
- Efficient Uncertainty Estimates for Monitoring and Control the Safety of Stochastic Systems
- Exploring Latent Space Structure of Generative Models
- Logics for Safe Multi-Agent RL
[1] Song, Y., Sohl-Dickstein, J., Kingma, D. P., Kumar, A., Ermon, S., & Poole, B. (2021). Score-Based Generative Modeling through Stochastic Differential Equations. ArXiv [Cs.LG].
[2] Austin, J., Johnson, D. D., Ho, J., Tarlow, D., & van den Berg, R. (2021). Structured Denoising Diffusion Models in Discrete State-Spaces. CoRR, abs/2107.03006.
[3] Sun, H., Yu, L., Dai, B., Schuurmans, D., & Dai, H. (2023). Score-based Continuous-time Discrete Diffusion Models. The Eleventh International Conference on Learning Representations.
[4] Gasse, Maxime, Didier Chételat, Nicola Ferroni, Laurent Charlin, and Andrea Lodi. “Exact combinatorial optimization with graph convolutional neural networks.” Advances in neural information processing systems 32 (2019).
[5] Crawshaw, Michael. “Multi-task learning with deep neural networks: A survey.” arXiv preprint arXiv:2009.09796 (2020).
[6] Sodhani, Shagun, Amy Zhang, and Joelle Pineau. “Multi-task reinforcement learning with context-based representations.” In International Conference on Machine Learning, pp. 9767-9779. PMLR, 2021.
[7] La Cava, William, Patryk Orzechowski, Bogdan Burlacu, Fabrício Olivetti de França, Marco Virgolin, Ying Jin, Michael Kommenda, and Jason H. Moore. “Contemporary symbolic regression methods and their relative performance.” arXiv preprint arXiv:2107.14351 (2021).
[8] Pietropolli, Gloria, et al. “Combining geometric semantic gp with gradient-descent optimization.” Genetic Programming: 25th European Conference, EuroGP 2022, Held as Part of EvoStar 2022, Madrid, Spain, April 20–22, 2022, Proceedings. Cham: Springer International Publishing, 2022
[9] Moraglio, Alberto, Krzysztof Krawiec, and Colin G. Johnson. “Geometric semantic genetic programming.” Parallel Problem Solving from Nature-PPSN XII: 12th International Conference, Taormina, Italy, September 1-5, 2012, Proceedings, Part I 12. Springer Berlin Heidelberg, 2012.
Robust and Bayesian Machine Learning
- Resilience to adversarial attacks (or lack thereof) as a proxy to Bayesianity: recent works aimed at assessing the robustness of Bayesian Neural Networks (BNNs) against adversarial attacks [1, 2, 3], mostly gradient-based, show clear resilience properties from both a mathematical and an experimental viewpoint. On the other hand, some variously (un)successful non-BNN models [e.g. 4, 5] have been proposed to ease the computational burden of BNN training, while preserving their ability to quantify uncertainty. Given the current lack of satisfactory strategies to quantify the degree of Bayesianity of a model, this thesis investigates the use of adversarial robustness benchmarks for BNNs as a proxy to accomplish such a goal.
- Better-than-random features projection without explicit learning: the very characteristic element of modern deep learning, beyond model size or complexity, is its ability to learn a suitable featurization from data (as opposed to, e.g., kernel methods, where features are handcrafted). Such learnability, though, comes at the expense of increased training cost and time – especially in large models. The use of random features [6], though practicable, is on the other hand discouraged due to their exponential scaling in the number of decision boundaries to be learned. Inspired by the neuroanatomical features of the early visual system [7], the approaches investigated in this thesis aim at augmenting projection on random features with a novel sampling scheme, to implicitly bias their selection towards more amenable representations for shallow (or even deep) learning. P.I.: Prof. Fabio Anselmi.
[1] G. Carbone et al., “Robustness of Bayesian Neural Networks to Gradient-Based Attacks”, 2020.
[2] G. Carbone et al., “Resilience of Bayesian Layer-Wise Explanations under Adversarial Attacks”, 2022.
[3] L. Bortolussi et al., “On the Robustness of Bayesian Neural Networks to Adversarial Attacks”, 2022.
[4] Y. Gal and Z. Ghahramani, “Dropout as a Bayesian Approximation: Representing Model Uncertainty in Deep Learning”, 2016.
[5] P. Izamilov et al., “Averaging Weights Leads to Wider Optima and Better Generalization”, 2018.
[6] A. Rahimi and B. Recht, “Random Features for Large-Scale Kernel Machines”, 2007.
[7] T.A. Poggio and F. Anselmi, “Visual Cortex and Deep Networks”, MIT Press, 2016.
AI for Sustainability and Health
- Explainable AI for anomalies in water distribution: this thesis aims at exploring and applying eXplainable AI techniques on anomaly detection for water distribution system data. By tackling anomaly detection as a prediction problem, explainability is needed to recognise the type of anomaly and/or provide examples to the user of other instances in which a similar anomaly occurred. In the thesis, some XAI methods will be investigated, including feature relevance measures and symbolic techniques.
AI for Industry
- Automatic routing for analog circuits: the aim of this master’s thesis is to address the routing problem within the context of analog circuits using automated techniques. The proposed approach will involve a literature review of established algorithms that are commonly used for automated routing of analog circuits. Subsequently, the selected algorithms will be implemented to solve the routing problem in an abstract context, which will involve using dummy floorplans with obstacles to avoid while maintaining the properties of an actual circuit. Furthermore, it will be explored whether the algorithm can be tested in a real-world scenario.
- Risk Factor Clustering and Macroeconomic Scenario Identification: the returns of stocks can be thought of as driven by a number of broad risk factors. While predicting directly stocks returns is an extremely difficult task, an alternative estimation is possible if we are able to identify the underlying risk factors. To do so, we build some historical risk factor clusters that we are able to interpret as macroeconomic scenarios. We are interested in estimating not only the clusters but also the transition probabilities between the clusters themselves. In this way, knowing the current cluster allows us to estimate a probability distribution on the clusters for the following time window.
[1] Brooke E Husic and Vijay S Pande. Markov state models: from an art to a science. Journal of the American Chemical Society, 2018.