Events

Abstract: We introduce an inexact two-level optimization framework (ItsOPT) for finding first- or second-order critical points of nonsmooth and nonconvex functions. The framework includes two levels: In the upper level, a smoothing technique (e.g., High-order Moreau envelope, high-order forward-backward envelope, high-order tensor envelope) will be applied to generate a smooth approximation of the objective function with the same minimizers. Then, first- or second-order methods will be introduced for minimizing the smoothing function. In the lower level, the corresponding high-order proximal subproblems (e.g., High-order proximal, forward-backward, and tensor subproblems) will be solved inexactly using subgradient or Bregman proximal methods. This will provide an approximated solution for the subproblems, leading to inexact smoothing information for the upper-level methods. We note that the complexity of solving the considered optimization problems is the multiplication of the complexities of both levels. Applying accelerated first- or second-order methods at the upper level and solving the subproblems with negligible complexity (e.g., logarithmic rate) may lead to the superfast methods attaining complexity better than worst-case complexity bounds. We finally introduce some algorithms and report preliminary numerical results.

Bio: Prof. Masoud Ahookhosh is an Assistant Professor at the Department of Mathematics at the University of Antwerp, Belgium. He received his PhD in Mathematics (optimization) from the University of Vienna, Austria, in 2016. From 2016 to 2018, he was a postdoctoral researcher at LCSB at the University of Luxemburg. In 2018, he joined KU Leuven as a postdoctoral researcher at STADIUS for two years, and then he was a senior postdoctoral researcher at UC Louvain for 10 months. His current research interests include designing, analyzing, and implementing optimization algorithms for large- to huge-scale nonsmooth and nonconvex problems appearing in a wide range of applications such as signal and image processing, compressed sensing, statistics, machine learning, and data science.

Abstract 

We discuss a new way of development of asymmetric Interior-Point Methods for solving primal-dual problems of Conic Optimization. It is very efficient for problems, where the dual formulation is simpler than the primal one. The problems of this type arise often in Semidefinite Optimization (SDO), for which we propose a new primal-dual method with very attractive computational cost.  In this approach, we do not need sophisticated Linear Algebra, restricting ourselves by standard Cholesky factorization. However, our complexity bounds correspond to the best-known polynomial-time results. Moreover, for symmetric cones the bounds automatically depend on the minimal barrier parameter between the primal and the dual feasible sets. We show by SDO-examples that the corresponding gain can be very big. We discuss some classes of SDO-problems, where the complexity bounds are proportional to the square root of the number of linear equality constraints and the computational cost of one iteration is as in Linear Optimization. Our theoretical developments are supported by encouraging numerical testing.

ABSTRACT: Finding the roots of systems of multivariate polynomial equations is a fundamental problem with applications ranging from robotics to statistics. A widely used class of methods for solving such systems is based on Macaulay matrices. The null space of these large matrices can be decomposed to obtain the roots. However, computing this null space is often the main computational bottleneck. In this talk, we present an algorithm that exploits the inherent structure of Macaulay matrices to compute null spaces faster — yielding improvements both in asymptotic complexity and practical performance. The presentation will be self-contained and accessible; only a basic familiarity with linear algebra is assumed.

URL: https://set.kuleuven.be/phd/seminars/raphaelwiddershoven

This presentation is based on the paper "Improved Topology-Independent Distributed Adaptive Node-Specific Signal Estimation for Wireless Acoustic Sensor Networks" by Paul Didier, Toon van Waterschoot, Simon Doclo, Jörg Bitzer, and Marc Moonen.

We address the challenge of topology-independent (TI) distributed adaptive node-specific signal estimation (DANSE) in wireless acoustic sensor networks (WASNs) where sensor nodes exchange only fused versions of their local signals. An algorithm named TI-DANSE has previously been presented to handle non-fully connected WASNs. However, its slow iterative convergence towards the optimal solution limits its applicability.

To address this, we proposed in this paper the TI-DANSE+ algorithm. At each iteration in TI-DANSE+, the node set to update its local parameters is allowed to exploit each individual partial in-network sums transmitted by its neighbors in its local estimation problem, increasing the available degrees of freedom and accelerating convergence with respect to TI-DANSE. Additionally, a tree-pruning strategy is proposed to further increase convergence speed. TI-DANSE+ converges as fast as the DANSE algorithm in fully connected WASNs while reducing transmit power usage.

The convergence properties of TI-DANSE+ will be shown and its real-life applicability will be discussed in the presentation.

Abstract: In this talk, I will review recent advancements in applying machine learning techniques to gravitational wave data analysis. In addition to discussing machine learning methods, I will highlight the software engineering challenges we encountered and share best practices in software development for fundamental science. 

Bio: Kaze W. K. Wong is a research assistant professor at the Applied Mathematics and Statistics Department, at Johns Hopkins University and a part-time software engineer at the Data Science and AI Institute, JHU. He is also a volunteer high jump coach for the track and field team in JHU. Prior to his appointments in JHU, he was a Flatiron Research Fellow at the Center for Computational Astrophysics, Flatiron Institute. His research primarily focuses on applying machine-learning methods to fundamental science problems (with an extra focus on astrophysics), understanding robustness and tuning properties of neural networks, and producing production-grade open-source scientific software. He earned his Ph.D. from Johns Hopkins University in physics and astronomy in 2021 and his bachelor's degree from The Chinese University of Hong Kong in 2017. He was awarded the GWIC Thesis Prize for his distinct contribution to building Machine-learning-enhanced tools for gravitational-wave data analysis. 

Estimating the position of animals over time provides useful additional information for understanding animal behavior and for ecology studies in general. A common approach for this task is to deploy microphone arrays (nodes) and use the acoustic signals to estimate the direction of arrival (DOA) of the sound source. DOAs from different nodes are then intersected to find the source’s position. However, when multiple sources are active, the DOA association problem (AP) arises as it becomes unclear which DOAs correspond to the same source. This problem is further exacerbated in bioacoustical scenarios where large distances increase the error in the DOA estimates, and sounds often overlap in both time and frequency. In this study, we propose a method to tackle the DOA AP in such challenging environments. Preliminary simulations suggest the potential of the proposed method for scenarios with missed detections and unknown number of sources, even when the number of microphones available at each node is limited.

Abstract:Understanding how genetic code gives rise to the complex traits we observe, remains one of the key missions of modern genetics. Although many milestones in understanding the link between genotype and phenotype have been reached, the genome yet continues to conceal many of its secrets, particularly for complex diseases shaped by a myriad of genetic and environmental factors. The limited clinical utility of state-of-the-art predictive models, polygenic risk scores based on the results of genome-wide association studies, further highlights this challenge. These additive models provide an oversimplified representation of the intricate landscape of molecular biology underlying complex diseases. Today's vast amount of genetic data is shifting the bottleneck from data availability to data interpretation. This dissertation addresses this challenge by introducing a novel neural network-based framework for genome interpretation, capable of processing whole exome sequences end-to-end while allowing for nonlinear interactions between the inputs.
Empirical evidence is provided that, for predicting genetic risk in inflammatory bowel disease, nonlinear deep learning models can outperform state-of-the-art additive models, if the amount of data is sufficiently large. We postulate that nonlinear models, able to capture complex interactions among the inputs, represent a more realistic approximation of real-life molecular biology, benefiting predictive performance as well as the extent to which insights into the true underlying molecular mechanisms can be derived. Key to successful machine learning modeling is finding the right position on the variance-bias spectrum to address the underdetermination in genetic datasets (number of parameters p >> number of samples n). This underdetermination is one of the major drivers for the apparent optimality of additive modeling in clinical genetics today. To tackle this issue while preserving the benefits of nonlinear modeling, we maximize n by leveraging large genetic datasets and at the same time minimize p by exploiting the sparsity of biological networks to constrain the complexity of the models. Further analysis demonstrates that the degree of sparsity is more decisive for predictive performance than the biological meaningfulness of the connections, although incorporating biological knowledge proved instrumental in the extraction of biological insights after the prediction phase.
In this thesis, we benchmark one gene-level and two variant-level whole exome sequence encodings with biologically sparsified neural network architectures, ranging from feedforward fully connected networks to graph neural networks, transformers and convolutional networks. By investigating the decision process leading towards the model’s predictions with Explainable AI methods, we identify pathways, genes and variants relevant to the disease. Additionally, we extend the framework to multiclass prediction in the context of inflammatory bowel disease subtype prediction, showcasing the framework's potential in Stratified Medicine. Possible future extensions, including the incorporation of other omics and environmental data, as well as accounting for population structure and sequencing batch effects, emphasize the value of the framework.
In the last part, we assess the framework's generalizability by applying it to two additional complex diseases: Type 2 Diabetes Mellitus and Schizophrenia. The results validate the nonlinear advantage in a large schizophrenia dataset, confirming the critical role of sample size and the complexity of the encoding and model in the prediction, next to the genetic component and heterogeneity of underlying molecular disease mechanisms. Although further extensive external validation remains necessary, this framework contributes to the groundwork for future research aimed at refining disease classifications, improving patient stratification, and ultimately paving the way for more personalized and effective therapeutic strategies in the era of precision medicine.

ABSTRACT: The performance estimation problem (PEP) methodology enables exact convergence analysis of first-order optimization methods. In this talk, we present the tight analysis of a single iteration of the difference-of-convex algorithm (DCA), also known as the convex-concave procedure, extended to accommodate weakly convex (or hypoconvex) functions in the second argument. Using this analysis, we establish sublinear convergence rates for DCA. Exploiting these insights, we propose a method that optimizes the objective function’s splitting through curvature-shifting technique. Since proximal gradient descent (PGD) is a special case of DCA, our analysis directly yields tight worst-case convergence rates for PGD. Notably, we accommodate any constant stepsize in PGD by allowing weak convexity in the second argument of the objective function’s splitting.

URL: https://set.kuleuven.be/phd/seminars/teodor-rotaru-on-tight-convergence-rates-for-the-difference-of-convex-algorithm-dca

ABSTRACT: The proximal point algorithm (PPA) is a widely used method for solving structured inclusion problems emerging in optimization and variational analysis. Despite its popularity, convergence guarantees have largely been limited to the monotone setting. In this seminar, we discuss the convergence of relaxed preconditioned PPA for a class of nonmonotone problems satisfying a so-called oblique weak Minty condition. The analysis is based on a simple projective interpretation of the algorithm, highlighting the important role of the relaxation parameter. Several classical methods, including Douglas-Rachford, Chambolle-Pock, and progressive decoupling, arise as special cases of our analysis, and we briefly discuss their convergence properties in the nonmonotone setting as well.

URL: https://set.kuleuven.be/phd/seminars/seminar-convergence-of-proximal-methods-without-monotonicity

Abstract: Sound field reconstruction aims to estimate pressure fields in areas lacking direct measurements. Existing techniques often rely on strong assumptions or face challenges related to data availability or the explicit modeling of physical properties. To bridge these gaps, this study introduces a zero-shot, physics-informed dictionary learning approach to perform sound field reconstruction. Our method relies only on a few sparse measurements to learn a dictionary, without the need for additional training data. Moreover, by enforcing the Helmholtz equation during the optimization process, the proposed approach ensures that the reconstructed sound field is represented as a linear combination of a few physically meaningful atoms. Evaluations on real-world data show that our approach achieves comparable performance to state-of-the-art dictionary learning techniques, with the advantage of requiring only a few observations of the sound field and no training on a dataset.

Abstract: Given a nonnegative matrix X and a factorization rank r, nonnegative matrix factorization (NMF) approximates the matrix X as the product of a nonnegative matrix W with r columns and a nonnegative matrix H with r rows. NMF has become a standard linear dimensionality reduction technique in data mining and machine learning. In this talk, we first introduce NMF and show how it can be used as an interpretable unsupervised data analysis tool in various applications. We then discuss the choice of the objective function, with a focus on beta divergences, which plays an instrumental role for obtaining meaningful factorizations. Motivated by these NMF models, we present a general block majorization-minimization (BMM) algorithm, called BMM with extrapolation (BMMe), for solving a class of multi-convex optimization problems. The extrapolation parameters of BMMe are updated using a novel adaptive update rule allowing us to establish subsequential convergence for BMMe. We use this method to design efficient algorithms to tackle NMF with beta-divergences (beta-NMF). These algorithms, which are multiplicative updates with extrapolation, benefit from our novel results that offer convergence guarantees. We also empirically illustrate the significant acceleration of BMMe for beta-NMF on several examples. 

Speaker: Piero Fariselli is an esteemed Italian physicist and bioinformatician, renowned for his significant contributions to computational biology and bioinformatics. He currently serves as a Full Professor at the Department of Medical Sciences at the University of Torino, Italy. Prof. Fariselli earned his Laurea degree in Physics, followed by a Ph.D. in Biophysics. His academic journey laid a strong foundation for his interdisciplinary approach, bridging physics and biology through computational methods. Throughout his career, Prof. Fariselli held various academic positions, including roles at the University of Bologna and the University of Padova, before joining the University of Torino. His research focuses on bioinformatics, computational biophysics, and the application of machine learning techniques to biological data. Fariselli has authored numerous publications, contributing significantly to the fields of protein structure prediction, membrane protein analysis, and genome structural annotation. His work has garnered over 15,000 citations, reflecting its impact and relevance in the scientific community.

Abstract

In today’s world, a vast number of devices, such as cell phones, personal computers, robots, wearable gadgets, etc., are contributing to our daily lives at an ever-increasing pace. These devices are mostly equipped with multiple sensors, capturing data in different modalities. For instance, microphones capture audio signals, cameras record images and videos, and biosensors monitor biomedical information. These devices also have memory and computational capacities, enabling efficient learning and signal processing, resulting in, e.g., accurate object recognition in images and powerful denoising techniques for better sound quality in calls. Although these processing capabilities enable each device to capture and analyze data efficiently, cooperation between devices can result in higher and more robust performance. Through proper distributed learning and signal estimation algorithms, sharing multi-modal data between devices guarantees better generalization in learning tasks—specifically for data-hungry learning tasks—and ensures better estimation of signal statistics, thereby improving signal processing quality. Furthermore, through efficient distributed optimization algorithms, sharing computational power enables each device to perform more demanding tasks in a more robust manner.

While cooperation offers each agent (device) various benefits in multi-agent systems, it brings multiple challenges in distributed learning, optimization, and signal estimation that need to be addressed to achieve higher performance compared to single-agent scenarios. Multi-agent sys- tems need to accommodate dynamic system configurations and adapt quickly to new conditions, as agents may freely enter or leave the system, each with dynamic demands and constraints. Co- operation also calls for devising frameworks that can address heterogeneity in data distribution and in the communication, memory, and computational capacities of agents. Communication between agents may also be unreliable, with random delays and limited resources, such as bandwidth; hence, asynchronicity and resource allocation (RA) need to be taken into account. Mutual interactions between agents, in the sense that each agent’s performance is affected by other agents’ actions and decisions, are necessary to be considered for higher performance. Pri- vacy concerns and addressing nonidealities, e.g.,, nonlinearities, are also among the imperative challenges in multi-agent systems.

The objective of this thesis is to address the aforementioned challenges in distributed learning, optimization, and signal estimation by developing relevant methods and algorithms. Mathemat- ical tools and methods such as zeroth-order optimization, biased stochastic gradient descent, variational inequalities, the Moreau envelope, and distributed adaptive node-specific signal esti- mation are considered to devise various distributed algorithms with convergence and adaptation guarantees for multiple applications across the chapters of the thesis. In the first three chapters, the developed distributed learning algorithms are applied to deep learning-based RA in wire- less communication networks, multi-task learning in generic multi-agent systems, and federated learning. A superlinear convergent optimization algorithm is developed in the next chapter for regularized nonconvex finite sum minimization, a problem at the core of many machine learning and signal processing applications. The developed optimization algorithm has favorable global convergence guarantees and can handle a broader class of nonconvex functions, specifically those that are non-Lipschitz differentiable. In the last chapter, distributed signal estimation is investigated to develop a cooperative uplink channel estimation algorithm in cell-free massive multi-input-multi-output communication networks.

Short Abstract:
"This dissertation focuses on leveraging multiple data types, such as images, text, and audio, to build more robust and accurate machine learning systems. It emphasizes effective combination strategies while preventing any single modality from dominating and impairing performance. Extensive research during this PhD explored new models, training objectives, and processes to optimize modality interactions, balance contributions, and ensure generalization across diverse tasks. This work advances multimodal learning by addressing these challenges and provides practical solutions for developing more efficient and adaptable AI systems."

Start: 22/01/2025, 11:00 - 12:00Location: B00.35
Title: Microphone Pair Selection Methods for Sound Source Localization in Large-Scale Spatially Distributed Microphones
ABSTRACT: In massive distributed microphone arrays, such as in conference systems, the use of all sensors leads to unnecessary energy consumption and limited network life, as some sensors may have a limited contribution to specific estimation tasks, such as sound source localization (SSL). In this work, we propose two microphone pair selection methods in steered response power (SRP) based SSL using massive arrays of spatially distributed microphones. The first method is based on thresholding the time-domain cross correlations (TDCCs) at lag 0, which results in a selection of broadside microphone pairs, and only the signals from those microphones are collected for use in the SRP algorithm. The second method is based on sparseness of the cross-correlation in which the pairs which have a high sparsity factor are selected for use in localization. Instead of computing the cross correlations that are used in the selection methods directly, we adopt a distributed consensus approach and obtain an approximation for the cross correlations. Additionally, we introduce a gossip protocol such that the threshold of pairwise cross correlations for the selection of microphone pairs are computed without relying on a central fusion center. Simulations show that we achieve a localization performance only using the selected microphone pairs that is comparable to the performance when all microphones are employed.