Self-supervised learning for image denoising problems in the presence of denaturation for noisy data is a crucial approach in machine learning. However, theoretical understanding of the performance of the approach that uses denatured data is lacking. To provide better understanding of the approach, in this paper, we analyze a self-supervised denoising algorithm that uses denatured data in depth through theoretical analysis and numerical experiments. Through the theoretical analysis, we discuss that the algorithm finds desired solutions to the optimization problem with the population risk, while the guarantee for the empirical risk depends on the hardness of the denoising task in terms of denaturation levels. We also conduct several experiments to investigate the performance of an extended algorithm in practice. The results indicate that the algorithm training with denatured images works, and the empirical performance aligns with the theoretical results. These results suggest several insights for further improvement of self-supervised image denoising that uses denatured data in future directions.

Acquiring plausible pathways on high-dimensional structural distributions is beneficial in several domains. For example, in the drug discovery field, a protein conformational pathway, i.e. a highly probable sequence of protein structural changes, is useful to analyze interactions between the protein and the ligands, helping to create new drugs. Recently, a state-of-the-art method in drug discovery was presented, which efficiently computes protein pathways using latent variables obtained from an isometric auto-encoding of the space of 3D density maps associated to protein conformations. However, our preliminary experiments show that there is room to significantly reduce the computing time. In this study, we use the Mapper algorithm, which is a Topological Data Analysis method, and present a novel variant to extract plausible conformational pathways from the isometric latent space with comparatively short running time. The extracted pathways are visualized as paths on the resulting Mapper graph. The methodological novelties are described as follows: firstly, the filter function of the Mapper algorithm is optimized so as to extract the pathways via minimization of an energy loss defined on the Mapper graph itself, while filter functions taken in the classical Mapper algorithm are fixed beforehand. The optimization is with respect to parameters of a deep neural network in the filter. Secondly, the clustering method, which defines the vertices and edges of the Mapper graph, of our algorithm is designed by incorporating domain prior knowledge to assist the extraction. In our numerical experiments, based on an isometric latent space built on the common 50S-ribosomal dataset, the resulting Mapper graph successfully includes all the well-recognized plausible pathways. Moreover, our running time is much shorter than the above state-of-the-art counterpart.

Advancing defensive mechanisms against adversar-ial attacks in generative models is a critical research topic in machine learning. Our study focuses on a specific type of generative models - Variational Auto-Encoders (VAEs). Contrary to common beliefs and existing literature which suggest that noise injection towards training data can make models more robust, our preliminary experiments revealed that naive usage of noise augmentation technique did not substantially improve VAE robustness. In fact, it even degraded the quality of learned representations, making VAEs more susceptible to adversarial perturbations. This paper introduces a novel framework that enhances robustness by regularizing the latent space divergence between original and noise-augmented data. Through incorpo-rating a paired probabilistic prior into the standard variational lower bound, our method significantly boosts defense against adversarial attacks. Our empirical evaluations demonstrate that this approach, termed Robust Augmented Variational Auto-EN coder (RAVEN), yields superior performance in resisting adversarial inputs on widely-recognized benchmark datasets.

Given a huge, online stream of time-evolving events with multiple attributes, such as online shopping logs: (item, price, brand, time), how can we summarize large, dynamic high-order tensor streams? How can we see any hidden patterns, rules, and anomalies? Our answer is to focus on two types of patterns, i.e., “regimes” and “components”, over high-order tensor streams, for which we present an efficient and effective method, namely CubeScope. Specifically, it identifies any sudden discontinuity and recognizes distinct dynamical patterns, “regimes” (e.g., weekday/weekend/holiday patterns). In each regime, it also performs multi-way summarization for all attributes (e.g., item, price, brand, and time) and discovers hidden “components” representing latent groups (e.g., item/brand groups) and their relationship. Thanks to its concise but effective summarization, CubeScope can also detect the sudden appearance of anomalies and identify the types of anomalies that occur in practice.

We consider the scenario of deep clustering, in which the available prior knowledge is limited. In this scenario, few existing state-of-the-art deep clustering methods can perform well for both noncomplex topology and complex topology data sets. To address the problem, we propose a constraint utilizing symmetric InfoNCE, which helps an objective of the deep clustering method in the scenario of training the model so as to be efficient for not only noncomplex topology but also complex topology data sets. Additionally, we provide several theoretical explanations of the reason that the constraint can enhances the performance of deep clustering methods. To confirm the effectiveness of the proposed constraint, we introduce a deep clustering method named MIST, which is a combination of an existing deep clustering method and our constraint. Our numerical experiments via MIST demonstrate that the constraint is effective. In addition, MIST outperforms other state-of-the-art deep clustering methods for most of the commonly used 10 benchmark data sets.

Domain shifts in the training data are common in practical applications of machine learning; they occur for instance when the data is coming from different sources. Ideally, a ML model should work well independently of these shifts, for example, by learning a domain-invariant representation. However, common ML losses do not give strong guarantees on how consistently the ML model performs for different domains, in particular, whether the model performs well on a domain at the expense of its performance on another domain. In this paper, we build new theoretical foundations for this problem, by contributing a set of mathematical relations between classical losses for supervised ML and the Wasserstein distance in joint space (i.e. representation and output space). We show that classification or regression losses, when combined with a GAN-type discriminator between domains, form an upper-bound to the true Wasserstein distance between domains. This implies a more invariant representation and also more stable prediction performance across domains. Theoretical results are corroborated empirically on several image datasets. Our proposed approach systematically produces the highest minimum classification accuracy across domains, and the most invariant representation.

Network densification via deploying various sorts of small base stations could improve spectral utilization, therefore the whole cellular network capacity is increased. Nevertheless, for moving user equipment (UE) (resp. the network), network densification could cause frequent handover (HO), which brings connection delay and disconnection risks (resp. huge energy and resource consumption). In our study, we consider a two-tier heterogeneous cellular network that is modeled by a Poisson Point Process and a Poisson Cluster Process. Inside the network, we consider a random walk-based UE with a time-based HO skipping technique. Based on the system model, we provide expected approximated analytical results for both HO rate and data rate. In addition, we conduct numerical experiments to verify the efficiency of our proposed model.