Deep learning has excelled in medical image classification, but its clinical application is limited by poor interpretability. Capsule networks, known for encoding hierarchical relationships and spatial features, show potential in addressing this issue. Nevertheless, traditional capsule networks often underperform due to their shallow structures, and deeper variants lack hierarchical architectures, thereby compromising interpretability. This paper introduces a novel capsule network, ParseCaps, which utilizes the sparse axial attention routing and parse convolutional capsule layer to form a parse-tree-like structure, enhancing both depth and interpretability. Firstly, sparse axial attention routing optimizes connections between child and parent capsules, as well as emphasizes the weight distribution across instantiation parameters of parent capsules. Secondly, the parse convolutional capsule layer generates capsule predictions aligning with the parse tree. Finally, based on the loss design that is effective whether concept ground truth exists or not, ParseCaps advances interpretability by associating each dimension of the global capsule with a comprehensible concept, thereby facilitating clinician trust and understanding of the model's classification results. Experimental results on three medical datasets show that ParseCaps not only outperforms other capsule network variants in classification accuracy and robustness, but also provides interpretable explanations, regardless of the availability of concept labels.

Many countries are facing energy shortages today and most of the global energy is consumed by HVAC systems in buildings. For the scenarios where the energy system is not sufficiently supplied to HVAC systems, a priority-based allocation scheme based on distributed model predictive control is proposed in this paper, which distributes the energy rationally based on priority order. According to the scenarios, two distributed allocation strategies, i.e., one-to-one priority strategy and multi-to-one priority strategy, are developed in this paper and validated by simulation in a building containing three zones and a building containing 36 rooms, respectively. Both priority-based strategies fully exploit the potential of predictive control solutions. The experiment shows that our scheme has good scalability and achieves the performance of the centralized strategy while making the calculation tractable. Note to Practitioners—The motivation of this paper is to develop a priority-based allocation strategy adapted to energy-limited systems. When energy is limited, the strategy can rationally allocate energy and satisfy the urgent need for energy supply in some specific zones. Two priority strategies are proposed for the case that a single subsystem corresponds to a particular priority and multiple subsystems correspond to the same priority, respectively. The developed strategies have been validated by co-simulation with MATLAB and EnergyPlus in a small-scale three-zone building and a large-scale 36-zone building to show their effectiveness.

Traffic signal control plays a crucial role in intelligent transportation systems, with cooperative control being challenging to implement but essential for its effectiveness. Many methods model multi-intersection traffic networks as grids and address the problem using multi-agent reinforcement learning (RL). Despite these existing studies, there is an opportunity to further enhance our understanding of the connectivity and globality of the traffic networks by capturing the spatiotemporal traffic information with efficient neural networks in deep RL. In this paper, we propose a novel multi-agent actor-critic framework based on an interpretable influence mechanism with a centralized learning and decentralized execution method. Specifically, we first construct an actor-critic framework, for which the piecewise linear neural network (PWLNN), named biased ReLU (BReLU), is used as the function approximator to obtain a more accurate and theoretically grounded approximation, and exhibits interpretability. Then, to model the relationships among agents in multi-intersection scenarios, we introduce an interpretable influence mechanism based on efficient hinging hyperplanes neural network (EHHNN), which derives weights by analysis of variance (ANOVA) decomposition among agents and extracts spatiotemporal dependencies of the traffic features. Finally, our proposed framework is validated on two synthetic traffic networks and a real road network to coordinate signal control between intersections, achieving lower traffic delays across the entire traffic network compared with benchmark performance.

Redundancy is a persistent challenge in Capsule Networks (CapsNet), leading to high computational costs and parameter counts. Although previous studies have introduced pruning after the initial capsule layer, dynamic routing's fully connected nature and non-orthogonal weight matrices reintroduce redundancy in deeper layers. Besides, dynamic routing requires iterating to converge, further increasing computational demands. In this paper, we propose an Orthogonal Capsule Network (OrthCaps) to reduce redundancy, improve routing performance and decrease parameter counts. Firstly, an efficient pruned capsule layer is introduced to discard redundant capsules. Secondly, dynamic routing is replaced with orthogonal sparse attention routing, eliminating the need for iterations and fully connected structures. Lastly, weight matrices during routing are orthogonalized to sustain low capsule similarity, which is the first approach to use Householder orthogonal decomposition to enforce orthogonality in CapsNet. Our experiments on baseline datasets affirm the efficiency and robustness of OrthCaps in classification tasks, in which ablation studies validate the criticality of each component. OrthCaps-Shallow outperforms other Capsule Network benchmarks on four datasets, utilizing only 110k parameters - a mere 1.25% of a standard Capsule Network's total. To the best of our knowledge,$it$achieves the smallest parameter count among existing Capsule Networks. Similarly, OrthCaps-Deep demonstrates competitive performance across four datasets, utilizing only 1.2% of the parameters required by its counterparts.

Energy shortage is a challenge for many countries, and building energy consumption accounts for a considerable proportion of global energy consumption. The main work of this paper is to optimize the energy consumption of heating, ventilating, and air conditioning (HVAC) systems in buildings based on economic model predictive control (EMPC). The cost in EMPC design includes energy consumption and predicted mean vote (PMV), which is an index that evaluates the thermal comfort of indoor occupants. In order to model the nonlinearity of the PMV index, we propose a lattice piecewise linear (PWL) approximation, which has high approximation precision and facilitates the resulting optimization problem, which is basically a piecewise quadratic programming problem. For the piecewise quadratic programming, we propose a descent algorithm that converges quickly and scales well with the length of the prediction horizon in the EMPC problem. The experimental results demonstrate that the proposed method saves 19.78% of the electricity cost compared to the conventional control strategy and significantly increases indoor comfort.Note to Practitioners— The motivation of this article is to provide a control strategy to reduce building energy consumption and ensure indoor thermal comfort. In most of the existing methods for air conditioning temperature control, the occupants’ comfort hasn’t been considered. In this paper, thermal comfort is described by the PMV index, which is basically nonlinear. In order to model the thermal comfort more accurately, in this paper, the PMV index is approximated piecewise linearly in order to meet the requirements of accuracy and computational efficiency. The resulting optimization problem is not hard to solve, and we provide an efficient algorithm for solving this optimization problem. Preliminary simulation experiments demonstrate that this approach is practical, i.e., it achieves energy reduction and ensures thermal comfort. Our strategy, however, has not yet been deployed in real buildings. In future research, we will propose similar techniques for large-scale systems in order to solve energy optimization problems containing multiple thermal zones and realize the proposed technique in real buildings.

Typical Siamese-based trackers focus on the target region and pay less attention to the background area. However, the background area can provide the tracker with prior knowledge about the target surroundings. Nonetheless, since the tracker can naturally utilize the target template for localization, importing additional background knowledge requires proper design so that the background area prior knowledge can be fully explored. Furthermore, the introduction of the entire background regions is redundant. Instead, the part background distractors in the regions are more meaningful for the discrimination of the tracker. In this work, we propose a background prior knowledge fully explored tracker for robust tracking. Firstly, we present a Transformer-based explicitly and fully background-utilizing scheme by boosting the tracker to independently exploit the background for localization. Specifically, a target-distractor independent decoder explicitly utilizes the background knowledge by making the target and the distractors independently perform fusion with the search feature. Secondly, we design a simple yet efficient discriminative distractors mining module to refine the background prior knowledge by replacing the whole background region with the mined background distractors. Extensive experiments demonstrate that the proposed method performs favorably against state-of-the-art trackers on nine benchmarks.

Cross-modal image-text retrieval is an important area of Vision-and-Language task that models the similarity of image-text pairs by embedding features into a shared space for alignment. To bridge the heterogeneous gap between the two modalities, current approaches achieve inter-modal alignment and intra-modal semantic relationship modeling through complex weighted combinations between items. In the intra-modal association and inter-modal interaction processes, the higher-weight items have a higher contribution to the global semantics. However, the same item always produces different contributions in the two processes, since most traditional approaches only focus on the alignment. This usually results in semantic changes and misalignment. To address this issue, this paper proposes Cross-modal Semantic Importance Consistency (CSIC) which achieves invariance in the semantic of items during aligning. The proposed technique measures the semantic importance of items obtained from intra-modal and inter-modal self-attention and learns a more reasonable representation vector by inter-calibrating the importance distribution to improve performance. We conducted extensive experiments on the Flickr30K and MS COCO datasets. The results show that our approach can significantly improve retrieval performance, proving the proposed approach’s superiority and rationality.

Traffic flow (TF) prediction is an important and yet a challenging task in transportation systems, since the TF involves high nonlinearities and is affected by many elements. Recently, neural networks have attracted much attention for TF prediction, but they are commonly black boxes with complex architectures and difficult to be interpreted, e.g., the contributions of specific traffic elements are not explicit, hardly providing informative guidance. In this paper, we aim at addressing more interpretable short-term TF prediction with joint consideration to high accuracy, and thus introduces a pragmatic method by applying the efficient hinging hyperplanes neural network (EHHNN) simply built upon sparse neuron connections. In the proposed method, different traffic factors are incorporated into the inputs, including their spatial-temporal information. Besides the pursuit of accuracy, we further extend the ANOVA decomposition of EHHNNs to the interpretation analysis with specifications to traffic data, in which the contributions concerning specific traffic variables are detected quantitatively. As such, the proposed method firstly applies the EHHNN to filter out more important traffic variables for dimensionality reduction while maintaining accurate prediction. Then, variable interpretation analysis is performed from different perspectives, e.g. to quantitatively investigate the influence of traffic factors and also their spatial-temporal impacts. Therefore, a predictor and an analyzing tool can both be attained for the TF by exerting the flexibility and extending the interpretability of EHHNNs, which is promising to provide informative guidance to future traffic control. Numerical experiments verify the effectiveness and potential of the proposed method in TF prediction and analysis.

The adaptive hinging hyperplane (AHH) model is a popular piecewise linear representation with a generalized tree structure and has been successfully applied in dynamic system identification. In this article, we aim to construct the deep AHH (DAHH) model to extend and generalize the networking of AHH model for high-dimensional problems. The network structure of DAHH is determined through a forward growth, in which the activity ratio is introduced to select effective neurons and no connecting weights are involved between the layers. Then, all neurons in the DAHH network can be flexibly connected to the output in a skip-layer format, and only the corresponding weights are the parameters to optimize. With such a network framework, the backpropagation algorithm can be implemented in DAHH to efficiently tackle large-scale problems and the gradient vanishing problem is not encountered in the training of DAHH. In fact, the optimization problem of DAHH can maintain convexity with convex loss in the output layer, which brings natural advantages in optimization. Different from the existing neural networks, DAHH is easier to interpret, where neurons are connected sparsely and analysis of variance (ANOVA) decomposition can be applied, facilitating to revealing the interactions between variables. A theoretical analysis toward universal approximation ability and explicit domain partitions are also derived. Numerical experiments verify the effectiveness of the proposed DAHH.

In this paper, we present an approach to nonlinear system approximation, called the lattice trajectory piecewise linear (LTPWL) model. The approach involves determining a lattice piecewise linear (PWL) approximation to the state trajectory of a nonlinear system. It has been shown in the literature that the lattice PWL expression can represent any PWL function in any dimension. After the LTPWL approximation has been obtained, the order of each model piece is reduced using a Krylov projection technique. Compared to existing trajectory piecewise linear (TPWL) models, which are quasi-PWL in the whole region, LTPWL models are virtually linear in each subregion. Besides, the single output LTPWL model can be seen as a special kind of TPWL model, in which only one weight is 1, and the other weights are 0. In general, for multiple output LTPWL model, the weights set to be 1 for each component are different, which makes the LTPWL model more flexible in approximation of nonlinear function. The proposed strategy is applied to simulate diode circuits, and the experimental results show that the performance of the LTPWL model is better than that of the traditional TPWL model in terms of approximation accuracy and generalization ability.

This paper describes an optimization strategy based on the model of adaptive hinging hyperplanes (AHH) and genetic algorithm (GA). The sample points of physical model are approximated by the AHH model, and the resulting model is minimized using a modified GA. In the modified GA, each chromosome corresponds to a local optimum. A criterion based on γ-valid cut is used to judge whether the global optimum is reached. Simulation results show that if the parameters are carefully chosen, the global optimum of AHH minimization is close to the optimum of the original function.

Centrifugal chiller plants (CCP) are widely used in air conditioning systems, its operation optimization can save lots of energy and has great significance in environmental protection. The optimization is a large-scale nonlinear problem and there is no practical algorithm until now. This paper proposes a new method to do this operation optimization using continuous piecewise linear programming (CPWLP). The main idea is transforming the nonlinear problem into a series of linear programmings by approximating the original system using piecewise linear representation. For CPWLP, some properties are discussed and an algorithm is given. Using CPWLP, CCP system is optimized and its energy performance is improved significantly.