The growing body of work on Physics-Informed Neural Networks (PINNs) seeks to use machine learning strategies to improve methods for solution discovery of Partial Differential Equations (PDEs). While classical solvers may remain the preferred tool of choice in the short-term, PINNs can be viewed as complementary. The expectation is that in some specific use cases, they can be effective, standalone. A key step in training PINNs is selecting domain points for loss evaluation, where Monte Carlo sampling remains the dominant but often suboptimal in low dimension settings, common in physics. We leverage recent advances in asymptotic expansions of quadrature nodes and weights (for weight functions belonging to the modified Gauss-Jacobi family) together with suitable adjustments for parameterization towards a data-driven framework for learnable quadrature rules. A direct benefit is a performance improvement of PINNs, relative to existing alternatives, on a wide range of problems studied in the literature. Beyond finding a standard solution for an instance of a single PDE, our construction enables learning rules to predict solutions for a given family of PDEs via hyper-networks, a useful capability for PINNs.

The idea of representing a signal as the weights of a neural network, called *Implicit Neural Representations* (INRs), has led to exciting implications for compression, view synthesis and 3D volumetric data understanding. One problem in this setting pertains to the use of INRs for downstream processing tasks. Despite some conceptual results, this remains challenging because the INR for a given image/signal often exists in isolation. What does the neighborhood around a given INR correspond to? Based on this question, we offer an operator theoretic reformulation of the INR model, which we call Operator INR (or O-INR). At a high level, instead of mapping positional encodings to a signal, O-INR maps one function space to another function space. A practical form of this general casting is obtained by appealing to Integral Transforms. The resultant model does not need multi-layer perceptrons (MLPs), used in most existing INR models -- we show that convolutions are sufficient and offer benefits including numerically stable behavior. We show that O-INR can easily handle most problem settings in the literature, and offers a similar performance profile as baselines. These benefits come with minimal, if any, compromise. Our code is available at https://github.com/vsingh-group/oinr.

Neural Controlled Differential equations (NCDE) are a powerful mechanism to model the dynamics in temporal sequences, e.g., applications involving physiological measures, where apart from the initial condition, the dynamics also depend on subsequent measures or even a different "control" sequence. But NCDEs do not scale well to longer sequences. Existing strategies adapt rough path theory, and instead model the dynamics over summaries known as log signatures. While rigorous and elegant, invertibility of these summaries is difficult, and limits the scope of problems where these ideas can offer strong benefits (reconstruction, generative modeling). For tasks where it is sensible to assume that the (long) sequences in the training data are a fixed length of temporal measurements – this assumption holds in most experiments tackled in the literature – we describe an efficient simplification. First, we recast the regression/classification task as an integral transform. We then show how restricting the class of operators (permissible in the integral transform), allows the use of a known algorithm that leverages non-standard Wavelets to decompose the operator. Thereby, our task (learning the operator) radically simplifies. A neural variant of this idea yields consistent improvements across a wide gamut of use cases tackled in existing works. We also describe a novel application on modeling tasks involving coupled differential equations.

Recent legislation has led to interest in machine unlearning, i. e., removing specific training samples from a predictive model as if they never existed in the training dataset. Unlearning may also be required due to corrupted/adversarial data or simply a user's updated privacy requirement. For models which require no training (k-NN), simply deleting the closest original sample can be effective. But this idea is inapplicable to models which learn richer representations. Recent ideas leveraging optimization-based updates scale poorly with the model dimension d, due to inverting the Hessian of the loss function. We use a variant of a new conditional independence coefficient, L-CODEC, to identify a subset of the model parameters with the most semantic overlap on an individual sample level. Our approach completely avoids the need to invert a (possibly) huge matrix. By utilizing a Markov blanket selection, we premise that L-CODEC is also suitable for deep unlearning, as well as other applications in vision. Compared to alternatives, L-CODEC makes approximate unlearning possible in settings that would otherwise be infeasible, including vision models used for face recognition, person reidentification and NLP models that may require unlearning samples identified for exclusion. Code is available at https://github.com/vsingh-group/LCODEC-deep-unlearning

Transformers have emerged as a preferred model for many tasks in natural langugage processing and vision. Recent efforts on training and deploying Transformers more efficiently have identified many strategies to approximate the self-attention matrix, a key module in a Transformer architecture. Effective ideas include various prespecified sparsity patterns, low-rank basis expansions and combinations thereof. In this paper, we revisit classical Multiresolution Analysis (MRA) concepts such as Wavelets, whose potential value in this setting remains underexplored thus far. We show that simple approximations based on empirical feedback and design choices informed by modern hardware and implementation challenges, eventually yield a MRA-based approach for self-attention with an excellent performance profile across most criteria of interest. We undertake an extensive set of experiments and demonstrate that this multi-resolution scheme outperforms most efficient self-attention proposals and is favorable for both short and long sequences. Code is available at \url{https://github.com/mlpen/mra-attention}.

The goal of our work is to use visual attention to enhance autonomous driving performance. We present two methods of predicting visual attention maps. The first method is a supervised learning approach in which we collect eye-gaze data for the task of driving and use this to train a model for predicting the attention map. The second method is a novel unsupervised approach where we train a model to learn to predict attention as it learns to drive a car. Finally, we present a comparative study of our results and show that the supervised approach for predicting attention when incorporated performs better than other approaches.

We introduce models for saliency prediction for mobile user interfaces. A mobile interface may include elements like buttons and text in addition to natural images which enable performing a variety of tasks. Saliency in natural images is a well studied topic. However, given the difference in what constitutes a mobile interface, and the usage context of these devices, we postulate that saliency prediction for mobile interface images requires a fresh approach. Mobile interface design involves operating on elements, the building blocks of the interface. We first collected eye-gaze data from mobile devices for a free viewing task. Using this data, we develop a novel autoencoder based multi-scale deep learning model that provides saliency prediction at the mobile interface element level. Compared to saliency prediction approaches developed for natural images, we show that our approach performs significantly better on a range of established metrics.

In standard IEEE 802.11 based systems, when the wireless client migrates away from the radio range of the currently associated access point (AP), network applications temporarily loose connectivity till the client is able to re-associate itself with a new AP. The delay that occurs during the break-off interval can vary from a few hundreds of microseconds to a few seconds. However, delay sensitive applications such as Voice over IP (VoIP) or streaming multimedia applications usually are unable to tolerate such long connectivity delays that fall beyond the range of 50 - 200 ms. This results in dropped calls or frozen video frames. In this paper we describe the design, implementation, and evaluation of a software based framework that facilitates seamless and transparent handoff between different APs in standard IEEE 802.11 based wireless local area networks (WLANs). Although different solutions are available in the literature that seek to address the handoff latency, most of them propose changes that are outside the purview of the current 802.11 standards. We have specifically kept such compatibility restrictions in mind and have devised a software based client side solution that is capable of reducing handoff delays to an average value of 20 ms. It is available as a driver update to the client and requires no additional support from the network. As part of our solution, we have successfully implemented and tested our proposed solution framework on Atheros AR5212 chipsets using the open source MadWifi driver.

In networks carrying large volume of traffic, accurate traffic characterization is necessary for understanding the dynamics and patterns of network resource usage. Previous approaches to flow characterization are based on random sampling of the packets (e.g., Cisco's NetFlow) or inferring characteristics solely based on long lived flows (LLFs) or on lossy data structures (e.g., bloom filters, hash tables). However, none of these approaches takes into account the heavy-tailed nature of the Internet traffic and separates the estimation algorithm from the flow measurement architecture.In this paper, we propose an alternate approach to traffic characterization by closely linking the flow measurement architecture with the estimation algorithm. Our measurement framework stores complete information related to short lived flows (SLFs) while collecting partial information related to LLFs. For real-time separation of LLFs and SLFs, we propose a novel algorithm based on typical sequences from information theory. The distribution (pdf) and sample space of the underlying traffic is estimated using the non-parametric Parzen window technique and likelihood function defined over the Coupon collector problem. We validate the accuracy and performance of our estimation technique using traffic traces from the internal LAN in our laboratory and from National Library for Applied Network Research (NLANR).

Emerging multi-rate wireless systems both cellular (cdma2000, high data rate (HDR)) and Wi-Fi like systems demand that the currently used channel estimation and scheduling techniques be revisited. Not only the channel estimation, prediction algorithms, and protocols need to be modified, the premise for evaluating their performance needs to be changed. Current techniques comprise of scheduling algorithms based on existing channel state. We recommend that for enhanced performance in terms of user satisfaction and system throughput not only channel state but also the demand rate needed for satisfying the user needs to be considered. In this paper, we devise new mechanisms to estimate the varying channel conditions using information theoretic techniques for multi-rate wireless systems. We utilized a non-parametric estimator of Renyi's entropy using the Parzen widowing technique to estimate the probability density function of the channel rate variation as experienced by every user in the system. Scheduling algorithms are proposed based on the channel conditions estimated by the proposed technique as well as the data rate needed for satisfying each user . The proposed mechanism ensures the highest number of satisfied users by maintaining the stipulated QoS requirements and fairness amonst users. We also demonstrate that maximizing the throughput does not necessarily result in maximizing the number of satisfied users. Results demonstrate that the proposed estimation techniques perform better than existing schemes both in terms of number of satisfied user and throughput.

Opportunistic scheduling algorithms are effective in exploiting channel variations and maximizing system throughput in multi-rate wireless networks. However, most scheduling algorithms ignore the per-user quality of service (QoS) requirements and try to allocate resources (i.e., the time slots) among multiple users. This leads to a phenomenon commonly referred to as the exposure problem wherein the algorithms fail to satisfy the minimum slot requirements of the users due to substitutability and complementarity requirement of user slots. To eliminate this exposure problem, we propose a novel scheduling algorithm based on two phase combinatorial reverse auction with the primary objective to maximize the number of satisfied users in the system. We also consider maximizing the system throughput as a secondary objective. In the proposed scheme, multiple users bid to acquire the required number of time slots, and the allocations are done to satisfy the two objectives in a sequential manner. We provide an approximate solution to the proposed scheduling problem which is a NP-complete problem. We prove that our proposed algorithm is (1 + log m) times the optimal solution, where m is the number of slots in a schedule cycle. We also present an extension to this algorithm which can support more satisfied users at the cost of additional complexity. Numerical results are provided to compare the proposed scheduling algorithms with other competitive schemes.

Opportunistic scheduling algorithms are effective in exploiting channel variations and maximizing system throughput in multi-rate wireless networks. However, most scheduling algorithms ignore the per-user quality of service (QoS) requirements and try to allocate resources (e.g., the time slots) among multiple users. This leads to a phenomenon commonly referred to as the exposure problem wherein the algorithms fail to satisfy the minimum slot requirements of the users due to substitutability and complementarity requirements of user slots. To eliminate this exposure problem, we propose a novel scheduling algorithm based on two-phase combinatorial reverse auction with the primary objective to maximize the number of satisfied users in the system. We also consider maximizing the system throughput as a secondary objective. In the proposed scheme, multiple users bid for the required number of time slots, and the allocations are done to satisfy the two objectives in a sequential manner. We provide an approximate solution to the proposed scheduling problem which is NP-complete. The proposed algorithm has an approximation ratio of (1 + logm) with respect to the optimal solution, where m is the number of slots in a schedule cycle. Simulation results are provided to compare the proposed scheduling algorithm with other competitive schemes.

The success of future generation wireless data services depends on the parameterized provisioning of quality of service (QoS) for applications whose demands and nature are highly heterogeneous. Also, user satisfaction plays a key role in the economic viability of wireless service deployments. In this paper, we present a QoS framework based on the paradigm of traffic class and user satisfaction. The proposed class-based QoS framework comprises a radio resource management scheme which considers user satisfaction based on the perceived QoS, and caters to heterogeneous applications such as voice and multiple types of data services, with diverse QoS requirements. The resource management scheme has two components: the admission control algorithm caters to the long term user satisfaction while the session-based rate and bandwidth allocation scheme manipulates the short term user satisfaction. Performance metrics have been specifically defined for each traffic class - blocking for voice communications, average rate jitter for streaming applications, bounded and negotiated delay for interactive data and background traffic. Extensive simulations using four types of traffic and three classes of users reveal that the proposed framework offer improved QoS without compromising the utilization of the system.

The deployment of 3G cellular networks is ushering in a variety of data services in conjunction with the existing voice services. A major proportion of data services will include Web browsing and file downloads categorized as nonreal time services, for which there exists no service level agreement between the users and the providers on the quality of data services. In this paper, we address how to deal with the subjectiveness of user satisfaction or expectation from service providers by defining user irritation factors with the help of Sigmoid functions. These factors reflect users' sensitivity and tolerance to delay. We propose a scheduling algorithm to provide a bound on the delivery time of the entire data content. This algorithm takes into consideration the dynamic nature of the irritation factors, channel conditions, user classes, and fairness within individual classes. Soft-reservation schemes are also proposed to cater to the higher paying users. Exhaustive simulations are conducted with synthetically generated HTTP traffic and three classes of users. Experimental results demonstrate that the proposed scheme is able to provide bounded delay even when the system utilization is as high as 80%.