In this article, a state-dependent piecewise reinforcement learning (PRL) method is proposed to learn an optimal control policy for state-dependent switched systems. This is an analogy to learning multiple piecewise continuous control inputs. We observe this method’s robust and impressive performance, explain it in rigorous mathematical language, and apply it to some canonical examples. Based on the deterministic policy gradient method, the PRL is compared with the vanilla policy gradient method in three customized demonstrative environments with switched dynamics. We theoretically demystify why PRL outperforms RL in these systems.

The overall student enrollment has declined over the past eight years in US. The administrators in enrollment management are applying ever more sophisticated analytical techniques in efforts to determine the characteristics of students that are most probable to enroll upon offering them monetary incentives (i.e., financial aid and/or scholarship). In this paper we present a framework that reveals the characteristics of such students using different machine learning models. Particularly, we use a genetic algorithm optimization method along with three different classification models: logistic regression (LR), support vector machines (SVMs) and bayesian networks (BNs). The results show that students with particular gender, ethnicity, socioeconomic and academic backgrounds are most probable to enroll upon offering them institutional money. We validated our results using data of actual students from a large public Tier 1 research university.

The curricular structure and the complexity of the prerequisite dependencies in a curriculum are essential factors that impact student progression, and ultimately graduation rates. However, we are not aware of any closed-form methods for quantifying the relationship between the complexity of a curriculum and the graduation rate of those attempting to complete the curriculum. This paper introduces a new method that quantifies this relationship using Markov Decision Processes (MDP). The non-deterministic nature of student progress along with their evolving states at each semester make MDP a suitable framework for this work. We propose a novel model that is useful due to the fact that it provides a closed-form solution approach that can be utilized to perform “what-if” analyses around student progress through a curriculum. The results confirm the inverse relationship between the complexity of a curriculum and the graduation rate of those students attempting to complete it. This is validated using a Monte Carlo simulation method. The results also provide useful insights that may guide future work in this area.

In this article, we propose a joint access selection and bandwidth allocation method for UAV-assisted wireless communication networks. We consider an application scenario where UAVs act as flying base stations to provide wireless services to ground mobile devices. The interactions among UAVs and mobile devices are modeled as a Stackelberg game, where the UAVs are the leaders and the mobile devices are the followers. The bandwidth allocation among UAVs is modeled as a noncooperative game, and the access selection of mobile devices is based on the evolutionary game theory. We explore the distributed evolutionary game and the synchronous update and asynchronous update's influence on the convergence results. In addition, a Q-learning approach for access selection is also used as a benchmark. We provide theoretical analysis to show the existence and uniqueness of equilibrium under this hierarchical framework. Finally, simulation results verify the performance of the proposed model.

In this paper, we develop a decentralized probabilistic method for performance optimization of cloud services. We focus on Infrastructure-as-a-Service where the user is provided with the ability of configuring virtual resources on demand in order to satisfy specific computational requirements. This novel approach is strongly supported by a theoretical framework based on tail probabilities and sample complexity analysis. It allows not only the inclusion of performance metrics for the cloud but the incorporation of security metrics based on cryptographic algorithms for data storage. To the best of the authors' knowledge this is the first unified approach to provision performance and security on demand subject to the Service Level Agreement between the client and the cloud service provider. The quality of the service is guaranteed given certain values of accuracy and confidence. We present some experimental results using the Amazon Web Services, Amazon Elastic Compute Cloud service to validate our probabilistic optimization method.

Many universities are working to improve their graduation rates. The factors that correlate to student success and hence graduation rates are many, varying from pre-institutional factors, including high school GPA and admissions scores, to institutional factors, including student support services and the quality of faculty. An essential institutional factor that is often overlooked is the structure of the curriculum. In this paper we consider the degree to which the underlying curriculum that a student must traverse in order to earn a degree impacts progress. Using data mining methods, complex network analysis and graph theory, this paper proposes a framework for analyzing university course enrollment networks at the program level. The analyses we provide are based on quantifying the importance of course enrollment sequences on a student's final GPA, a metric that is highly correlated to graduation rates. In particular, we investigate the orderings of courses enrollment sequences that best contribute to student performance and achievement. Experimental results, using data from the University of New Mexico, show that Electrical Engineering students who graduated with "high GPA" values tend to follow a common course enrollment sequence that is quite different than that of students who graduated with relatively "low GPA" values. This work may be useful to both students and decision makers at universities as it presents a robust framework for analyzing the ease of flow of students through curricula, which may lead to improvements that facilitate improved student success.

Security and resource optimization are two of the most critical concerns in cloud computing. A cloud provider must ensure customers with appropriate security, while optimizing the use of cloud resources. In this paper, we present a framework which optimizes both the use of cloud resources and security provided to the customers in an infrastructure as a service (IaaS) cloud. Our framework offers secure usage control of sensitive data within secure virtual machines (VMs), which are dynamically instantiated while optimizing both security and resources allocated to the VMs. These resources are then allocated to the VMs using an optimization model based upon randomized algorithms. We demonstrate that both security and resources can be efficiently optimized within a cloud setting using our formal mathematical model and usage management framework.

Colleges and universities are increasingly interested in tracking student progress as they monitor and work to improve their retention and graduation rates. Ideally, early indicators of student progress, or lack thereof, can be used to provide appropriate interventions that increase the likelihood of student success. In this paper we present a framework that uses machine learning, and in particular, a Bayesian Belief Network (BBN), to predict the performance of students early in their academic careers. The results obtained show that the proposed framework can predict student progress, specifically student grade point average (GPA) within the intended major, with minimal error after observing a single semester of performance. Furthermore, as additional performance is observed, the predicted GPA in subsequent semesters becomes increasingly accurate, providing the ability to advise students regarding likely success outcomes early in their academic careers.

Many universities have recently focused significant efforts on enhancing their graduation rates. Numerous factors may impact a student's ability to succeed and ultimately graduate, including pre-university preparation, as well as the student support services provided by a university. However, even the best efforts to improve in these areas may fail if other institutional factors overwhelm their ability to facilitate student progress. Specifically, in this paper we consider degree to which the underlying curriculum that a student must traverse in order to earn a degree impacts progress. Using complex network analysis and graph theory, this paper proposes a framework for analyzing university course networks at the university, college and departmental levels. The analyses we provide are based on quantifying the importance of a course based on its delay and blocking factors, as well as the number of curricula that incorporate the course, leading to a metric we refer to as the course cruciality. Experimental results, using data from the University of New Mexico, show that the distribution of course cruciality follows a power law distribution. Applications of the proposed framework are extended to study the complexity of curricula within colleges as well as the tendency of a university's disciplines to associate with others that are unlike them. This work may be useful to both students and decision makers at universities as it presents a robust framework for analyzing the ease of flow of students through curricula, which may lead to improvements that facilitate improved student success.

Colleges and universities are increasingly interested in tracking student progress as they monitor and work to improve their retention and graduation rates. Ideally, early indicators of student progress, or lack thereof, can be used to provide appropriate interventions that increase the likelihood of student success. In this paper we present a framework that uses data mining and machine learning techniques, and in particular, linear regression and a Markov network (MN), to predict the performance of students early in their academic careers. The results obtained show that the proposed framework can predict student progress, specifically student grade point average (GPA) within the intended major, with minimal error after observing a single semester of performance. Furthermore, as additional performance is observed, the predicted GPA in subsequent semesters becomes increasingly accurate, providing the ability to advise students regarding likely success outcomes early in their academic careers.

Our goal in this paper is to study the performance of the game-theoretic power control algorithms for wireless data introduced by Saraydar et al [1] in two realistic channels: (a1) fast flat fading channel and (a2) Slow flat fading channel. The fading coefficients under both (a1) and (a2) are studied under an appropriate small scale channel model that is used in the CDMA cellular systems, namely Nakagami channel model. To do so, we derive a closed- form expression of the average utility function which represents the number of bits received correctly at the receiver per one Joule expended. Then, using this expression we study the existence, uniqueness of Nash equilibrium (NE), and the social desirability of NE in the Pareto sense.

Supporting mobility in IP networks is a crucial step towards satisfying the nomadic communication paradigms on the current Internet. The Session Initiation Protocol (SIP) presents one approach towards supporting IP mobility and is increasingly gaining in popularity as the next generation multimedia signaling and session establishment protocol. In this paper, we explore the design of an efficient approach to inter-domain SIP mobility in an attempt to improve personal and terminal mobility schemes. We apply a persistent identification framework to application level SIP addressing by introducing a level of indirection on top of the traditional SIP architecture. We show how this approach helps achieve efficient inter-domain authentication and call routing towards providing inter-domain mobility. This paper presents the design of H-SIP, while its implementation is described in a companion paper.

We present a generalization of Benford’s law for the first significant digit. This generalization is based on keeping two terms of the Fourier expansion of the probability density function of the data in the modular logarithmic domain. We prove that images in the Discrete Cosine Transform domain closely follow this generalization. We use this property to propose an application in image steganalysis, namely, detecting that a given image carries a hidden message.

Over the past decade, multimedia services have gained significant acceptance and played an important role in the convergence of IP networks. The proliferation of mobile devices and the nomadic user and computing lifestyles on current networks makemobilitysupporta crucial ingredient of current IP-based multimedia systems. The Session Initiation Protocol (SIP) presents one approach towards supporting IP mobility. Additionally, SIP is increasingly gaining in popularity as the next generation multimedia signaling and session establishment protocol, and the SIP infrastructure is anticipated to be extensively deployed all over the Internet. We have lately proposed an approach to inter-domain SIP mobility which we call H-SIP. H-SIP is a user-controlled mobility scheme that improves personal and terminal mobility. H-SIP uses persistent identifiers and leverages the traditional SIP architecture to abstract any domain binding from users. This paper expands on our previous work and experimentally proves the efficiency of H-SIP in achieving inter-domain authentication and call routing through modeling and real-time measurements.

In many distributed computing systems that are prone to either induced or spontaneous node failures, the number of available computing resources is dynamically changing in a random fashion. A load-balancing (LB) policy for such systems should therefore be robust, in terms of workload re-allocation and effectiveness in task completion, with respect to the random absence and re-emergence of nodes as well as random delays in the transfer of workloads among nodes. In this paper two LB policies for such computing environments are presented: The first policy takes an initial LB action to preemptively counteract the consequences of random failure and recovery of nodes. The second policy compensates for the occurrence of node failure dynamically by transferring loads only at the actual failure instants. A probabilistic model, based on the concept of regenerative processes, is presented to assess the overall performance of the system under these policies. Optimal performance of both policies is evaluated using analytical, experimental and simulation-based results. The interplay between node-failure/recovery rates and the mean load-transfer delay are highlighted

In a previous paper we proposed a bilateral teleoperation framework of a wheeled mobile robot over communication channel with constant time delay. In this paper we present experimental results. Our goal is to illustrate and validate the properties of the proposed scheme as well as to present practical implementation issues and the adopted solutions. In particular, the bilaterally teleoperated system is passive and the system is stable in the presence of time delay. Internet has been used as the communication channel and a buffer has been implemented to maintain a constant time delay and to handle packet order

In this paper we consider a scheme of packet scheduling for the forward link of a MIMO cellular system, to increase system capacity by exploiting multiuser diversity based on spatial multiplexing (SM). The proposed scheme exhibits considerable increases in system capacity compared to commonly used schemes, while, at the same time, giving a fair chance of channel access to all users regardless of their channel fading status

The proliferation of computational resources and data networks has enabled the conception and solution of information management problems of an unprecedented scale. These technologies expose vulnerabilities that may be exploited through happenstance or design. The engineering problems encountered in the design of networked data systems and software are familiar in the controls community: resource allocation and management, distributed decision-making, time-delay systems, discrete-event and hybrid systems analysis and design, detection and identification, stability, robustness and reliability, and large-scale systems.

A complete analytical solution to one-shot load balancing in a two-node distributed system is presented and verified experimentally. The model takes into account uncertainty in network delays and the variability in the processors speeds. The system consisting of two Transmeta processors connected through a wireless LAN, utilizing an 802.11b access point, is used as the testbed. The statistics of the network delays are estimated experimentally and used in the analytical model to show the effect of the balancing instant and gain on the overall completion times of various workloads. The analytical results are verified by comparing them to actual experiments and Monte-Carlo simulation. The effect of the communication delay on the choice of the gain and the balancing instant is clearly observed. Finally, the model is used to determine the optimal gain and balancing instant for different initial load distributions.

The paper presents a multiclass, multilabel implementation of least squares support vector machines (LS-SVM) for direction of arrival (DOA) estimation in a CDMA system. For any estimation or classification system, the algorithm's capabilities and performance must be evaluated. Specifically, for classification algorithms, a high confidence level must exist along with a technique to tag misclassifications automatically. The presented learning algorithm includes error control and validation steps for generating statistics on the multiclass evaluation path and the signal subspace dimension. The error statistics provide a confidence level for the classification accuracy.

In this paper we use statistical learning theory to evaluate the performance of game theoretic power control algorithms for wireless data in arbitrary channels, i.e., no presumed channel model is required. To show the validity of statistical learning theory in this context, we studied a flat fading channel, and more specifically, we simulated the case of Rayleigh flat fading channel. With the help of a relatively small number of training samples, the results suggest the learnability of the utility function classes defined by changing the user power (adjusted parameter) for each user's utility function.

Wireless LAN systems based on IEEE 802.11 have been a proved and sought after technological innovation, over the past few years. In this paper, we discuss some of the technical aspects for testing and deploying a wireless LAN system at the University of New Mexico, Albuquerque, New Mexico. This shall serve as a model test bed for future research and development, under the aegis of Ibero American Science and Technology Education Consortium (ISTEC).

This paper presents a comparison study of the computational complexity of the general job shop protocol and the more structured flow line protocol in a flexible manufacturing system. It is shown that the representative problem of finding resource invariants is NP-complete in the case of the job shop, while in the flow line case it admits a closed form solution. The importance of correctly selecting part flow and job routing protocols in flexible manufacturing systems to reduce complexity is thereby conclusively demonstrated.

The authors have developed an interactive environment for the creation and maintenance of dynamic, active multimedia-based teaching mechanisms. The environment is designed to be user-friendly and to facilitate the creation of educational material. This tool has already been used to create courses in multidimensional signal processing (MDSP) by researchers working together while geographically separated.

A set of nonlinear differential equations that describe the dynamics of the ART1 model are presented, along with the motivation for their use. These equations are extensions of those developed by Carpenter and Grossberg (1987). It is shown how these differential equations allow the ART1 model to be realized as a collective nonlinear dynamical system. Specifically, we present an ART1-based neural network model whose description requires no external control features. That is, the dynamics of the model are completely determined by the set of coupled differential equations that comprise the model. It is shown analytically how the parameters of this model can be selected so as to guarantee a behavior equivalent to that of ART1 in both fast and slow learning scenarios. Simulations are performed in which the trajectories of node and weight activities are determined using numerical approximation techniques.