This paper introduces a novel reference-free (RF) audio quality metric called the RF-Generative Machine Listener (RF-GML), designed to evaluate coded mono, stereo, and binaural audio at a 48 kHz sample rate. RF-GML leverages transfer learning from a state-of-the-art full-reference (FR) Generative Machine Listener (GML) with minimal architectural modifications. The term "generative" refers to the model’s ability to generate an arbitrary number of simulated listening scores. Unlike existing RF models, RF-GML accurately predicts subjective quality scores across diverse content types and codecs. Extensive evaluations demonstrate its superiority in rating unencoded audio and distinguishing different levels of coding artifacts. RF-GML’s performance and versatility make it a valuable tool for coded audio quality assessment and monitoring in various applications, all without the need for a reference signal.

Prior art investigating task-oriented dialog and automatic generation of such dialogs have focused on single-user dialogs between a single user and an agent.However, there is limited study on adapting such AI agents to multiuser conversations (involving multiple users and an agent).Multi-user conversations are richer than single-user conversations containing social banter and collaborative decision making.The most significant challenge impeding such studies is the lack of suitable multiuser task-oriented dialogs with annotations of user belief states and system actions.One potential solution is multi-user dialog generation from single-user data.Many single-user dialogs datasets already contain dialog state information (intents, slots), thus making them suitable candidates.In this work, we propose a novel approach for expanding single-user taskoriented dialogs (e.g.MultiWOZ) to multiuser dialogs in a zero-shot setting.

Goal-oriented dialogue agents can comfortably utilize the conversational context and understand its users' goals. However, in visually driven user experiences, these conversational agents are also required to make sense of the screen context in order to provide a proper interactive experience. In this paper, we propose a novel multimodal conversational framework where the dialogue agent's next action and their arguments are derived jointly conditioned both on the conversational and the visual context. We demonstrate the proposed approach via a prototypical furniture shopping experience for a multimodal virtual assistant.

Learning from multimodal data has become a popular research topic in recent years. Multimodal coreference resolution (MCR) is an important task in this area. MCR involves resolving the references across different modalities, e.g., text and images, which is a crucial capability for building next-generation conversational agents. MCR is challenging as it requires encoding information from different modalities and modeling associations between them. Although significant progress has been made for visual-linguistic tasks such as visual grounding, most of the current works involve single turn utterances and focus on simple coreference resolutions. In this work, we propose an MCR model that resolves coreferences made in multi-turn dialogues with scene images. We present GRAVL-BERT, a unified MCR framework which combines visual relationships between objects, background scenes, dialogue, and metadata by integrating Graph Neural Networks with VL-BERT. We present results on the SIMMC 2.0 multimodal conversational dataset, achieving the rank-1 on the DSTC-10 SIMMC 2.0 MCR challenge with F1 score 0.783. Our code is available at https://github.com/alexa/gravl-bert.

Next-Gen Intel Xeon Scalable Processor New Standard for Data Center Architecture Designed for Microservices & AI Workloads Pioneering Advanced Memory & IO Transitions

Audio codecs are typically transform-domain based and efficiently code stationary audio signals, but they struggle with speech and signals containing dense transient events such as applause. Specifically, with these two classes of signals as examples, we demonstrate a technique for restoring audio from coding noise based on generative adversarial networks (GAN). A primary advantage of the proposed GAN-based coded audio enhancer is that the method operates end-to-end directly on decoded audio samples, eliminating the need to design any manually-crafted frontend. Furthermore, the enhancement approach described in this paper can improve the sound quality of low-bit rate coded audio without any modifications to the existent standard-compliant encoders. Subjective tests illustrate that the proposed enhancer improves the quality of speech and difficult to code applause excerpts significantly.

E-commerce companies like Amazon, Alibaba and Flip-kart have an extensive catalogue comprising of billions of products. Matching customer search queries to plausible products is challenging due to the size and diversity of the catalogue. These challenges are compounded in apparel due to the semantic complexity and a large variation of fashion styles, product attributes and colours. Providing aids that can help the customer visualise the styles and colours matching their "search queries" will provide customers with necessary intuition about what can be done next. This helps the customer buy a product with the styles, embellishments and colours of their liking. In this work, we propose a Generative Adversarial Network (GAN) for generating images from text streams like customer search queries. Our GAN learns to incrementally generate possible images complementing the fine-grained style, colour of the apparel in the query. We incorporate a novel colour modelling approach enabling the GAN to render a wide spectrum of colours accurately. We compile a dataset from an e-commerce website to train our model. The proposed approach outperforms the baselines on qualitative and quantitative evaluations.

Classical parametric speech coding techniques provide a compact representation for speech signals. This affords a very low transmission rate but with a reduced perceptual quality of the reconstructed signals. Recently, autoregressive deep generative models such as WaveNet and SampleRNN have been used as speech vocoders to scale up the perceptual quality of the reconstructed signals without increasing the coding rate. However, such models suffer from a very slow signal generation mechanism due to their sample-by-sample modelling approach. In this work, we introduce a new methodology for neural speech vocoding based on generative adversarial networks (GANs). A fake speech signal is generated from a very compressed representation of the glottal excitation using conditional GANs as a deep generative model. This fake speech is then refined using the LPC parameters of the original speech signal to obtain a natural reconstruction. The reconstructed speech waveforms based on this approach show a higher perceptual quality than the classical vocoder counterparts according to subjective and objective evaluation scores for a dataset of 30 male and female speakers. Moreover, the usage of GANs enables to generate signals in one-shot compared to autoregressive generative models. This makes GANs promising for exploration to implement high-quality neural vocoders.

Instructional videos are one of the most popular ways of teaching and learning in an online setting. However, navigation in videos is linear as compared to other instructional resources such as textbooks, where a table of topics and a multi-faceted index of different anchor points i.e., list of figures, tables aid in efficiently navigating to a desired point of interest. There is a lack of appropriate techniques and interfaces which can support such textbook-style navigation in instructional videos. This paper presents a novel approach to automatically localize and classify different anchor points in a video including figures, tables, equations, flowcharts, code snippets and charts. Our approach uses a deep convolution neural network in a semi-supervised fashion where the training data is obtained from the unconstrained Internet images. On an anchor point dataset of about 10K images, the proposed algorithm leads to a classification accuracy of 86%. Further, we designed a system ViZig that uses these localized anchor points along with a automatically generated list of topics for non-linear video navigation and studied its effectiveness in real-world. Our user studies with 18 participants establish that the proposed video navigation mechanism provides statistically significant time savings as compared to the popularly used time-synched transcript along with youtube-style timeline scrubbing.

The rate of particle induced soft errors in a processor increases in proportion to the number of bits. This soft error rate (SER) can limit the performance of a system by placing an effective limit on the number of cores, nodes or clusters. The vulnerability of bits in a processor to soft errors can be represented by their architectural vulnerability factor (AVF), defined as the probability that a bit corruption results in a user-visible error. Analytical models such as architecturally correct execution (ACE) lifetime analysis enable AVF estimation at high speed by operating at a level of abstraction well above that of RTL. However, sequential elements do not lend themselves to this type of analysis because these bits are not typically included in the abstracted ACE model. Brute force methods, such as statistical fault injection (SFI), enable register level detail but at the expense of computation speed. We have developed a novel approach that marries the computational speed of the analytical approach with the level of detail of the brute force approach. Our methodology introduces the concept of "port AVFs" computed by ACE analysis on a performance model and applies these values to a node graph extracted from RTL. We employ rules derived from set theory that let us propagate these port AVFs throughout the node graph using an iterative relaxation technique. This enables us to generate statistically significant AVFs for all sequential nodes in a given processor design in a fast and accurate manner. We use this approach to compute the sequential AVF for all nodes in a modern microprocessor and show good correlation with beam test measurements on silicon.

In this paper we propose a multimodal method called MMToC for automatically creating a table of content for educational videos. MMToC defines and quantifies word saliency for visual words extracted from the slides and spoken words obtained from the speech transcript. The saliency scores from these two modalities are combined to obtain a ranked list of salient words. These ranked words along with their saliency scores are used to formulate a topic segmentation cost function. The cost function is optimized using a dynamic program framework to obtain the topic segments of the video. These segments are labelled with their corresponding topic names for creating the table of content. We perform experiments on 24 hours of lectures spread across 23 videos ranging over 20-75 minutes duration each. We compare the proposed method with LDA-based video segmentation approaches and show that the proposed MMToC method is significantly better (F-score improvement of 0.19 and 0.24 on two datasets). We also perform a user study to demonstrate the effectiveness of MMToC for navigating educational videos.

Active learning provides useful tools to reduce annotation costs without compromising classifier performance. However it traditionally views the supervisor simply as a labeling machine. Recently a new interactive learning paradigm was introduced that allows the supervisor to additionally convey useful domain knowledge using attributes. The learner first conveys its belief about an actively chosen image e.g. "I think this is a forest, what do you think?". If the learner is wrong, the supervisor provides an explanation e.g. "No, this is too open to be a forest". With access to a pre-trained set of relative attribute predictors, the learner fetches all unlabeled images more open than the query image, and uses them as negative examples of forests to update its classifier. This rich human-machine communication leads to better classification performance. In this work, we propose three improvements over this set-up. First, we incorporate a weighting scheme that instead of making a hard decision reasons about the likelihood of an image being a negative example. Second, we do away with pre-trained attributes and instead learn the attribute models on the fly, alleviating overhead and restrictions of a pre-determined attribute vocabulary. Finally, we propose an active learning framework that accounts for not just the label-but also the attributes-based feedback while selecting the next query image. We demonstrate significant improvement in classification accuracy on faces and shoes. We also collect and make available the largest relative attributes dataset containing 29 attributes of faces from 60 categories.

We propose a method of clustering images that combines algorithmic and human input. An algorithm provides us with pairwise image similarities. We then actively obtain selected, more accurate pairwise similarities from humans. A novel method is developed to choose the most useful pairs to show a person, obtaining constraints that improve clustering. In a clustering assignment elements in each data pair are either in the same cluster or in different clusters. We simulate inverting these pairwise relations and see how that affects the overall clustering. We choose a pair that maximizes the expected change in the clustering. The proposed algorithm has high time complexity, so we also propose a version of this algorithm that is much faster and exactly replicates our original algorithm. We further improve run-time by adding heuristics, and show that these do not significantly impact the effectiveness of our method. We have run experiments in two different domains, namely leaf images and face images, and show that clustering performance can be improved significantly.

The huge investment in the design and production of multicore processors may be put at risk because the emerging highly miniaturized but unreliable fabrication technologies will impose significant barriers to the life-long reliable operation of future chips. Extremely complex, massively parallel, multi-core processor chips fabricated in these technologies will become more vulnerable to: (a) environmental disturbances that produce transient (or soft) errors, (b) latent manufacturing defects as well as aging/wearout phenomena that produce permanent (or hard) errors, and (c) verification inefficiencies that allow important design bugs to escape in the system. In an effort to cope with these reliability threats, several research teams have recently proposed multicore processor architectures that provide low-cost dependability guarantees against hardware errors and design bugs. This paper focuses on dependable multicore processor architectures that integrate solutions for online error detection, diagnosis, recovery, and repair during field operation. It discusses taxonomy of representative approaches and presents a qualitative comparison based on: hardware cost, performance overhead, types of faults detected, and detection latency. It also describes in more detail three recently proposed effective architectural approaches: a software-anomaly detection technique (SWAT), a dynamic verification technique (Argus), and a core salvaging methodology.

We have discovered that processors can experience a super-linear increase in detected unrecoverable errors (DUE) when the write-back L2 cache is doubled in size. This paper explains how an increase in the cache tag's Architectural Vulnerability Factor or AVF caused such a super-linear increase in the DUE rate. AVF expresses the fraction of faults that become user-visible errors. Our hypothesis is that this increase in AVF is caused by a super-linear increase in ¿dirty¿ data residence times in the L2 cache. Using proton beam irradiation, we measured the DUE rates from the write-back cache tags and analyzed the data to show that our hypothesis holds. We utilized a combination of simulation and measurements to help develop and prove this hypothesis. Our investigation reveals two methods by which dirty line residency causes super-linear increases in the L2 cache tag's AVF. One is a reduction in the miss rates as we move to the larger cache part, resulting in fewer evictions of data required for architecturally correct execution. The second is the occurrence of strided cache access patterns, which cause a significant increase in the ¿dirty¿ residency times of cache lines without increasing the cache miss rate.

Very recently bacterial foraging has emerged as a powerful technique for solving optimization problems. In this paper, we introduce a micro-bacterial foraging optimization algorithm, which evolves with a very small population compared to its classical version. In this modified bacterial foraging algorithm, the best bacterium is kept unaltered, whereas the other population members are reinitialized. This new small population mu-BFOA is tested over a number of numerical benchmark problems for high dimensions and we find this to outperform the normal bacterial foraging with a larger population as well as with a smaller population.

Microprocessor power has become a first-order constraint at run-time. Designers must employ aggressive power-management techniques at run-time to keep a processor's ballooning power requirements under control. Effective power management benefits from knowledge of run-time microprocessor power consumption in both the core and individual microarchitectural structures, such as caches, queues, and execution units. Increasingly feasible per-structure power-control techniques, such as fine-grain clock gating, power gating, and dynamic voltage/frequency scaling (DVFS), become more effective from run-time estimates of per-structure power. However, run-time computation of per-structure power estimates based on utilization requires daunting numbers of input statistics, which makes per-structure monitoring of run-time power a challenging problem. To address the challenges of estimating per-structure power in hardware, we propose a new technique, called Common Activity-based Model for Power (CAMP), to estimate activity factors and power for microarchitectural structures. Despite using a relatively few input parameters-specifically nine-based on general microprocessor utilization statistics (e.g., IPC and load rate), our linear-regression-based model estimates activity and dynamic power for over 100 structures in an out-of-order x86 pipeline and core power with an average error of 8%. Because the computations utilize few inputs, CAMP is simple enough to implement in hardware, providing run-time structure and core power estimates for dynamic power management. Because the input statistics are generic in nature and the model remains accurate across incremental microarchitectural refinements, CAMP provides simple intuitive equations relating global microarchitectural statistics to structure activity and power. These equations provide a simple technique that can equate changes in one structure's activity to power variations in other structures across the pipeline.

The incidence of hard errors in CPUs is a challenge for future multicore designs due to increasing total core area. Even if the location and nature of hard errors are known a priori, either at manufacture-time or in the field, cores with such errors must be disabled in the absence of hard-error tolerance. While caches, with their regular and repetitive structures, are easily covered against hard errors by providing spare arrays or spare lines, structures within a core are neither as regular nor as repetitive. Previous work has proposed microarchitectural core salvaging to exploit structural redundancy within a core and maintain functionality in the presence of hard errors. Unfortunately microarchitectural salvaging introduces complexity and may provide only limited coverage of core area against hard errors due to a lack of natural redundancy in the core.

In his seminal paper published in 2002, Passino pointed out how individual and groups of bacteria forage for nutrients and how to model it as a distributed optimization process, which he called the bacterial foraging optimization algorithm (BFOA). One of the major driving forces of BFOA is the chemotactic movement of a virtual bacterium that models a trial solution of the optimization problem. This paper presents a mathematical analysis of the chemotactic step in BFOA from the viewpoint of the classical gradient descent search. The analysis points out that the chemotaxis employed by classical BFOA usually results in sustained oscillation, especially on flat fitness landscapes, when a bacterium cell is close to the optima. To accelerate the convergence speed of the group of bacteria near the global optima, two simple schemes for adapting the chemotactic step height have been proposed. Computer simulations over several numerical benchmarks indicate that BFOA with the adaptive chemotactic operators shows better convergence behavior, as compared to the classical BFOA. The paper finally investigates an interesting application of the proposed adaptive variants of BFOA to the frequency-modulated sound wave synthesis problem, appearing in the field of communication engineering.

Bacterial-foraging optimization algorithm (BFOA) attempts to model the individual and group behavior of E.Coli bacteria as a distributed optimization process. Since its inception, BFOA has been finding many important applications in real-world optimization problems from diverse domains of science and engineering. One key step in BFOA is the computational chemotaxis, where a bacterium (which models a candidate solution of the optimization problem) takes steps over the foraging landscape in order to reach regions with high-nutrient content (corresponding to higher fitness). The simulated chemotactic movement of a bacterium may be viewed as a guided random walk or a kind of stochastic hill climbing from the viewpoint of optimization theory. In this paper, we first derive a mathematical model for the chemotactic movements of an artificial bacterium living in continuous time. The stability and convergence-behavior of the said dynamics is then analyzed in the light of Lyapunov stability theorems. The analysis indicates the necessary bounds on the chemotactic step-height parameter that avoids limit cycles and guarantees convergence of the bacterial dynamics into an isolated optimum. Illustrative examples as well as simulation results have been provided in order to support the analytical treatments.

One of the major driving forces of bacterial foraging optimization algorithm (BFOA) is the reproduction phenomenon of virtual bacteria each of which models one trial solution of the optimization problem. During reproduction, the least healthier bacteria (with a lower accumulated value of the objective function in one chemotactic lifetime) die and the other healthier bacteria each split into two, which then starts exploring the search place from the same location. This keeps the population size constant in BFOA. The phenomenon has a direct analogy with the selection mechanism of classical evolutionary algorithms. In this article, we provide a simple mathematical analysis of the effect of reproduction on bacterial dynamics. Our analysis reveals that the reproduction event contributes to the quick convergence of the bacterial population near optima.

Differential evolution (DE) is well known as a simple and efficient algorithm for global optimization over continuous spaces. This article provides a simple mathematical model of the underlying evolutionary dynamics of a one-dimensional DE. The model relates the search process of DE with the classical gradient descent search and also analyzes the convergence behavior of a DE population, very near to optima.

Some researchers have illustrated how individual and groups of bacteria forage for nutrients and to model it as a distributed optimization process, which is called the bacterial foraging optimization (BFOA). One of the major driving forces of BFOA is the chemotactic movement of a virtual bacterium, which models a trial solution of the optimization problem. In this article, we analyze the chemotactic step of a one dimensional BFOA in the light of the classical gradient descent algorithm (GDA). Our analysis points out that chemotaxis employed in BFOA may result in sustained oscillation, especially for a flat fitness landscape, when a bacterium cell is very near to the optima. To accelerate the convergence speed near optima we have made the chemotactic step size C adaptive. Computer simulations over several numerical benchmarks indicate that BFOA with the new chemotactic operation shows better convergence behavior as compared to the classical BFOA.

Differential Evolution (DE) has recently emerged as a simple yet very powerful technique for real parameter optimization. This article describes an application of DE for the design of Fractional-Order Proportional-Integral-Derivative (FOPID) Controllers involving fractional order integrator and fractional order differentiator. FOPID controllers' parameters are composed of the proportionality constant, integral constant, derivative constant, derivative order and integral order, and its design is more complex than that of conventional integer order PID controller. Here the controller synthesis is based on user-specified peak overshoot and rise time and has been formulated as a single objective optimization problem. In order to digitally realize the fractional order closed loop transfer function of the designed plant, Tustin operator-based CFE (continued fraction expansion) scheme was used in this work. Simulation examples as well as comparisons of DE with two other state-of-the-art optimization techniques (Particle Swarm Optimization and Bacterial Foraging Optimization Algorithm) over the same problems demonstrate the superiority of the proposed approach especially for actuating fractional order plants.

This article presents an algorithm for the automatic detection of circular shapes from complicated and noisy images. The algorithm is based on a recently developed swarm-intelligence technique, well known as the Bacterial Foraging Optimization (BFO). A new fuzzy objective function has been derived for the edge map of a given image. Minimization of this function with an adaptive version of the BFO algorithm leads to the automatic detection of circles on the image.

This article presents an algorithm for the automatic detection of circular shapes from complicated and noisy images. The algorithm is based on a hybrid technique composed of simulated annealing and differential evolution. A new fuzzy objective function has been derived for the edge map of a given image. Minimization of this function with a hybrid annealed differential evolution algorithm leads to the automatic detection of circles on the image. Simulation results over several synthetic as well as natural images with varying range of complexity validate the efficacy of the proposed technique in terms of its final accuracy, speed and robustness.

Linear prediction (LP) is a widely used technique in single-channel speech coding. LP can also be applied to stereo (or multi-channel) audio coding, but it requires a scheme for quantization and efficient transmission of stereo-LP parameters. We propose to transmit the (forward) normalized reflection matrices together with the zero-lag correlation matrix. Furthermore, we select a parameterization for the matrices and a quantization strategy per parameter. The parameterization for the normalized reflection matrices is a variant of the singular value decomposition (SVD). The proposed quantization scheme is evaluated for first-order stereo-LP systems. As criterion, we use a spectral distortion (SD) measure based on the norm of the transfer matrix of the synthesis filter. Simulations show that the distortions can easily be controlled as a function of bit rate

Processor designers require estimates of the architectural vulnerability factor (AVF) of on-chip structures to make accurate soft error rate estimates. AVF is the fraction of faults from alpha particle and neutron strikes that result in user-visible errors. This paper shows how to use a performance model to calculate the AVF of address-based structures, using a data cache, a data translation buffer, and a store buffer as examples. We describe how to perform a detailed breakdown of lifetime components (e.g., fill-to-read, read-to-evict) of bits in these structures into ACE (required for architecturally correct execution), un-ACE (unnecessary for ACE), and unknown components. This lifetime analysis produces best estimate AVFs for these three structures' data arrays of 6%, 36%, and 4%, respectively. We then present a new technique, hamming-distance-one analysis, and show that it predicts surprisingly low best estimate AVFs of 0.41%, 3%, and 7.7% for the structures' tag arrays. Finally, using our lifetime analysis framework, we show how two AVF reduction techniques - periodic flushing and incremental scrubbing - can reduce the AVF by converting ACE lifetime components into un-ACE without affecting performance significantly.