Multimodal foundation models, such as Gemini and ChatGPT, have revolutionized human-machine interactions by seamlessly integrating various forms of data. Developing a universal spoken language model that comprehends a wide range of natural language instructions is critical for bridging communication gaps and facilitating more intuitive interactions. However, the absence of a comprehensive evaluation benchmark poses a significant challenge. We present Dynamic-SUPERB Phase-2, an open and evolving benchmark for the comprehensive evaluation of instruction-based universal speech models. Building upon the first generation, this second version incorporates 125 new tasks contributed collaboratively by the global research community, expanding the benchmark to a total of 180 tasks, making it the largest benchmark for speech and audio evaluation. While the first generation of Dynamic-SUPERB was limited to classification tasks, Dynamic-SUPERB Phase-2 broadens its evaluation capabilities by introducing a wide array of novel and diverse tasks, including regression and sequence generation, across speech, music, and environmental audio. Evaluation results show that no model performed well universally. SALMONN-13B excelled in English ASR and Qwen2-Audio-7B-Instruct showed high accuracy in emotion recognition, but current models still require further innovations to handle a broader range of tasks. We open-source all task data and the evaluation pipeline at https://github.com/dynamic-superb/dynamic-superb.

Using large language models (LLMs) for automatic evaluation has become an important evaluation method in NLP research.However, it is unclear whether these LLM-based evaluators can be applied in real-world classrooms to assess student assignments.This empirical report shares how we use GPT-4 as an automatic assignment evaluator in a university course with 1,028 students.Based on student responses, we find that LLM-based assignment evaluators are generally acceptable to students when students have free access to these LLM-based evaluators.However, students also noted that the LLM sometimes fails to adhere to the evaluation instructions.Additionally, we observe that students can easily manipulate the LLM-based evaluator to output specific strings, allowing them to achieve high scores without meeting the assignment rubric.Based on student feedback and our experience, we provide several recommendations for integrating LLM-based evaluators into future classrooms.Our observation also highlights potential directions for improving LLM-based evaluators, including their instruction-following ability and vulnerability to prompt hacking.

Warning: This paper may contain texts with uncomfortable content.Large Language Models (LLMs) have achieved remarkable performance in various tasks, including those involving multimodal data like speech. However, these models often exhibit biases due to the nature of their training data. Recently, more Speech Large Language Models (SLLMs) have emerged, underscoring the urgent need to address these biases. This study introduces Spoken Stereoset, a dataset specifically designed to evaluate social biases in SLLMs. By examining how different models respond to speech from diverse demographic groups, we aim to identify these biases. Our experiments reveal significant insights into their performance and bias levels. The findings indicate that while most models show minimal bias, some still exhibit slightly stereotypical or anti-stereotypical tendencies.

Speech Integrated Large Language Models (SILLMs) combine large language models with speech perception to perform diverse tasks, such as emotion recognition to speaker verification, demonstrating universal audio understanding capability. However, these models may amplify biases present in training data, potentially leading to biased access to information for marginalized groups. This work introduces a curated spoken bias evaluation toolkit and corresponding dataset. We evaluate gender bias in SILLMs across four semantic-related tasks: speech-to-text translation (STT), spoken coreference resolution (SCR), spoken sentence continuation (SSC), and spoken question answering (SQA). Our analysis reveals that bias levels are language-dependent and vary with different evaluation methods. Our findings emphasize the necessity of employing multiple approaches to comprehensively assess biases in SILLMs, providing insights for developing fairer SILLM systems.

Complex steerable pyramid (CSP) is widely used to decompose images into muti-scale and oriented subbands for phase-based processing, such as video magnification, frame interpolation, and view synthesis. The conventional implementation is based on frequency-domain bandpass filtering which relies on fast Fourier transform (FFT). However, FFT requires high-precision computation and complex memory access for hardware implementation. In this paper, we study computation- and memory-efficient finite impulse response filter implementation for CSP. We use Kaiser windowing for filter designs and adopt 9-tap radial and 11-tap angular filters which can achieve 38.6 dB of PSNR for frame interpolation. We then discuss about VLSI architecture designs and, in particular, propose a stripe-based computation flow for 2-D CSP to reduce the line buffer size down to 15%. For evaluation, we implement two VLSI circuits using TSMC 40nm technology. One is a 1-D CSP engine working at 4K UHD 30 fps, and it saves 67.8% of logic gates compared to a FFT-based design. The other is a 2-D CSP engine working at FHD 60 fps. It uses 32-KB SRAM and 3.5M-gate logic. We also implement a FPGA system for 2-D CSP engine, and it operates at 80 MHz and provides 1024×1024 resolution video at 16 fps.

This paper presents an ac-dc rectifier for piezoelectric vibration energy harvesting application. The proposed rectifier is designed and implemented with TSMC 0.25 μm 1P3M 5 V CMOS process. Active control is proposed to eliminate the voltage drop from the traditional diode forward voltage and the reverse current from the output capacitor to input source. The simultaneously turn on and oscillation on power MOS MN1-MN2gate signal VGN1-VGN2has been overcome by a 40ns delay time which generated by a non-overlapping control. The experimental results indicated maximum power efficiency 95% and voltage conversion ratio 99% are achieved at 20kΩ and 200kΩ loading resistor.

View synthesis is one of the important techniques utilized in 3D TV devices. Traditional methods such as depth image-based rendering usually rely on accurate depth maps which require computation-intensive stereo matching. In this work, we present a hardware system of phase-based view synthesis that is able to convert stereoscopic videos to multi-view content with low-resolution depth maps. When compared to the view synthesis reference software, the phase-based method does not suffer from severe arifacts on object boundaries and provides higher quality views in our experimental result. There are two major contributions in our implementation. First, we propose a cross-band disparity correction scheme that not only enables the usage of low-resolution disparity maps but also improves the quality of novel views. Second, we propose a hardware-friendly wavelet re-projection engine to reduce the hardware complexity. We implemented a VLSI circuit for 8-view 4K Ultra-HD (UHD) 3DTV in TSMC 40nm technology. It delivers 30 frames per second (fps) for UHD display when operating at 200MHz. It uses 228-KB SRAM and 2M-gate logic . We also implemented the system on FPGA, and it can provide 4K UHD multi-view content at 12 fps.

Besides designing channel-hopping (CH) sequences, finding good methods of communication rendezvous is important for improving spectral efficiency and alleviating traffic load of cognitive radio (CR) networks. In this paper, new asynchronous-symmetric CH sequences are constructed and their (channel overlap, even-channel-use, and pairwise shift invariance) properties are investigated. Their operations in three transmission scenarios involving different basic collision-avoidance mechanisms are analyzed and compared in terms of maximum time-to-rendezvous, rendezvous-success (RS) rate, and RS variance. This paper shows that the new sequences have shorter period, larger cardinality, and smaller RS variations than other CH sequences, thus supporting greater number of secondary users and more frequent and uniform rendezvous in the CR networks.

This paper presents a single-inductor multi-output (SIMO) dc-dc converter for gigascale systems, which includes two different kinds of control loops, the comparator loop and the error amplifier (EA) loop. The proposed converter can provide four output voltages with only five power switches, two comparator loops, and two error amplifier loops. To save additional power switches and lower the complexity, the proposed converter uses the comparator loops on the first and the third control loops. The two outputs of the comparator loops can be either step-up or step-down outputs, depending on the requirements of different applications. The controller can effectively distribute the energy of inductor to four outputs with low complexity. The proposed converter was designed by using TSMC 0.18μm 1P6M CMOS process with the supply voltage of 1.8V and provides outputs of 2V, 2.5V, 1.5V and 2.2V. The proposed converter yields a maximum efficiency of 83.76% with only one inductor and five power switches.

This paper presents 3D IC design of a fully integrated four-phase buck converter. The control circuit was implemented in the TSMC 0.18μm CMOS Mixed Signal RF General Purpose MiM Al 1P6M 1.8 & 3.3V process while the passive components were implemented in the tMt Glass Substrate Integrated Passive Device (IPD) Process, and then these two dies are stacked. Thus, a high output loading current can be obtained with a small chip area. The supply voltage is converted from 1.8V down to 1V with output ripple less than 70mV. The switching frequency is 50MHz and the maximum load current is 500mA, with 5mV/A load regulation and 2mV/V line regulation.

Many modern embedded systems, such as cyber-physical systems, feature close integration of computation and physical components. Configurability, efficiency, adaptability, reliability, and usability are essential features for such systems. A workflow engine is a software application that manages workflows. It helps developers separate control flows from activities of the system, and thus is able to enhance configurability and development efficiency. This work aims at the design and implementation of a workflow engine for cyber-physical systems so as to configure workflows with less efforts. To meet timing requirements, the engine is designed to schedule activities to meet their timing requirements, and provides admission control so that these requirements of a set of workflows are guaranteed as long as it is admitted. A humanoid robot is used as the test-bed for the workflow engine. We model robot applications as workflows, and show that how the workflow engine provides real time guarantee and enhances its configurability.

In this article, we propose a 2D to 3D video conversion scheme for MPEG videos. The difficulty for 2D/3D conversion problem lies on depth estimation/ assignment with insufficient information. Our depth assignment is based on the analyses of multiple cues, e.g., motion parallax, atmospheric perspective, texture gradient, linear perspective, and relative height, separately for the foreground objects and the background area. To fit more kinds of videos, the proposed depth assignment scheme is content-adaptive by segmenting a video into shots and classifying each of them into three categories for different conversion schemes. Subjective experiments show that the 3D stereo video generated by using our depth assignment scheme and the Depth Image Based Rendering (DIBR) technique presents little difference to that created based on the depth ground truths.

Radio frequency identification has been developed and used in many applications in the real world. Due to the shared wireless channel between tags and the reader during communication, the tag collision arbitration is a significant issue for reducing the communication overhead. This paper presents a novel anti-collision algorithm named New Enhanced Anti-Collision Algorithm (NEAA) using counters and stack to reduce the probability of collision efficiently and to make it possible to identify multiple passive tags in a timeslot. The upper bound of total timeslots for identifying N passive tags is first derived in this paper; suppose the length of a tag ID is n, the upper bound of total timeslots for identifying N (N= 2n) passive tags is derived to be 2n-1- n + 4, when n > 2. This bound is quite tight. Compared to the existing methods proposed by other researchers, the performance evaluation shows that the proposed scheme in this paper consumes fewer timeslots and has better performance for identifying tags.