Teasers are an effective tool for promoting content in entertainment, commercial and educational fields. However, creating an effective teaser for long videos is challenging for it requires long-range multimodal modeling capability for the input videos, while necessitating maintaining audiovisual alignments, managing scene transitions and preserving factual accuracy for the output teasers. Due to the lack of a publicly-available dataset, progress along this research direction has been hindered. In this work, we present DocumentaryNet, a collection of 1,269 documentaries paired with their teasers, featuring multimodal data streams of video, speech, music, sound effects and narrations. With DocumentaryNet, we propose a new two-stage system for generating teasers from long documentaries. The proposed TeaserGen system first generates the teaser narration from the transcribed narration from the documentary using a pretrained large language model, and then selects the most relevant visual content to accompany the generated narration through language-vision models. For narration-video matching, we explore two approaches: a pretraining-based model using pretrained contrastive language-vision models and a deep sequential model that learns the mapping between the narrations and visuals. Our experimental results show that the pretraining-based approach is more effective at identifying relevant visual content than directly trained deep autoregressive models.

Short videos are an effective tool for promoting contents and improving knowledge accessibility. While existing extractive video summarization methods struggle to produce a coherent narrative, existing abstractive methods cannot `quote' from the input videos, i.e., inserting short video clips in their outputs. In this work, we explore novel video editing models for generating shorts that feature a coherent narrative with embedded video insertions extracted from a long input video. We propose a novel retrieval-embedded generation framework that allows a large language model to quote multimodal resources while maintaining a coherent narrative. Our proposed REGen system first generates the output story script with quote placeholders using a finetuned large language model, and then uses a novel retrieval model to replace the quote placeholders by selecting a video clip that best supports the narrative from a pool of candidate quotable video clips. We examine the proposed method on the task of documentary teaser generation, where short interview insertions are commonly used to support the narrative of a documentary. Our objective evaluations show that the proposed method can effectively insert short video clips while maintaining a coherent narrative. In a subjective survey, we show that our proposed method outperforms existing abstractive and extractive approaches in terms of coherence, alignment, and realism in teaser generation.

This paper presents a millimeter-wave (mmWave) Integrated Communication and Sensing (ICAS) channel analysis for the Frequency Range 2 (FR2) band at 28 GHz in a street intersection scenario. ICAS has emerged as a potential new feature in sixth-generation (6G) communication systems, offering the possibility of unlocking new markets using Third Generation Partnership Project (3GPP) technology. Additionally, the advanced sensing capabilities of 6G technology can help address the growing hardware demands of ultra-fast mmWave communications. For example, an mmWave-V2X communication module may provide radar data to enhance or validate existing sensors. Furthermore, standardized ICAS architectures can help mitigate radar interference challenges in automotive environments. This paper focuses on the analysis of the dynamic effects of Vulnerable Road Users (VRUs) in a crowded street intersection scenario. To understand the challenges of ICAS system in dynamic environments, ray tracing simulations are conducted at 28 GHz, to obtain the stochastic channel parameters, such as delay spread, angular spreads, and received power. The dynamic evolution of the channel is further analyzed for the potential sensing purposes such as in-vehicle sensor fusion and bi-static sensing for hidden object detection. The results provide insights into the development and testing of 6G ICAS systems for automotive use cases.

Music composition and analysis is an inherently creative task, involving a combination of heart and mind. However, the vast majority of algorithmic music models completely ignore the "heart" component of music, resulting in output that often lacks the rich emotional direction found in human-composed music. Models that try to incorporate musical sentiment rely on a "valence-arousal" model, which insufficiently characterizes emotion in two dimensions. Furthermore, existing methods typically adopt a black-box, music agnostic approach, treating music-theoretical and sentimental understanding as a by-product that can be inferred given sufficient data. In this study, we introduce two major novel elements: a nuanced mixture-based representation for musical sentiment, including a web tool to gather data, as well as a sentiment- and theory-driven harmonization model, SentHYMNent. SentHYMNent employs a novel Hidden Markov Model based on both key and chord transitions, as well as sentiment mixtures, to provide a probabilistic framework for learning key modulations and chordal progressions from a given melodic line and sentiment. Furthermore, our approach leverages compositional principles, resulting in a simpler model that significantly reduces computational burden and enhances interpretability compared to current state-of-the-art algorithmic harmonization methods. Importantly, as shown in our experiments, these improvements do not come at the expense of harmonization quality. We also provide a web app where users can upload their own melodies for SentHYMNent to harmonize.

The bunching mitigation and eco-driving of buses have traditionally been treated separately in the literature and formulated in different frameworks. Although a large amount of research has been conducted aiming at providing eco-driving advisory to drivers, impacts of bus operation are not considered. Ignoring the constraints induced by buses could result in transit service irregularity. The present work aims to bridge this gap by formulating optimal control model of buses’ eco-speed and service regularity as a non-linear program. In this study, an integrated bus eco-driving speed and holding time controller is developed to obtain the optimal eco-driving advisory and operational schedule of bus fleet. A bi-level optimization model is formulated for the whole transit line with multiple signalized intersections. The proposed model is solved by a hybrid solution algorithm, which embeds the particle swarm optimization for the upper-level problem and the pseudo-spectral method for the lower-level problem. The proposed controller is tested on real-world cases with the Simulation of Urban MObility (SUMO), and is compared to non-control, speed control, and holding control methods. Simulation results show that the proposed controller reduces the headway deviation by 78.27% and reduces total fuel consumption by 23.95%.

The “pretraining-and-finetuning” paradigm has become a norm for training domain-specific models in natural language processing and computer vision. In this work, we aim to examine this paradigm for symbolic music generation through leveraging the largest ever symbolic music dataset sourced from the MuseScore forum. We first pretrain a large unconditional transformer model using 1.5 million songs. We then propose a simple technique to equip this pretrained unconditional music transformer model with instrument and genre controls by finetuning the model with additional control tokens. Our proposed representation offers improved high-level controllability and expressiveness against two existing representations. The experimental results show that the proposed model can successfully generate music with user-specified instruments and genre. In a subjective listening test, the proposed model outperforms the pretrained baseline model in terms of coherence, harmony, arrangement and overall quality.

This article presents a mathematical model to design a demand-responsive connector (DRC) bus operational network for improving the service quality and accessibility of public transportation systems. The proposed model features an integrated framework that simultaneously guides passengers to reach their nearest bus stops and routes buses to transport passengers at selected bus stops to connected stations of major transit systems. Passengers' preferred time for pick up is fully considered in the model. For this purpose, this study proposes a multi-objective mixed-integer linear programing model to effectively capture the interactions between users with their predefined service time windows and DRC bus network. This study further develops a three-stage heuristic to yield suboptimal solutions to the model in a reasonable time. Case study results demonstrate the effectiveness of the proposed model.