Current dual-fusion models for 6D pose estimation often lead to increased computational complexity and risk of overfitting with the addition of more networks. To address this, we propose a Cross-modal Fusion Network (CFN), which extracts robust dual-modal features while reducing computation energy and overfitting risks. The CFN consists of multiple Cross-modal Fusion Modules (CFM), featuring two key components: 1) the Spiking-based Cross-Attention Block (SCA), which utilizes only mask and addition operations, significantly lowering computational energy compared to traditional self-attention; and 2) the Specificity Preserving Block (SPB), designed to mitigate overfitting from multiple CFM layers. Additionally, we introduce a density-peak keypoint localization (DKL) method that resists noise and sparse data, eliminating the need for iterative processes. Extensive experiments on multiple 6D pose estimation benchmarks demonstrate that our CFN method significantly outperforms existing state-of-the-art approaches.

With the development of the Android system and increasing screen size, the use of multi-window mode has become prevalent among users. However, the security and privacy implications associated with this mode have not been thoroughly investigated. This paper uncovers severe and unique security vulnerabilities in Android's multi-window mode, revealing several high-risk side-channels that facilitate diverse cross-window attacks, leading to significant breaches of user privacy. In detail, our research introduces WinSpy, a framework leveraging a newly discovered resource contention side-channel in multi-window mode to fingerprint app launches, web pages, and in-app activities, all without violating Android's permission framework. Our extensive evaluations demonstrate that WinSpy achieves high accuracy (from 70 to 80% detecting website and app launches to over 97% recognizing critical in-app activities). Additionally, we reveal that due to Android's lenient permission management for this mode, window apps can also use Inertial Measurement Unit sensors to launch attacks, such as inferring the user's touch positions outside the window with high precision. Furthermore, we propose systematic mitigations against these vulnerabilities.

OpenAI has enabled third-party developers to build applications around ChatGPT, known as GPTs, to expand its capability to handle complex and specialized tasks. A key feature of GPTs is Retrieval-Augmented Generation (RAG), which allows developers to upload documents containing domain knowledge or application context, referred to as file knowledge. However, these documents often contain sensitive information, and the security mechanisms governing access control in GPTs remains an underexplored area.

We present a framework to generate past and future processes for drawing process videos. Given a canvas image uploaded by a user, the framework can generate both preceding and succeeding states of the drawing process, and the generated states can be reused as inputs for further state generation. We observe that the user queries typically have one-to-one or many-to-many states, and in many cases, involve non-contiguous states. This necessitates a backend that solves a set-to-set problem with arbitrary combinations of past or future states. To this end, we repurpose video diffusion models to learn the set-to-set mapping with pretrained video priors. We implement the system with strong diffusion transformer backbones ( e.g. , CogVideoX and LTXVideo) and high-quality data processing ( e.g. , sampling short shots from long videos of real drawing records). Experiments show that the generated states are diverse in drawing contexts and resemble human drawing processes. This capability may aid artists in visualizing potential outcomes, generating creative inspirations, or refining existing workflows.

Virtual personal assistants (VPA) services encompass a large number of third-party applications (or apps) to enrich their functionalities. These apps have been well examined to scrutinize their data collection behaviors against their declared privacy policies. Nonetheless, it is often overlooked that most users tend to ignore privacy policies at the installation time. Dishonest developers thus can exploit this situation by embedding excessive declarations to cover their data collection behaviors during compliance auditing.

ChatGPT has enabled third-party developers to create plugins to expand ChatGPT's capabilities. These plugins are distributed through OpenAI's plugin store, making them easily accessible to users. With ChatGPT as the backbone, this app ecosystem has illustrated great business potential by offering users personalized services in a conversational manner. Nonetheless, many crucial aspects regarding app development, deployment, and security of this ecosystem have yet to be thoroughly studied in the research community, potentially hindering a broader adoption by both developers and users. In this work, we conduct the first comprehensive study of the ChatGPT app ecosystem, aiming to illuminate its landscape for our research community. Our study examines the distribution and deployment models in the integration of LLMs and third-party apps, and assesses their security and privacy implications. We uncover an uneven distribution of functionality among ChatGPT plugins, highlighting prevalent and emerging topics. We also identify severe flaws in the authentication and user data protection for third-party app APIs integrated within LLMs, revealing a concerning status quo of security and privacy in this app ecosystem. Our work provides insights for the secure and sustainable development of this rapidly evolving ecosystem.

Shadowing allows artists to convey realistic volume and emotion of characters in comic colorization. While AI technologies have the potential to improve professionals’ shadowing experience, current practice is manual and time-consuming. To understand how we can improve their shadowing experience, we conducted interviews with 5 professionals. We found that professionals’ level of engagement can vary depending on semantics, such as characters’ faces or hair. We also found they spent time on shadow “landscaping”—deciding where to put big shadow regions to make a realistic volumetric presentation—while the final results can dramatically vary depending on their “staging” and “attention guiding” needs. We found they would accept AI suggestions for less engaging semantic parts or landscaping, while they would need to have the capability to adjust details. Based on our observations, we built ShadowMagic that (1) generates AI-driven shadows based on typically used light directions, (2) enables a user to selectively choose the results depending on the semantics, and (3) allows users to finish shadow areas by themselves for further perfection. Through a summative evaluation with 5 professionals, we found that they were significantly more satisfied with our AI-driven results than a baseline. We also found ShadowMagic’s “step by step” workflow helps participants more easily adopt AI-driven results. We conclude by providing implications.

An app ecosystem built around virtual personal assistant (VPA) services becomes flourishing in recent years, fueled by the booming of the Internet of Things (IoT). A large number of functionality-rich VPA applications (or apps for short) have been released through app stores, and become easily-accessible by users through their smart speakers. In response to the increasingly stringent data protection regulations around the world, VPA service providers require app developers to include a privacy policy that declares their data handling practices. These privacy policies serve as the de facto agreement between developers and users, and may be taken as the basis in resolving conflicts in the event of a data breach. Therefore, it is essential that privacy policy documents are crafted in a clear, easy-to-understand, and unambiguous way. In this work, we conduct the first systematic study on the quality of privacy policies in the VPA app domain. Based on our review of literature and documents from standard working groups, we identify four metrics that enable the quality of the privacy policy to become measurable, including timeliness, availability, completeness and readability. We then develop QuPer, which extracts the meta features (e.g., update history) and linguistic features (e.g., sentence semantics) from privacy policies, and assesses their quality. Our analysis reveals that the status of the quality of privacy policies in the VPA app domain is concerning. For instance, only 1.17% of privacy policies completely cover all contents that are regarded as privacy concerns by legislation (e.g., GDPR article 13) and relevant literature. Our findings are expected to raise an alert among the VPA app developers and provide them with guidelines for creating high-quality privacy policy documents. We also encourage app store operators to implement a vetting process that ensures the quality of privacy policies before apps are released to the public.

We introduce an algorithm for sketch vectorization with state-of-the-art accuracy and capable of handling complex sketches. We approach sketch vectorization as a surface extraction task from an unsigned distance field, which is implemented using a two-stage neural network and a dual contouring domain post processing algorithm. The first stage consists of extracting unsigned distance fields from an input raster image. The second stage consists of an improved neural dual contouring network more robust to noisy input and more sensitive to line geometry. To address the issue of under-sampling inherent in grid-based surface extraction approaches, we explicitly predict undersampling and keypoint maps. These are used in our post-processing algorithm to resolve sharp features and multi-way junctions. The keypoint and undersampling maps are naturally controllable, which we demonstrate in an interactive topology refinement interface. Our proposed approach produces far more accurate vectorizations on complex input than previous approaches with efficient running time.

Creating digital comics involves multiple stages, some creative and some menial. For example, coloring a comic requires a labor-intensive stage known as ‘flatting,’ or masking segments of continuous color, as well as creative shading, lighting, and stylization stages. The use of AI can automate the colorization process, but early efforts have revealed limitations—technical and UX—to full automation. Via a formative study of professionals, we identify flatting as a bottleneck and key target of opportunity for human-guided AI-driven automation. Based on this insight, we built FlatMagic, an interactive, AI-driven flat colorization support tool for Photoshop. Our user studies found that using FlatMagic significantly reduced professionals’ real and perceived effort versus their current practice. While participants effectively used FlatMagic, we also identified potential constraints in interactions with AI and partially automated workflows. We reflect on implications for comic-focused tools and the benefits and pitfalls of intermediate representations and partial automation in designing human-AI collaboration tools for professionals.

A large number of functionality-rich and easily accessible applications have become popular among various virtual personal assistant (VPA) services such as Amazon Alexa. VPA applications (or VPA apps for short) are accompanied by a privacy policy document that informs users of their data handling practices. These documents are usually lengthy and complex for users to comprehend, and developers may intentionally or unintentionally fail to comply with them. In this work, we conduct the first systematic study on the privacy policy compliance issue of VPA apps. We develop Skipper, which targets Amazon Alexa skills. It automatically depicts the skill into the declared privacy profile by analyzing their privacy policy documents with Natural Language Processing (NLP) and machine learning techniques, and derives the behavioral privacy profile of the skill through a black-box testing. We conduct a large-scale analysis on all skills listed on Alexa store, and find that a large number of skills suffer from the privacy policy noncompliance issues.

Rollover is a potential dangerous factor for mobile robots to accomplish a task. However, to our best knowledge, there still lacks a systematic research on the rollover mechanism and active rollover prevention control of mobile robots in the literature. This paper aims to propose a general control framework for rollover prevention of high-speed wheeled mobile robots. First, the lateral dynamics of the robot is modelled and the Load Transfer Ratio (LTR) is used as an index to measure the rollover level. Second, an optimal algorithm-based reference governor (RG) is developed, by which the wheel speed command that satisfies the constraint is induced, retaining the actual LTR within the threshold safety value. In addition, an integral sliding mode (ISM) wheel speed tracking controller is proposed. Lastly, simulations results show that for both trajectory tracking and path following cases, the controlled robot avoids possible rollover successfully.

Sketching is a foundational step in the design process. Decades of sketch processing research have produced algorithms for 3D shape interpretation, beautification, animation generation, colorization, etc. However, there is a mismatch between sketches created in the wild and the clean, sketch-like input required by these algorithms, preventing their adoption in practice. The recent flurry of sketch vectorization, simplification, and cleanup algorithms could be used to bridge this gap. However, they differ wildly in the assumptions they make on the input and output sketches. We present the first benchmark to evaluate and focus sketch cleanup research. Our dataset consists of 281 sketches obtained in the wild and a curated subset of 101 sketches. For this curated subset along with 40 sketches from previous work, we commissioned manual vectorizations and multiple ground truth cleaned versions by professional artists. The sketches span artistic and technical categories and were created by a variety of artists with different styles. Most sketches have Creative Commons licenses; the rest permit academic use. Our benchmark's metrics measure the similarity of automatically cleaned rough sketches to artist-created ground truth; the ambiguity and messiness of rough sketches; and low-level properties of the output parameterized curves. Our evaluation identifies shortcomings among state-of-the-art cleanup algorithms and discusses open problems for future research.

In this paper, a novel deep generative model-based approach for video error concealment is proposed. Our method is comprised of completion network and two critics. The frame completion network is trained to fool the both the local and global critics, which requires completion network to conceal frame distortions with regard to overall consistency as well as in details. Specifically, mask attention convolution layer is proposed, which utilize not only the temporal information of the previous frame, but also the intact pixels of the current distorted frame to mask and re-normalize convolution features. Then, both qualitative and quantitative experiments validate the effectiveness and generality of our approach in advancing the error concealment on low resolution video.

In this paper, a novel weighted boundary matching error concealment schema for HEVC is proposed, which is based on the CU depths and PU partitions in reference frame. Firstly, the information of CU depths in reference frames is used for lost slices. For each LCU in a lost slice, the LCUs surrounding to the co-located LCU are used to calculate summed CU-depth weight, which is used to determine the conceal order of each CU. Then, the co-located partition decision from the reference frame is adopted for PUs in each lost CU. The sequence of PUs to conceal is sorted based on the texture randomness index weight and the PU with the largest weight will be concealed next. Finally, the best estimated motion vector for the lost PU is selected for concealment. The experimental results show that our method achieves higher PSNR gains and has a better visual quality than the state-of-the-art methods.

The Navypsilas future electric ships will contain a number of pulsed power loads for high-energy applications such as radar, railguns, and advanced weapons. This pulse energy demand has to be provided by the ship energy sources, while not impacting the operation of the rest of the system. It is clear from studies carried out earlier that disturbances are created at the generator ac bus. This paper describes an online design and laboratory hardware implementation of an optimal excitation controller using particle swarm optimization (PSO) to minimize the effects of pulsed loads. The PSO algorithm has been implemented on a digital signal processor. Laboratory results show that the PSO designed excitation controller provides an effective control of a generatorpsilas terminal voltage during pulsed loads, restoring and stabilizing it quickly.