Open world perception expands traditional closed-set frameworks, which assume a predefined set of known categories, to encompass dynamic real-world environments. Open World Object Detection (OWOD) and Open Vocabulary Object Detection (OVD) are two main research directions, each addressing unique challenges in dynamic environments. However, existing studies often focus on only one of these tasks, leaving the combined challenges of OWOD and OVD largely underexplored. In this paper, we propose a novel detector, OW-OVD, which inherits the zero-shot generalization capability of OVD detectors while incorporating the ability to actively detect unknown objects and progressively optimize performance through incremental learning, as seen in OWOD detectors. To achieve this, we start with a standard OVD detector and adapt it for OWOD tasks. For attribute selection, we propose the Visual Similarity Attribute Selection (VSAS) method, which identifies the most generalizable attributes by computing similarity distributions across annotated and unannotated regions. Additionally, to ensure the diversity of attributes, we incorporate a similarity constraint in the iterative process. Finally, to preserve the standard inference process of OVD, we propose the Hybrid Attribute-Uncertainty Fusion (HAUF) method. This method combines attribute similarity with known class uncertainty to infer the likelihood of an object belonging to an unknown class. We validated the effectiveness of OW-OVD through evaluations on two OWOD benchmarks, M-OWODB and S-OWODB. The results demonstrate that OW-OVD outperforms existing state-of-the-art models, achieving a +15.3 improvement in unknown object recall (U-Recall) and a +15.5 increase in unknown class average precision (U-mAP). Our code is available at: https://github.com/xxyzll/OW_OVD.

Open World Object Detection (OWOD) often uses objectness scores as a key metric for identifying unknown objects, which leads to label bias issues. To address this problem, we applied Unsupervised Domain Adaptation (UDA) to OWOD and built an unbiased foreground/background predictor. However, current self-training methods in UDA often rely on fixed thresholds to calculate unsupervised loss, which fails to adapt to the learning difficulties of different categories. Thus, we propose a Dynamic Learning-driven Unsupervised Domain Adaptation (DLDUA) strategy to meet this challenge. Furthermore, we generate initial pseudo-labels for unknown objects using large models to recall more unknown objects. Experimental results show that our method significantly outperforms existing state-of-the-art (SOTA) methods in unknown object recall (improving by 15.2 U-Recall) and leads in inference speed by over 10 FPS compared to based on deformable DETR SOTA methods. Additionally, our method accelerates model convergence.

Monitoring the Mean Opinion Score (MOS) of video conferencing is critical for Internet Service Providers (ISPs) to ensure user satisfaction. However, a significant technical challenge arises: MOS is a subjective measure, while ISPs primarily rely on deep packet inspection (DPI) data for performance monitoring, making direct mapping between MOS and DPI data nearly impossible. To address this gap, we develop DePI-MOSE, a novel solution that leverages sub-application-level video context clues to build machine-learning models. By inferring the type of end devices and identifying the motion level within video content during a conference session, DePI-MOSE can estimate MOS values from DPI data accurately. We implemented and tested DePI-MOSE in a real-world ISP network, and experimental results show that DePI-MOSE is more accurate than state-of-the-art methods. We also built a network resource management platform for ISPs to dynamically adjust users' network resources by precisely monitoring users' video conferencing QoE.

Unsupervised domain adaptation (UDA) has been developed to transfer knowledge from a fully labeled source domain to an unlabeled target domain, mitigating domain shift. The presence of category shift has prompted the research in open-oet domain adaptation (ODA) and universal domain adaptation (UNDA). However, current ODA and UNDA models often treat all novel classes as a unified unknown class or set a priori number for novel classes, which fall short in the novel category discovery in domain adaptation (NCD in DA) problem. Accurately detecting the actual number of novel classes and learning their discriminative features in the DA process are two critical issues underexplored in the problem of NCD in DA. To address these issues, we propose a two-stage framework named Contrastive Adaptive Network (CAN) for the NCD in ODA and UNDA tasks. In Stage 1, we employ two types of contrastive learning for representation learning followed by k-means clustering with various values of k to search for the most probable novel class number. In Stage 2, we propose a known-unknown mapping protocol and group all the classes into several "known-unknown" pairs. Based on them, we design a novel Adaptive Classifier, the size of which is set according to the novel class number predicted in Stage 1. Then we design two tailored loss functions for the training of Adaptive Classifier to learn the discriminative features of novel classes while mitigating the domain shift of common classes, with source and target domain, respectively. Extensive experimental results demonstrate that CAN outperforms the performance of existing SOTA methods on ODA and UNDA problems and showcases significant improvements in the discrimination of novel class features. Implementation is available at https://github.com/yushengyuann/CAN.

Open-World Object Detection (OWOD) is a challenging task that requires the detector to identify unlabeled objects and continuously demands the detector to learn new knowledge based on existing ones. Existing methods primarily focus on recalling unknown objects, neglecting to explore the reasons behind them. This paper aims to understand the model's behavior in predicting the unknown category. First, we model the text attribute and the positive sample probability, obtaining their empirical probability, which can be seen as the detector's estimation of the likelihood of the target with certain known attributes being predicted as the foreground. Then, we jointly decide whether the current object should be categorized in the unknown category based on the empirical, the in-distribution, and the out-of-distribution probability. Finally, based on the decision-making process, we can infer the similarity of an unknown object to known classes and identify the attribute with the most significant impact on the decision-making process. This additional information can help us understand the behavior of the model's prediction in the unknown class. The evaluation results on the Real-World Object Detection (RWD) benchmark, which consists of five real-world application datasets, show that we surpassed the previous state-of-the-art (SOTA) with an absolute gain of 5.3 mAP for unknown classes, reaching 20.5 mAP. Our code is available at https://github.com/xxyzll/UMB.

The complexity required in the software development of machine learning (ML) applications introduces additional challenges to requirement engineering (RE) activities. RE researchers expressed concerns and the need for more discussions on RE for ML, requiring additional real-world case studies to evaluate RE activities for practical ML-enabled applications. This study aims to observe the RE activities for ML-enabled systems in a real-world context, taking action research in the ML-enabled FinTech project where the RE activities are being adjusted by engaging the data scientists to help and clarify ML-related requirements. This paper discussed the difficulties of RE activities from the perspectives of the data scientist and requirement engineer. Considering data and model relevance in developing the ML-enabled FinTech application, a RE framework iteratively made active changes according to the parameters is proposed, which includes the selected ML-related requirement characteristics to pursue and complete RE activities for ML-enabled application development. The feedback from the practitioners indicates that such practices address the difficulties of improving data quality and verifying model requirements in RE activities. The lessons learned by researchers and practitioners are also presented, which provides practical suggestions to the SE and RE communities with similar concerns in the related context.

Smart contracts vulnerability auditing is vitally critical to ensure transaction execution in normal on blockchain. The current data-driven approaches normally tokenize smart contracts into a series of sequences according to only one tokenization standard for vulnerability detection purpose, resulting some of the semantic contexts could not be reflected within restricted sequence length. To address this limitation, we generate sequences from smart contracts in three tokenization standards for which we utilize n-gram language model to capture semantic contexts respectively, and finally exploiting our effective combination strategy of Intersection or Union to integrate the audited results from multiple semantic contexts. In order to evaluate the proposed approach, we applied it on over 7200 Ethereum smart contract samples. Experimental result shows our proposed method is capable of detecting vulnerabilities and competitive with the baseline in test sets, with improved precision of over 44% when Intersection is applied in their results, as well as improved Recall measure up by over 300% and F-measure up by 220% when Union is applied. Our proposed method for smart contract vulnerability detection, an important tool for developing quality decentralized software applications, is able to analyze multiple semantic contexts and successfully detects more true vulnerabilities with high precision, outperforming that of the baseline approaches.

Due to the limited battery capacity of mobile devices, various CPU power governors and dynamic frequency adjustment schemes have been proposed to reduce CPU energy consumption. However, most such schemes are app-oblivious, ignoring an important fact that real-world applications often exhibit multiple execution phases that perform different functionality and may request different amounts of hardware resources. Having a unified app-level frequency setting for different phases of an application may not be energy efficient enough and may even violate the desirable latency performance required by certain phases. Motivated by this observation, in this paper, we present PIFA, which is an intelligent Phase Identification and Frequency Adjustment framework for energy-efficient and time-sensitive mobile computing. PIFA addresses two major challenges of fully automatically identifying different execution phases of an application and efficiently integrating the phase identification results for runtime frequency adjustment. We have fully implemented PIFA on the Android platform. An extensive set of experiments using real-world Android applications from multiple app categories demonstrate that PIFA achieves closely better performance than the desired latency requirement specified for each phase, while dramatically reducing energy consumption (e.g., >30% energy reduction for most apps) and incurring rather small runtime overhead (e.g., <;5% overhead for most apps).