Camera-based 3D semantic scene completion (SSC) provides dense geometric and semantic perception for autonomous driving. However, images provide limited information making the model susceptible to geometric ambiguity caused by occlusion and perspective distortion. Existing methods often lack explicit semantic modeling between objects, limiting their perception of 3D semantic context. To address these challenges, we propose a novel method VLScene: Vision-Language Guidance Distillation for Camera-based 3D Semantic Scene Completion. The key insight is to use the vision-language model to introduce high-level semantic priors to provide the object spatial context required for 3D scene understanding. Specifically, we design a vision-language guidance distillation process to enhance image features, which can effectively capture semantic knowledge from the surrounding environment and improve spatial context reasoning. In addition, we introduce a geometric-semantic sparse awareness mechanism to propagate geometric structures in the neighborhood and enhance semantic information through contextual sparse interactions. Experimental results demonstrate that VLScene achieves rank-1st performance on challenging benchmarks—SemanticKITTI and SSCBench-KITTI-360, yielding remarkably mIoU scores of 17.52 and 19.10, respectively.

Text-driven hair editing on 3D heads is a challenging problem in computer vision and graphics. In this paper, we propose TextHair3D, a NeRF-based text-driven 3D hair editing method that uses 3D perception to generate priors, edit hair attributes from user-provided text, and preserve facial features. TextHair3D uses the Contrastive Language-Image Pre-training (CLIP) model to encode textual conditions. To address the complexity and roughness of local editing, we design a combined conditional mapping module to map image and text conditions into latent space for learning generative priors. This enables high-quality, photo-realistic hair editing and 3D head reproduction. Extensive experiments show Tex-tHair3D’s superiority in visual realism and attribute accuracy.

Creating high-quality 3D object representations from a single-view image is challenging. Existing methods tend to infer the geometry and texture information simultaneously within a shared network. However, decoding geometry and texture from a unified network often leads to their entanglement, causing geometric structure collapse or floating artifacts. After revisiting this task, we propose a single-view reconstruction framework based on 3D Gaussian Splatting. The key idea is to decouple Gaussian position attribute generation from texture feature generation. Technically, our framework combines a Geometry Generator, a Texture Generator, and a Gaussian Attributes Decoder. Two parallel branches, Geometry Generator and Texture Generator, aim for point cloud prediction and texture optimization, respectively. Then the Gaussian Attributes Decoder integrates the generated position and texture attributes into a coherent Gaussian point cloud, facilitating efficient novel view synthesis. Extensive qualitative and quantitative evaluations of public datasets demonstrate that our method consistently outperforms existing methods in terms of reconstruction quality and inferring efficiency.

Point cloud completion aims to reconstruct complete shapes from partial observations. Although current methods have achieved remarkable performance, they still have some limitations: Supervised methods heavily rely on ground truth, which limits their generalization to real-world datasets due to the synthetic-to-real domain gap. Unsupervised methods require complete point clouds to compose unpaired training data, and weakly-supervised methods need multi-view observations of the object. Existing self-supervised methods frequently produce unsatisfactory predictions due to the limited capabilities of their self-supervised signals. To overcome these challenges, we propose a novel self-supervised point cloud completion method. We design a set of novel self-supervised signals based on multi-view augmentations of the single partial point cloud. Additionally, to enhance the model’s learning ability, we first incorporate Mamba into self-supervised point cloud completion task, encouraging the model to generate point clouds with better quality. Experiments on synthetic and real-world datasets demonstrate that our method achieves state-of-the-art results.

3D Semantic Scene Completion (SSC) provides comprehensive scene geometry and semantics for autonomous driving perception, which is crucial for enabling accurate and reliable decision-making. However, existing SSC methods are limited to capturing sparse information from the current frame or naively stacking multi-frame temporal features, thereby failing to acquire effective scene context. These approaches ignore critical motion dynamics and struggle to achieve temporal consistency. To address the above challenges, we propose a novel temporal SSC method FlowScene: Learning Temporal 3D Semantic Scene Completion via Optical Flow Guidance. By leveraging optical flow, FlowScene can integrate motion, different viewpoints, occlusions, and other contextual cues, thereby significantly improving the accuracy of 3D scene completion. Specifically, our framework introduces two key components: (1) a Flow-Guided Temporal Aggregation module that aligns and aggregates temporal features using optical flow, capturing motion-aware context and deformable structures; and (2) an Occlusion-Guided Voxel Refinement module that injects occlusion masks and temporally aggregated features into 3D voxel space, adaptively refining voxel representations for explicit geometric modeling. Experimental results demonstrate that FlowScene achieves state-of-the-art performance, with mIoU of 17.70 and 20.81 on the SemanticKITTI and SSCBench-KITTI-360 benchmarks.

The vision-based semantic scene completion task aims to predict dense geometric and semantic 3D scene representations from 2D images. However, the presence of dynamic objects in the scene seriously affects the accuracy of the model inferring 3D structures from 2D images. Existing methods simply stack multiple frames of image input to increase dense scene semantic information, but ignore the fact that dynamic objects and non-texture areas violate multi-view consistency and matching reliability. To address these issues, we propose a novel method, CDScene: Vision-based 3D Semantic Scene Completion via Capturing Dynamic Representations. First, we leverage a large multi-modal model to extract 2D explicit semantics and align them into 3D space. Second, we exploit the characteristics of monocular and stereo depth to decouple scene information into dynamic and static features. The dynamic features contain structural relationships around dynamic objects, and the static features contain dense contextual spatial information. Finally, we design a dynamic-static adaptive fusion module to effectively extract and aggregate complementary features, achieving robust and accurate semantic scene completion in autonomous driving scenarios. Extensive experimental results on the SemanticKITTI, SSCBench-KITTI360, and SemanticKITTI-C datasets demonstrate the superiority and robustness of CDScene over existing state-of-the-art methods.

Vision-based semantic scene completion task aims to predict dense geometric and semantic 3D scene representations from 2D images. However, 3D modeling from a single view is an ill-posed problem, limited by the field of view and occlusion problems caused by image input. Moreover, existing methods tend to produce erroneous scene hallucinations and overly smooth boundary segmentation due to a lack of information. To address this problem, we propose MixSSC, which mixes the sparsity of forward projection with the denseness of depth-prior backward projection. The aim is to use sparse features to fill information-poor regions and dense features to enhance visible regions. Specifically, we develop the forward-backward mixture module, which enables the generation of scene mixture voxel representation by leveraging the benefits of both forward and backward projection. Subsequently, we design the semantic-spatial fusion module, which utilizes a coarse-to-fine approach to process mixture voxel features at the semantic-spatial level. Extensive experimental results on the SemanticKITTI, SSCBench-KITTI-360 and nuScenes datasets demonstrate the superiority of MixSSC. Our code is available on https://github.com/willemeng/MixSSC.

Constraints may scatter the Pareto optimal solutions of a constrained multiobjective optimization problem (CMOP) into multiple feasible regions. To avoid getting trapped in local optimal feasible regions or a part of the global optimal feasible regions, a constrained multiobjective evolutionary algorithm (CMOEA) should consider both the escape force and the expansion force carefully during the search process. However, most CMOEAs fail to provide these two forces effectively. As a remedy for this limitation, this article proposes a method called TPEA. TPEA maintains three populations, termed Pop1, Pop2, and Pop3. Pop1 is a regular population, updated with a constrained NSGA-II variant. Pop2 and Pop3 are two auxiliary populations, containing the innermost and outermost nondominated infeasible solutions, respectively. The analysis reveals that these two types of nondominated infeasible solutions can contribute to the generation of escape and expansion forces, respectively. Due to these two forces, TPEA is likely to identify more global optimal feasible regions, which is crucial for constrained multiobjective optimization. Also, a mating selection strategy is developed in TPEA to coordinate the interaction among these three populations. Extensive experiments on 58 benchmark CMOPs and 35 real-world ones demonstrate that TPEA is significantly superior or comparable to six state-of-the-art CMOEAs on most test instances.

Long-term occlusion is one of the most formidable challenges in Multi-Object Tracking (MOT). The motion models of existing SORT-based trackers are unreliable in estimating the motion states of long-term occluded targets. This is mainly because as the occlusion period increases, the increases speed of estimation errors in the motion model increases faster. In practical applications, we believe that the estimation error of the tracker during long-term occlusion is mainly concentrated in the estimation error of the motion model on the velocity of the occluded target. In this work, we have demonstrated that in the long-term occlusion period, appropriately correcting the estimated values of the motion model on the target motion velocity and fully utilizing the temporal and attribute information of the target’s historical trajectory as calculation indicators of correlation are beneficial for improving the robustness of the tracker in long-term occlusion. We refer to our proposed motion correction-based framework as MC-SORT, which mainly consists of a Momentum Compensation Module (MCM) and a Backtracking Re-association (BRA) module. The former can correct the estimated value of the target’s motion state during long-term occlusion, the latter uses the temporal and attribute information of the target’s historical trajectory during long-term occlusion as correlation indicators to measure the degree of correlation between the target and trajectory. Our proposed MC-SORT has the characteristics of simplicity, online, real-time, and plug-and-play, particularly improving the robustness of the tracker in long-term occlusion. The extensive experimental results on the MOT17 and MOT20 datasets demonstrate the robustness and superiority of our framework.

Task assignment is a fundamental problem in spatial crowdsourcing. In many spatial crowdsourcing platforms, such as Didi, AMAP, and Uber, there are hybrid tasks, including real-time and reservation-type tasks, which are with different constraints and unevenly distributed in spatial and temporal. For these hybrid tasks, most existing studies suffer from low task completion rate and low profit for two reasons: firstly, they focus on homogeneous tasks with uniform constraints, and assign hybrid tasks separately; secondly, they cannot effectively address the uneven distribution of hybrid tasks. Inspired by this, we delve into the problem of online hybrid task assignment (HyTAO) with the goal of maximizing total revenue by simultaneously assigning both real-time and reservation-type tasks online for the first time. We prove the NP-hardness of the offline version of the HyTAO problem. To solve HyTAO effectively, we utilize a cross-platform cooperation model to tackle the challenge of non-uniform distribution. Following this, we design a binary tree-based search algorithm, namely BTS, which is capable of uniformly processing various types of tasks and quickly searching for available workers. Additionally, we discuss the parallel optimization strategies of BTS. To further enhance performance, we develop TBTS, which identifies tasks with high increased revenue based on a threshold. Finally, we conduct a comprehensive analysis of the complexity and competitive ratio of both BTS and TBTS. Extensive experiments are performed to demonstrate the efficiency of our approaches.

We propose DeformingNet, an effective 3D point cloud completion network. Unlike existing methods that complete partial point cloud by directly learning the morphing function from 2D grids to 3D shapes, which limits the model’s inference capability due to the intrinsic gaps between feature spaces with different dimensions, we design a deforming-based point generator that emulates the deforming from multiple uniform 3D priors (i.e., pre-defined 3D point clouds in cube shape) into 3D shapes. In addition, we design a MAE-based encoder, which introduces the MAE encoder from point-MAE pre-trained on ShapeNet dataset to learn the correlation of local regions and fuse local and global features to enrich the latent representation. The 3D shapes generated by DeformingNet have both accurate local details and faithful global structure with less noise. Experiments demonstrate that our DeformingNet outperforms state-of-the-art methods in terms of quantitative metrics and visual quality.

Remarkable breakthroughs but daunting complexities of deep learning have aroused widespread interest in dedicated deep neural network (DNN) acceleration hardware, among which optical accelerators (OAs) are particularly promising thanks to their unprecedentedly high-performance-per-watt. However, the development of OAs is much slower than that of electrical accelerators due to threefold challenges. First, the OA design space is ample and discrete, making it tough for OA optimization; Second, the ecosystem that facilitates OA development is still in its infancy. Techniques to support OA design remain less explored, limiting both the achievable performance and the innovative development of OAs; and Third, OAs are highly sensitive to fabrication-induced process variations and thermal fluctuations (i.e., PTVs), which degrades OAs’ inference robustness and even renders them unusable in practice. In this article, we develop AutOAS, the first-of-its-kind framework for Automated Optical Accelerator Search, in order to jointly boost acceleration efficiency, inference robustness, and development speed. Our AutOAS comprises four enabling components: 1) a holistic OA search space, which takes full consideration of OAs’ micro-architectures (e.g., the type, shape and size of core functional units for data computation and data access), dataflow choices, DNN-to-accelerator mapping methods, memory hierarchy and PTV mitigation techniques; 2) a PTV Regulator, which can emulate the impact of PTVs on OAs’ inference accuracy based on given PTV profiles, and enables energy-efficient PTV mitigation on OAs; 3) an O-Performance Predictor, which enables accurate yet efficient predictions of an OA’s energy, throughput (latency) and chip area according to the DNN model and OA architecture parameters; and 4) two O-Search Engines (i.e., a differentiable search engine and an evolutionary search engine), which can automatically explore the large design space of OAs and identify the optimal accelerators to maximize the acceleration targets. Based on 10 DNN models widely applied in both computer vision and sequence modeling tasks, extensive experiments and ablation studies validate the effectiveness of our PTV Regulator, O-Performance Predictor, and O-Search Engines, as well as the superior performance of AutOAS-generated OAs.

3-D placement is an important stage in 3-D physical synthesis. In addition to the need to place the standard cells or Macros inside the die, the placement of interdie connections also needs to be considered. As the density of the interdie connections gradually increases, their placement becomes more critical. However, unlike standard cells, the interdie connection does not need to be aligned into rows, which leads to a larger legalization solution space. Legalization aims at minimizing the total and maximum displacement to maintain the quality of global placement on the premise of no violation of the physical circuit constraint. In this article, we propose a two-stage legalization approach (NGLIC) for nonaligned-row interdie connections. In the initial stage, a single-row height legalization algorithm is used for dense placement. Afterward, the unequal multirow height legalization (UML) is designed for sparse placement in the post-optimization stage. A pruning scheme is adopted to identify and eliminate redundant computations. The performance, effectiveness, and configuration are analyzed empirically based on ICCAD 2022 benchmarks. Compared to the state-of-the-art multirow or single-row height legalization, our approach outperforms well-known algorithms, such as multirow global legalization (MGL) and Abacus by at least 24% averaged total displacement, 3% averaged maximum displacement, and 31% averaged HPWL Growth. Our case study also illustrates the effectiveness of UML and NGLIC. In addition, the effectiveness of pruning is also validated by experiments that show savings of 33% redundant computations.

The goal of constrained multiobjective evolutionary optimization is to obtain a set of well-converged and well-distributed feasible solutions. To achieve this goal, a delicate tradeoff must be struck among feasibility, diversity, and convergence. However, balancing these three elements simultaneously through a single tradeoff model is nontrivial, mainly because the significance of each element varies in different evolutionary phases. As an alternative approach, we adapt distinct tradeoff models in various phases and introduce a novel algorithm named adaptive tradeoff model with reference points (ATM-R). In the infeasible phase, ATM-R takes the tradeoff between diversity and feasibility into account, aiming to move the population toward feasible regions from diverse search directions. In the semi-feasible phase, ATM-R promotes the transition from "the tradeoff between feasibility and diversity" to "the tradeoff between diversity and convergence." This transition is instrumental in discovering an adequate number of feasible regions and accelerating the search for feasible Pareto optima in succession. In the feasible phase, ATM-R places an emphasis on balancing diversity and convergence to obtain a set of feasible solutions that are both well-converged and well-distributed. It is worth noting that the merits of reference points are leveraged in ATM-R to accomplish these tradeoff models. Also, in ATM-R, a multiphase mating selection strategy is developed to generate promising solutions beneficial to different evolutionary phases. Systemic experiments on a diverse set of benchmark test functions and real-world problems demonstrate that ATM-R is effective. When compared to eight state-of-the-art constrained multiobjective optimization evolutionary algorithms, ATM-R consistently demonstrates its competitive performance.

Monocular depth estimation has made considerable advances under clear weather conditions. However, how to learn accurate scene depth under rain and fog conditions and alleviate the negative influence of occlusion, light, visibility, etc., is an open problem. To address this problem, in this article, we split the adverse weather depth estimation network into two subbranches: the depth prediction branch and the masked encoding branch. The depth prediction branch is used for depth estimation. The masked encoding branch, inspired by masked image modeling, uses random masks to simulate occlusion or low visibility often seen in rain and fog, forcing this branch to learn to infer the prediction of masked regions from the context. In order to make the masked encoding better enhance the depth prediction, we designed the mask feature fusion module, which can fuse the depth and spatial context features of the two branches to produce a fine-level depth map. The experimental results on the Foggy Cityscapes and RainCityscapes datasets demonstrate that our method achieves state-of-the-art performance, significantly outperforming previous methods across all evaluation metrics.

The construction of undetectable adversarial examples with few perturbances remains a difficult problem in adversarial attacks. At present, most solutions use the standard gradient optimization algorithm to build adversarial examples by applying global perturbations to benign samples and then launch attacks on the targets (e.g., face recognition systems). However, when the perturbance size is limited, the performance of these approaches suffers substantially. The content of crucial places in an image, on the other hand, will impact the final prediction; if these areas can be investigated and limited perturbances introduced, an acceptable adversarial example will be constructed. Based on the foregoing research, this article offers a dual attention adversarial network (DAAN) to produce adversarial examples with limited perturbations. DAAN initially searches for effective areas in an input image using the spatial attention network and channel attention network, and then creates space and channel weights. Following that, these weights direct an encoder and a decoder to generate effective perturbation, which is then combined with the input to produce an adversarial example. Finally, the discriminator determines if the created adversarial examples are true or false, and the attacked model is utilized to determine whether the generated samples fit the attack targets. Extensive studies on various datasets show that DAAN not only delivers the best attack performance across all comparison algorithms with few perturbations, but it can also significantly improve the defensiveness of the attacked models.

Multiobjective-based constraint-handling techniques are popular in evolutionary constrained single-objective optimization. However, most of these techniques run into troubles when dealing with constrained multiobjective optimization problems (CMOPs). That is, they have difficulty optimizing too many objective functions, are ineffective in maintaining population diversity, or are challenged in establishing appropriate additional objective functions. As a remedy to these limitations, we propose a novel technique called NRC for handling CMOPs. The novelty of NRC lies in its three sorting procedures: 1) nondominated sorting; 2) reversed nondominated sorting; and 3) constrained crowding distance sorting, which are performed in sequence to provide driving forces toward the Pareto front (PF) of a transformed unconstrained multiobjective optimization problem (treating the overall constraint violation as an additional objective function), the boundary front, and the constrained PF, respectively. With the combination of these three different forces, NRC can conveniently approach the desired PF from diverse search directions. The effectiveness of NRC is experimentally verified. Also, we incorporate NRC into a two-archive mechanism and develop a novel constrained multiobjective evolutionary algorithm, called NRC2. Comprehensive experiments on 49 benchmark CMOPs and 21 real-world ones demonstrate that NRC2 is significantly superior or comparable to six state-of-the-art constrained evolutionary multiobjective optimizers on most test instances.

MapReduce offers a promising programming model for big data processing. One significant issue in practical applications is data skew, its an important reason for the emergence of stragglers which makes the data assigned to each reducer imbalance. This paper presents CSRA, an efficient resource allocation algorithm in MapReduce considering data skew. CSRA aims at reducing the running time and coefficient of variation by reordering the task list and splitting the big clusters. Through thinking over the actual status of tasks, this method largely squares up the resource utilization. After we implement CSRA in Hadoop, the experiments show that CSRA has negligible overhead and can speed up the execution time of some popular applications obviously.