Many LiDAR-based methods for detecting large objects, single-class object detection, or under easy situations were claimed to perform well. However, due to their failure to exploit image semantics, their performance in detecting small targets or under challenging conditions does not exceed that of fusion-based approaches. In order to elevate the detection performance in a complex environment, this paper proposes a multi-modal and multi-class 3D object detection network, named Voxel-Pixel Fusion Network (VoPiFNet). Within this network, we design a key novel component called the Voxel-Pixel Fusion Layer, which takes advantage of the geometric relation of a voxel-pixel pair and effectively fuses voxel features and pixel features with the cross-modal attention mechanism. Moreover, after considering the characteristics of the voxel-pixel pair, we design four parameters to guide and enhance this fusion effect. This proposed layer can be integrated with voxel-based 3D LiDAR detectors and 2D image detectors. Finally, the proposed method is evaluated on the public KITTI benchmark dataset for multi-class 3D object detection at different levels. Extensive experiments show that our method outperforms the state-of-the-art methods in detecting challenging pedestrian category and achieve promising performance in overall 3D mean average precision (mAP).

Advanced Driver Assistance Systems (ADAS) consists of two basic functions. One is the object detection for preventing vehicles from hitting pedestrians or other obstacles. The other is image segmentation for recognizing drivable areas and guiding the vehicle forward. For the latter, unlike those traditional image segmentation methods, image semantic segmentation based on deep learning architecture can handle the irregularly shaped road areas better, guiding a vehicle to drive in a more complex environment. With the popularity of Convolution Neural Networks (CNNs) in recent year, the traditional hand-crafted features methods have shown to be outperformed. However, deep CNN models are difficult to implement on vehicle application because the severe cost of time for complex processing. Although some proposed methods, such as Efficient neural network (Enet), achieved higher speed by removing some layers, it also led to the decrease of segmentation accuracy. In this research work, we propose a novel semantic segmentation network, Edgenet, which contains a class-aware edge loss module and a channel-wise attention mechanism, aiming to improve the accuracy with no harm to inference speed. We evaluate Edgenet on Cityscapes dataset, which is the most challenging and authoritative on-road semantic segmentation dataset. The results show that our proposed method can achieve over 70% mean IOU on Cityscapes test set and run at over 30 FPS in a single GTX Titan X (Maxwell) GPU.

Pedestrian detection is an important issue in the field of intelligent transportation systems. As a pedestrian is not an apparent object at nighttime, it brings about critical difficulties in effectively detecting a pedestrian for a driving assistant vision system. While using an infrared projector to enhance the illumination contrast, objects in a nighttime environment might reflect the infrared projected by the emitted spotlight. In some cases, however, the clothes on a pedestrian might absorb most of the infrared, thus causing the pedestrian to be partially invisible. To deal with this problem, a nighttime part-based pedestrian detection method is proposed. It divides a pedestrian into parts for a moving vehicle with a camera and a near-infrared lighting projector. Due to a high computation load, selecting effective parts becomes imperative. By analyzing the spatial relationship between every pair of parts, the confidence of the detected parts can be enhanced even when some parts are occluded. At the last stage of this system, the pedestrian detection result is refined by a block-based segmentation method. The system is verified by experiments, and the appealing results are demonstrated.

To robustly detect people in a video sequence is hard due to various challenges. One of the most successful discriminative features for finding people goes to the Histograms of Oriented Gradients (HOG). Although the major contour information is encoded in the HOG feature well, the background clutter disturbs the gradient information. Thus, an extension of HOG, called histograms of oriented gradient of granules (HOGG), is proposed. Instead of collecting gradient information at each pixel, the histograms of gradients in small regions are computed. HOGG with different granularity can describe the contour while ignoring the noisy edges. Moreover, the clutter background problem can be solved by encoding extra region information. With the help of the integral image technique, the evaluation of HOGG can be efficient. The final HOG+HOGG classifier obtains 92% detection rate at 10-4false positive per window in the experiments.

This work investigates a semantic-driven human detection algorithm, which employs global human template matching to inspire the local features based Adaboosting algorithm. We use distance transform to analyze distances between training samples and human contour template to obtain a classifier based on human outline features. At the training stage, the global outline feature will be coordinated into the Adaboost framework to guide the learning of a set of HOGs local classifiers. In other words, we make a stronger human classifier by dynamically tuning the hyper-plane of the support vector machine to combine both global and local features for attaining a better accuracy. A global fused error rate is also proposed to enhance the modified Adaboost iterative calculation such that a human detector called strong classifier can be obtained. The experiments illustrate that the detection rate received from the presented framework is above 20% better than original HOGs-based human detector.

Changing lanes while having no information about the blind spot area can be dangerous. We propose a vision-based vehicle detection system for a lane changing assistance system to monitor the potential sedan vehicle in the blind-spot area. To serve our purpose, we select adequate features, which are directly obtained from vehicle images, to detect possible vehicles in the blind-spot area. This is challenging due to the significant change in the view angle of a vehicle along with its location throughout the blind-spot area. To cope with this problem, we propose a method to combine two kinds of part-based features that are related to the characteristics of the vehicle, and we build multiple models based on different viewpoints of a vehicle. The location information of each feature is incorporated to help construct the detector and estimate the reasonable position of the presence of the vehicle. The experiments show that our system is reliable in detecting various sedan vehicles in the blind-spot area.

This paper presents a probabilistic approach for automatically segmenting foreground objects from a video sequence. In order to save computation time and be robust to noise effects, a region detection algorithm incorporating edge information is first proposed to identify the regions of interest, within which the spatial relationships are represented by a region adjacency graph. Next, we consider the motion of the foreground objects and, hence, utilize the temporal coherence property in the regions detected. Thus, the foreground segmentation problem is formulated as follows. Given two consecutive image frames and the segmentation result priorly obtained, we simultaneously estimate the motion vector field and the foreground segmentation mask in a mutually supporting manner by maximizing the conditional joint probability density function of these two elements. To represent the conditional joint probability density function in a compact form, a Bayesian network is adopted, which is derived to model the interdependency of these two elements. Experimental results for several video sequences are provided to demonstrate the effectiveness of the proposed approach.

We introduce an Augmented Histograms of Oriented Gradients (AHOG) feature for human detection from a nonstatic camera. We increase the discriminating power of original Histograms of Oriented Gradients (HOG) feature by adding human shape properties, such as contour distances, symmetry, and gradient density. Based on the biological structure of human shape, we impose the symmetry property on HOG features by computing the similarity between itself and its’ symmetric pair to weight HOG features. After that, the capability of describing human features is much better than the original one, especially when the humans are moving across. We also augment the gradient density into features to mitigate the influences caused by repetitive backgrounds. In the experiments, our method demonstrates most reliable performance at any view of targets.

This paper presents a new approach to automatic segmentation of foreground objects from an image sequence by integrating techniques of background subtraction and motion-based foreground segmentation. First, a region-based motion segmentation algorithm is proposed to obtain a set of motion-coherence regions and the correspondence among regions at different time instants. Next, we formulate the classification problem as a graph labeling over a region adjacency graph based on Markov random fields (MRFs) statistical framework. A background model representing the background scene is built and then is used to model a likelihood energy. Besides the background model, a temporal coherence is also maintained by modeling it as the prior energy. On the other hand, color distributions of two neighboring regions are taken into consideration to impose spatial coherence. Then, the a priori energy of MRFs takes both spatial and temporal coherence into account to maintain the continuity of our segmentation. Finally, a labeling is obtained by maximizing the a posteriori energy of the MRFs. Under such formulation, we integrate two different kinds of techniques in an elegant way to make the foreground detection more accurate. Experimental results for several video sequences are provided to demonstrate the effectiveness of the proposed approach.

In this paper, the problem of human pose estimation is formulated as a multi-objective optimization problem so as to fuse multiple cues more properly, which is in contrast with the hypothesis that the cues are mutually independent so that their consolidation can be solely through the product of their individual likelihood distributions. An evolutionary algorithm for optimizing the defined objectives optimization is applied to evolve a set of non-dominated alternative solutions, known as the Pareto-optimal set. For convergence improvement of the evolutionary algorithm, the DD-MCMC method is used to generate a set of good initial solutions. Evaluating solutions by using relative dominant relation rather than quantitative absolute difference in value makes the solution exploration not dominated by the poor cue and result in more effective solutions for further decision making. Experimental results to the images obtained from different scene are provided to demonstrate the effectiveness and efficiency of our proposed framework

This paper describes a 5/spl times/5 window-based, power-of-two approximation algorithm for 2D Gaussian smoothing filter and its generic hardware reference design. With a small change in standard deviation /spl sigma/, the Gaussian smoothing filter is able to provide different levels of smoothing and noise reduction capability, which is highly desirable in the early stage of an image processing flow. By using power-of-two terms, the digital-approximated 2D Gaussian filter can be implemented by simple hardware shifters. A hardware reference design is also proposed to achieve such functionality. Experiments in using the proposed filter with an existing edge detection algorithm show the flexibility and effectiveness of the proposed smoothing mask. The synthesized hardware of such combination shows a 200MHz high operating frequency under UMC 0.18/spl mu/m technology, which proves the proposed filter is able to achieve the real-time high frame-rate and high-resolution video processing requirements.

This paper presents a probabilistic approach for automatically segmenting foreground objects from a video sequence. A Bayesian network is presented to model the interactions among three variables, such as foreground segmentation mask, motion segmentation field, and motion vector held. Given two consecutive images, the conditional joint probability density of the three variables is maximized iteratively to simultaneously achieve foreground segmentation and motion segmentation in a mutually beneficial manner. The solution to the optimization problems are obtained by using iterative conditional mode (ICM) and graph cut algorithm. Incorporating motion information with background subtraction technique makes the segmentation perform in a more semantic level and obtain more accurate results. Experimental results for two video sequences are provided to demonstrate the effectiveness of the proposed approach.

Lane departure warning systems are expected to improve safety on the roads. In the present paper, we proposed the design of a handheld real-time lane departure warning system. Such a device can be used on every road vehicle and dramatically improve safety. In order to attain our goal, we adopted the technology of embedded systems. In recent years, the enhancement of embedded calculating technology has allowed the performance of embedded systems to gain enough capability to carry out the huge calculations for lane departure warning algorithms. Finally, we implemented a real-time lane departure warning system on an image-processing platform that we also developed and which can be validated both in actual road tests and on a small-sized PCB board.

This paper presents a new approach to automatic segmentation of the foreground objects from the sequence of images by integrating techniques of background subtraction and motion-based segmentation. At first, a background model is built to represent information of both color and motion of the background scene. Based on temporal and spatial information, an initial partition of each image is obtained. Next, we formulate the classification problem as a graph labeling over a region adjacency graph (RAG) based on Markov random fields (MRFs) statistical framework. The Bhattacharyya distance for estimating the similarity between color and motion distributions of the background model and the currently obtained regions are used to model the likelihood energies. The object tracking strategy for finding the correspondence between region at different time instant is used to maintain the temporal coherence of the segmentation. For spatial coherence, the length of the common boundaries of two regions is taken into consideration for classification. Both spatial and temporal coherence are incorporated into the prior energy to maintain the continuity of the segmentation. Finally, a labeling is obtained by maximizing a posterior probability of the MRFs. Under such formulation, we integrate two different kinds of framework in an elegant way to make the foreground detection become more accurate. Experimental results for two image sequences including the hall monitoring and our e-home demo site are provided to demonstrate the effectiveness of the proposed approach.

The objectives of this research are to develop a driving assistance system that can locate the positions of the lane boundaries and detect the existence of the front-vehicle. By providing warning mechanism, the system can protect drivers from dangerousness. In lane recognition, Gaussian filter, peak-finding procedure, and line-segment grouping procedure are used to detect land markers successfully and effectively. On the other hand, vehicle detection is achieved by using three features, such as underneath, vertical edge, and symmetry property. The proposed system is shown to work well under various conditions on the roadway. The vehicle detection rate is higher than 97%. Besides, the computation cost is inexpensive and the system's response is almost real time. Thus, the results of the present research work can improve traffic safety for on-road driving.

This work presents a region-based approach for background modeling to economize the use of space. In order to be immune to noise and changes resulting from illumination or motion, the partial directed Hausdorff distance is adopted while subtracting the foreground objects from the scene robustly. Instead of determining the threshold values manually, we use an adaptive method to automatically choose the threshold values. Finally, experimental results of applying our approach on a sequence of an indoor scene are provided to demonstrate the effectiveness of the proposed method.

Query-based learning (QBL) has been introduced for training a supervised network model with additional queried samples. Experiments demonstrated that the classification accuracy is further increased. Although QBL has been successfully applied to supervised neural networks, it is not suitable for unsupervised learning models without external supervisors. In this paper, an unsupervised QBL (UQBL) algorithm using selective-attention and self-regulation is proposed. Applying the selective-attention, we can ask the network to respond to its goal-directed behavior with self-focus. Since there is no supervisor to verify the self-focus, a compromise is then made to environment-focus with self-regulation. In this paper, we introduce UQBL1 and UQBL2 as two versions of UQBL; both of them can provide fast convergence. Our experiments indicate that the proposed methods are more insensitive to network initialization. They have better generalization performance and can be a significant reduction in their training size.

In this paper, a three-layer force-directed self-organizing map is designed to resolve the circuit placement problem with arbitrarily shaped rectilinear modules. The proposed neural model with an additional hidden layer can easily model a rectilinear module by a set of hidden neurons to correspond the partitioned rectangles. With the collective computing from hidden neurons, these rectilinear modules can correctly interact with each other and finally converge to a good placement result. In this paper, multiple contradictory criteria are accounted simultaneously during the placement process, in which, both the wire length and the module overlap are reduced. The proposed model has been successfully exploited to solve the time consuming rectilinear module placement problem. The placement results of real rectilinear test examples are presented, which demonstrate that the proposed method is better than the simulated annealing approach in the total wire length. The appropriate parameter values which yield good solutions are also investigated.

For a building block placement, the routing space will be fully defined into channels and switchboxes because the definition of switchboxes releases all the cycles in a channel precedence graph and further yields a safe routing ordering. However, switchbox routing is more difficult than channel routing. Due to the requirements in the routing phase, it is important for us to minimize the number of switchboxes in the definition of channels and switchboxes. In this paper, a region definition and ordering assignment (RDAOA) algorithm on minimizing the number of switchboxes is proposed. First, the routing space is modeled as a floorplan graph. According to the routing precedence in one "T" type junction, the graph is further transformed into a channel precedence graph. Hence, the problem of minimizing the number of switchboxes in region definition will correspond to the minimum feedback vertex set (MFVS) problem in the channel precedence graph. Second, based on the cyclic properties in a channel precedence graph, the MFVS problem is solved by the minimal-cycle phase and the long-cycle phase. All the minimal cycles and most of the long cycles are broken in the minimal-cycle phase, and the remaining long cycles are further broken in the long-cycle phase. As a result, fewer switchboxes are defined, and these channels and switchboxes are assigned to guarantee a safe routing ordering. The time complexity of the algorithm is proved to be in O(n) time, where n is the number of line segments in a given floorplan graph. Finally, several examples have been tested on the proposed algorithm and other published algorithms, and the experimental results show that our algorithm defines fewer switchboxes than other algorithms.

In this paper, two routability crossing distribution problems based on the non-crossing relations, vertical constraint relations and geometry relations are proposed to improve routing performance of one T-type junction region. For the routability problem, a routability ordering graph can be built to decide a net ordering on the boundary in O(n/sup 2/) time. Furthermore, for the routability quota problem, if the number of crossings for the routability problem is more than the quota, the net ordering in the routability problem must be adjusted by a net interchange operation to satisfy the quota requirement in the routability quota problem in O(n) time. Since a net ordering is obtained in the routability problem or the routability quota problem, the global nets will be assigned onto the boundary in O(n) time by interleaving vacant terminals between any pair of global nets for the floating terminal assignment of the boundary.>

In this paper, a three-layer self-organizing neural network is designed to resolve the cell placement problem with arbitrarily-shaped rectilinear cells. The proposed model with additional hidden layer can easily model the rectilinear cells by a set of hidden neurons which correspond to the partitioned rectangles of cells, called "molecule model". With the collective computing property and solid-connection of neural networks, these rectilinear cells can correctly interact with each other and finally converge to a good placement result. The placement results of rectilinear test examples have been presented In this paper, multiple contradictory criteria are accounted simultaneously during the placement process, in which, total wire length is reduced without module overlap.>

We present a new efficient algorithm for both two and three-layer over-the-cell channel routing in the standard cell design technology. Our approach considers both density distribution in the channel and longest path in vertical constraint graph. Besides, we use vacant terminals to eliminate cycles in the vertical constraint graph as well as to reduce the maximum cliques in the horizontal constraint graph for selecting net segments to be routed over the cells. For the PRIMARY 1 benchmark examples, our router reduced the total channel height by 39.1% and 61.0% for two-layer and three-layer routing model, respectively.>

For a building block placement, the routing space can be partitioned into channels and switchboxes. The definition of switchboxes can release the cyclic channel precedence constraints and further yield a safe routing ordering process. However, switchbox routing is always more difficult than channel routing. In this paper, an O(nlogN) region definition and ordering assignment algorithm is proposed to minimize the number of switchboxes, where N is the number of vertices in a channel precedence graph. Several examples have been tested on this algorithm, the experimental results are listed and compared.>

An efficient algorithm for over-the-cell routing in the standard cell layout design technology is presented. Two variations are discussed: one aims to minimize the channel density with fewest tracks over the cells while the other aims to minimize the final channel width. The algorithm can fit both the two-layer and three-layer routing models. With the two-layer model, there is a single routing layer over the cells for intercell connections. With the three-layer model, there are two disjoint routing layers over the cells for intercell connections. In this approach, the problem is decomposed into two phases: (1) over-the-cell routing and (2) conventional channel routing. The over-the-cell routing phase, which is executed iteratively, consists of two steps, routing over the cells and choosing net segments within the channel. For each iteration in the over-the-cell routing phase, the algorithm removes a net or a subnet which intersects the column with the highest column density and routes it over the cells according to some prioritized criteria. In comparison with the previous researches, this approach achieved the best effectiveness and has used the least CPU-time.>

The authors present a novel plane-sweep algorithm based on spatial data structures with region query operations. Such an algorithm is applicable to the problems of VLSI layout design and image processing. It has a computing time of O(N log N) and allows the functional operations of region search of the spatial data structures to be useful in solving some of the specified problems. This algorithm has been successfully implemented in C language; it was based on two kinds of spatial data structures; multiple storage quad tree and quad list quad tree. This plane-sweep algorithm also has been successfully applied to problems of layout compaction, design rule checking, and minimum reliable partition.>

A graph-generating algorithm and the experimental results of a hierarchical mask-layout-compaction scheme based on a plane-sweep algorithm, a fast region-query and a space-efficient data structure called the hierarchical multiple-storage quad tree are presented. For a mask-layout design, a rectangle is used as the primary element of the layout. Hence, in the hierarchical mask-compaction scheme, the graph-generating algorithm is based on the edges of rectangles rather than the central lines of symbols for the symbolic-compaction design. The plane-sweep algorithm is also called a dynamic event scheduling algorithm and can be applied to solve some other problems in the field of computational geometry and image processing. The efficiencies of the plane-sweep algorithm and the graph-generating algorithm are dependent on the region-query operations of the spatial data structure. By using the improved multiple storage quad tree as the spatial data structure in the system, the mask-layout compactor has been accomplished in a practically linear time performance in terms of the rectangles in the source layout.>

A rule-based system, to be used in place of the conventional approaches using algorithms, is proposed to explore the feasibility of using expert system technology to automate the compaction process by reasoning about the layout design using the sophisticated expert rules contained in its knowledge base. It has been demonstrated that, given a process-independent mask layout, the design rules, user specified constraints, and circuit electrical requirements can be checked over and rearranged into an optimized state. The system considered consists of approximately 100 Kbytes of source code and contains 120 rules. The compactor is written in a combination of the Lisp and OPS5 languages.>