Multiple mobile device-based collaborative computing emerges with the rapid proliferation of various smart mobile devices such as smartphones and tablets, which provide always-on connectivity, information and communication. However, due to severe resource poverty and poor network connectivity, lots of traditional embedded electronic devices with attracting features cannot be incorporated into this computing paradigm conveniently. In this paper, an uneven-dual-core SoC, which integrates a CPU core and a MCU core on a single chip with multiple operating system support, is proposed to realize loosely-coupled multiple heterogeneous device collaboration. A network file system, MRFS (Multi-client Raindrop File System), and FAT-X (File Allocation Table eXtension) are also proposed to provide client-centric cross-device data consistency and virtual file access respectively. Comprehensive mobile services are enabled by offloading appropriate tasks from existing smart mobile devices to involved traditional embedded devices. The SoC is implemented onto a 16.65 mm2silicon with 65 nm CMOS technology. This paper also presents three typical applications to illustrate the universality and huge potential for innovative usage model of the proposed system.

Single instruction multiple data (SIMD) execution is in no doubt an efficient way to exploit the data level parallelism in image and video applications. However, SIMD execution bottlenecks must be tackled in order to achieve high execution efficiency. We first analyze in this paper the implementation of two major kernel functions of H.264/AVC namely, SATD and subpel interpolation, in conventional SIMD architectures to identify the bottlenecks in traditional approaches. Based on the analysis results, we propose a new SIMD architecture with two novel features: 1) parallel memory structure with variable block size and word length support, and 2) configurable SIMD structure. The proposed parallel memory structure allows great flexibility for programmers to perform data access of different block sizes and different word lengths. The configurable SIMD structure allows almost “random” register file access and slightly different operations in ALUs inside SIMD. The new features greatly benefit the realization of H.264/AVC kernel functions. For instance, the fractional motion estimation, particularly the half to quarter pixel interpolation, can now be executed with minimal or no additional memory access. When comparing with the conventional SIMD systems, the proposed SIMD architecture can have a further speedup of 2.1X to 4.6X when implementing H.264/AVC kernel functions. Based on Amdahl's law, the overall speedup of H.264/AVC encoding application can be projected to be 2.46X. We expect significant improvement can also be achieved when applying the proposed architecture to other image and video processing applications.

The extraction of component line segments and circular arcs from freehand strokes along with their relations is a prerequisite for sketch understanding. Existing approaches usually take three stages to segment a stroke: first identifying segmentation points, then classifying the substroke between each pair of adjacent segmentation points, and, finally, obtaining graphical representations of substrokes by fitting graphical primitives to them. Since a stroke inevitably contains noises, the first stage may produce wrong or inaccurate segmentation points, resulting in the wrong substroke classification in the second stage and inaccurately fitted parameters in the third stage. To overcome the noise sensitivity of the three-stage method, the segmental homogeneity feature is emphasized in this paper. We propose a novel approach, which first extracts graphical primitives from a stroke by a connected segment growing from a seed-segment and then utilizes relationships between the primitives to refine their control parameters. We have conducted experiments using real-life strokes and compared the proposed approach with others. Experimental results demonstrate that the proposed approach is effective and robust.

Text in video is a very compact and accurate clue for video indexing and summarization. Most video text detection and extraction methods hold assumptions on text color, background contrast, and font style. Moreover, few methods can handle multilingual text well since different languages may have quite different appearances. This paper performs a detailed analysis of multilingual text characteristics, including English and Chinese. Based on the analysis, we propose a comprehensive, efficient video text detection, localization, and extraction method, which emphasizes the multilingual capability over the whole processing. The proposed method is also robust to various background complexities and text appearances. The text detection is carried out by edge detection, local thresholding, and hysteresis edge recovery. The coarse-to-fine localization scheme is then performed to identify text regions accurately. The text extraction consists of adaptive thresholding, dam point labeling, and inward filling. Experimental results on a large number of video images and comparisons with other methods are reported in detail.

Merged characters are the major cause of recognition errors. We classify the merging relationship between two involved characters into three types: "linear," "nonlinear," and "overlapped." Most segmentation methods handle the first type well, however, their capabilities of handling the other two types are limited. The weakness of handling the nonlinear and overlapped types results from character segmentation by linear, usually vertical, cuts assumed in these methods. This paper proposes a novel merged character segmentation and recognition method based on forepart prediction, necessity-sufficiency matching and character-adaptive masking. This method utilizes the information obtained from the forepart of merged characters to predict candidates for the leftmost character, and then applies character-adaptive masking and character recognition to verifying the prediction. Therefore, the arbitrary-shaped cutting path will follow the right shape of the leftmost character so as to preserve the shape of the next character. This method handles the first two types well and greatly improves the segmentation accuracy of the overlapped type. The experimental results and the performance comparisons with other methods demonstrate the effectiveness of the proposed method.

Arc segmentation plays an important role in the process of graphics recognition from scanned images. The GREC arc segmentation contest shows there is a lot of room for improvement in this area. This paper proposes a multiresolution arc segmentation method based on our previous seeded circular tracking algorithm which largely depends on the OOPSV model. The newly-introduced multiresolution paradigm can handle arcs/circles with large radii well. We describe new approaches for arc seed detection, arc localization, and arc verification, making the proposed method self-contained and more efficient. Moreover, this paper also brings major improvement to the dynamic adjustment algorithm of circular tracking to make it more robust. A systematic performance evaluation of the proposed method has been conducted using the third-party evaluation tool and test images obtained from the GREC arc segmentation contests. The overall performance over various arc angles, arc lengths, line thickness, noises, arc-arc intersections, and arc-line intersections has been measured. The experimental results and time complexity analyses on real scanned images are also reported and compared with other approaches. The evaluation result demonstrates the stable performance and the significant improvement on processing large arcs/circles of the MAS method.

Video text extraction and recognition are prerequisite tasks for video indexing and retrieval. Color polarity classification of video text is very important to these tasks. Most existing text extraction methods assume that the text color is always light (or dark). Obviously, this assumption restricts the application of these methods to some specific domains. Only a few methods can detect the color polarity in the condition that the background is clear. However, many real video texts have various appearances and complex backgrounds that existing methods cannot handle. The paper proposes a statistical color polarity classification method that is robust to various background complexities, font styles, stroke widths, and languages. We discover the intrinsic relationships between text edges and background edges, and then develop an efficient measurement to detect the color polarity. The experimental results show that the proposed method achieves a much higher accuracy, 98.5%, than existing methods.

Video text extraction and recognition are prerequisite tasks for video indexing and retrieval. Color polarity classification of video text is very important to these tasks. Most existing text extraction methods assume that the text color is always light (or dark). Obviously, this assumption restricts the application of these methods to some specific domains. Only a few methods can detect the color polarity on condition that the background is clear. However, many real video texts have various appearances and complex backgrounds that existing methods cannot handle. This paper proposes a statistic color polarity classification method that is robust to various background complexities, font styles, stroke widths, and languages. We discover the intrinsic relationships between text edges and background edges, and then develop an efficient measurement to detect the color polarity. The experimental results show that the proposed method achieves a much higher accuracy, 98.5%, than existing methods.

Wilh the rapid deployment of videoconference, the fast-accumulated personal videoconference archives need to be effectively indexed. This paper proposes a well-designed indexing system - PVCAIS that integrates many multimedia-indexing techniques to manage personal videoconference archives. Firstly, the contents of video, audio, text and whiteboard communications are stored after removing the redundancies. Next, more information, e.g., participants, title, keywords and slides, is extracted by face detection and recognition, speech recognition, OCR, automatic title generation, keyword selection, etc. Then, an XML index file containing the summary of the videoconference is also generated. The whole indexing process is automatic except that the face of a new contact needs to be interactively identified for only once. Finally, we demonstrate a graphical user interface which allows the user to search and browse the indexed videoconference archives conveniently.

Text detection is fundamental to video information retrieval and indexing. Existing methods cannot handle well those texts with different contrast or embedded in a complex background. To handle these difficulties, this paper proposes an efficient text detection approach, which is based on invariant features, such as edge strength, edge density, and horizontal distribution. First, it applies edge detection and uses a low threshold to filter out definitely non-text edges. Then, a local threshold is selected to both keep low-contrast text and simplify complex background of high-contrast text. Next, two text-area enhancement operators are proposed to highlight those areas with either high edge strength or high edge density. Finally, coarse-to-fine detection locates text regions efficiently. Experimental results show that this approach is robust for contrast, font-size, font-color, language, and background complexity.

Existing vectorization systems for engineering drawings usually take a two-phase workflow: convert a raster image to raw vectors and recognize graphic objects from the raw vectors. The first phase usually separates aground truth graphic object that intersects or touches other graphic objects into several parts, thus, the second phase faces the difficulty of searching for and merging raw vectors belonging to the same object. These operations slow down vectorization and degrade the recognition quality. Imitating the way humans read engineering drawings, we propose an efficient one-phase object-oriented vectorization model that recognizes each class of graphic objects from their natural characteristics. Each ground truth graphic object is recognized directly in its entirety at the pixel level. The raster image is progressively simplified by erasing recognized graphic objects to eliminate their interference with subsequent recognition. To evaluate the performance of the proposed model, we present experimental results on real-life drawings and quantitative analysis using third party protocols. The evaluation results show significant improvement in speed and recognition rate.

This paper presents a novel approach for recognizing and interpreting dimensions in engineering drawings. It starts by detecting potential dimension frames, each comprising only the line and text components of a dimension, then verifies them by detecting the dimension symbols. By removing the prerequisite of symbol recognition from detection of dimension sets, our method is capable of handling low quality drawings. We also propose a reconstruction algorithm for rebuilding the drawing entities based on the recognized dimension annotations. A coordinate grid structure is introduced to represent and analyze two-dimensional spatial constraints between entities; this simplifies and unifies the process of rectifying deviations of entity dimensions induced during scanning and vectorization.

In this paper, an efficient global algorithm for vectorizing line drawings is presented. It first extracts a seed segment of a graphic entity from a raster image to obtain its direction and width, then tracks the pixels under the guidance of the direction so that the tracking can track through functions and is not affected by noise and degradation of image quality. Thus, an entity will be vectorized in one step without postprocessing. The relations among lines are also used to realize the continuous vectorization of a line net. The speed and quality of vectorization are greatly improved with this algorithm. The performance evaluation is carried out both by theoretical analysis and by experiments. Comparisons with other vectorization algorithms are also made.