In the processing of high-resolution remote sensing images, multiscale feature fusion techniques are commonly employed to construct change detection models, aiming to capture the details and characteristics of target objects at different scales. However, current change detection methods often neglect the subtle features of small-scale targets and edge information during feature fusion. To address this issue, this article proposes a deep feature fusion network (DFFNet) for full-scale change detection in remote sensing images. DFFNet enhances the accuracy of boundary information in change regions and enables the extraction of full-scale change features. By integrating phased difference extraction techniques with an enhanced attention mechanism, a dual temporal difference enhancement module (DT-DEM) is designed to comprehensively extract change detection information across scales. In addition, combining an inverted pyramid technique with an attention mechanism, a screening function inverted pyramid network (S-FIPN) is proposed, which efficiently extracts and fuses full-scale features while significantly improving the extraction of subtle changes and weak edge features. Comparison experiments with ten state-of-the-art (SOTA) algorithms on four publicly available datasets, namely, LEVIR-CD, WHU-CD, NJDS, and MSRS-CD, show that DFFNet achieves the best$F1$values on all datasets, which are 92.50%, 90.55%, 81.04%, and 77.44%, respectively. These results underscore the superiority and effectiveness of DFFNet in full-scale feature extraction and edge information retrieval.

In this paper, an unsupervised cycle image registration convolutional neural network named CIRNet is developed for 3D medical image registration. Different from most deep learning based registration methods that require known spatial transforms, our proposed method is trained in an unsupervised way and predicts the dense displacement vector field. The CIRNet is composed by two image registration modules which have the same architecture and share the parameters. A cycle identical loss is designed in the CIRNet to provide additional constraints to ensure the accuracy of the predicted dense displacement vector field. The method is evaluated by the registration in 4D (3D+t) cardiac CT and MRI images respectively. Quantitative evaluation results demonstrate that our method performs better than the other two existing image registration algorithms. Especially, compared to the traditional image registration methods, our proposed network can finish 3D image registration in less than one second.

Accounting for nearly 2% of all adults, renal cell carcinomas are sensitive to laparoscopic partial nephrectomy (LPN) which needs an accurate diagnosis and localization before operation. Faced with various intensity distribution, erratic location, irregular shape, etc, the image classification and semantic segmentation on CT scans of renal tumor are challenges. This paper presents a multi-task network, segmentation and classification convolutional neural network (SCNet), for preoperative assessment of renal tumor. Via the combination of two tasks, semantic features are fed to the classification network and classification results give segmentation network feedbacks in return. Besides, a 2-step segmentation strategy is conducted to the segmentation module which improves the result by 2.8%. Our experimental results of classification and segmentation achieve 100% accuracy and 0.882 dice coefficient of tumor region respectively, which are better than the results of a single classification network and segmentation network.

Renal cancer is one of ten most common cancers in human beings. The laparoscopic partial nephrectomy (LPN) becomes the main therapeutic approach in treating renal cancer. Accurate kidney and tumor segmentation in CT images is a prerequisite step in the surgery planning. However, automatic and accurate kidney and renal tumor segmentation in CT images remains a challenge. In this paper, we propose a new method to perform a precise segmentation of kidney and renal tumor in CT angiography images. This method relies on a three-dimensional (3D) fully convolutional network (FCN) which combines a pyramid pooling module (PPM). The proposed network is implemented as an end-to-end learning system directly on 3D volumetric images. It can make use of the 3D spatial contextual information to improve the segmentation of the kidney as well as the tumor lesion. The experiments conducted on 140 patients show that these target structures can be segmented with a high accuracy. The resulting average dice coefficients obtained for kidney and renal tumor are equal to 0.931 and 0.802 respectively. These values are higher than those obtained from the other two neural networks.

User-managed-events is a popular feature on social networks. Take Facebook Events as an example: over 135 million events were created in 2015 and over 550 million people use events each month. In this work, we consider the heavy sparseness in both user and event feedback history caused by short lifespans (transiency) of events and user participation patterns in a production event system. We propose to solve the resulting cold-start problems by introducing a joint representation model to project users and events into the same latent space. Our model based on parallel Convolutional Neural Networks captures semantic meaning in event text and also utilizes heterogeneous user knowledge available in the social network. By feeding the model output as user and event representation into a combiner prediction model, we show that our representation model improves the prediction accuracy over existing techniques (+6% AUC lift). Our method provides a generic way to match heterogeneous information from different domains and applies to a wide range of applications in social networks.

Personalization is a long-standing problem in data mining and machine learning. Companies make personalized product recommendations to millions of users every second. In addition to the recommendation problem, with the emerging of personal devices, many conventional problems, e.g., recognition, need to be personalized as well. Moreover, as the number of users grows huge, solving personalization becomes quite challenging. In this paper, we formalize the generic personalization problem as an optimization problem. We propose several ADMM algorithms to solve this problem in a distributed way including a new Asynchronous ADMM that removes all synchronous barriers to maximize the training throughput. We provide a mathematical analysis to show that the proposed Asynchronous ADMM algorithm holds a linear convergence rate which is the best to our knowledge. The distributed personalization allows training to be performed in either a cluster or even on a user's device. This can improve the privacy protection as no personal data is uploaded, while personal models can still be shared with each other. We apply this approach to two industry problems, \emph{Facial Expression Recognition} and \emph{Job Recommendation}. Experiments demonstrate more than 30\% relative error reduction on both problems. Asynchronous ADMM allows faster training for problems with millions of users since it eliminates all network I/O waiting time to maximize the cluster CPU throughput. Experiments demonstrate 4 times faster than original synchronous ADMM algorithm.

In order to overcome the phenomenon that the existence of end-to-end path is no longer guaranteed because of intermittent connection in the opportunistic network, it is desirable to employ a special node called ferry, which moves in a specific trace and forwards data for disconnected nodes, to provide communication opportunities. In this paper, Global Ferry Scheme (GFS), which exploits multiple local ferries and a global ferry to deliver messages, is proposed to minimize the average message delivery delay. The performance of GFS is evaluated and compared to the existing ferry-based routing method by analysis and simulation. The results show that GFS can improve the performance of opportunistic networks in terms of average delay and delivery ratio.

Instructional videos are used intensively in universities for remote education and e-learning, and a typical university course consists of videos of more than two thousand minutes in total length. This paper presents a novel graphics user interface for indexing and browsing such extensive but thematically related content. We present how the interface automatically extracts semantic indices from the visual content, and then presents both high- and low-level cues from five different conceptual viewpoints. We detail each of these novel UI units, and show how they are integrated into a user-adjustable main framework, and interconnected and navigated through user mouse events.

Handwritten text frames appear frequently in educational videos and can be used as an important cue for semantic analysis of educational videos. We detect text frames using a motion pattern analyzing algorithm. Then, we extract binary handwritten word images from the text frames in various visual formats: handwritten slides, electronic slides, handwriting on chalkboard, etc. We propose a handwritten word recognition method, using combined dynamic programming stroke-based character segmentation with optimal statistical handwritten character recognition. In parallel, we construct a small vocabulary from topic words taken from table-of-contents of course materials such as the course textbook. We use the handwritten word recognition results to query this table-of-contents structure, implemented as latent semantic analysis matrix operations. We are able to spot the most likely discussed chapters and topic words for each frame. We evaluate the overall approach on 12 videos of two courses, and the results are encouraging.

Videotext can be an efficient semantic index and summary for instructional videos. However, videotext usually appears in different visual formats: handwritten slides, electronic slides, book pages, web pages, handwriting on chalkboard, etc. We propose a unified approach to handle all these kinds of videotext in three steps. First, we detect still video segments by analyzing motion energy patterns in instructional videos, and construct a quality-enhanced candidate text frame for each still video segment. Then, we use a trained SVM classifier to verify the candidate text frames, as well as to segment the text region and individual text blocks from the verified frames. Finally, we filter redundant text frames with similar text content by a Hausdorff distance-based image comparison algorithm. The resulting text frames are automatically organized into HTML and PDF documents to serve as an imagery summarization of the instructional videos. We show the application of our method to 75 instructional videos of five different courses, and discuss its applications.

Efficient indexing and retrieval of digital videos are important needs within instructional video databases. Semantic indexing for instructional videos can be achieved by combining the analysis of the instructor's handwriting in the video with domain knowledge taken from course support materials such as the course textbook, syllabus, or slides. We propose such a semantic indexing method, by combining handwritten word recognition with information retrieval techniques. We first present a novel handwritten word segmentation and recognition approach for instructional videos. Then we construct a table-of-contents (TOC) structure from course materials. We use word recognition results to query the TOC, implemented as matrix operations, and spot the most likely discussed chapters and topic words for each video. We evaluate the overall approach on 12 videos of two courses, and the results are encouraging.

Efficient indexing and retrieval of digital videos are important needs within instructional video databases. We construct a word-based video index by spotting words taken from the table of contents (TOC) of the textbooks for the course given in the video. We present a segment-free method of word spotting based on a two step analysis. First, for each candidate word in the vocabulary and each unknown handwritten word image, we search for the location, orientation, and scale of every character of the candidate word; we use a modified Hausdorff distance (MHD) to robustly locate the character's skeleton in the skeletons of the handwritten word image. Second, we verify words by creating a template image according to the result of character analysis, and by matching this template image to the skeleton of the handwritten word using the MHD. Results for recognition of handwritten words from real videos are shown (greater than 70% accuracy with 100 words). Applying these methods to three contiguous videos (of 25) of a course, we find that they recognize nearly disjoint subsets of highly descriptive indexing keywords.

We demonstrate a system that combines steerable projection and computer vision technologies to create "touch-screen" style interactive displays on any flat surface in a space. A high-end version of the system -- the Everywhere Display (ED) -- combines an LCD projector with motorized focus and zoom, a computer controlled pan-tilt mirror, and a pan-tilt zoom camera to enable steering of interactive projections around space. A low-end version (ED-lite) enables creation of interactive displays using a portable projector and camera attached to a laptop computer. Unlike traditional augmented reality systems, the ED systems enable delivery of interactive multimedia content on ordinary objects without requiring users to wear head mounted displays or carry special input devices.

We present an automatic tool for compact representation and cross-referencing of long video sequences, which is based on a novel visual abstraction of semantic content. Our highly compact hierarchical representation results from the non-temporal clustering of scene segments into a new conceptual form grounded in the recognition of real-world backgrounds. We represent shots and scenes using mosaics and employ a novel method for the comparison of scenes based on these representative mosaics. We then cluster scenes together into a higher level of abstraction-the physical setting. We demonstrate our work using situation comedies (sitcoms), where each half-hour episode is well structured by rules governing background use. Consequently, browsing, indexing and comparison across videos by physical setting is very fast. Further, we show that physical settings lead to a higher-level contextual identification of the main plots in each video. We demonstrate these contributions with a browsing tool whose top-level single page displays the settings of several episodes. This page expands to display windows for each episode, and each episode menu summary is further expanded into scenes and shots, all by mouse-clicking on appropriate plots and settings according to user interests.