Demonstration venue · read-only. Every page can be browsed; the buttons that would change it are switched off. Create an account to run TaxoReview on your own data. A simulation from public publication records: the people listed were not invited and did not agree to review.

Tian-Tsong Ng

dblp:28/3620 · DBLP profile ↗
← Back
38ranked-venue papers
5as first author
1since 2021 · last 2023
0000-0002-6713-2310ORCID · verified

Domains — the database's venue-derived domains; a paper can count in several

Graphics, computer vision, multimedia, augmented reality and games · 26 · 4 first-authorArtificial intelligence and machine learning · 19 · 3 first-author · 1 since 2021Security and privacy · 3Systems, architecture and hardware · 2

Expertise — from the expertise taxonomy: the topics of the expert's papers under the CCF categories. A weight counts papers with recency: 1 for a paper about the topic, 0.3 when the topic is its context, halved every five years.

Computer graphics and multimedia
14 papers
Image and video processing · 24% Rendering · 22% Computational photography and imaging · 20%
Artificial intelligence
7 papers
Video understanding and tracking · 61% Trustworthy machine learning · 14% Knowledge representation and reasoning · 8%
Network and information security
2 papers
Digital forensics and information hiding · 33% Privacy and data protection · 29% Security and privacy of machine learning · 29%

Topics — the 30 heaviest of 45, each with the papers that count most for it

TopicWeightPapersLastEvidence papers
Computer vision › Video understanding and tracking
action recognition
0.622018
Deep Multimodal Feature Analysis for Action Recognition in RGB+D Videos · IEEE Trans. Pattern Anal. Mach. Intell. 2018
Multimodal Multipart Learning for Action Recognition in Depth Videos · IEEE Trans. Pattern Anal. Mach. Intell. 2016
Machine learning › Trustworthy machine learning
interpretability
0.422016
A Paradigm for Building Generalized Models of Human Image Perception through Data Fusion · CVPR 2016
An Automated Estimator of Image Visual Realism Based on Human Cognition · CVPR 2014
Rendering
light transport
0.442012
From the Rendering Equation to Stratified Light Transport Inversion · Int. J. Comput. Vis. 2012
A Dual Theory of Inverse and Forward Light Transport · ECCV (2) 2010
Radiometric compensation using stratified inverses · ICCV 2009
Security and privacy of machine learning › adversarial attack
adversarial perturbation
0.412019
Human-imperceptible Privacy Protection Against Machines · ACM Multimedia 2019
Privacy and data protection
image privacy
0.412019
Human-imperceptible Privacy Protection Against Machines · ACM Multimedia 2019
Computer vision › Video understanding and tracking › action recognition
multimodal action recognition
0.312018
Deep Multimodal Feature Analysis for Action Recognition in RGB+D Videos · IEEE Trans. Pattern Anal. Mach. Intell. 2018
Image and video coding
image quality assessment
0.312018
Image Visual Realism: From Human Perception to Machine Computation · IEEE Trans. Pattern Anal. Mach. Intell. 2018
Image and video processing › perceptual modeling
visual perception modeling
0.312018
Image Visual Realism: From Human Perception to Machine Computation · IEEE Trans. Pattern Anal. Mach. Intell. 2018
Computer vision › Video understanding and tracking › action recognition › 3d action recognition
depth-based action recognition
0.212016
Multimodal Multipart Learning for Action Recognition in Depth Videos · IEEE Trans. Pattern Anal. Mach. Intell. 2016
Computer vision › Video understanding and tracking › activity recognition
human activity recognition
0.212016
NTU RGB+D: A Large Scale Dataset for 3D Human Activity Analysis · CVPR 2016
Knowledge, reasoning and agents › Knowledge representation and reasoning
information fusion
0.212016
A Paradigm for Building Generalized Models of Human Image Perception through Data Fusion · CVPR 2016
Computer vision › Video understanding and tracking › action recognition › multimodal action recognition
RGB-D action recognition
0.212016
NTU RGB+D: A Large Scale Dataset for 3D Human Activity Analysis · CVPR 2016
Computer vision › Video understanding and tracking › action recognition
skeleton-based action recognition
0.212016
NTU RGB+D: A Large Scale Dataset for 3D Human Activity Analysis · CVPR 2016
Image and video coding
image compression
0.212015
Statistical Model of JPEG Noises and Its Application in Quantization Step Estimation · IEEE Trans. Image Process. 2015
Image and video processing
image forensics
0.212015
Statistical Model of JPEG Noises and Its Application in Quantization Step Estimation · IEEE Trans. Image Process. 2015
Geometric modeling and processing › feature recognition
lattice detection
0.212015
PatchMatch-Based Automatic Lattice Detection for Near-Regular Textures · ICCV 2015
Image and video processing
texture analysis
0.212015
PatchMatch-Based Automatic Lattice Detection for Near-Regular Textures · ICCV 2015
Digital forensics and information hiding › digital forensics › multimedia forensics
image forensics
0.212015
Revealing the Trace of High-Quality JPEG Compression Through Quantization Noise Analysis · IEEE Trans. Inf. Forensics Secur. 2015
Digital forensics and information hiding › digital forensics › multimedia forensics › image forensics
JPEG compression forensics
0.212015
Revealing the Trace of High-Quality JPEG Compression Through Quantization Noise Analysis · IEEE Trans. Inf. Forensics Secur. 2015
Computational photography and imaging
image stitching
0.112011
Smoothly varying affine stitching · CVPR 2011
Computational photography and imaging › physics-based vision › light transport analysis
interreflection removal
0.112011
On the Duality of Forward and Inverse Light Transport · IEEE Trans. Pattern Anal. Mach. Intell. 2011
Rendering › light transport
inverse light transport
0.112011
On the Duality of Forward and Inverse Light Transport · IEEE Trans. Pattern Anal. Mach. Intell. 2011
Computer vision › Face, body and person analysis
face recognition
0.112010
Factorization towards a classifier · CVPR 2010
Machine learning › Representation and self-supervised learning › matrix factorization
nonnegative matrix factorization
0.112010
Factorization towards a classifier · CVPR 2010
Rendering
inverse rendering
0.112010
A Dual Theory of Inverse and Forward Light Transport · ECCV (2) 2010
Machine learning › Deep learning architectures and training
autoencoder
0.112018
Deep Multimodal Feature Analysis for Action Recognition in RGB+D Videos · IEEE Trans. Pattern Anal. Mach. Intell. 2018
Computational photography and imaging › projector-camera systems
radiometric compensation
0.112009
Radiometric compensation using stratified inverses · ICCV 2009
Computer vision › Face, body and person analysis › human body analysis
body part analysis
0.112016
Multimodal Multipart Learning for Action Recognition in Depth Videos · IEEE Trans. Pattern Anal. Mach. Intell. 2016
Computational photography and imaging › camera characterization
camera response function estimation
0.112007
Using Geometry Invariants for Camera Response Function Estimation · CVPR 2007
Geometric modeling and processing › computational geometry
geometric invariants
0.112007
Using Geometry Invariants for Camera Response Function Estimation · CVPR 2007

Methods — techniques the papers use, named apart from their topics

statistical modeling · 1.0data imputation · 0.5variance analysis · 0.4patchmatch · 0.4particle belief propagation · 0.4k-nearest-neighbor correspondence · 0.4image perturbation model · 0.4human sensitivity map · 0.4structured sparsity learning · 0.3shared-specific feature factorization · 0.3mixed norms · 0.3deep autoencoder · 0.3convolutional neural network · 0.3recurrent neural network · 0.2hierarchical mixed norm · 0.2group feature selection · 0.2deep learning · 0.2sufficient statistic · 0.2
YearPublicationVenuePosition
2023 When and Why Static Images Are More Effective Than Videos
abstract
People often prefer videos over images in research and applications, believing that videos are more effective for eliciting human emotions and building machine intelligence. However, our research shows that this assumption is not always correct when it comes to evoking emotions in human observers. In this article, we compare thirteen emotions and two perceptions elicited by short videos (2-6 second, silent video clips) versus static frames extracted from the videos. We show that static frames and videos elicit most emotions similarly, but static frames elicit negative emotions more strongly than videos. We test two complementary explanations: differential activation of suspense and the peak-end rule. These findings help us to computationally model human reactions more faithfully with fewer video frames. Our interdisciplinary results have important implications for methods, theory, and applications in diverse fields, including social psychology, computer vision, mass media, and marketing.
Shaojing Fan, Zhiqi Shen 0002, Bryan L. Koenig, Tian-Tsong Ng, Mohan Kankanhalli
IEEE Trans. Affect. Comput.4
2019 Human-imperceptible Privacy Protection Against Machines
abstract
Privacy concerns with social media have recently been under the spotlight, due to a few incidents on user data leakage on social networking platforms. With the current advances in machine learning and big data, computer algorithms often act as a first-step filter for privacy breaches, by automatically selecting content with sensitive information, such as photos that contain faces or vehicle license plate. In this paper we propose a novel algorithm to protect the sensitive attributes against machines, meanwhile keeping the changes imperceptible to humans. In particular, we first conducted a series of human studies to investigate multiple factors that influence human sensitivity to the visual changes. We discover that human sensitivity is influenced by multiple factors, from low-level features such as illumination, texture, to high-level attributes like object sentiment and semantics. Based on our human data, we propose for the first time the concept of human sensitivity map. With the sensitivity map, we design a human-sensitivity-aware image perturbation model, which is able to modify the computational classification results of sensitive attributes while preserving the remaining attributes. Experiments on real world data demonstrate the superior performance of the proposed model on human-imperceptible privacy protection.
Zhiqi Shen 0002, Shaojing Fan, Yongkang Wong, Tian-Tsong Ng, Mohan Kankanhalli
ACM Multimedia4
2018 Image Visual Realism: From Human Perception to Machine Computation
abstract
Visual realism is defined as the extent to which an image appears to people as a photo rather than computer generated. Assessing visual realism is important in applications like computer graphics rendering and photo retouching. However, current realism evaluation approaches use either labor-intensive human judgments or automated algorithms largely dependent on comparing renderings to reference images. We develop a reference-free computational framework for visual realism prediction to overcome these constraints. First, we construct a benchmark dataset of 2,520 images with comprehensive human annotated attributes. From statistical modeling on this data, we identify image attributes most relevant for visual realism. We propose both empirically-based (guided by our statistical modeling of human data) and deep convolutional neural network models to predict visual realism of images. Our framework has the following advantages: (1) it creates an interpretable and concise empirical model that characterizes human perception of visual realism; (2) it links computational features to latent factors of human image perception.
Shaojing Fan, Tian-Tsong Ng, Bryan L. Koenig, Jonathan S. Herberg, Ming Jiang 0019, Zhiqi Shen 0002, Qi Zhao 0001
IEEE Trans. Pattern Anal. Mach. Intell.2
2018 Deep Multimodal Feature Analysis for Action Recognition in RGB+D Videos
abstract
Single modality action recognition on RGB or depth sequences has been extensively explored recently. It is generally accepted that each of these two modalities has different strengths and limitations for the task of action recognition. Therefore, analysis of the RGB+D videos can help us to better study the complementary properties of these two types of modalities and achieve higher levels of performance. In this paper, we propose a new deep autoencoder based shared-specific feature factorization network to separate input multimodal signals into a hierarchy of components. Further, based on the structure of the features, a structured sparsity learning machine is proposed which utilizes mixed norms to apply regularization within components and group selection between them for better classification performance. Our experimental results show the effectiveness of our cross-modality feature analysis framework by achieving state-of-the-art accuracy for action classification on five challenging benchmark datasets.
Amir Shahroudy, Tian-Tsong Ng, Yihong Gong, Gang Wang 0012
IEEE Trans. Pattern Anal. Mach. Intell.2
2018 Locating 3D Object Proposals: A Depth-Based Online Approach
abstract
2D object proposals, quickly detected regions in an image that likely contain an object of interest, are an effective approach for improving the computational efficiency and accuracy of object detection in color images. In this paper, we propose a novel online method that generates 3D object proposals in an RGB-D video sequence. Our main observation is that depth images provide important information about the geometry of the scene. Diverging from the traditional goal of 2D object proposals to provide a high recall, we aim for precise 3D proposals. We leverage on depth information per frame and multiview scene information to obtain accurate 3D object proposals. Using efficient but robust registration enables us to combine multiple frames of a scene in near real time and generate 3D bounding boxes for potential 3D regions of interest. Using standard metrics, such as precision-recall (P-R) curves and F-measure, we show that the proposed approach is significantly more accurate than the current state-of-the-art techniques. Our online approach can be integrated into simultaneous localization and mapping-based video processing for quick 3D object localization. Our method takes less than a second in MATLAB on the UW-RGBD scene data set on a single thread CPU and, thus, has potential to be used in low-power chips in unmanned aerial vehicles, quadcopters, and drones.
Ramanpreet Singh Pahwa, Jiangbo Lu, Nianjuan Jiang, Tian-Tsong Ng, Minh N. Do
IEEE Trans. Circuits Syst. Video Technol.4
2016 A Paradigm for Building Generalized Models of Human Image Perception through Data Fusion
abstract
In many sub-fields, researchers collect datasets of human ground truth that are used to create a new algorithm. For example, in research on image perception, datasets have been collected for topics such as what makes an image aesthetic or memorable. Despite high costs for human data collection, datasets are infrequently reused beyond their own fields of interest. Moreover, the algorithms built from them are domain-specific (predict a small set of attributes) and usually unconnected to one another. In this paper, we present a paradigm for building generalized and expandable models of human image perception. First, we fuse multiple fragmented and partially-overlapping datasets through data imputation. We then create a theoretically-structured statistical model of human image perception that is fit to the fused datasets. The resulting model has many advantages. (1) It is generalized, going beyond the content of the constituent datasets, and can be easily expanded by fusing additional datasets. (2) It provides a new ontology usable as a network to expand human data in a cost-effective way. (3) It can guide the design of a generalized computational algorithm for multi-dimensional visual perception. Indeed, experimental results show that a model-based algorithm outperforms state-of-the-art methods on predicting visual sentiment, visual realism and interestingness. Our paradigm can be used in various visual tasks (e.g., video summarization).
Shaojing Fan, Tian-Tsong Ng, Bryan L. Koenig, Ming Jiang 0019, Qi Zhao 0001
CVPR2
2016 NTU RGB+D: A Large Scale Dataset for 3D Human Activity Analysis
abstract
Recent approaches in depth-based human activity analysis achieved outstanding performance and proved the effectiveness of 3D representation for classification of action classes. Currently available depth-based and RGB+Dbased action recognition benchmarks have a number of limitations, including the lack of training samples, distinct class labels, camera views and variety of subjects. In this paper we introduce a large-scale dataset for RGB+D human action recognition with more than 56 thousand video samples and 4 million frames, collected from 40 distinct subjects. Our dataset contains 60 different action classes including daily, mutual, and health-related actions. In addition, we propose a new recurrent neural network structure to model the long-term temporal correlation of the features for each body part, and utilize them for better action classification. Experimental results show the advantages of applying deep learning methods over state-of-the-art handcrafted features on the suggested cross-subject and cross-view evaluation criteria for our dataset. The introduction of this large scale dataset will enable the community to apply, develop and adapt various data-hungry learning techniques for the task of depth-based and RGB+D-based human activity analysis.
Amir Shahroudy, Jun Liu 0036, Tian-Tsong Ng, Gang Wang 0012
CVPR3
2016 COVERAGE - A novel database for copy-move forgery detection
abstract
We present COVERAGE - a novel database containing copy-move forged images and their originals with similar but genuine objects. COVERAGE is designed to highlight and address tamper detection ambiguity of popular methods, caused by self-similarity within natural images. In COVERAGE, forged-original pairs are annotated with (i) the duplicated and forged region masks, and (ii) the tampering factor/similarity metric. For benchmarking, forgery quality is evaluated using (i) computer vision-based methods, and (ii) human detection performance. We also propose a novel sparsity-based metric for efficiently estimating forgery quality. Experimental results show that (a) popular forgery detection methods perform poorly over COVERAGE, and (b) the proposed sparsity based metric best correlates with human detection performance. We release the COVERAGE database to the research community.
Bihan Wen, Subramanian Ramanathan, Tian-Tsong Ng, Xuanjing Shen, Stefan Winkler 0001
ICIP4
2016 Multimodal Multipart Learning for Action Recognition in Depth Videos
abstract
The articulated and complex nature of human actions makes the task of action recognition difficult. One approach to handle this complexity is dividing it to the kinetics of body parts and analyzing the actions based on these partial descriptors. We propose a joint sparse regression based learning method which utilizes the structured sparsity to model each action as a combination of multimodal features from a sparse set of body parts. To represent dynamics and appearance of parts, we employ a heterogeneous set of depth and skeleton based features. The proper structure of multimodal multipart features are formulated into the learning framework via the proposed hierarchical mixed norm, to regularize the structured features of each part and to apply sparsity between them, in favor of a group feature selection. Our experimental results expose the effectiveness of the proposed learning method in which it outperforms other methods in all three tested datasets while saturating one of them by achieving perfect accuracy.
Amir Shahroudy, Tian-Tsong Ng, Qingxiong Yang, Gang Wang 0012
IEEE Trans. Pattern Anal. Mach. Intell.2
2015 PatchMatch-Based Automatic Lattice Detection for Near-Regular Textures
abstract
In this work, we investigate the problem of automatically inferring the lattice structure of near-regular textures (NRT) in real-world images. Our technique leverages the PatchMatch algorithm for finding k-nearest-neighbor (kNN) correspondences in an image. We use these kNNs to recover an initial estimate of the 2D wallpaper basis vectors, and seed vertices of the texture lattice. We iteratively expand this lattice by solving an MRF optimization problem. We show that we can discretize the space of good solutions for the MRF using the kNNs, allowing us to efficiently and accurately optimize the MRF energy function using the Particle Belief Propagation algorithm. We demonstrate our technique on a benchmark NRT dataset containing a wide range of images with geometric and photometric variations, and show that our method clearly outperforms the state of the art in terms of both texel detection rate and texel localization score.
Tian-Tsong Ng, Kalyan Sunkavalli, Minh N. Do, Eli Shechtman, Nathan Carr 0001
ICCV2
2015 Revealing the Trace of High-Quality JPEG Compression Through Quantization Noise Analysis
abstract
To identify whether an image has been JPEG compressed is an important issue in forensic practice. The state-of-the-art methods fail to identify high-quality compressed images, which are common on the Internet. In this paper, we provide a novel quantization noise-based solution to reveal the traces of JPEG compression. Based on the analysis of noises in multiple-cycle JPEG compression, we define a quantity called forward quantization noise. We analytically derive that a decompressed JPEG image has a lower variance of forward quantization noise than its uncompressed counterpart. With the conclusion, we develop a simple yet very effective detection algorithm to identify decompressed JPEG images. We show that our method outperforms the state-of-the-art methods by a large margin especially for high-quality compressed images through extensive experiments on various sources of images. We also demonstrate that the proposed method is robust to small image size and chroma subsampling. The proposed algorithm can be applied in some practical applications, such as Internet image classification and forgery detection.
Bin Li 0011, Tian-Tsong Ng, Xiaolong Li 0001, Shunquan Tan, Jiwu Huang
IEEE Trans. Inf. Forensics Secur.2
2015 Statistical Model of JPEG Noises and Its Application in Quantization Step Estimation
abstract
In this paper, we present a statistical analysis of JPEG noises, including the quantization noise and the rounding noise during a JPEG compression cycle. The JPEG noises in the first compression cycle have been well studied; however, so far less attention has been paid on the statistical model of JPEG noises in higher compression cycles. Our analysis reveals that the noise distributions in higher compression cycles are different from those in the first compression cycle, and they are dependent on the quantization parameters used between two successive cycles. To demonstrate the benefits from the analysis, we apply the statistical model in JPEG quantization step estimation. We construct a sufficient statistic by exploiting the derived noise distributions, and justify that the statistic has several special properties to reveal the ground-truth quantization step. Experimental results demonstrate that the proposed estimator can uncover JPEG compression history with a satisfactory performance.
Bin Li 0011, Tian-Tsong Ng, Xiaolong Li 0001, Shunquan Tan, Jiwu Huang
IEEE Trans. Image Process.2
2014 An Automated Estimator of Image Visual Realism Based on Human Cognition
abstract
Assessing the visual realism of images is increasingly becoming an essential aspect of fields ranging from computer graphics (CG) rendering to photo manipulation. In this paper we systematically evaluate factors underlying human perception of visual realism and use that information to create an automated assessment of visual realism. We make the following unique contributions. First, we established a benchmark dataset of images with empirically determined visual realism scores. Second, we identified attributes potentially related to image realism, and used correlational techniques to determine that realism was most related to image naturalness, familiarity, aesthetics, and semantics. Third, we created an attributes-motivated, automated computational model that estimated image visual realism quantitatively. Using human assessment as a benchmark, the model was below human performance, but outperformed other state-of-the-art algorithms.
Shaojing Fan, Tian-Tsong Ng, Jonathan S. Herberg, Bryan L. Koenig, Cheston Tan, Rangding Wang
CVPR2
2014 Epitomic image colorization
abstract
Image colorization adds color to grayscale images. It not only increases the visual appeal of grayscale images, but also enriches the information conveyed by scientific images that lack color information. We develop a new image colorization method, epitomic image colorization, which automatically transfers color from the reference color image to the target grayscale image by a robust feature matching scheme using a new feature representation, namely the heterogeneous feature epitome. As a generative model, heterogeneous feature epitome is a condensed representation of image appearance which is employed for measuring the dissimilarity between reference patches and target patches in a way robust to noise in the reference image. We build a Markov Random Field (MRF) model with the learned heterogeneous feature epitome from the reference image, and inference in the MRF model achieves robust feature matching for transferring color. Our method renders better colorization results than the current state-of-the-art automatic colorization methods in our experiments.
Yingzhen Yang, Xinqi Chu, Tian-Tsong Ng, Alex Yong Sang Chia, Jianchao Yang, Hailin Jin, Thomas S. Huang
ICASSP3
2014 Calibration of depth cameras using denoised depth images
abstract
Depth sensing devices have created various new applications in scientific and commercial research with the advent of Microsoft Kinect and PMD (Photon Mixing Device) cameras. Most of these applications require the depth cameras to be pre-calibrated. However, traditional calibration methods using a checkerboard do not work very well for depth cameras due to the low image resolution. In this paper, we propose a depth calibration scheme which excels in estimating camera calibration parameters when only a handful of corners and calibration images are available. We exploit the noise properties of PMD devices to denoise depth measurements and perform camera calibration using the denoised depth as additional set of measurements. Our synthetic and real experiments show that our depth denoising and depth based calibration scheme provides significantly better results than traditional calibration methods.
Ramanpreet Singh Pahwa, Minh N. Do, Tian-Tsong Ng, Binh-Son Hua
ICIP3
2014 Learning deep features for multiple object tracking by using a multi-task learning strategy
abstract
Model-free object tracking is still challenging because of the limited prior knowledge and the unexpected variation of the target object. In this paper, we propose a feature learning algorithm for model-free multiple object tracking. First, we pre-learn generic features invariant to diverse motion transformations from auxiliary video data by using a deep network of anto-encoder. Then, we adapt the pre-learned features according to multiple target objects respectively in a multi-task learning manner. We treat the feature adaptation for each target object as one single task. We simultaneously learn the common feature shared by all target objects and the individual feature of each object. Experimental results demonstrate that our feature learning algorithm can significantly improve multiple object tracking performance.
Li Wang 0057, Nam Trung Pham, Tian-Tsong Ng, Gang Wang 0012, Kap Luk Chan, Karianto Leman
ICIP3
2014 Human Perception of Visual Realism for Photo and Computer-Generated Face Images
abstract
Computer-generated (CG) face images are common in video games, advertisements, and other media. CG faces vary in their degree of realism, a factor that impacts viewer reactions. Therefore, efficient control of visual realism of face images is important. Efficient control is enabled by a deep understanding of visual realism perception: the extent to which viewers judge an image as a real photograph rather than a CG image. Across two experiments, we explored the processes involved in visual realism perception of face images. In Experiment 1, participants made visual realism judgments on original face images, inverted face images, and images of faces that had the top and bottom halves misaligned. In Experiment 2, participants made visual realism judgments on original face images, scrambled faces, and images that showed different parts of faces. Our findings indicate that both holistic and piecemeal processing are involved in visual realism perception of faces, with holistic processing becoming more dominant when resolution is lower. Our results also suggest that shading information is more important than color for holistic processing, and that inversion makes visual realism judgments harder for realistic images but not for unrealistic images. Furthermore, we found that eyes are the most influential face part for visual realism, and face context is critical for evaluating realism of face parts. To the best of our knowledge, this work is a first realism-centric study attempting to bridge the human perception of visual realism on face images with general face perception tasks.
Shaojing Fan, Rangding Wang, Tian-Tsong Ng, Cheston Tan, Jonathan S. Herberg, Bryan L. Koenig
ACM Trans. Appl. Percept.3
2013 Reconstruction of depth and normals from interreflections
abstract
While geometry reconstruction has been extensively studied, several shortcomings still exist. First, traditional geometry reconstruction methods such as geometric or photometric stereo only recover either surface depth or normals. Second, such methods require calibration. Third, such methods cannot recover accurate geometry in the presence of interreflections. In order to address these problems in a single system, we propose an approach to reconstruct geometry from light transport data. Specifically, we investigate the problem of geometry reconstruction from interreflections in a light transport matrix. We show that by solving a system of polynomial equations derived directly from the interreflection matrix, both surface depth and normals can be fully reconstructed. Our system does not require projector-camera calibration, but only make use of a calibration object such as a checkerboard in the scene to pre-determine a few known points to simplify the polynomial solver. Our experimental results show that our system is able to reconstruct accurate geometry from interreflections up to a certain noise level. Our system is easy to set up in practice.
Binh-Son Hua, Tian-Tsong Ng, Kok-Lim Low
ICIP2
2012 Automatic Grading of Cortical and PSC Cataracts Using Retroillumination Lens Images
Xinting Gao, Damon Wing Kee Wong, Tian-Tsong Ng, Carol Yim-lui Cheung, Ching Yu Cheng, Tien Yin Wong
ACCV (2)3
2012 From the Rendering Equation to Stratified Light Transport Inversion
Tian-Tsong Ng, Ramanpreet Singh Pahwa, Jiamin Bai, Kar-Han Tan, Ravi Ramamoorthi
Int. J. Comput. Vis.1
2011 Compressive Inverse Light Transport
abstract
This paper explores the possibility of acquiring inverse li ght transport directly. The current strategy of obtaining an inverse light transport ma trix involves two steps: First, acquire the forward light transport matrix (f-LTM) and then calculate the inverse of the f-LTM. Both steps of the strategy requires considerable computational power. In addition to computational cost, the measurement error incurred at the first step inevitably propagates to or potentially gets amplified in the matrix inv ersion step. In this paper, we propose a sensing strategy that acquires the inverse light transport matrix (i-LTM) directly, without reconstructing the f-LTM. Our direct strategy reduces both computational error and cost of acquiring i-LTM. For that, we propose a compressive inverse theory. Following the compressible property of i-LTM, a reconstruction condition for i-LTM is introduced. This new framework implies a trade-off between two factors: condition numbers of submatrices of f-LTM and the isometry constant of the illumination pattern. Our direct i-LTM reconstruction method is then demonstrated with a 2nd-bounce separation experiment on an M-shaped panel scene. Finally by quantitatively comparing our method with the existing two-stage approach, our method shows higher accuracy with lower complexity. The proofs of main theorem/lemma are contained in the supplementary material. The compressive inverse theory is general and potentially useful for wider application.
Xinqi Chu, Tian-Tsong Ng, Ramanpreet Singh Pahwa, Tony Q. S. Quek, Thomas S. Huang
BMVC2
2011 Smoothly varying affine stitching
abstract
Traditional image stitching using parametric transforms such as homography, only produces perceptually correct composites for planar scenes or parallax free camera motion between source frames. This limits mosaicing to source images taken from the same physical location. In this paper, we introduce a smoothly varying affine stitching field which is flexible enough to handle parallax while retaining the good extrapolation and occlusion handling properties of parametric transforms. Our algorithm which jointly estimates both the stitching field and correspondence, permits the stitching of general motion source images, provided the scenes do not contain abrupt protrusions.
Wen-Yan Lin, Yasuyuki Matsushita, Tian-Tsong Ng, Loong Fah Cheong
CVPR4
2011 Distinguishing Photographic Images and Photorealistic Computer Graphics Using Visual Vocabulary on Local Image Edges
Rong Zhang 0007, Rangding Wang, Tian-Tsong Ng
IWDW3
2011 On the Duality of Forward and Inverse Light Transport
abstract
Inverse light transport seeks to undo global illumination effects, such as interreflections, that pervade images of most scenes. This paper presents the theoretical and computational foundations for inverse light transport as a dual of forward rendering. Mathematically, this duality is established through the existence of underlying Neumann series expansions. Physically, it can be shown that each term of our inverse series cancels an interreflection bounce, just as the forward series adds them. While the convergence properties of the forward series are well known, we show that the oscillatory convergence of the inverse series leads to more interesting conditions on material reflectance. Conceptually, the inverse problem requires the inversion of a large light transport matrix, which is impractical for realistic resolutions using standard techniques. A natural consequence of our theoretical framework is a suite of fast computational algorithms for light transport inversion--analogous to finite element radiosity, Monte Carlo and wavelet-based methods in forward rendering--that rely at most on matrix-vector multiplications. We demonstrate two practical applications, namely, separation of individual bounces of the light transport and fast projector radiometric compensation, to display images free of global illumination artifacts in real-world environments.
Manmohan Krishna Chandraker, Jiamin Bai, Tian-Tsong Ng, Ravi Ramamoorthi
IEEE Trans. Pattern Anal. Mach. Intell.3
2010 Factorization towards a classifier
abstract
In practice, nonnegative data factorization is often performed for data dimensionality reduction prior to a classification task using a classifier which is effective in low dimensional space such as nearest neighbor classifier. In this work, we propose a novel formulation to learn a multi-class classifier directly through a supervised nonnegative data factorization. This new formulation has the following properties: 1) the nonnegative data matrix is approximated as the product of a nonnegative basis matrix and a coefficient matrix where the nonnegative bases distinctively capture the common characteristics of all classes apart from that specific to individual classes; 2) a regularization term is imposed on nonnegative data factorization so that each datum can be predominantly reconstructed by the common basis vectors and its corresponding class-specific basis vectors; and 3) the coefficient vector for each datum is assumed to be transformed from a mapped kernel space, and the l2norm of the class-specific coefficients reveals the relative confidence of classes, which then directly leads to a multi-class classifier. We also present an iterative optimization technique for our formulation and analytically show its convergence property. Extensive experiments on face recognition, head pose estimation, and handwritten digit recognition tasks clearly demonstrate the advantages of the proposed classifier over the conventional two-step approach of nonnegative data factorization followed by a classic classifier.
Qiang Chen 0007, Shuicheng Yan, Tian-Tsong Ng
CVPR3
2010 A Dual Theory of Inverse and Forward Light Transport
Jiamin Bai, Manmohan Krishna Chandraker, Tian-Tsong Ng, Ravi Ramamoorthi
ECCV (2)3
2010 Shape from Second-Bounce of Light Transport
Tian-Tsong Ng, Yasuyuki Matsushita
ECCV (2)2
2010 Single-view recaptured image detection based on physics-based features
abstract
In daily life, we can see images of real-life objects on posters, television, or virtually any type of smooth physical surfaces. We seldom confuse these images with the objects per se mainly with the help of the contextual information from the surrounding environment and nearby objects. Without this contextual information, distinguishing an object from an image of the object becomes subtle; it is precisely an effect that a large immersive display aims at achieving. In this work, we study and address a problem that mirrors the above-mentioned recognition problem, i.e., distinguishing images of true natural scenes and those from recapturing. Being able to detect recaptured images, robot vision can be more intelligent and a single-image-based counter-measure for re-broadcast attack on a face authentication system becomes feasible. This work is timely as the face authentication system is getting common on consumer mobile devices such as smart phones and laptop computers. In this work, we present a physical model for image recapturing and the features derived from the model are used in a recaptured image detector. Our physics-based method out-performs a statistics-based method by a significant margin on images of VGA (640×480) and QVGA (320×240) resolutions which are common for mobile devices. In our study, we find that apart from the contextual information, the unique properties for the recaptured image rendering process are crucial for the recognition problem.
Xinting Gao, Tian-Tsong Ng, Shih-Fu Chang
ICME2
2010 Rake transform and edge statistics for image forgery detection
abstract
In this paper, an effective framework for passive-blind color image forgery detection is proposed. It is a combination of image features extracted from image luminance by applying a rake-transform and from image chroma by using edge statistics. The efficacy of the image features has been tested over two color image datasets established for tampering detection. The proposed framework outweighs the state of the arts over the small-scale dataset, and performs well on the newly established large-scaled dataset (likely the first reported test result on this dataset). The initial tests on some real image forgery cases available in the website and those reported in the literature on image composition with advanced image and vision technologies indicate the promise possessed as well as the challenge faced by the community of image forgery detection.
Patchara Sutthiwan, Yun Q. Shi 0001, Wei Su 0001, Tian-Tsong Ng
ICME4
2010 Is physics-based liveness detection truly possible with a single image?
abstract
Face recognition is an increasingly popular method for user authentication. However, face recognition is susceptible to playback attacks. Therefore, a reliable way to detect malicious attacks is crucial to the robustness of the system. We propose and validate a novel physics-based method to detect images recaptured from printed material using only a single image. Micro-textures present in printed paper manifest themselves in the specular component of the image. Features extracted from this component allows a linear SVM classifier to achieve 2.2% False Acceptance Rate and 13% False Rejection Rate (6.7% Equal Error Rate). We also show that the classifier can be generalizable to contrast enhanced recaptured images and LCD screen recaptured images without re-training, demonstrating the robustness of our approach.
Jiamin Bai, Tian-Tsong Ng, Xinting Gao, Yun Q. Shi 0001
ISCAS2
2010 New developments in color image tampering detection
abstract
In this paper, an efficient framework for passive-blind color image tampering detection is presented. Statistical features are extracted from a given test image and a set of 2-D arrays derived by applying multi-size block discrete cosine transform to the given test image. Image features are extracted from Cr channel, a chroma channel in YCbCr color space, because of its observed sensitivity to color image tampering. A support vector machine is employed to evaluate the effectiveness of image features over a color image dataset recently established for tampering detection. Boosting feature selection is applied to having feature dimensionality reduced so as to make detection accuracy generalizable and computational complexity decreased. Experimental results have demonstrated that the proposed framework applied to the aforementioned dataset outperforms the state of the arts by distinct margins.
Patchara Sutthiwan, Yun Q. Shi 0001, Jing Dong 0003, Tieniu Tan, Tian-Tsong Ng
ISCAS5
2010 A Smart Phone Image Database for Single Image Recapture Detection
Xinting Gao, JingJing Shen, Tian-Tsong Ng, Yun Q. Shi 0001
IWDW4
2009 Radiometric compensation using stratified inverses
abstract
Through radiometric compensation, a projector-camera system can project a desired image onto a non-flat and non-white surface. This can be achieved by computing the inverse light transport of a scene. A light transport matrix is in general large and on the order of 106× 106elements. Therefore, computing the inverse light transport matrix is computationally and memory intensive. Two prior methods were proposed to simplify matrix inversion by ignoring scene inter-reflection between individual or clusters of camera pixels. However, compromising scene inter-reflection in spatial domain introduces spatial artifacts and how to systematically adjust the compensation quality is not obvious. In this work, we show how scene inter-reflection can be systematically approximated by stratifying the light transport of a scene. The stratified light transport enables a similar stratification in the inverse light transport. We can show that the stratified inverse light transport converges to the true inverse. For radiometric compensation, the set of stratified inverse light transport provides a systematic way of quantifying the tradeoff between computational efficiency and accuracy. The framework of stratified matrix inversion is general and can have other applications, especially for applications that involve large-size sparse matrices.
Tian-Tsong Ng, Ramanpreet Singh Pahwa, Jiamin Bai, Tony Q. S. Quek, Kar-Han Tan
ICCV1
2008 Recaptured photo detection using specularity distribution
abstract
Detection of planar surfaces in a generic scene is difficult when the illumination is complex and less intense, and the surfaces have non-uniform colors (e.g., a movie poster). As a result, the specularity, if appears, is superimposed with the surface color pattern, and hence the observation of uniform specularity is no longer sufficient for identifying planar surfaces in a generic scene as it does under a distant point light source. In this paper, we address the problem of planar surface recognition in a single generic-scene image. In particular, we study the problem of recaptured photo recognition as an application in image forensics. We discover that the specularity of a recaptured photo is modulated by themesostructure of the photo surface, and its spatial distribution can be used for differentiating recaptured photos from the original photos. We validate our findings in real images of generic scenes. Experimental results show that there is a distinguishable feature of natural scene and recaptured images. Given the definition of specular ratio as the percentage of specularity in the overall measured intensity, the distribution of specular ratio image’s gradient of natural images is Laplacian-like while that of recaptured images is Rayleigh-like.
Tian-Tsong Ng, Qibin Sun
ICIP2
2007 Using Geometry Invariants for Camera Response Function Estimation
abstract
In this paper, we present a new single-image camera response function (CRF) estimation method using geometry invariants (GI). We derive mathematical properties and geometric interpretation for GI, which lend insight to addressing various algorithm implementation issues in a principled way. In contrast to the previous single-image CRF estimation methods, our method provides a constraint equation for selecting the potential target data points. Comparing to the prior work, our experiment is conducted over more extensive data and our method is flexible in that its estimation accuracy and stability can be improved whenever more than one image is available. The geometry invariance theory is novel and may be of wide interest.
Tian-Tsong Ng, Shih-Fu Chang, Mao-Pei Tsui
CVPR1
2005 Physics-motivated features for distinguishing photographic images and computer graphics
abstract
The increasing photorealism for computer graphics has made computer graphics a convincing form of image forgery. Therefore, classifying photographic images and photorealistic computer graphics has become an important problem for image forgery detection. In this paper, we propose a new geometry-based image model, motivated by the physical image generation process, to tackle the above-mentioned problem. The proposed model reveals certain physical differences between the two image categories, such as the gamma correction in photographic images and the sharp structures in computer graphics. For the problem of image forgery detection, we propose two levels of image authenticity definition, i.e., imaging-process authenticity and scene authenticity, and analyze our technique against these definitions. Such definition is important for making the concept of image authenticity computable. Apart from offering physical insights, our technique with a classification accuracy of 83.5% outperforms those in the prior work, i.e., wavelet features at 80.3% and cartoon features at 71.0%. We also consider a recapturing attack scenario and propose a counter-attack measure. In addition, we constructed a publicly available benchmark dataset with images of diverse content and computer graphics of high photorealism.
Tian-Tsong Ng, Shih-Fu Chang, Jessie Hsu, Lexing Xie, Mao-Pei Tsui
ACM Multimedia1
2004 A model for image splicing
abstract
The ease of creating image forgery using image-splicing techniques will soon make our naive trust on image authenticity a tiling of the past. In prior work, we observed the capability of the bicoherence magnitude and phase features for image splicing detection. To bridge the gap between empirical observations and theoretical justifications, in this paper, an image-splicing model based on the idea of bipolar signal perturbation is proposed and studied. A theoretical analysis of the model leads to propositions and predictions consistent with the empirical observations.
Tian-Tsong Ng, Shih-Fu Chang
ICIP1
2004 Predicting optimal operation of MC-3DSBC multidimensional scalable video coding using subjective quality measurement
abstract
Recently we have witnessed a growing interest in the development of the subband/wavelet coding (SBC) technology, partly due to the superior scalability of SBC. Scalable coding provides great synergy with the universal media access applications, where media content is delivered to client devices of diverse types through heterogeneous channels. In this respect, SBC system provides flexibility in realizing different ways of media scaling, including scaling dimensions of SNR, spatial, and temporal. However, the selection of specific scalability operations given the bit rate constraint has always been ad hoc - a systematic methodology is missing. In this paper, we address this open issue by applying our content-based optimal scalability selection framework and adopting subjective quality evaluation. For this purpose we firstly explore the behavior of SNR-Spatial-Temporal scalability using Motion Compensated (MC) SBC systems. Based on the system behavior, we propose an efficient method for the optimal selection of scalability operator through content-based prediction. Our experiment results demonstrate that the proposed method can efficiently predict the optimal scalability operation with an excellent accuracy.
Tian-Tsong Ng, Mihaela van der Schaar, Shih-Fu Chang
VCIP2