In this paper, we present RigAnyFace (RAF), a scalable neural auto-rigging framework for facial meshes of diverse topologies, including those with multiple disconnected components. RAF deforms a static neutral facial mesh into industry-standard FACS poses to form an expressive blendshape rig. Deformations are predicted by a triangulation-agnostic surface learning network augmented with our tailored architecture design to condition on FACS parameters and efficiently process disconnected components. For training, we curated a dataset of facial meshes, with a subset meticulously rigged by professional artists to serve as accurate 3D ground truth for deformation supervision. Due to the high cost of manual rigging, this subset is limited in size, constraining the generalization ability of models trained exclusively on it. To address this, we design a 2D supervision strategy for unlabeled neutral meshes without rigs. This strategy increases data diversity and allows for scaled training, thereby enhancing the generalization ability of models trained on this augmented data. Extensive experiments demonstrate that RAF is able to rig meshes of diverse topologies on not only our artist-crafted assets but also in-the-wild samples, outperforming previous works in accuracy and generalizability. Moreover, our method advances beyond prior work by supporting multiple disconnected components, such as eyeballs, for more detailed expression animation.

Vector fields are widely used to represent and model flows for many science and engineering applications. This paper introduces a novel neural network architecture for learning tangent vector fields that are intrinsically defined on manifold surfaces embedded in 3D. Previous approaches to learning vector fields on surfaces treat vectors as multi-dimensional scalar fields, using traditional scalar-valued architectures to process channels individually, thus fail to preserve fundamental intrinsic properties of the vector field. The core idea of this work is to introduce a trainable vector heat diffusion module to spatially propagate vector-valued feature data across the surface, which we incorporate into our proposed architecture that consists of vector-valued neurons. Our architecture is invariant to rigid motion of the input, isometric deformation, and choice of local tangent bases, and is robust to discretizations of the surface. We evaluate our Vector Heat Network on triangle meshes, and empirically validate its invariant properties. We also demonstrate the effectiveness of our method on the useful industrial application of quadrilateral mesh generation.

We examine opportunities for ubiquitous technologies in retail shopping, jewelry shopping in this case, to supplement the unique information needs inherent to physical trials of tactile products. We describe an iterative design approach to develop a mirror system that records and matches images across jewelry trials called the Countertop Responsive Mirror. The key technological distinction of our system from prior technologies is the use of "matched access," which automatically retrieves images that match a scene shown in separately accessed images. This not only helps shoppers compare jewelry but also promotes interactions among all parties during shopping. We report qualitative findings from multiple field trials of the system. This paper contributes to a body of research on the design and introduction of new technologies into retail shopping that provide value to all users without disruption to their normative practices and behaviors.

In this paper, we describe the architecture of the vision system for the Responsive Mirror, a novel system for retail fitting rooms that enables online social fashion comparisons in physical stores based on multi-camera perception. This vision system provides implicitly controlled real-time interaction for “self ” and “social ” clothing comparisons by automatically tracking user’s motion as she tries on clothes. We describe the key components of the motion-tracking and clothes-recognition systems and evaluate their effectiveness against images collected during a previous user study and a dataset of images representing content from a social fashion network. Author Keywords Ubiquitous computing, computer vision, multi-camera,

This paper investigates the problem of designing decentralized representations to support monitoring and inferences in sensor networks. State-space models of physical phenomena such as those arising from tracking multiple interacting targets, while commonly used in signal processing and control, suffer from the curse of dimensionality as the number of phenomena of interest increases. Furthermore, mapping an inference algorithm onto a distributed sensor network must appropriately allocate scarce sensing and communication resources. We address the state-space explosion problem by developing a distributed state-space model that switches between factored and joint state spaces as appropriate. We develop a collaborative group abstraction as a mechanism to effectively support the information ow within and across subspaces of the state-space model, which can be efficiently supported in a communication-constrained network. The approach has been implemented and demonstrated in a simulation of tracking multiple interacting targets.

As sensing technology becomes cheaper and more effective, large networks of intelligent sensors with rich sensing modalities, such as video, will extend into busy large-scale environments in applications such as traffic safety, urban surveillance, and situational awareness. In these unstructured and highly active environments, we consider the problem of detecting, identifying, and monitoring anomalous or suspicious activities among a large number of behaviors of people or vehicles going about their everyday business. Monitoring the activities of all objects in these environments is overly intrusive as well as impractical due to physical limitations of sensing devices and computing power. Therefore, a system to be deployed in such environments must be able to simultaneously (1) ignore objects behaving normally, (2) search for newly emerging anomalous behaviors, and (3) track and monitor objects behaving abnormally. In such systems, the large areas covered and huge amount of data generated require us to implement the system on a distributed platform with limited sensing, processing, and communication resources.Our solution is motivated by the visual attention system in humans, where the eye is attracted to interesting stimuli which pop from the environment, and the brain interprets the most important stimuli in greater detail by focusing the eyes and its mental faculties on them. Our distributed attention architecture consists of a layered information processing module that interprets data from multiple perspectives and a resource allocation module to allocate limited sensing resources based on the system's current knowledge. Thus, distributed attention implements a complete loop, consisting of data observation, information extraction, and finally, sensor control for the next set of observations.The information processing tasks are layered and range from evaluating simple predicates to verifying hypotheses about complex behaviors. A low observation, such as detecting a cluster of pixels moving at an anomalous speed may spawn a vehicle localization and tracking task. This may, in turn, spawn high hypotheses about its behavior or intent, or may evaluate its behavior in relation to other nearby vehicles. Reasoning about these hypotheses may reassure the system that the behavior is normal so that the lower level anomaly detection can be adjusted to ignore similar behavior in the future, or it may lead to further focusing of resources to investigate. By processing data incrementally, the system can use intermediate information to make decisions on what to ignore and on what to focus.Executing a task requires data from some local region of the world. Since there are multiple tasks with different, possibly conflicting data requirements, we have developed a resource allocation module to optimize each task's access to limited sensing resources to maximize the global usefulness of the sensor configuration. In our current implementation, all requests are tagged with a utility value based on the importance of the task they are accomplishing, and sensors are assigned to observe the regions which maximize the sum of the utilities of the all the tasks served. Our testbed to demonstrate this distributed attention system consists of a network of 6 or more OpenBrick x86 Linux boxes, outfitted with steerable pan-tilt cameras which observe a rectangular board (approx. 6' x 8') on the floor where the objects of interest move. The field of view (FOV) of each camera is too small to see the entire board at once. The cameras pan and tilt to discrete directions, so that any one camera can only sense a small portion of the board. Some regions may be observed by multiple cameras, while others only have single coverage. This work is not primarily focused on image understanding, so vehicles and people are represented here as moving, colored dots, performing various behaviors under automatic or manual control. Information processing tasks can jump from node to node, issuing requests to the resource allocator, which points the cameras appropriately.We will demonstrate a scenario with many moving dots flowing normally around the board. One or more objects then exhibit anomalous behavior, like pushing other objects out of the way. The layered information processing module detects anomalous behavior, looking for local regions with statistically unlikely deviations from the learned velocity field, hypothesizing candidate objects that caused the anomalous behavior, tracking them to gather more information, and finally, identifying the behavior based on existing high-level models of anomalous behavior. From the intermediate processing results, the tasks generate data requests for viewing a subset of FOV's, and the resource allocation module then maximizes the utility of the competing tasks by formulating the set of utility tagged requests over possible camera pan-tilt directions as a factor graph and applying the max-sum algorithm (a variant of the sum-product algorithm). The cameras are then moved, new images captured, and the cycle is repeated.