Video Anomaly Detection (VAD) is essential for computer vision and multimedia research. Existing VAD methods utilize either reconstruction-based or prediction-based frameworks. The former excels at detecting irregular patterns or structures, whereas the latter is capable of spotting abnormal deviations or trends. We address pose-based video anomaly detection and introduce a novel framework called Dual Conditioned Motion Diffusion (DCMD), which enjoys the advantages of both approaches. The DCMD integrates conditioned motion and conditioned embedding to comprehensively utilize the pose characteristics and latent semantics of observed movements, respectively. In the reverse diffusion process, a motion transformer is proposed to capture potential correlations from multi-layered characteristics within the spectrum space of human motion. To enhance the discriminability between normal and abnormal instances, we design a novel United Association Discrepancy (UAD) regularization that primarily relies on a Gaussian kernel-based time association and a self-attention-based global association. Finally, a mask completion strategy is introduced during the inference stage of the reverse diffusion process to enhance the utilization of conditioned motion for the prediction branch of anomaly detection. Extensive experiments conducted on four datasets demonstrate that our method dramatically outperforms state-of-the-art methods and exhibits superior generalization performance.

Various types of surgical robots have gradually occupied more positions in modern medical care due to safety and stability reasons. Retinal surgical robots are still in the development stage, so there is still a lack of some surgical indicators to evaluate the safety and stability of surgical robots. In order to evaluate the performance of robot-assisted surgery more accurately, this paper proposes a method to evaluate the tremor of tools based on video collected by a microscope in surgery. This method is mainly dedicated to evaluating the tremor level of tools in clinical retinal surgery, in order to verify the pulling of the wound during surgery. The method is mainly divided into three parts: tool semantic segmentation, key point extraction, tip-end reasoning and tremor evaluation. Firstly, the method extracts the outline label of the tool through semantic segmentation. Then, some key points on the outline, which could reflect the posture, could be obtained by skeleton extraction, edge straight line fitting. After that, these points could be used to assess the tilt angle of the needle. Also, according to the angle and the actual length of the needle tip, the position of the needle tip could be assessed in the image, and finally the algorithm could evaluate the tremor level based on the change in position. This method could evaluate the tremor of the tool in clinical surgery, which can help to evaluate the safety and stability of the retinal injection surgery.

Existing visual assistive technologies are built for simple and common use cases, and have few avenues for blind people to customize their functionalities. Drawing from prior work on DIY assistive technology, this paper investigates end-user programming as a means for users to create and customize visual access programs to meet their unique needs. We introduce ProgramAlly, a system for creating custom filters for visual information, e.g., ‘find NUMBER on BUS’, leveraging three end-user programming approaches: block programming, natural language, and programming by example. To implement ProgramAlly, we designed a representation of visual filtering tasks based on scenarios encountered by blind people, and integrated a set of on-device and cloud models for generating and running these programs. In user studies with 12 blind adults, we found that participants preferred different programming modalities depending on the task, and envisioned using visual access programs to address unique accessibility challenges that are otherwise difficult with existing applications. Through ProgramAlly, we present an exploration of how blind end-users can create visual access programs to customize and control their experiences.

Blind or Visually Impaired (BVI) people often encounter flat, inaccessible interfaces. Current solutions lack cost-effectiveness, portability, and robustness in real-world settings. We introduce VizLens, a fully-automated, full-stack mobile application powered by computer vision algorithms. The system is deployed and publicly available through the Apple App Store (https://vizlens.org/). From May to August 2023, we had 665 users, who uploaded 1,320 interface images. We aim to use it to study usage patterns and possible challenges BVI users may encounter with flat interfaces through a large-scale study in real-world settings. With in-depth analysis of user data and activity logs, our study will provide insights into BVI users’ interface interests, preferred assistance modes, and potential challenges due to system limitations or users’ diverse abilities. Our goal is to enhance the understanding of how BVI users interact with inaccessible, flat interfaces, and inform future assistive technology design.

Existing assistive technologies (AT) often fail to support the unique needs of blind and visually impaired (BVI) people. Thus, BVI people have become domain experts in customizing and ‘hacking’ AT, creatively suiting their needs. We aim to understand this behavior in depth, and how BVI people envision creating future DIY personalized AT. We conducted a multi-part qualitative study with 12 blind participants: an interview on unique uses of AT, a two-week diary study to log use cases, and a scenario-based design session to imagine creating future technologies. We found that participants work to design new AT both implicitly through creative use cases, and explicitly through regular ideation and development. Participants envisioned creating a variety of new technologies, and we summarize expected benefits and concerns of using a DIY technology approach. From our results, we present design considerations for future DIY technology systems to support existing customization and ‘hacking’ behaviors.