Instance detection (InsDet) aims to localize specific object instances within a novel scene imagery based on given visual references. Technically, it requires proposal detection to identify all possible object instances, followed by instance-level matching to pinpoint the ones of interest. Its open-world nature supports its broad applications from robotics to AR/VR but also presents significant challenges: methods must generalize to unknown testing data distributions because (1) the testing scene imagery is unseen during training, and (2) there are domain gaps between visual references and detected proposals. Existing methods tackle these challenges by synthesizing diverse training examples or utilizing off-the-shelf foundation models (FMs). However, they only partially capitalize the available open-world information. In contrast, we approach InsDet from an Open-World perspective, introducing our method IDOW. We find that, while pretrained FMs yield high recall in instance detection, they are not specifically optimized for instance-level feature matching. Therefore, we adapt pretrained FMs for improved instance-level matching using open-world data. Our approach incorporates metric learning along with novel data augmentations, which sample distractors as negative examples and synthesize novel-view instances to enrich the visual references. Extensive experiments demonstrate that our method significantly outperforms prior works, achieving >10 AP over previous results on two recently released challenging benchmark datasets in both conventional and novel instance detection settings.

In our increasingly diverse society, everyday physical interfaces often present barriers, impacting individuals across various contexts. This oversight, from small cabinet knobs to identical wall switches that can pose different contextual challenges, highlights an imperative need for solutions. Leveraging low-cost 3D-printed augmentations such as knob magnifiers and tactile labels seems promising, yet the process of discovering unrecognized barriers remains challenging because disability is context-dependent. We introduce AccessLens, an end-to-end system designed to identify inaccessible interfaces in daily objects, and recommend 3D-printable augmentations for accessibility enhancement. Our approach involves training a detector using the novel AccessDB dataset designed to automatically recognize 21 distinct Inaccessibility Classes (e.g., bar-small and round-rotate) within 6 common object categories (e.g., handle and knob). AccessMeta serves as a robust way to build a comprehensive dictionary linking these accessibility classes to open-source 3D augmentation designs. Experiments demonstrate our detector’s performance in detecting inaccessible objects.

The widespread consumer-grade 3D printers and learning resources online enable novices to self-train in remote settings. While troubleshooting plays an essential part of 3D printing, the process remains challenging for many remote novices even with the help of well-developed online sources, such as online troubleshooting archives and online community help. We conducted a formative study with 76 active 3D printing users to learn how remote novices leverage online resources in troubleshooting and their challenges. We found that remote novices cannot fully utilize online resources. For example, the online archives statically provide general information, making it hard to search and relate their unique cases with existing descriptions. Online communities can potentially ease their struggles by providing more targeted suggestions, but a helper who can provide custom help is rather scarce, making it hard to obtain timely assistance. We propose 3DPFIX, an interactive 3D troubleshooting system powered by the pipeline to facilitate Human-AI Collaboration, designed to improve novices' 3D printing experiences and thus help them easily accumulate their domain knowledge. We built 3DPFIX that supports automated diagnosis and solution-seeking. 3DPFIX was built upon shared dialogues about failure cases from Q&A discourses accumulated in online communities. We leverage social annotations (i.e., comments) to build an annotated failure image dataset for AI classifiers and extract a solution pool. Our summative study revealed that using 3DPFIX helped participants spend significantly less effort in diagnosing failures and finding a more accurate solution than relying on their common practice. We also found that 3DPFIX users learn about 3D printing domain-specific knowledge. We discuss the implications of leveraging community-driven data in developing future Human-AI Collaboration designs.

Instance detection (InsDet) is a long-lasting problem in robotics and computer vision, aiming to detect object instances (predefined by some visual examples) in a cluttered scene. Despite its practical significance, its advancement is overshadowed by Object Detection, which aims to detect objects belonging to some predefined classes. One major reason is that current InsDet datasets are too small in scale by today's standards. For example, the popular InsDet dataset GMU (published in 2016) has only 23 instances, far less than COCO (80 classes), a well-known object detection dataset published in 2014. We are motivated to introduce a new InsDet dataset and protocol. First, we define a realistic setup for InsDet: training data consists of multi-view instance captures, along with diverse scene images allowing synthesizing training images by pasting instance images on them with free box annotations. Second, we release a real-world database, which contains multi-view capture of 100 object instances, and high-resolution (6k$\times$8k) testing images. Third, we extensively study baseline methods for InsDet on our dataset, analyze their performance and suggest future work. Somewhat surprisingly, using the off-the-shelf class-agnostic segmentation model (Segment Anything Model, SAM) and the self-supervised feature representation DINOv2 performs the best, achieving $>$10 AP better than end-to-end trained InsDet models that repurpose object detectors (e.g., FasterRCNN and RetinaNet).

This paper investigates how touchscreen exploration and verbal feedback can be used to support blind people to access visual artwork. We present two artwork exploration modalities. The first one, attribute-based exploration, extends prior work on touchscreen image accessibility, and provides fine-grained segmentation of artwork visual elements; when the user touches an element, the associated attributes are read. The second one, hierarchical exploration, is designed with domain experts and provides multi-level segmentation of the artwork; the user initially accesses a general description of the entire artwork and then explores a coarse segmentation of the visual elements with the corresponding high-level descriptions; once selected, coarse segments are subdivided into fine-grained ones, which the user can access for more detailed descriptions.

The authors propose a time series model that predicts future values of various types of liquid cargo traffic based on long short-term memory (LSTM), a deep learning technique. Existing liquid cargo traffic prediction models are based on traditional time series models, such as autoregressive integrated moving average (ARIMA) and vector autoregression (VAR). Some of these models, which do not consider linear dependencies among the values of different types of liquid cargo traffic, have limitations on their prediction performance, because the values of different types of liquid cargo traffic are dependent on one another. These models’ prediction performance are also limited due to the problem of vanishing gradients, which hinders the learning of long-range time series records. Missing values that exist on real-world liquid cargo traffic records reduce prediction performance as well. The proposed LSTM-based time series model handles missing values on liquid cargo traffic records and predicts future values of liquid cargo traffic. In addition, additional indices, such as inflation rates, exchange rates for dollars, GDP values, and the international prices of oil, are used to improve prediction performance. A case study involving real-world liquid cargo traffic records at the Port of Ulsan, Republic of Korea, for 216 months is used to validate prediction performance of the proposed LSTM-based prediction model compared with traditional ARIMA-based and VAR-based prediction models.

One of the reasons that people with visual impairments have difficulties in enjoying artworks is the limited number of accessible artworks. To enable people with visual impairments to explore and understand various artworks independently, we built a touchscreen-based mobile application as a prototype focusing on 2D paintings which plays object-level verbal descriptions upon users' touch. To confirm the needs and to collect initial feedback from potential users of such a system, we conducted an exploratory study with 8 participants with visual impairments using the prototype as a design probe. Overall, participants appreciated the prototype as they can learn artworks with less physical and time constraints while listening for details of contents as they explore each painting by touch.

This study aims to identify the way researchers collaborate with other researchers in the course of the scientific research life cycle and provide information to the designers of e‐Science and e‐Research implementations. On the basis of in‐depth interviews with and on‐site observations of 24 scientists and a follow‐up focus group interview in the field of bioscience/nanoscience and technology in Korea, we examined scientific collaboration using the framework of the scientific research life cycle. We attempt to explain the major motivations, characteristics of communication and information sharing, and barriers associated with scientists' research collaboration practices throughout the research life cycle. The findings identify several notable phenomena including motivating factors, the timing of collaboration formation, partner selection, communication methods, information‐sharing practices, and barriers at each phase of the life cycle. We find that specific motivations were related to specific phases. The formation of collaboration was observed throughout the entire process, not only in the beginning phase of the cycle. For communication and information‐sharing practices, scientists continue to favor traditional means of communication for security reasons. Barriers to collaboration throughout the phases included different priorities, competitive tensions, and a hierarchical culture among collaborators, whereas credit sharing was a barrier in the research product phase.

Abstract In this study a perceived gap between the ideal and the reality of a community network (CN) is examined. Most proponents of CNs state that building a better physical community is their major service goal. However, there has been a concern that citizens might use the service simply as a means to connect to the Internet rather than as a means to connect to their communities. Using a survey research method (n = 213), users' perceptions of community aspects of CN service and the influence of such perceptions on their use were investigated. User demographics and alternative service accessibility were also examined as predictors of use. The present study found that the respondents were using the service mainly for general Internet features. More than two thirds of the respondents were not aware of the community content aspect of the service. Approximately 20% of respondents were identified as those whose perceptions of the community aspects actually affected their use of the service. They were both aware of community contents and using an additional Internet service provider. Findings suggest that the providers did not fully communicate the community aspects of the service with the users, while the user perception of community aspects is a key to further promotion of the service.

Abstract Understanding users' attitudes and perceptions and their influence on behavior is crucial to predict the use of community information and communication technology. In this study, the authors attempt to uncover this process by elaborating on I. Ajzen's ( 1985 , 1991 ) theory of planned behavior (TPB), a widely applied social behavior model. The developed structural equation model (SEM) was tested using a sample of 417 users of a community network. The final selected model, which was called thecommunity network use model, included seven predictors of use: three behavioral beliefs (i.e., Learning, Social Interactions, and Community Connection), Normative Beliefs Individuals, Attitude, Subjective Norm, and Intention. In particular, Intention moderated the relationships between Use and the other six variables. Further, beliefs in both Community Connection and Social Interactions were directly related to community network use, as was Attitude. Belief in Community Connection was indirectly and positively related to community network use via Intention; Belief in Community Connection was directly and negatively related to Use. These latter two findings suggest that Belief in Community Connection serves as both a facilitator and inhibitor of community network use depending on whether belief is followed by intention. Implications are discussed.