The exponential growth of data-driven systems and AI technologies has intensified the demand for high-quality web-sourced datasets. While existing datasets have proven valuable, conventional web data collection approaches face significant limitations in terms of human effort and scalability. Current data collecting solutions fall into two categories: wrapper-based methods that struggle with adaptability and reproducibility, and large language model (LLM)-based approaches that incur substantial computational and financial costs. To address these challenges, we propose AutoData, a novel multi-agent system for Automated web Data collection, that requires minimal human intervention, i.e., only necessitating a natural language instruction specifying the desired dataset. In addition, AutoData is designed for a robust multi-agent architecture, featuring a novel oriented message hypergraph coordinated by a central task manager, to efficiently organize agents across research and development squads. Besides, we introduce a novel hypergraph cache system to advance the multi-agent collaboration process that enables efficient automated data collection and mitigates the token cost issues prevalent in existing LLM-based systems. Moreover, we introduce Instruct2DS, a new benchmark dataset supporting live data collection from web sources across three domains: academic, finance, and sports. Comprehensive evaluations over Instruct2DS and three existing benchmark datasets demonstrate AutoData's superior performance compared to baseline methods. Case studies on challenging tasks such as picture book collection and paper extraction from surveys further validate its applicability.

As the demand for precision medicine rapidly grows, companion diagnostics is proposed to monitor and evaluate therapeutic effects for adjusting medicine plans in time. Although a set of clinical companion diagnostics tools (e.g., polymerase chain reaction) have been investigated, they are expensive and only accessible in a lab environment, which hinders the promotion to broader patients. In light of this situation, we take the first steps towards developing a real-world companion diagnostic tool by leveraging mobile technology. In this paper, we present TherapyPal, a privacy-preserving medicine effectiveness computational framework by harnessing semantic hashing-based digital symptomatic phenotyping. Specifically, sensor data captured from daily-life activities is first transformed into spectrograms. Then, we develop a hashing learning network to extract privacy-masked symptomatic phenotypes on smartphones. Afterward, symptomatic hashes at different medicine states are fed to a contrastive learning network in the cloud for treatment effectiveness detection. To evaluate the performance, we conduct a clinical study among 65 Parkinson's disease (PD) patients under dopaminergic drug treatment. The results show that TherapyPal can achieve around 84.1% medicine effectiveness detection accuracy among patients and above 0.925 privacy-masked scores for protecting each private attribute, which validates the reliability and security of TherapyPal to be used as a real-world companion diagnostics tool.

Monitoring the changes in gait patterns is important to individuals’ health. Gait analysis should be taken as early as possible to prevent gait impairments and improve gait quality. Accurate stride-length estimation and gait rehabilitation activity recognition are fundamental components in gait monitoring, gait analysis, and long-term gait care. This article proposes a novel multimodality deep learning architecture to investigate the applications of stride length (SL) estimation and rehabilitation activity recognition. In order to verify this architecture, we have conducted the data collection and data labeling with our customized wearable sensing system. The sensing system can provide sensor readings from 96 sensors-based pressure array and 3-channels accelerometer and gyroscope. Many experiments with multiple perspective analysis are implemented to evaluate the models’ precision, robustness, and reliability. The multimodality deep learning architecture can map multiple sensor readings to the resulting SL with a mean absolute error of 3.89 cm and accurately detect the gait activity with an accuracy of 97.08%. It correlates the step length estimation and gait activity recognition to fulfill comprehensive long-term gait information statistic. The proposed applications’ implementation enriched our previous gait study and brought insights for clinically relevant wearable gait monitoring and gait analysis.

In the era of big data, data-driven based classification has become an essential method in smart manufacturing to guide production and optimize inspection. The industrial data obtained in practice is usually time-series data collected by soft sensors, which are highly nonlinear, nonstationary, imbalanced, and noisy. Most existing soft-sensing machine learning models focus on capturing either intra-series temporal dependencies or pre-defined inter-series correlations, while ignoring the correlation between labels as each instance is associated with multiple labels simultaneously. In this paper, we propose a novel graph based soft-sensing neural network (GraSSNet) for multivariate time-series classification of noisy and highly-imbalanced soft-sensing data. The proposed GraSSNet is able to 1) capture the inter-series and intra-series dependencies jointly in the spectral domain; 2) exploit the label correlations by superimposing label graph that built from statistical co-occurrence information; 3) learn features with attention mechanism from both textual and numerical domain; and 4) leverage unlabeled data and mitigate data imbalance by semi-supervised learning. Comparative studies with other commonly used classifiers are carried out on Seagate soft sensing data, and the experimental results validate the competitive performance of our proposed method.

"IEEE BigData 2021 Cup: Soft Sensing at Scale" is a data mining competition organized by Seagate Technology, in association with the IEEE BigData 2021 conference. The scope of this challenge is to tackle the task of classifying soft sensing data with machine learning techniques. In this paper we go into the details of the challenge and describe the data set provided to participants. We define the metrics of interest, baseline models, and describe approaches we found meaningful which may be a good starting point for further analysis. We discuss the results obtained with our approaches and briefly summarize the submissions from the participants.Students, researchers, and anyone interested in working on a major industrial problem are welcome to participate in this challenge and any future challenges!

The growing availability of the data collected from smart manufacturing is changing the paradigms of production monitoring and control. The increasing complexity and content of the wafer manufacturing process in addition to the time-varying unexpected disturbances and uncertainties, make it infeasible to do the control process with model-based approaches. As a result, data-driven soft-sensing modeling has become more prevalent in wafer process diagnostics. Recently, deep learning has been utilized in soft sensing system with promising performance on highly nonlinear and dynamic time-series data. Despite its successes in soft-sensing systems, however, the underlying logic of the deep learning framework is hard to understand. In this paper, we propose a deep learning-based model for defective wafer detection using a highly imbalanced dataset. To understand how the proposed model works, the deep visualization approach is applied. Additionally, the model is then fine-tuned guided by the deep visualization. Extensive experiments are performed to validate the effectiveness of the proposed system. The results provide an interpretation of how the model works and an instructive fine-tuning method based on the interpretation.

Over the last few decades, modern industrial processes have investigated several cost-effective methodologies to improve the productivity and yield of semiconductor manufacturing. While playing an essential role in facilitating real-time monitoring and control, the data-driven soft-sensors in industries have provided a competitive edge when augmented with deep learning approaches for wafer fault-diagnostics. Despite the success of deep learning methods across various domains, they tend to suffer from bad performance on multi-variate soft-sensing data domains. To mitigate this, we propose a soft-sensing ConFormer (CONvolutional transFORMER) for wafer fault-diagnostic classification task which primarily consists of multi-head convolution modules that reap the benefits of fast and light-weight operations of convolutions, and also the ability to learn the robust representations through multi-head design alike transformers. Another key issue is that traditional learning paradigms tend to suffer from low performance on noisy and highly-imbalanced soft-sensing data. To address this, we augment our soft-sensing ConFormer model with a curriculum learning-based loss function, which effectively learns easy samples in the early phase of training and difficult ones later. To further demonstrate the utility of our proposed architecture, we performed extensive experiments on various toolsets of Seagate Technology’s wafer manufacturing process which are shared openly along with this work. To the best of our knowledge, this is the first time that curriculum learning-based soft-sensing ConFormer architecture has been proposed for soft-sensing data and our results show strong promise for future use in soft-sensing research domain.

With the rapid development of AI technology in recent years, there have been many studies with deep learning models in soft sensing area. However, the models have become more and more complex, yet, the data sets remain limited: researchers are fitting million-parameter models with hundreds of data samples, which is insufficient to exercise the effectiveness of their models. To solve this long-lasting problem, we are providing large-scale high-dimensional time series manufacturing sensor data from Seagate Technology to the public. We demonstrate the challenges and effectiveness of modeling industrial big data by a Soft Sensing Transformer model on these data sets. Transformer is used because, it has outperformed state-of-the-art techniques in Natural Language Processing, and since then has also performed well in the direct application to computer vision without introduction of image- specific inductive biases. We observe the similarity of a sentence structure to the sensor readings and process the multi-variable sensor readings in a similar manner of sentences in natural language. The high-dimensional time series data is formatted into the same shape of embedded sentences and fed into the transformer model. The results show that transformer model outperforms the benchmark models in soft sensing field based on auto-encoder and long short-term memory (LSTM). To the best of our knowledge, we are the first team in academia or industry to benchmark the performance of original transformer model with large-scale numerical soft sensing data. Additionally, In contrast to the natural language processing or computer vision tasks where human-level performances are regarded as golden standards, our large-scale soft sensing study is an example that transformer is able to interpret high-dimensional numerical data which is not interpretable by human.

Deep learning has been becoming a promising focus in data mining research. With deep learning techniques, researchers can discover deep properties and features of events from quantitative mobile sensor data. However, many data sources are geographically separated and have strict privacy, security, and regulatory constraints. Upon releasing the privacy-sensitive data, these data sources generally no longer physically possess their data and cannot interfere with the way their personal data being used. Therefore, it is necessary to explore distributed data mining architecture which is able to conduct consensus learning based on needs. Accordingly, we propose a distributed deep learning optimized system which contains a cloud server and multiple smartphone devices with computation capabilities and each device is served as a personal mobile data hub for enabling mobile computing while preserving data privacy. The proposed system keeps the private data locally in smartphones, shares trained parameters, and builds a global consensus model. The feasibility and usability of the proposed system are evaluated by three experiments and related discussion. The experimental results show that the proposed distributed deep learning system can reconstruct the behavior of centralized training. We also measure the cumulative network traffic in different scenarios and show that the partial parameter sharing strategy does not only preserve the performance of the trained model but also can reduce network traffic. User data privacy is protected on two levels. First, local private training data do not need to be shared with other people and the user has full control of their personal training data all the time. Second, only a small fraction of trained gradients of the local model are selected for sharing, which further reduces the risk of information leaking.

With the development of the Internet of Things (IoT), wearable technologies have been proposed to measure gait parameters in everyday life. However, since both diseases and activities could influence gait patterns, clinicians cannot use the measured gait parameters for clinical applications without knowing the corresponding activities. To address this problem, a novel gait analysis method—“gait analysis in terms of activities of daily living (ADLs)”—was proposed based on a wearable Smart Insole system. Twenty six gait parameters were extracted to realize a systematic gait analysis. Novel activity recognition algorithms based on characteristics of human gait were proposed to recognize ADLs, including “sitting,” “standing,” “walking,” “running,” “ascend stairs,” and “descend stairs” with high accuracy and low computation load. To evaluate the performance of “gait analysis in terms of ADLs,” an experiment consisting of a sequence of different ADLs was designed to simulate the scenario of everyday life. In the result, gait parameters measured during different activities were automatically highlighted with different colors, which made it easy to see whether the gait pattern change was caused by activities or diseases. Besides, a refined gait analysis could be realized by individually extracting and analyzing the gait parameters of a specific activity. The results indicate that “gait analysis in terms of ADLs” is a feasible method to reach the aim of bringing gait lab to everyday life.

With the development of Internet of Things (IoT), indoor localization has been a research focus in recent years. For inertial measurement unit (IMU)-based indoor localization method, zero velocity update (ZUPT) uses the known velocity at stationary epoch as a benchmark to calibrate the velocity drift. However, stationary epoch only takes up 24% of a whole gait cycle time, and the velocity drift at the remaining 76% time is usually estimated according to an assumption that velocity has a linear drift over time, which would introduce errors. In this paper, a two-step velocity calibration method was proposed based on human gait characteristics with Smart Insole: known velocity update (KUPT) and double-foot position calibration (DFPC). KUPT could measure the velocity from heel-strike to toe-off based on the recorded real-time foot angle and the shoe dimensions, which increases the time period when the velocity could be measured from 24% to 62% of a whole gait cycle time. DFPC method could fuse the position information of both feet based on the symmetrical characteristic of human gait to further increase the reliability of the localization results. The statistical result of a 20 times 20-m walking experiment showed that KUPT method was more accurate and reliable than ZUPT method for both feet, and DFPC method could further improve the result of KUPT method. Another experiment about walking in an indoor environment for 91 m showed that the proposed KUPT+DFPC method had an error of about 0.78 m which is acceptable for most IoT applications.