Omnidirectional videos (ODVs) present distinct challenges for accurate audio-visual saliency prediction due to their immersive nature, which combines spatial audio with panoramic visuals to enhance the user experience. While auditory cues are crucial for guiding visual attention across the panoramic scene, the interaction between audio and visual stimuli in ODVs remains underexplored. Existing models primarily focus on spatiotemporal visual cues and treat audio signals separately from their spatial and temporal contexts, often leading to misalignments between audio and visual content and undermining temporal consistency across frames. To bridge these gaps, we propose a novel audio-induced saliency prediction model for ODVs that holistically integrates audio and visual inputs through a multi-modal encoder, an audio-visual interaction module, and an audio-visual transformer. Unlike conventional methods that isolate audio cue locations and attributes, our model employs a query-based framework, where learnable audio queries capture comprehensive audio-visual dependencies, thus enhancing saliency prediction by dynamically aligning with audio cues. Besides, we introduce a novel consistency loss to enforce temporal coherence in saliency regions across frames. Extensive experiments demonstrate that our model outperforms state-of-the-art methods in predicting audio-visual salient regions in ODVs, establishing its robustness and superior performance.

This paper presents a comparative study evaluating the performance of Large Language Models (LLMs) in generating timeline summaries from construction delay documents. We assessed seven open-source LLMs and two commercial chatbots (ChatGPT and Claude) on their ability to extract, organize, and summarize delay events from twenty-one carefully curated synthetic snippets of text. The evaluation framework combined automatic metrics (BERTScore and ROUGE scores) with expert human assessment across four dimensions: event description accuracy, date accuracy, event capture completeness, and language quality.Results demonstrate that while commercial solutions, particularly Claude, achieved superior performance, several open-source alternatives showed comparable capabilities. Notably, Llama-3.1-70B-Instruct showed robust performance in event capture and source tracking, while Llama-3.1-8B-Instruct offered efficient processing with balanced performance among smaller models. A critical finding was the widespread challenge in temporal information processing, with only Claude achieving complete accuracy in date extraction and event association. The study's findings suggest that open-source LLMs can serve as practical tools for construction document analysis, although model selection is a critical consideration based on specific accuracy and efficiency requirements, and resource constraints.

Text embeddings are numerical representations of text data, where words, phrases, or entire documents are converted into vectors of real numbers. These embeddings capture semantic meanings and relationships between text elements in a continuous vector space. The primary goal of text embeddings is to enable the processing of text data by machine learning models, which require numerical input. Numerous embedding models have been developed for various applications. This paper presents our work in evaluating different embeddings through a comprehensive comparative analysis of four distinct models, focusing on their text classification efficacy. We employ both K-Nearest Neighbors (KNN) and Logistic Regression (LR) to perform binary classification tasks, specifically determining whether a text snippet is associated with 'delay' or 'not delay' within a labeled dataset. Our research explores the use of text snippet embeddings for training supervised text classification models to identify delay-related statements during the document review process of construction delay disputes. The results of this study highlight the potential of embedding models to enhance the efficiency and accuracy of document analysis in legal contexts, paving the way for more informed decision-making in complex investigative scenarios.

Tumor homing peptides (THPs) play significant role in recognizing and specifically binding to tumor cells. Traditional experimental methods can accurately identify THPs but often suffer from high measurement costs and long experimental cycles. In-silico methods can rapidly screen THPs and accelerate the THPs identification process. Existing THPs prediction methods primarily rely on constructing peptide sequence features and machine learning approaches. These tools rely on manual feature extraction, and the combination of features is crucial for both model performance and robustness. In this study, we propose a method called PLMTHP based on protein language model and integrate multiple machine learning models. Compared to existing models, PLMTHP improves predictions by incorporating high-dimensional features encoded through a protein language model. Our method achieved impressive performance on an independent test set, with ACC of 0.915, MCC of 0.831, and AUC of 0.964. PLMTHP can be downloaded from the following website: https://github.com/Chenyb939/PLMTHP

Email threading is a commonly used tool by legal practitioners to streamline document review and classification in legal proceedings. Threading organizes component pieces of an email thread together to effectively reduce a dataset. The most inclusive threads and their associated document attachments are maintained, and non-inclusive or duplicative thread components are set aside. However, as data volumes continue to grow and outpace deadlines for legal proceedings, practitioners often look to incorporate multiple cost-effective and defensible technology solutions to further reduce or otherwise accelerate document review and classification. One such technology is predictive modeling – known in the legal industry as ‘predictive coding’ or ‘Technology Assisted Review (TAR)’ – which is a popular tool used to augment a manual document review and classification process. Like email threading, predictive coding has become more commonplace recently for its proven ability to minimize manual document classification, thus reducing the time and cost associated with this aspect of legal proceedings.In this study, we explore the performance impacts of layering predictive modeling onto an email threading reduction workflow. Generally, a predictive model is established first, and email threading is layered onto the scored output to further reduce and streamline document review. Our research evaluates a reversed workflow, where a population is initially reduced to its inclusive email threads, and this limited dataset is used to train and apply a predictive model to the larger population of documents for classification. Using classified data from four real-world legal proceedings, we compare the performance impact of email threading on predictive modeling by 1) training a model using all positive and negative examples, 2) training a model using positive and negative examples from only inclusive email threads, and 3) training a model using positive examples from only inclusive email threads, but negative examples from all emails. The results of our research provide thoughtful, empirical insights for legal practitioners to review when exploring the deployment of both email threading and predictive modeling into a single, cohesive document classification strategy for their legal proceedings.

The increased integration of Large Language Models (LLMs) across industry sectors is enabling domain experts with new text classification optimization methods. These LLMs are pretrained on exceedingly large amounts of data; however, practitioners can perform additional training, or “fine-tuning,” to improve their text classifier’s results for their own use cases. This paper presents a series of experiments comparing a standard, pretrained DistilBERT model and a fine-tuned DistilBERT model, both leveraged for the downstream NLP task of text classification. Tuning the model using domain-specific data from real-world legal matters suggests fine-tuning improves the performance of LLM text classifiers.To evaluate the performance of text classification models, using these two Large Language Models, we employed two distinct approaches that 1) score a whole document’s text for prediction and 2) score snippets (sentence-level components of a document) of text for prediction. When comparing the two approaches we found that one prediction method outperforms the other, depending on the use case.

In the domain of natural language processing and text analysis, the Bidirectional Encoder Representations from Transformers (BERT) has emerged as a powerful tool for discerning the intricate nuances of textual data. Nonetheless, BERT’s inherent token limit of 512 tokens presents a noteworthy challenge when confronted with exceedingly long documents. Lengthy documents are commonly encountered during legal document review and often exceed the 512-token constraint. In response to this challenge, this study empirically compares two distinct applications of BERT, leveraging real-world, construction industry legal data. The approach compares applications of BERT to the entire document and on segmented text portions from each document. In the latter approach, the highest-scoring text segment from each document represents the document’s score. This research offers practical insights for effectively utilizing BERT in scenarios where document length exceeds the token limit. Our results allow practitioners and researchers to make informed choices when confronted with documents of significant length, thus contributing to a more effective and insightful application of BERT for text analysis.

With the rapid development of artificial intelligence, especially deep learning technology, various new technologies and applications based on face images have emerged. Obstructive sleep apnea (OSA) is the disease with the highest morbidity and the most serious long-term harm among childhood sleep breathing disorders, and it is increasingly receiving common attention from families and society. Children with the disease have a special facial appearance and require early identification and treatment to prevent it. However, the current diagnostic methods have problems such as being invasive, time-consuming, and expensive. The purpose of this article is to use graph neural network technology based on face images to establish an OSA auxiliary diagnosis strategy for children to achieve OSA screening and analysis. Therefore, this article first takes the facial landmarks as the analysis object, divides the face into six key areas, and selects important landmarks in these regions. On this basis, to better consider the relationship between important landmarks, a global collaborative recognition strategy is proposed. By extracting the implicit relationship between landmarks, face graph structure data is established. Finally, the OSA-GNN model is established to achieve OSA screening and auxiliary analysis in children. Compared with other related studies, this strategy not only has a stronger representation and generalisation ability but can also carry out clinical applications better, providing doctors with diagnostic suggestions.

In recent years, transformer-based, large-scale language models have greatly advanced deep learning in NLP tasks. These models allow transfer learning to be performed on NLP in similar methods to what was previously done for computer vision. Two of the latest models are DistilBERT and Longformer -- the former is a distilled version of BERT, which enables faster training and inferencing on off-cloud servers, and the latter offers capability of working with long texts. In this paper, we study empirical comparisons of the effectiveness of the two deep learning methods along with a logistical regression method for text classification, using three real-world datasets from legal document reviews. The study shows that Longformer performs better (up to 10%) than or at par with the other two methods. Cross-dataset evaluation is leveraged as well to validate the performance of Longformer as a viable method when labeled data is not available.

Relation extraction (RE) is a sub-field of information extraction, which aims to extract the relation between two given named entities (NEs) in a sentence and thus requires a good understanding of contextual information, especially the entities and their surrounding texts. However, limited attention is paid by most existing studies to re-modeling the given NEs and thus lead to inferior RE results when NEs are sometimes ambiguous. In this paper, we propose a RE model with two training stages, where adversarial multi-task learning is applied to the first training stage to explicitly recover the given NEs so as to enhance the main relation extractor, which is trained alone in the second stage. In doing so, the RE model is optimized by named entity recognition (NER) and thus obtains a detailed understanding of entity-aware context. We further propose the adversarial mechanism to enhance the process, which controls the effect of NER on the main relation extractor and allows the extractor to benefit from NER while keep focusing on RE rather than the entire multi-task learning. Experimental results on two English benchmark datasets for RE demonstrate the effectiveness of our approach, where state-of-the-art performance is observed on both datasets.

Aspect-based sentiment analysis (ABSA) aims to predict the sentiment polarity towards a given aspect term in a sentence on the fine-grained level, which usually requires a good understanding of contextual information, especially appropriately distinguishing of a given aspect and its contexts, to achieve good performance. However, most existing ABSA models pay limited attention to the modeling of the given aspect terms and thus result in inferior results when a sentence contains multiple aspect terms with contradictory sentiment polarities. In this paper, we propose to improve ABSA by complementary learning of aspect terms, which serves as a supportive auxiliary task to enhance ABSA by explicitly recovering the aspect terms from each input sentence so as to better understand aspects and their contexts. Particularly, a discriminator is also introduced to further improve the learning process by appropriately balancing the impact of aspect recovery to sentiment prediction. Experimental results on five widely used English benchmark datasets for ABSA demonstrate the effectiveness of our approach, where state-of-the-art performance is observed on all datasets.

Chinese word segmentation (CWS) and named entity recognition (NER) are two important tasks in Chinese natural language processing. To achieve good model performance on these tasks, existing neural approaches normally require a large amount of labeled training data, which is often unavailable for specific domains such as the Chinese medical domain due to privacy and legal issues. To address this problem, we have developed a Chinese medical corpus named ChiMST which consists of question-answer pairs collected from an online medical healthcare platform and is annotated with word boundary and medical term information. For word boundary, we mainly follow the word segmentation guidelines for the Penn Chinese Treebank (Xia, 2000); for medical terms, we define 9 categories and 18 sub-categories after consulting medical experts. To provide baselines on this corpus, we train existing state-of-the-art models on it and achieve good performance. We believe that the corpus and the baseline systems will be a valuable resource for CWS and NER research on the medical domain.

As an important task to analyze the semantic structure of a sentence, semantic role labeling (SRL) aims to locate the semantic role (e.g., agent) of noun phrases with respect to a given predicate and thus plays an important role in downstream tasks such as dialogue systems. To achieve a better performance in SRL, a model is always required to have a good understanding of the context information. Although one can use advanced text encoder (e.g., BERT) to capture the context information, extra resources are also required to further improve the model performance. Considering that there are correlations between the syntactic structure and the semantic structure of the sentence, many previous studies leverage auto-generated syntactic knowledge, especially the dependencies, to enhance the modeling of context information through graph-based architectures, where limited attention is paid to other types of auto-generated knowledge. In this paper, we propose map memories to enhance SRL by encoding different types of auto-generated syntactic knowledge (i.e., POS tags, syntactic constituencies, and word dependencies) obtained from off-the-shelf toolkits. Experimental results on two English benchmark datasets for span-style SRL (i.e., CoNLL-2005 and CoNLL-2012) demonstrate the effectiveness of our approach, which outperforms strong baselines and achieves state-of-the-art results on CoNLL-2005.

During the past decade breakthroughs in GPU hardware and deep neural networks technologies have revolutionized the field of computer vision, making image analytical potentials accessible to a range of real-world applications. Technology Assisted Review (TAR) in electronic discovery though traditionally has dominantly dealt with textual content, is witnessing a rising need to incorporate multimedia content in the scope. We have developed innovative image analytics applications for TAR in the past years, such as image classification, image clustering, and object detection, etc. In this paper, we discuss the use of image clustering applications to facilitate TAR based on our experiences in serving clients. We describe our general workflow on leveraging image clustering in tasks and use statistics from real projects to showcase the effectiveness of using image clustering in TAR. We also summarize lessons learned and best practices on using image clustering in TAR.

Aspect-based sentiment analysis (ABSA) predicts the sentiment polarity towards a particular aspect term in a sentence, which is an important task in real-world applications.To perform ABSA, the trained model is required to have a good understanding of the contextual information, especially the particular patterns that suggest the sentiment polarity.However, these patterns typically vary in different sentences, especially when the sentences come from different sources (domains), which makes ABSA still very challenging.Although combining labeled data across different sources (domains) is a promising solution to address the challenge, in practical applications, these labeled data are usually stored at different locations and might be inaccessible to each other due to privacy or legal concerns (e.g., the data are owned by different companies).To address this issue and make the best use of all labeled data, we propose a novel ABSA model with federated learning (FL) adopted to overcome the data isolation limitations and incorporate topic memory (TM) proposed to take the cases of data from diverse sources (domains) into consideration.Particularly, TM aims to identify different isolated data sources due to data inaccessibility by providing useful categorical information for localized predictions.Experimental results on a simulated environment for FL with three nodes demonstrate the effectiveness of our approach, where TM-FL outperforms different baselines including some well-designed FL frameworks. 1 * Equal contribution.

Most recent studies for relation extraction (RE) leverage the dependency tree of the input sentence to incorporate syntax-driven contextual information to improve model performance, with little attention paid to the limitation where high-quality dependency parsers in most cases unavailable, especially for indomain scenarios.To address this limitation, in this paper, we propose attentive graph convolutional networks (A-GCN) to improve neural RE methods with an unsupervised manner to build the context graph, without relying on the existence of a dependency parser.Specifically, we construct the graph from n-grams extracted from a lexicon built from pointwise mutual information (PMI) and apply attention over the graph.Therefore, different word pairs from the contexts within and across n-grams are weighted in the model and facilitate RE accordingly.Experimental results with further analyses on two English benchmark datasets for RE demonstrate the effectiveness of our approach, where state-of-the-art performance is observed on both datasets.1

In eDiscovery, it is critical to ensure that each page produced in legal proceedings conforms with the requirements of court or government agency production requests. Errors in productions could have severe consequences in a case, putting a party in an adverse position. The volume of pages produced continues to increase, and tremendous time and effort has been taken to ensure quality control of document productions. This has historically been a manual and laborious process. This paper demonstrates a novel automated production quality control application which leverages deep learning-based image recognition technology to extract Bates Number and Confidentiality Stamping from legal case production images and validate their correctness. Effectiveness of the method is verified with an experiment using a real-world production data.

Though technology assisted review in electronic discovery has been focusing on text data, the need of advanced analytics to facilitate reviewing multimedia content is on the rise. In this paper, we present several applications of deep learning in computer vision to Technology Assisted Review of image data in legal industry. These applications include image classification, image clustering, and object detection. We use transfer learning techniques to leverage established pretrained models for feature extraction and fine tuning. These applications are first of their kind in the legal industry for image document review. We demonstrate effectiveness of these applications with solving real world business challenges.

Research has shown that Convolutional Neural Networks (CNN) can be effectively applied to text classification as part of a predictive coding protocol. That said, most research to date has been conducted on data sets with short documents that do not reflect the variety of documents in real world document reviews. Using data from four actual reviews with documents of varying lengths, we compared CNN with other popular machine learning algorithms for text classification, including Logistic Regression, Support Vector Machine, and Random Forest. For each data set, classification models were trained with different training sample sizes using different learning algorithms. These models were then evaluated using a large randomly sampled test set of documents, and the results were compared using precision and recall curves. Our study demonstrates that CNN performed well, but that there was no single algorithm that performed the best across the combination of data sets and training sample sizes. These results will help advance research into the legal profession's use of machine learning algorithms that maximize performance.

Predictive coding has been widely used in legal matters to find relevant or privileged documents in large sets of electronically stored information. It saves the time and cost significantly. Logistic Regression (LR) and Support Vector Machines (SVM) are two popular machine learning algorithms used in predictive coding. Recently, deep learning received a lot of attentions in many industries. This paper reports our preliminary studies in using deep learning in legal document review. Specifically, we conducted experiments to compare deep learning results with results obtained using a SVM algorithm on the four datasets of real legal matters. Our results showed that CNN performed better with larger volume of training dataset and should be a fit method in the text classification in legal industry.

Web traffic is a valuable data source, typically used in the marketing space to track brand awareness and advertising effectiveness. However, web traffic is also a rich source of information for cybersecurity monitoring efforts. To better understand the threat of malicious cyber actors, this study develops a methodology to monitor and evaluate web activity using data archived from Google Analytics. Google Analytics collects and aggregates web traffic, including information about web visitors' location, date and time of visit, visited webpages, and searched keywords. This study seeks to streamline analysis of this data and uses rule-based anomaly detection and predictive modeling to identify web traffic that deviates from normal patterns. Rather than evaluating pieces of web traffic individually, the methodology seeks to emulate real user behavior by creating a new unit of analysis: the user session. User sessions group individual pieces of traffic from the same location and date, which transforms the available information from single point-in-time snapshots to dynamic sessions showing users' trajectory and intent. The result is faster and better insight into large volumes of noisy web traffic.

Financial forecasting is the basis for budgeting activities and estimating future financing needs. Applying machine learning and data mining models to financial forecasting is both effective and efficient. Among different kinds of machine learning models, kernel methods are well accepted since they are more robust and accurate than traditional models, such as neural networks. However, learning from multiple data sources is still one of the main challenges in the financial forecasting area. In this paper, we focus on applying the multiple kernel learning models to the multiple major international stock indexes. Our experiment results indicate that applying multiple kernel learning to the financial forecasting problem suffers from both the short training period problem and non-stationary problem. Therefore we propose a novel multiple kernel learning model to address the challenge by introducing the Gompertz model and considering a non-linear combination of different kernel matrices. The experiment results show that our Gompertz multiple kernel learning model addresses the challenges and achieves better performance than the original multiple kernel learning model and single SVM models.

Abnormal events, such as security attacks, misconfigurations, or electricity failures, could have severe consequences toward the normal operation of the Border Gateway Protocol (BGP) that is in charge of the delivery of packets between different autonomous domains, a key operation for the Internet to function. Unfortunately, it has been a difficult task for network security researchers and engineers to classify and detect these events. In our previous work, we have shown that with classification (which relies on the labeling with domain knowledge from BGP experts), it is feasible to effectively detect and distinguish some worms and blackouts from normal BGP behaviors. In this paper, we move one important step forward—we show that we can automatically detect and classify between different abnormal BGP events based on a hierarchy discovered by clustering. As a systematic application of data mining, we devise a clustering method based on normalized BGP data that forms a tree-like hierarchy of abnormal BGP event classes. We then obtain a set of classification rules for each class (node) in the hierarchy, thus able to label unknown BGP data to a closest class. Our method works even as the BGP dynamics evolve over time, as shown in our experiments with seven different abnormal events during a four-year period. Our work, in a more general context, shows it is promising to conduct an interdisciplinary research between network security and data mining in solving real-world problems.