While Large Language Models (LLMs) possess remarkable generative capabilities, generating diversified and engaging ad headlines in industrial applications remains challenging. Conventional training paradigms often suffer from mode collapse, converging on dominant data patterns and yielding homogeneous outputs. Meanwhile, existing diversity-enhancing techniques like stochastic decoding frequently compromise semantic coherence and controllability. To break this trade-off, we propose DIVER, an automated training framework that internalizes diversity as an intrinsic model capability. DIVER employs an automatic data pipeline to synthesize high-quality, multi-faceted training pairs and utilizes multi-objective reinforcement learning to effectively co-optimize diversity with advertising metrics such as faithfulness and click-through rate (CTR). Unlike personalized approaches, our framework generates diverse content for general users without relying on heavy and costly user-behavior modeling, ensuring efficient inference for large-scale real-time systems. Real-world deployment on Xiaohongshu's Explore Feed demonstrates significant commercial impact, increasing advertiser value (ADVV) by 4.0% and CTR by 1.4%.

With the advent of the smart medical era, medical images play a central role in clinical diagnosis and treatment. However, the existence of low-quality medical images has significantly impacted the progress of smart medicine. In order to solve the problems of noise, fuzziness, and insufficient contrast faced by traditional convolutional neural networks when processing low-quality medical images, this article innovatively adopts the probabilistic linguistic term sets to characterize the fuzzy degree of the image, and proposes a probabilistic linguistic convolutional neural network (PL-CNN) which fuses probabilistic linguistic information. For this new PL-CNN, we provide a complete calculation process including forward propagation, backward propagation, and parameter update. Finally, we apply the PL-CNN to the classification of CIFAR-10 series datasets and breast datasets, demonstrating the versatility and effectiveness of the proposed method. This article not only provides a universal deep-learning method for image classification, but also offers a new idea and method for the processing of low-quality medical images in smart healthcare.

Tool learning aims to extend the capabilities of large language models (LLMs) with external tools. A major challenge in tool learning is how to support a large number of tools, including unseen tools. To address this challenge, previous studies have proposed retrieving suitable tools for the LLM based on the user query. However, previously proposed methods do not consider the differences between seen and unseen tools, nor do they take the hierarchy of the tool library into account, which may lead to suboptimal performance for tool retrieval. Therefore, to address the aforementioned issues, we propose ToolRerank, an adaptive and hierarchy-aware reranking method for tool retrieval to further refine the retrieval results. Specifically, our proposed ToolRerank includes Adaptive Truncation, which truncates the retrieval results related to seen and unseen tools at different positions, and Hierarchy-Aware Reranking, which makes retrieval results more concentrated for single-tool queries and more diverse for multi-tool queries. Experimental results show that ToolRerank can improve the quality of the retrieval results, leading to better execution results generated by the LLM.

Black-box prompt tuning employs derivative-free optimization algorithms to learn prompts within low-dimensional subspaces rather than back-propagating through the network of Large Language Models (LLMs). Recent studies reveal that black-box prompt tuning lacks versatility across tasks and LLMs, which we believe is related to the suboptimal choice of subspaces. In this paper, we introduceBlack-box prompt tuning withSubspaceLearning (BSL) to enhance the versatility of black-box prompt tuning. Based on the assumption that nearly optimal prompts for similar tasks reside in a common subspace, we propose identifying such subspaces through meta-learning on a collection of similar source tasks. Consequently, for a target task that shares similarities with the source tasks, we expect that optimizing within the identified subspace can yield a prompt that performs well on the target task. Experimental results confirm that our BSL framework consistently achieves competitive performance across various downstream tasks and LLMs.

Large-scale group decision making becomes increasingly common with the rapid development of society and the increasing complexity of practical problems. However, it is difficult to distinguish the semantic differences between the same linguistic term, and original linguistic term may not express flexible semantics, so that this will affect the precise of decision-making results. With the help of granular computing, this article adopts a new format of linguistic term, named as hesitant fuzzy linguistic term set with granularity level, to endow preference information with flexibility and at a specific granularity. Then, in this study, we propose a novel intelligent biobjective optimization method for large-scale group decision making, considering group consensus degree and group risk degree in the decision-making process, where group risk degree is measured from the motivation of portfolio risk. Differential evolution is used to handle biobjective optimization method to determine the optimal results. We also introduce an additive consistency measure and develop a method to derive the corresponding threshold values through Monte Carlo simulation. Finally, the case study and comparison results are covered to demonstrate the practicality and superiority of the proposed method. This work has some original points: 1) Hesitant fuzzy linguistic term set with granularity level brings flexibility to the decision-making process. 2) Group consensus degree and group risk degree are involved in biobjective optimization method, where the group risk degree is measured from the motivation of portfolio risk. 3) A novel additive consistency measure is proposed and different threshold values of preference relations in different dimensions are derived.

Jianhai Zhang, Mieradilijiang Maimaiti, Gao Xing, Yuanhang Zheng, Ji Zhang. Proceedings of the 2022 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies. 2022.

Deep learning presents excellent learning ability in constructing learning model and greatly promotes the development of artificial intelligence, but its conventional models cannot handle uncertain or imprecise circumstances. Fuzzy systems, can not only depict uncertain and vague concepts widely existing in the real world, but also improve the prediction accuracy in deep learning models. Thus, it is important and necessary to go through the recent contributions about the fusion of deep learning and fuzzy systems. At first, we introduce the deep learning into fuzzy community from two perspectives: statistical results of relevant publications and conventional deep learning algorithms. Then, the fusing framework and graphic form of deep learning and fuzzy systems are constructed. Followed by, are the current situations of several types of fuzzy techniques used in deep learning, some reasons why use fuzzy techniques in deep learning, and the application fields of the fusion, respectively. Finally, some discussions and future challenges are provided regarding the fusion technology of deep learning and fuzzy systems, the application scenarios of fusing deep learning and fuzzy systems, and some limitations of the current fusion, respectively. After summarizing the recent contributions, we have found that this field is an emerging research direction and it is increasingly paying much more attention. Especially, fuzzy systems make great effects on deep learning models in the aspect of classification, prediction, natural language processing, auto-control, etc., and the fusion is applied into different fields, like but not limited to computer science, natural language, medical system, smart energy management systems and manufacturing industry.

Considering the conditions that: 1) same linguistic term means different things for different people; 2) flexible semantics cannot be represented by original linguistic term; and 3) some semantics given by decision makers are possible to be changed during the consistency improving process, we bring some flexibility and personality into hesitant fuzzy linguistic preference matrix structures by allowing the linguistic preference matrices to be granular rather than numeric, providing a new characterization of linguistic preference matrices. Inspired by the thought of granular computing, this article proposes a new hesitant fuzzy linguistic method to deal with issues when a lot of decision makers provide hesitant and uncertain preference information in the decision-making process. First, we design a multiplicative consistency index and calculate its thresholds corresponding to different dimensions of preference matrix by the Monte Carlo experiment. Then, we construct a hesitant fuzzy linguistic model with granularity level, so as to recharacterize original assessment information and improve the consistency of preference matrices as far as possible. Considering the features of some large-scale group decision-making situation, where the decision makers have little opportunity to take part in multiple consensus reaching processes, hesitant fuzzy linguistic fuzzy$C$-means clustering algorithm is developed to integrate the assessment information given by decision makers. Finally, the final decision-making results are derived. An illustrative example of assessing psychological situation of some COVID-19 infected persons clarifies the reasonability of the proposed method. Finally, we complete some comparative studies and simulation experiments to demonstrate the method’s validity and advantages.

Mieradilijiang Maimaiti, Yang Liu, Yuanhang Zheng, Gang Chen, Kaiyu Huang, Ji Zhang, Huanbo Luan, Maosong Sun. Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing. 2021.

Yuanhang Zheng, Zhixing Tan, Meng Zhang, Mieradilijiang Maimaiti, Huanbo Luan, Maosong Sun, Qun Liu, Yang Liu. Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing. 2021.

In multiattribute decision making, the analytic network process (ANP) is an important methodology to derive the subjective weights of attributes when the dependence and feedback relations exist between attributes, and the number of attributes should be no more than seven in a comparison matrix. To reduce the dimensions of attributes, we propose a hybrid hesitant fuzzy linguistic factor analysis method to cluster the attributes into main factors. The method takes multiple forms of decision-making information into consideration, such as single linguistic terms, hesitant fuzzy linguistic terms, and numeric values. Meanwhile, the objective weights of the main factors are obtained as well. As for the subjective weights of main factors, the incomplete probabilistic linguistic ANP is developed after improving the incomplete probabilistic linguistic preference relation with multiplicative consistency. At last, the final weights of the main factors are calculated by combining the objective and subjective weights. A questionnaire survey about assessing the weights of the main factors influencing graduate students' physical health is designed to explain the application of the proposed methodology. To sum up, the main importance and contributions of this study are as following: (1) developing a hybrid hesitant fuzzy linguistic factor analysis method and incomplete probabilistic linguistic ANP, (2) proposing a novel weight-derived method from both objective and subjective perspectives, and (3) applying it to graduate students' physical health assessment.

In this paper, minimization of the statistical dependence is exploited for acoustic source localization purposes. Originally developed for the separation of signal mixtures, we show that Independent Component Analysis (ICA) can also be successfully applied to localize multiple simultaneously active sound sources, with possibly less sensors than sources. First, the recently proposed Averaged Directivity Pattern (ADP) and State Coherence Transform (SCT) methods are reviewed. Similarities and differences between both approaches are underlined and analyzed, leading to a new method merging elements from both concepts, which we call the Modified ADP (MADP). Since the investigated methods do not suffer from the permutation ambiguity, they can be applied in combination with any narrowband or broadband ICA algorithm, without the need to solve the still challenging permutation issue. Experimental results are presented for speech sources in a reverberant environment.

In this paper, we show that minimization of the statistical dependence using broadband independent component analysis (ICA) can be successfully exploited for acoustic source localization. As the ICA signal model inherently accounts for the presence of several sources and multiple sound propagation paths, the ICA criterion offers a theoretically more rigorous framework than conventional techniques based on an idealized single-path and single-source signal model. This leads to algorithms which outperform other localization methods, especially in the presence of multiple simultaneously active sound sources and under adverse conditions, notably in reverberant environments. Three methods are investigated to extract the time difference of arrival (TDOA) information contained in the filters of a two-channel broadband ICA scheme. While for the first, the blind system identification (BSI) approach, the number of sources should be restricted to the number of sensors, the other methods, the averaged directivity pattern (ADP) and composite mapped filter (CMF) approaches can be used even when the number of sources exceeds the number of sensors. To allow fast tracking of moving sources, the ICA algorithm operates in block-wise batch mode, with a proportionate weighting of the natural gradient to speed up the convergence of the algorithm. The TDOA estimation accuracy of the proposed schemes is assessed in highly noisy and reverberant environments for two, three, and four stationary noise sources with speech-weighted spectral envelopes as well as for moving real speech sources.