The emergence of large language models (LLMs) has substantially changed the artificial intelligence field, enabling its wide use over different domains. As various LLM alternatives have been developed, the current study proposes a novel decision‐support framework for evaluating and benchmarking LLMs based on multicriteria decision‐making (MCDM) techniques. In the proposed framework, an improved version of the best‐worst method (BWM) is proposed to effectively reduce the computational complexity of assigning a critical weight for the evaluation criteria of LLMs. Then, the improved BWM is integrated with the combined compromise solution (CoCoSo) method for ranking LLM alternatives. Findings show that the improved BWM successfully computes the criteria weights with low computational complexity compared to the original BWM. According to the enhanced BWM, the ‘factual errors’ criterion received the highest significant weight (0.2681), while the ‘logical inconsistencies’ criteria obtained the lowest (0.0827). The rest of the criteria were distributed in between that range. Subsequently, CoCoSo ranked the involved LLM alternatives in two different runs based on the extracted weights. Sensitivity analysis was employed to evaluate the effect of the assessment criteria on LLMs’ evaluation.

Developing a comprehensive data-driven strategy for evaluating the organisational culture in companies to foster digital innovation involves a multi-criteria decision-making (MCDM) problem. This needs to consider various organisational culture characteristics that influence digital innovation success, assign significance weights to each characteristic, and recognise that distinct organisational cultures may excel in different aspects necessitates the proper handling of data variations. Hence, to provide organisations seeking to align cultural practises with digital innovation objectives with valuable insights, this study aims to develop an MCDM model for evaluating and benchmarking organisational culture in companies to foster digital innovation. The benchmarking decision matrix is formulated based on the intersection of evaluation characteristics and a list of organisational culture aspects in companies. The MCDM model is developed in two phases. Firstly, a new weighting model, q-rung picture fuzzy-weighted zero-inconsistency (q-RPFWZIC), is formulated for assessing the evaluation characteristics under the q-rung picture fuzzy sets environment. Secondly, the simple additive weighting (SAW) model is formulated for benchmarking the organisational culture in companies using the extracted weights of the evaluation characteristics. The results indicate that characteristic C6 (corporate entrepreneurship) has the highest weight, with a value of 0.161, while characteristic C3 (employee participation, agility and organizational structures) and C7 (digital awareness and necessity of innovations) has the lowest weight of 0.088. Company A2 secures the top rank with a score of 0.911, satisfying eight evaluation characteristics, whereas company A7 holds the last rank order, satisfying only one evaluation characteristic, obtaining a score of 0.101. In model evaluation, several scenarios were considered in a sensitivity analysis test based on a 100% increment in weight values for each characteristic to validate the reliability of the model results.

Given the tremendous potential and influence of artificial intelligence (AI) and algorithmic decision‐making (DM), these systems have found wide‐ranging applications across diverse fields, including education, business, healthcare industries, government, and justice sectors. While AI and DM offer significant benefits, they also carry the risk of unfavourable outcomes for users and society. As a result, ensuring the safety, reliability, and trustworthiness of these systems becomes crucial. This article aims to provide a comprehensive review of the synergy between AI and DM, focussing on the importance of trustworthiness. The review addresses the following four key questions, guiding readers towards a deeper understanding of this topic: (i) why do we need trustworthy AI? (ii) what are the requirements for trustworthy AI? In line with this second question, the key requirements that establish the trustworthiness of these systems have been explained, including explainability, accountability, robustness, fairness, acceptance of AI, privacy, accuracy, reproducibility, and human agency, and oversight. (iii) how can we have trustworthy data? and (iv) what are the priorities in terms of trustworthy requirements for challenging applications? Regarding this last question, six different applications have been discussed, including trustworthy AI in education, environmental science, 5G‐based IoT networks, robotics for architecture, engineering and construction, financial technology, and healthcare. The review emphasises the need to address trustworthiness in AI systems before their deployment in order to achieve the AI goal for good. An example is provided that demonstrates how trustworthy AI can be employed to eliminate bias in human resources management systems. The insights and recommendations presented in this paper will serve as a valuable guide for AI researchers seeking to achieve trustworthiness in their applications.

Considering the coronavirus disease 2019 (COVID-19) pandemic, the government and health sectors are incapable of making fast and reliable decisions, particularly given the various effects of decisions on different contexts or countries across multiple sectors. Therefore, leaders often seek decision support approaches to assist them in such scenarios. The most common decision support approach used in this regard is multiattribute decision-making (MADM). MADM can assist in enforcing the most ideal decision in the best way possible when fed with the appropriate evaluation criteria and aspects. MADM also has been of great aid to practitioners during the COVID-19 pandemic. Moreover, MADM shows resilience in mitigating consequences in health sectors and other fields. Therefore, this study aims to analyse the rise of MADM techniques in combating COVID-19 by presenting a systematic literature review of the state-of-the-art COVID-19 applications. Articles on related topics were searched in four major databases, namely, Web of Science, IEEE Xplore, ScienceDirect, and Scopus, from the beginning of the pandemic in 2019 to April 2021. Articles were selected on the basis of the inclusion and exclusion criteria for the identified systematic review protocol, and a total of 51 articles were obtained after screening and filtering. All these articles were formed into a coherent taxonomy to describe the corresponding current standpoints in the literature. This taxonomy was drawn on the basis of four major categories, namely, medical (n = 30), social (n = 4), economic (n = 13) and technological (n = 4). Deep analysis for each category was performed in terms of several aspects, including issues and challenges encountered, contributions, data set, evaluation criteria, MADM techniques, evaluation and validation and bibliography analysis. This study emphasised the current standpoint and opportunities for MADM in the midst of the COVID-19 pandemic and promoted additional efforts towards understanding and providing new potential future directions to fulfil the needs of this study field.

This study evaluated the benchmarking process of active queue management (AQM) methods, which consider a multicriteria decision-making (MCDM) problem using multidimensional criteria. Academic studies have benchmarked the AQM methods using MCDM techniques. However, these studies have used existing MCDM techniques, which face considerable theoretical challenges. The latest MCDM method called fuzzy decision by opinion score (FDOSM) was published in the Journal of Applied Soft Computing in 2020 to address the theoretical challenges of the existing MCDM methods. However, FDOSM continues to encounter serious issues. That is, it exclusively depends on the direct aggregation MCDM approach based on arithmetic mean (AM) operator. However, performing other operators (i.e., geometric mean, harmonic mean, and root mean square), in addition to applying other MCDM approaches (i.e., distance measurement and compromise rank), may result in different ranking results. Hence, this study mainly proposes an extension of FDOSM through the following aspects: (1) application of different aggregation techniques in the direct aggregation MCDM approach, (2) discussion of the effectiveness of each type on the final AQM benchmarking, and (3) use of varying MCDM approaches on FDOSM to reach the optimum result when benchmarking the AQM methods. The current research methodology is based on two sequential phases. The first phase provides the decision matrix used in benchmarking the AQM methods. The decision matrix was constructed based on the AQM evaluation criteria and a list of AQM methods. The second phase presents two stages, namely, data transformation unit and data processing. Findings of the AQM benchmarking are as follows. (1) In the individual FDOSM, two main configurations are recommended when using the AQM benchmarking: direct aggregation MCDM approach with AM operator and compromise rank approach. Benchmarking results of both configurations based on six decision makers are nearly similar, with the AQM BLUE method being ranked the best. The exception is for the results of the compromise rank approach based on the third decision maker, which revealed that the AQM ERED method is the best. (2) Results of the group FDOSM showed a relatively similar order for the AQM methods in both configurations, with the AQM BLUE method being the best. (3) Lastly, significant differences were found among the groups' scores, thereby indicating the validity of the FDOSM-based AQM benchmarking results.

The benchmarking of smart e-tourism data management applications falls under the problem of multicriteria decision-making (MCDM). This claim is supported by three issues: 12 smart key concepts need to be considered in the evaluation, criteria importance, and data variation among these criteria. Thus, an MCDM solution is essential to overcome problem complexity. To end this, this study presents a decision-making framework on the basis of the extension of interval type 2 trapezoidal-fuzzy weighted with zero inconsistency (IT2TR-FWZIC) integrated with the Vlsekriterijumska Optimizcija I Kaompromisno Resenje (VIKOR) method for evaluating and benchmarking the smart e-tourism data management applications. Our methodology comprises two consecutive phases. In the first phase, a decision matrix is constructed using the intersection between the 12 key concepts and smart e-tourism data management applications of each category and subcategory in smart e-tourism. In the second phase, the integration of the IT2TR-FWZIC formulation and VIKOR is presented to compute the weights for the 12 key concepts and benchmark the smart e-tourism data management applications for each category. The results are as follows: (1) A clear difference is found among the criteria weights (12 smart key concepts). Specifically, the real-time criterion achieves the highest importance weight (0.098), whereas augmented reality obtains the lowest weight (0.068). The context-awareness and recommender systems have the same weight value (0.087), and the other eight criteria are distributed in between. (2) The smart e-tourism data management applications are evaluated and benchmarked effectively per category and subcategories. (3) Benchmarked applications in each category are subjected to a systematic ranking in the evaluation process. The sensitivity analysis has shown high correlation outcomes to the systematic ranking results over the 31 scenarios of criteria weight changing. Moreover, a comparative analysis of the proposed work with other existing studies is also discussed.