Tian Xiao

dblp:129/8250 · DBLP profile ↗
← Back
28ranked-venue papers
4as first author
22since 2021 · last 2025
—ORCID · conflict

Domains — the database's venue-derived domains; a paper can count in several

Security and privacy · 12 · 2 first-author · 12 since 2021Systems, architecture and hardware · 8 · 1 first-author · 6 since 2021Applied, interdisciplinary, general and emerging computing · 3 · 3 since 2021Computer networks · 2Software engineering, systems software and programming languages · 2 · 1 first-author
YearPublicationVenuePosition
2025 Differentiated Service Data-Driven Intelligent Adjustment Method for 5G/5G-A Broadcast Beams
abstract
Artificial Intelligence (AI) technology has been increasingly integrated into Radio Access Networks (RAN), thereby boosting the intelligence capabilities of the air interface in mobile communications. Specifically, AI-driven beam management technology achieves multiple performance breakthroughs. These include spectral efficiency improvement and coverage quality enhancement, enabled by dynamic optimization of beam direction and intelligent resource multiplexing. This paper proposes a differentiated service datadriven intelligent adjustment method for 5G-A broadcast beams. By considering service-level performance requirements and traffic popularity, the method dynamically and intelligently adjusts broadcast beam parameters. The method aims to provide differential network coverage guarantee for different services under fixed resources and enhance the precision of broadcast beam adjustment.
Zixiang Di, Lu Zhi, Tian Xiao, Feibi Lyu, Zhaoxing Li, Songbai Liang, Jinjian Qiao
HPCC4
2025 The Application of SSB Frequency Offset in Low-Altitude Network
abstract
During the During the National People's Congress and the Chinese Political Consultative Conference in 2024, the ‘low-altitude economy’ was included in the government work report as a significant factor driving new quality productive forces. In the New Radio (NR) network, when a terminal is accessed, the base station uses SSB (Synchronization Signal Block) beam sweeping to detect the optimal beam for the terminal. After the terminal accesses and obtains the configuration information of the reference signal, it feeds back the channel state information (CSI), and the base station uses the optimal beam from CSI-RS (Channel State Information Reference Signal) beam sweeping. In low-altitude communications, 5G antennas flexibly configure the number of beams, considering horizontal and vertical dimensions. Combining SSB frequency offset technology, the SSB frequency points of the low-altitude network can be staggered with the configuration of the ground network, forming a virtual airground heterogeneous frequency network. This approach enhances performance by reducing handover times and interference.
Zixiang Di, Tian Xiao, Zhaoning Wang, Feibi Lv, Hongbing Ma, Jiajia Zhu 0005, Guanghai Liu 0002, Lexi Xu, Xiaomeng Zhu 0001
HPCC3
2025 Combining Large and Small Models to Empower Handling of User Complaints of 5G Network
abstract
This paper investigates the workflow and requirement of telecommunications operators in handling 4G/5G user network quality complaints and proposes a solution that combines large and small models to achieve more intelligent complaint handling. The large model is responsible for comprehensively analyzing unstructured data such as user complaint texts, extracting key information, and understanding user intentions. Small models are used for indepth processing of structured data related to network performance indicators, conducting root cause analysis, and providing targeted solutions. The models and systems are applied to current network operations, significantly reducing network maintenance optimization work orders, saving labor costs, and improving work efficiency.
Feibi Lyu, Songbai Liang, Zixiang Di, Tian Xiao, Lu Zhi, Jiajia Zhu 0005, Lexi Xu, Zhaoning Wang
HPCC4
2025 AI-Based 5G Beam Weight Optimization Scheme for Coverage Improvement in Low-Altitude Scenarios
abstract
This paper analyzes the typical service requirements of low-altitude scenarios and proposes an intelligent weight optimization scheme for 5G beams in low-altitude scenarios using artificial bee colonies and genetic algorithms. Based on key indicators such as coverage quality, interference level, and service perception, joint optimization was conducted and validated in low-altitude networking pilot areas, resulting in significant improvements in computational efficiency and optimization results. This scheme achieved the optimal solution for the weight of contiguous areas, resulting in sound application effects.
Tian Xiao, Zixiang Di, Feibi Lyu, Lu Zhi, Chenrui Zang, Lexi Xu
HPCC1
2025 Research on Host Classification Based on Language Models in Mobile Communication Networks
abstract
In mobile communication networks, host classification plays a critical role in constructing user profiles and ensuring network security. Traditional approaches, which rely on rule-based matching and shallow feature engineering, face significant limitations in coping with the high-frequency dynamic variations of hostnames and the labor-intensive maintenance of manual rules. To address these challenges, this paper proposes a novel frequency-aware hybrid-granularity tokenization method, specifically designed to capture both the semantic structure and statistical patterns of hostnames. By leveraging semi-supervised learning on large-scale host sequence data collected from real-world network environments, the proposed method enables effective service classification through vectorized host representations. This work not only offers an efficient and scalable solution for host analysis in personalized recommendation systems and mobile network security but also provides valuable insights into the design of pretrained tokenizers tailored for dynamic data scenarios.
Yuhui Han, Zixiang Di, Lexi Xu, Tian Xiao, Guoguang Zhang
HPCC10
2025 Transformer-Based Temporal Feature Pyramid Network for Temporal Action Proposal Generation
abstract
Temporal action proposal generation plays a vital role in the analysis of untrimmed videos and has garnered growing interest from researchers. Nevertheless, the presence of long-term temporal dependencies and the large variation in action durations within untrimmed videos pose significant challenges for accurately localizing action boundaries. To overcome the aforementioned issues, we design a novel Transformer-based Temporal Feature Pyramid Network (TTFPN) tailored for generating action proposals. Specifically, we introduce a local transformer to capture longterm temporal information while reducing computational complexity through the substitution of conventional selfattention with a localized variant. Subsequently, a temporal feature pyramid is built to produce multi-scale representations, enabling the model to effectively handle action instances of varying durations. Based on this temporal feature pyramid, we employ a convolutional network-based predictor to generate action proposals in an anchor-free manner. We evaluate TTFPN on THUMOS14, a standard benchmark for temporal action detection, to validate its effectiveness. The results show that TTFPN achieves competitive performance and significantly outperforms previous methods.
Tian Xiao, Lu Zhi, Feibi Lv, Jiajia Zhu 0005, Zhaoning Wang, Zixiang Di, Lexi Xu
HPCC2
2025 QuadrantSearch: A Novel Method for Registering UAV and Backpack LiDAR Point Clouds in Forested Areas
abstract
Unmanned aerial vehicle (UAV) laser scanning (ULS) and backpack laser scanning (BLS) are two commonly employed technologies in precision forestry. However, data acquired by these two types of light detection and ranging (LiDAR) are distinct, with one capturing point clouds beneath the canopy and the other above. Consequently, there is minimal overlap in the point clouds collected by both methods, especially in dense forests, presenting significant challenges for data registration. Furthermore, many trees in forests (particularly broadleaf trees) have the tree tops and trunk centers not aligned vertically, which greatly increases the difficulty of the data registration methods based on tree position. To solve the above-mentioned problems, we here propose a novel and robust method to register ULS and BLS point clouds in forested areas. Our method consists of three key steps, that is, tree location extraction, quadrant search-based minimum spanning tree (MST) matching, and registration. The quadrant searching strategy dynamically searches for potential candidates in four quadrants centered on the initial tree locations. By constructing MSTs for the potential tree locations, triangle constraints require only four topologically similar tree locations to find one-to-one correspondences during the stepwise MST matching process. The proposed method was evaluated in five urban forest sample plots and one natural forest sample plot located in China, covering both coniferous and broadleaf forests. The results show that our method obtained good registration results on all six sample plots, with an averaged rotation error, translation error, pointwise error, and root-mean-square error (RMSE) of 0.012 rad, 0.354, 0.378, and 0.379 m, respectively. Comparative studies indicate that our method outperformed existing registration methods, demonstrating its effectiveness and robustness. Our method allows for the creation of a more complete picture of forest vertical structure and holds great potential for informing sustainable forest management practices and supporting critical ecological assessments.
Guorong Li, Bin Wu 0010, Zhan Pan, Linxin Dong, Guochun Shen, Tian Xiao, Lefeng Zhang, Bailang Yu
IEEE Trans. Geosci. Remote. Sens.9
2023 A Novel Algorithm and System of Customer Value Evaluation based on Telecom Operator Big Data
abstract
With the increasing market competition, telecom operators need to improve the level of services, ensure the quality of experience, as well as reduce the cost of enterprise. Therefore it is crucial to evaluate the value of telecom customers accurately. The traditional method of telecom customer value assessment is mainly based on ARPU (Average Revenue Per User), which is one-dimensional and cannot evaluate customer value comprehensively. This paper proposes a multidimensional customer value assessment method, including two perspectives, Current value and Potential value to improve the accuracy and comprehensiveness of evaluation. Also, an intelligent method based on swarm intelligence algorithm is presented to calculate the indicator weight for each characteristic field of customer value. The practical results show that the algorithm, which is applied to realistic scenarios of network operation, enables telecom operator to achieve a comprehensive and accurate customer value assessment in many issues such as customer churn warning and accurate recommendation, ultimately increasing the effectiveness and efficiency of decision-making closed loop for telecom operators. At last but not the least, the algorithm and system can benefit other industries to improve their intelligence level of customer service.
Xinzhou Cheng, Jinyou Dai, Feibi Lyu, Tian Xiao
TrustCom9
2023 An AI-driven Dockerized Lightweight Framework for Smart Home Service Orchestration
abstract
We are going to enter the most intelligent era than ever before. Intelligent electronics network is infiltrating into our life and making it more convenient. Nonetheless, users always want smart home be more intelligent and complete more features. users’ issues are endless. Modular packaging device services and effective choreography algorithms can flexible fit different issues. Many organizations have been proving, implementing and managing business solutions for many specific individual industries. However, when comes to smart home for end users, there are numerous limitations in process, tooling, and skills. In the paper, we provide a lightweight visualized service creating tool and an AI-driven service flow construction model. It helps end users to create services though drag-and-drop, and then deploy new services automatically. And in the end a case study will be introduced.
Zhaoning Wang, Jiajia Zhu 0005, Bo Cheng 0001, Xinzhou Cheng, Feibi Lyu, Guoping Xu, Jinjian Qiao, Lu Zhi, Tian Xiao
TrustCom9
2022 Estimation of Internal Wave Parameters based on SAR Images
abstract
Since the synthetic aperture radar (SAR) images of internal wave contain additive and multiplicative noise under the ocean background, it is necessary to extract the internal wave to obtain the parameter information. This paper proposes a variational mode decomposition (VMD) method for separating and extracting internal waves. To solve the ambiguity in the estimation of direction, normalized radon transform in the elliptic domain is combined with the VDM, which can estimate the propagation direction of the internal waves accurately. Then performing experiments on Andaman Sea internal wave data collected by ERS-1 shows that VMD is least disturbed by the background information, and the estimation result of wavelength is closest to the measured value. At the same time, correct results are obtained by using the proposed method to estimate the propagation direction of the measured data from various sea areas, which proves the effectiveness of the proposed method.
Haiyue Dong, Rongqing Xu, Tian Xiao, Yun Zhang 0023
IGARSS3
2022 Joint LSTM and Periodic Decision Algorithm for 5G Massive MIMO
abstract
Massive MIMO (Multiple Input and Multiple Output) is a key technology for improving 5G (the 5thGeneration) system capacity and spectrum utilization. This paper introduces the basic principles of Massive MIMO. Then, this paper proposes a novel LSTM-PD (LSTM and Periodic Decision) algorithm. The proposed LSTM-PD algorithm belongs to the category of periodic decision method with predictive properties. In addition, we also design a monitoring exit mechanism to improve the entire algorithm. The current network data results show that when the physical resource block (PRB) utilization rate is greater than 30%, the spectral efficiency of the LSTM-PD algorithm is significantly higher than that of the traditional algorithm. In addition, when the PRB utilization reaches 60%, the CPU utilization of the LSTM-PD algorithm can be reduced by nearly 30%, compared with the traditional algorithm.
Yi Li 0053, Feihu Yang, Lexi Xu, Tian Xiao, Yuwei Jia, Xinzhou Cheng, Guanghai Liu 0002
IWCMC6
2022 Telecom Big Data assisted Algorithm and System of Campus Safety Management
abstract
Recently, information and digital technology are widely used in thousands of industries, leading to intelligent transformation, traditional methods, which lacks intelligent instrument. The safety of college students has attracted widespread attention from all walks of life, while campus safety management still adopts manual and traditional methods, which lacks intelligent instrument and big data resources and technologies are not fully utilized. In this paper, we propose a system of campus safety management based on telecom big data and data fusion architecture, providing solutions for intelligent campus management. In addition, a prediction algorithm of student behavior intent considering time spans has been proposed, proving the advantages in accuracy metrics and F1-score compared with traditional prediction algorithms.
Xinzhou Cheng, Shikun Jiang, Yuhui Han, Lijuan Cao, Yuwei Jia, Tian Xiao
TrustCom9
2022 Research on Voice Quality Evaluation Method Based on Artificial Neural Network
abstract
With the gradual commercialization of 5G VoNR, VoLTE and VoNR will become the main methods of voice services. How to efficiently evaluate the quality of voice service is the focus of telecom operators. This paper proposes an intelligent combined evaluation method of VoLTE and VoNR voice quality based on artificial neural network. In the proposed method, the artificial neural network model is fitted by the call level time slice sample data of voice, and then the prediction model is established. The prediction results of voice quality of mobile networks are obtained by using the prediction model at call level, grid level and area level. Meanwhile, the proposed method can address the shortcomings of traditional evaluation method based on road test, such as high cost, low timeliness and limited area. Finally, through theoretical verification and comparison with the real test results, the effectiveness of the prediction method is verified.
Zixiang Di, Tian Xiao, Yi Li 0053, Xinzhou Cheng, Lexi Xu, Xiaomeng Zhu 0001, Lu Zhi
TrustCom2
2022 Research on Capability Building of Mobile Network Data Analysis and Visualization
abstract
In order to meet the needs of data analysis and visualization to assist mobile network operation decision-making, telecom operators have established several mobile network index analysis tools or platforms. However, the network data analysis efficiency of planning, construction, maintenance and optimization is still low, and demand-oriented visualization means are still insufficient. This paper designs a mobile network data analysis and visualization system. The designed system aims at addressing the problems that mobile network has various types of data. The designed system can make data easy to manage, improve the data analysis efficiency and the flexibility of data visualization for telecom operators.
Xinzhou Cheng, Kun Chao, Yuwei Jia, Lexi Xu, Tian Xiao
TrustCom8
2022 Research on OTFS Systems for 6G
abstract
The 6G communication system is expected to achieve seamless global coverage. The orthogonal time-frequency space (OTFS) is generally considered as the main candidate waveform for 6G communication. Especially, in the air-space-ground integrated communication system, OTFS is more suitable for air interface modulation waveforms for high mobility communication scenarios than OFDM system. This paper focuses on OTFS technology, which conveniently adapts to the channel constantly changing via modulating information. In the paper, comparative analysis under different rate scenarios is performed, and potential future application scenarios are proposed, such as applying artificial intelligence based on vehicle network.
Tian Xiao, Lexi Xu, Guanghai Liu 0002, Zixiang Di
TrustCom2
2022 Coverage Estimation of Wireless Network Using Attention U-Net
abstract
MDT data have been widely used for 4G/5G wireless network coverage estimation. Whereas the sparsity of the MDT data makes coverage rate bias when it applied into realistic network coverage analysis. To achieve a more precise coverage estimation, this paper proposes an approach that adding geographical and landform information to network coverage estimation in order to refine the coverage rate. An attention U-Net model was applied to landforms recognition from online satellite map with low cost. It can effectively assists telecom operators to filter out areas, which are users inaccessible or do not require signal coverage.
Feibi Lyu, Xinzhou Cheng, Lexi Xu, Jinjian Qiao, Lu Zhi, Zixiang Di, Tian Xiao
TrustCom8
2022 Research on Intelligent 5G Remote Interference Avoidance and Clustering Scheme
abstract
This paper investigates on the remote interference problem in the TDD network and proposes an intelligent 5G Remote Interference Avoidance and Clustering Scheme (RIAC), on the basis of RIM-RS (remote interference management-reference signal) and clustering algorithm. This paper adopts the GBLA-DBSACN (the grid-based local adaptive DBSCAN) algorithm based on the traditional DBSCAN algorithm (Density—Based Spatial Clustering of Application with Noise) to improve the accuracy of interference base station (BS) clustering, which considers the dispersion of interference sources. This scheme helps to locate interference problems and potential sources through testing in the existing network quickly and effectively. By taking corresponding optimization means for these problems, network operators can effectively reduce the interference level in the target area and improve the quality of network construction.
Tian Xiao, Zixiang Di, Guanghai Liu 0002, Lexi Xu, Zhaoning Wang, Yi Li 0053
TrustCom1
2022 AI based Collaborative Optimization Scheme for Multi-Frequency Heterogeneous 4G/5G Networks
abstract
With the continuous expansion of network construction, 4G/5G networks have gradually developed into hybrid multi-frequency heterogeneous networks, while the difficulty of inter-RAT mobility assurance is gradually increasing. Traditional interoperability optimization requires enormous labor costs, and the accuracy is low. This paper proposes an AI-based collaborative optimization scheme under multi-frequency heterogeneous 4G/5G networks based on the XGBoost prediction model and DNN algorithm. It aims to comprehensively improve the performance of different users in multi-frequency heterogeneous 4G/5G networks in terms of 4G/5G neighborhood re-organization and intelligent optimization of 4G/5G interoperability parameters. The results show that the proposed scheme has high accuracy and strong generalization, which is critical in improving user mobility perception under complex network structures. The scheme contributes to the network operators’ efficiency improvement and intelligent transformation process.
Tian Xiao, Guoping Xu, Lexi Xu, Xinzhou Cheng, Feibi Lyu, Guanghai Liu 0002
TrustCom1
2022 Big Data based Potential Fixed-Mobile Convergence User Mining
abstract
With the disappearance of the demographic dividend and the saturation of the public telecom market, telecom operators need new development strategies urgently. New services formed by the convergence of mobile network services and broadband network services (referred to as fixed-mobile convergence services) have become an important strategy. Through business innovation, telecom operators can bundle mobile services with broadband services, which can enhance user stickiness and increase business revenue. Based on the joint analysis of mobile network data and broadband network data, this paper proposes a rule-based and model-based integrated method for mining potential fixed-mobile convergence target users. After applying this method to the real market for single mobile contract user transferring to convergent contract, results show that the proposed method for exploiting potential target users can increase the conversion rate of convergence users.
Tao Zhang 0100, Shikun Jiang, Yuhui Han, Xinzhou Cheng, Tian Xiao
TrustCom9
2022 Telecom Customer Chum Prediction based on Half Termination Dynamic Label and XGBoost
abstract
With the rapid progress of the telecom industry and fierce competition among telecom operators, telecom companies pay more attention to customer retention. Telecom companies developed multiple solutions to predict churn customers before customers move to another telecom operator. However, the existing prediction solutions have some disadvantages in the real-world use cases. For example, churn definition is limited to moving from one telecom operator to another, which is too late for preventing customer churn. The main contribution of the paper is to introduce the new definition of customer chum for the telecom industry, and to propose a Half Termination Dynamic Label (HTDL) that improves the churn prediction solution with XGBoost. Experiment results showed that the proposed solution improved the model performance, which significantly outperforms traditional solution, in terms of churn prediction on F1-score. The new solution also sidelines more active customers for retention.
Chuntao Song, Xinzhou Cheng, Lexi Xu, Tian Xiao
TrustCom7
2022 Mahalanobis Distance and Pauta Criterion based Log Anomaly Detection Algorithm for 5G Mobile Network
abstract
In the 5G era, mobile networks gradually become complex, and there are also high requirements for network operation and maintenance. As log data is important information to reflect the status of network devices, the monitoring of log data generated by network devices has become an important part of network operation and maintenance. But the massive amount of log data generated by large-scale network devices has already exceeded the range of human processing capabilities. And the introduction of artificial intelligence algorithms can optimize the detection of log anomalies and reduce network operation and maintenance costs under the challenges of high complexity of 5G networks. This paper proposes a Mahalanobis distance and Pauta criterion based log anomaly detection (MPLAD) algorithm for 5G mobile network. On the basis of solving the shortcomings of the existing log anomaly detection algorithms, it innovatively integrates the Mahalanobis distance algorithm and the Pauta criterion. Meanwhile, it also introduces the negative sample mechanism and the principal component analysis (PCA) method to achieve high accuracy, high efficiency and high compatibility towards 5G mobile network log anomaly detection.
Yi Li 0053, Yuchao Jin, Xiaomeng Zhu 0001, Lexi Xu, Tian Xiao, Xinzhou Cheng
TrustCom6
2022 A Stepwise Minimum Spanning Tree Matching Method for Registering Vehicle-Borne and Backpack LiDAR Point Clouds
abstract
Vehicle-borne Laser Scanning (VLS) and Backpack Laser Scanning (BLS) are two emerging mobile mapping technologies for capturing detailed spatial information near ground in urban built environments. BLS has flexible mobility and usually provides point clouds in a local coordinate system. Therefore, a mismatch between VLS and BLS point clouds data is quite common. Fusing VLS and BLS data in different coordinate systems could provide a comprehensive survey of urban built environments. Because of the complexity of urban road environments and the difference in data acquisition methods, traditional registration approaches based on point-level correspondences are likely to fail and sometimes involve substantial manual efforts. In this paper, we propose a novel registration approach that finds the optimal transformation between the respective point clouds based on a unique tree distribution pattern defined by tree trunk centers. The proposed method consists of three key steps, i.e., trunk center extraction, stepwise minimum spanning tree (MST) matching, and transformation estimation. Stepwise MST matching is an essential step in finding the one-to-one correspondences using a topological similarity between the two LiDAR datasets. We evaluated our method with five real-world datasets collected in the Shanghai city, China. The results showed that the proposed method performed well in all five experiment sites with an average rotation error of less than 0.06° and an average translation error of less than 0.05 m. Moreover, the reported mean position deviation in the five sites are 0.112 m, 0.144 m, 0.176 m, 0.148 m, and 0.184 m, respectively. Our proposed method has a great potential for registering multiplatform LiDAR data that could provide comprehensive and essential 3D information for numerous urban applications.
Bin Wu 0010, Qiusheng Wu, Yi Zhao 0032, Zhan Pan, Tian Xiao, Bailang Yu
IEEE Trans. Geosci. Remote. Sens.6
2018 A Stop-Wait Collaborative Charging Scheme for Mobile Wireless Rechargeable Sensor Networks
abstract
The mobile wireless sensor networks have been used in many popular applications. However, the limited battery capacity of sensor nodes is still one of the key issues in wireless sensor networks (WSNs). The new and effective way is to use wireless energy transfer and rechargeable lithium batteries to solve this problem. To improve the charging energy effectiveness, we introduce a stop-wait scheme that allows the mobile charger (MC) to stop and wait to charge the coming mobile sensors. The lifetime of the WSNs is considered to assure that each sensor will not run out of energy. The simulation experiment shows that our algorithm improves the energy usage effectiveness.
He Li 0041, Tian Xiao, Yihua Lan, Qinglei Qi
ICCCN2
2018 Image Compression Task Coordination Mechanism Based on Dynamic Alliance in Mobile Wireless Multimedia Sensor Networks
abstract
In mobile wireless multimedia sensor networks (MWMSNs) image compression task coordination, the existing methods without considering the dynamic changes of the processing ability and location of cooperative nodes, which will cause frequent interruptions of image compression tasks and task data re-transmission. To solve these problems, an image compression task cooperation algorithm based on dynamic alliance (ATDA) is proposed. The image compression task is divided into image transfer subtask and image compression subtask based on the task stable execution time and the principles and constraints of task decomposition. Simulation results show that the proposed algorithm can realize the task load balancing of the alliance collaboration nodes and reduce the execution time of the image compression task and energy consumption.
He Li 0041, Tian Xiao, Yihua Lan, Qinglei Qi
SenSys2
2016 DRDDR: a lightweight method to detect data races in Linux kernel
Yunyun Jiang, Yi Yang 0033, Tian Xiao, Tianwei Sheng
J. Supercomput.3
2015 A Power-Conserving Online Scheduling Scheme for Video Streaming Services
Yunyun Jiang, Tian Xiao, Jidong Zhai
ICA3PP (1)2
2014 Cybertron: pushing the limit on I/O reduction in data-parallel programs
abstract
I/O reduction has been a major focus in optimizing data-parallel programs for big-data processing. While the current state-of-the-art techniques use static program analysis to reduce I/O, Cybertron proposes a new direction that incorporates runtime mechanisms to push the limit further on I/O reduction. In particular, Cybertron tracks how data is used in the computation accurately at runtime to filter unused data at finer granularity dynamically, beyond what current static-analysis based mechanisms are capable of, and to facilitate a new mechanism called constraint based encoding for more efficient encoding. Cybertron has been implemented and applied to production data-parallel programs; our extensive evaluations on real programs and real data have shown its effectiveness on I/O reduction over the existing mechanisms at reasonable CPU cost, and its improvement on end-to-end performance in various network environments.
Tian Xiao, Hucheng Zhou, Xu Zhao 0004, Chencheng Ye 0001, Xi Wang 0005, Wei Lin 0016, Lidong Zhou
OOPSLA1
2013 A characteristic study on failures of production distributed data-parallel programs
abstract
SCOPE is adopted by thousands of developers from tens of different product teams in Microsoft Bing for daily web-scale data processing, including index building, search ranking, and advertisement display. A SCOPE job is composed of declarative SQL-like queries and imperative C# user-defined functions (UDFs), which are executed in pipeline by thousands of machines. There are tens of thousands of SCOPE jobs executed on Microsoft clusters per day, while some of them fail after a long execution time and thus waste tremendous resources. Reducing SCOPE failures would save significant resources. This paper presents a comprehensive characteristic study on 200 SCOPE failures/fixes and 50 SCOPE failures with debugging statistics from Microsoft Bing, investigating not only major failure types, failure sources, and fixes, but also current debugging practice. Our major findings include (1) most of the failures (84.5%) are caused by defects in data processing rather than defects in code logic; (2) table-level failures (22.5%) are mainly caused by programmers' mistakes and frequent data-schema changes while row-level failures (62%) are mainly caused by exceptional data; (3) 93% fixes do not change data processing logic; (4) there are 8% failures with root cause not at the failure-exposing stage, making current debugging practice insufficient in this case. Our study results provide valuable guidelines for future development of data-parallel programs. We believe that these guidelines are not limited to SCOPE, but can also be generalized to other similar data-parallel platforms.
Hucheng Zhou, Haoxiang Lin, Tian Xiao, Wei Lin 0016, Tao Xie 0001
ICSE4