Recent advancements in Vision-Language (VL) research have sparked new benchmarks for complex visual reasoning, challenging models’ advanced reasoning ability. Traditional Vision-Language models (VLMs) perform well in visual perception tasks while struggling with complex reasoning scenarios. Conversely, Large Language Models (LLMs) demonstrate robust text reasoning capabilities; however, they lack visual acuity. To bridge this gap, we propose Complex Visual Reasoning Large Language Models (CVR-LLM), capitalizing on VLMs’ visual perception proficiency and LLMs’ extensive reasoning capability. Unlike recent multimodal large language models (MLLMs) that require a projection layer, our approach transforms images into detailed, context-aware descriptions using an iterative self-refinement loop and leverages LLMs’ text knowledge for accurate predictions without extra training. We also introduce a novel multi-modal in-context learning (ICL) methodology to enhance LLMs’ contextual understanding and reasoning. Additionally, we introduce Chain-of-Comparison (CoC), a step-by-step comparison technique enabling contrasting various aspects of predictions. Our CVR-LLM presents the first comprehensive study across a wide array of complex visual reasoning tasks and achieves SOTA performance among all.

Federated learning (FL) protects training data in clients by collaboratively training local machine learning models of clients for a global model, instead of directly feeding the training data to the server. However, existing studies show that FL is vulnerable to various attacks, resulting in training data leakage or interfering with the model training. Specifically, an adversary can analyze local gradients and the global model to infer clients’ data, and poison local gradients to generate an inaccurate global model. It is extremely challenging to guarantee strong privacy protection of training data while ensuring the robustness of model training. None of the existing studies can achieve the goal. In this paper, we propose a robust privacy-preserving federated learning framework (PILE), which protects the privacy of local gradients and global models, while ensuring their correctness by gradient verification where the server verifies the computation process of local gradients. In PILE, we develop a verifiable perturbation scheme that makes confidential local gradients verifiable for gradient verification. In particular, we build two building blocks of zero-knowledge proofs for the gradient verification without revealing both local gradients and global models. We perform rigorous theoretical analysis that proves the security of PILE and evaluate PILE on both passive and active membership inference attacks. The experiment results show that the attack accuracy under PILE is between$[50.3\%,50.9\%]$, which is close to the random guesses. Particularly, compared to prior defenses that incur the accuracy losses ranging from 2% to 13%, the accuracy loss of PILE is negligible, i.e., only$\pm 0.3\%$accuracy loss.

Deep learning-based video segmentation methods can offer a good performance after being trained on the large-scale pixel labeled datasets. However, a pixel-wise manual labeling of animal images is challenging and time consuming due to irregular contours and motion blur. To achieve desirable tradeoffs between the accuracy and speed, a novel one-shot learning-based approach is proposed in this article to segment animal video with only one labeled frame. The proposed approach consists of the following three main modules: guidance frame selection utilizes “BubbleNet” to choose one frame for manual labeling, which can leverage the fine-tuning effects of the only labeled frame; Xception-based fully convolutional network localizes dense prediction using depthwise separable convolutions based on one single labeled frame; and postprocessing is used to remove outliers and sharpen object contours, which consists of two submodules—test time augmentation and conditional random field. Extensive experiments have been conducted on the DAVIS 2016 animal dataset. Our proposed video segmentation approach achieved mean intersection-over-union score of 89.5% on the DAVIS 2016 animal dataset with less run time, and outperformed the state-of-art methods (OSVOS and OSMN). The proposed one-shot learning-based approach achieves real-time and automatic segmentation of animals with only one labeled video frame. This can be potentially used further as a baseline for intelligent perception-based monitoring of animals and other domain-specific applications.11The source code, datasets, and pre-trained weights for this work are publicly [Online]. Available:https://github.com/tengfeixue-victor/One-Shot-Animal-Video-Segmentation.

Proof-of-Work based blockchain wastes an enormous amount of electricity and requires expensive mining equipment. This phenomenon is getting worse and worse. We propose a new consensus protocol for cryptocurrency built on top of Bitcoin protocol, which combines Proof-of-Work component with our new algorithm Proof-of-Contribution(PoC). PoC algorithm reduces the energy consumption for Bitcoin mining by rewarding the calculation difficulty of a cryptographic puzzle. Further, when miners no longer get block rewards, they receive difficulty rewards along with the regular mining fee which motivates mining. We explored different attacking scenarios and show that the network resilient to these attacks is comparable to Poof-of-Work(PoW). Finally, we designed two experiments to simulate mining and detect the changes of difficulty to prove PoC offering a new cryptocurrency algorithm which is energy-efficient.

The wall cracks of damaged building caused by earthquake are important to predict the extent of building damage and to reveal the process of building damage in an earthquake. As a new technology of non-contact measurement method, gound-based LiDAR can provide a new way to extract the quantitative information of wall cracks of earthquake damage buildings. In this article, one of damaged buildings of Beichuan County, which suffered Wenchuan earthquake on May 12th, 2008, were took as an example to study the quantitative extraction method of wall cracks information of earthquake damaged buildings based on gound-based lidar. And, according to the extraction results, the influence of extraction accuracy by point cloud density is analyzed as well.