Contrastive Language-Image Pre-training (CLIP) models as a milestone of modern multimodal intelligence, its gener-alization mechanism grasped massive research interests in the community. While existing studies limited in the scope of pre-training knowledge, hardly underpinned its generalization to countless open-world concepts absent from the pre-training regime. This paper dives into such Out-of-Pre-training (OOP) generalization problem from a holistic perspective. We propose LAION-Beyond benchmark to isolate the evaluation of OOP concepts from pre-training knowledge, with regards to OpenCLIP and its reproducible variants derived from LAION datasets. Empirical analysis evidences that despite image features of OOP concepts born with significant category margins, their zero-shot transfer significantly fails due to the poor image-text alignment. To this, we elaborate the "name-tuning" methodology with its theoretical merits in terms of OOP generalization, then propose few-shot name learning (FSNL) and zero-shot name learning (ZSNL) algorithms to achieve OOP generalization in a data-efficient manner. LAION-Beyond dataset and codes: http://m-huangx.github.io/laion_beyond/.

Large Vision-Language Models (LVLMs) have exhibited remarkable progress. However, deficiencies remain compared to human intelligence, such as hallucination and shallow pattern matching. In this work, we aim to evaluate a fundamental yet underexplored intelligence: association, a cornerstone of human cognition for creative thinking and knowledge integration. Current benchmarks, often limited to closed-ended tasks, fail to capture the complexity of open-ended association reasoning vital for real-world applications. To address this, we present MM-OPERA, a systematic benchmark with 11,497 instances across two open-ended tasks: Remote-Item Association (RIA) and In-Context Association (ICA), aligning association intelligence evaluation with human psychometric principles. It challenges LVLMs to resemble the spirit of divergent thinking and convergent associative reasoning through free-form responses and explicit reasoning paths. We deploy tailored LLM-as-a-Judge strategies to evaluate open-ended outputs, applying process-reward-informed judgment to dissect reasoning with precision. Extensive empirical studies on state-of-the-art LVLMs, including sensitivity analysis of task instances, validity analysis of LLM-as-a-Judge strategies, and diversity analysis across abilities, domains, languages, cultures, etc., provide a comprehensive and nuanced understanding of the limitations of current LVLMs in associative reasoning, paving the way for more human-like and general-purpose AI. The dataset and code are available at https://github.com/MM-OPERA-Bench/MM-OPERA.

With the widespread adoption of online education platforms, an increasing number of students are gaining new knowledge through Massive Open Online Courses (MOOCs). Exercise recommendation have made strides toward improving student learning outcomes. However, existing methods not only struggle with high dropout rates but also fail to match the diverse learning pace of students. They frequently face difficulties in adjusting to inactive students' learning patterns and in accommodating individualized learning paces, resulting in limited accuracy and diversity in recommendations. To tackle these challenges, we propose Neural Re-ranking for Diversified Exercise Recommendation (in short, NR4DER). NR4DER first leverages the mLSTM model to improve the effectiveness of the exercise filter module. It then employs a sequence enhancement method to enhance the representation of inactive students, accurately matches students with exercises of appropriate difficulty. Finally, it utilizes neural re-ranking to generate diverse recommendation lists based on individual students' learning histories. Extensive experimental results indicate that NR4DER significantly outperforms existing methods across multiple real-world datasets and effectively caters to the diverse learning pace of students.

Intrusion detection is crucial for safeguarding intelligent vehicle systems, aiming to identify abnormal network traffic and operational anomalies. Traditional methods primarily focus on spatial features of attacks, often neglecting temporal dynamics essential for detecting complex, evolving threats. Additionally, the effectiveness of existing techniques is limited by the scope and quality of available datasets, reducing their ability to detect novel, unseen attacks. To address these challenges, this article introduces a Transformer-based transfer learning intrusion detection system (TIDS), designed to capture and analyze spatiotemporal sequence features from vehicle data. TIDS generates high-dimensional feature representations of intricate intrusion patterns, improving the detection of known attack types through instance-based transfer learning, enhancing domain adaptability. Moreover, we proposed a novel self-supervised box classification method that enhances the system’s capability to detect previously unknown attacks, thereby increasing the overall robustness of the intrusion detection process. Comparative experiments demonstrate that TIDS outperforms traditional methods in detection speed and accuracy across various intrusion scenarios, effectively responding to emerging threats in intelligent vehicle networks.

In this study, we tackle the challenge of parcel damage detection and present YTCNet to improve both accuracy and real-time performance. We enhance the Yolov5 algorithm by integrating C3TR modules into the architecture to capture more comprehensive feature information. Furthermore, we introduce the CBAM attention module to focus on critical areas in parcel images, enhancing the model’s feature extraction capabilities. By effectively combining C3TR and CBAM modules, we successfully developed an efficient parcel damage detection algorithm. Our experimental results demonstrate the superior performance of YTCNet in accuracy and real-time processing compared to various object detection models across multiple parcel damage scenarios. Our research significantly contributes to the logistics supply chain industry by aligning with IoT development. It enables practical applications to more effectively identify parcel damage, thereby enhancing customer satisfaction.

In the realm of short-term portfolio optimization, the integration of machine learning with exponential growth rate techniques is gaining prominence. This paper introduces a novel approach for short-term portfolio optimization, termed Short-term Portfolio Optimization using Doubly Regularized EGR (SPODR), to address the challenges posed by limited data availability. SPODR utilizes radial basis functions for the effective identification of market trends, enabling improved stock market forecasts. The approach uniquely combines ℓ1and ℓ2-regularization, adhering to empirical financial principles, to strike a balance between risk and return in short-term portfolios. A key aspect of SPODR is addressing the complexity of its ElasticNet-like objective, which poses a challenge for traditional methods due to its online learning nature. To overcome this, we have developed an algorithm based on the log barrier interior-point method. This algorithm is adept at efficiently optimizing portfolio allocation, taking into account the specific constraints inherent in our approach. Extensive comparative experiments across five benchmark datasets demonstrate that SPODR significantly outperforms existing short-term portfolio optimization models. It achieves a right balance between return and risk. Furthermore, SPODR showcases efficient computational speed, enhancing its applicability in real-world financial settings.

The express delivery industry has become integral to modern social life, but supply chain parcels, especially those made of corrugated cardboard, are at risk of damage during transportation. Although corrugated cardboard boxes offer some impact resistance, they can still experience deformation and penetration damage. To address this issue, we propose a hybrid model called Parcels-DNet. Parcels-DNet adopts a lightweight feature extraction backbone network, enabling deployment in resource-constrained scenarios like mobile or embedded devices. Additionally, our experimental results demonstrate that Parcels-DNet effectively captures the features of parcel deformation and penetration damage. This improves the safety and efficiency of express supply chain parcel transportation, offering greater convenience and economic benefits for the logistics industry.

Security vulnerabilities, illustrated by the menace of track traffic or vehicle hacking, present a substantial risk to the Controller Area Network (CAN) bus, enabling unauthorized remote access and intrusion. Nevertheless, existing vehicle intrusion detection models encounter challenges in capturing temporal aspects, compromising the preservation of temporal attributes in the data. Addressing this predicament, we propose a novel Vehicle Intrusion Detection System (PTIDS) model based on Principal Component Analysis (PCA) and Transformer architecture, equipped with multi-type classification decisions. This model utilizes the PCA algorithm to preprocess the data and employs a multi-head self-attention mechanism to simulate the continuous input of the real-world environment in vehicle or track traffic data. Experimental results demonstrate that the PTIDS model outperforms traditional neural networks regarding accuracy, precision, recall, and F1-score for vehicle intrusion detection, confirming the importance of temporal variables in intrusion detection. However, owing to the high accuracy and low discrimination of the PTIDS model on the Car Hacking dataset, we migrated the model to the M-CAN and B-CAN intrusion datasets and conducted ablation experiments. The results reveal that the model achieved an accuracy of 90.14% on the M-CAN dataset, surpassing other models by 54.5%, demonstrating robust generalization ability.

Multi-Modal Large Language Models (MM-LLMs) have demonstrated powerful reasoning abilities in various visual question-answering tasks. However, they face the challenge of lacking rigorous reasoning and precise arithmetic, when solving geometry questions. To address this challenge, we propose a novel prompting method, namely Reason-and-Execute (R&E), to enhance the accuracy of solving geometry questions by MM-LLMs. Specifically, the R&E method includes two templates: reasoning template and execution template. We first adopt a reverse-thinking approach to construct a rigorous reasoning template so that it guides MM-LLMs to start reasoning from the most relevant domain knowledge of the question and ultimately identify the arithmetic requirements. We then make use of program-assisted thought to construct execution template in order to guide MM-LLMs to understand the arithmetic requirements from reasoning template and generate executable code block. The answer is finally obtained by executing the code block. We evaluate our prompting method on 9 models in answering questions on 6 datasets (including four geometry datasets and two science datasets) compared to Chain-of-Thought (CoT) and Program-Aided Language (PAL) prompting methods. R&E method shows up to 12.8% improvement compared to CoT and PAL, proving strong reasoning and arithmetic abilities for solving geometry questions of our method. Moreover, we further analyze the answering accuracy from the different perspectives on solving geometric questions, including domain knowledge, geometry shapes, question length, and language. Through multiple analysis, our method is able to enhance the ability of MM-LLMs to solve geometry questions.

Relative radiometric normalization (RRN) is a widely used method for enhancing the radiometric consistency among multi-temporal satellite images. Diverse satellite images enhance the information for observing the Earth’s surface and bring additional uncertainties in the applications using multi-sensor images, such as change detection, multi-temporal analysis, image fusion, etc. To address this challenge, we developed a multi-rule-based RRN method for multi-sensor satellite images, which involves the identification of spectral- and spatial-invariant pseudo-invariant features (PIFs) and a Partial least-squares (PLS) regression-based RRN modeling using neighboring target pixels around PIFs. The proposed RRN method was validated on four datasets and demonstrated excellent effectiveness in identifying high-quality PIFs with spectral- and spatial-invariant properties, estimating precise regression models, and enhancing the radiometric consistency of reference-target image pair. Our method outperformed six RRN methods and effectively processed well-registered medium- and high-resolution images from the same sensor. This letter highlights the potential of our method for generating more comparable bi-temporal multi-sensor images.

This article investigates observability/controllability of a networked dynamic system (NDS) in which system matrices of its subsystems are expressed through linear fractional transformations (LFTs). Some relations have been obtained between this NDS and descriptor systems about their observability/controllability. A necessary and sufficient condition is established with the associated matrices depending affinely on subsystem parameters/connections. An attractive property of this condition is that all the required calculations are performed independently on each individual subsystem. Except well posedness, not any other conditions are asked for subsystem parameters/connections. This is in sharp contrast to recent results on structural observability/controllability, which is proven to be NP-hard. Some characteristics are established for a subsystem that are helpful in constructing an observable/controllable NDS. It has been made clear that subsystems with an input matrix of full column rank are helpful in constructing an observable NDS while subsystems with an output matrix of full row rank are helpful in constructing a controllable NDS. These results are extended to an NDS with descriptor form subsystems. As a byproduct, the full normal rank condition of previous works on network observability/controllability has been completely removed. On the other hand, satisfaction of this condition is shown to be appreciative in building an observable/controllability NDS.

Satellite-derived nighttime light (NTL) data from the Defense Meteorological Satellite Program's Operational Linescan System (DMSP-OLS) and the Suomi National Polar-orbiting Partnership Visible Infrared Imaging Radiometer Suite (NPP-VIIRS) have been extensively used for monitoring human activities and urbanization processes. Differences of these two datasets in their spatial and radiometric properties make it difficult for a temporally consistent analysis using these two datasets together. In this article, we developed a new approach to integrate these two datasets and generated a temporally consistent NTL dataset from 1992 to 2018. First, we performed the pixel-level spatial resampling of VIIRS data using a kernel density method after preprocessing the raw VIIRS data. Second, we conducted a logarithmic transformation of the aggregated VIIRS data. Third, we proposed a sigmoid function between DMSP and processed VIIRS data to characterize their relationship. Using the proposed method, we generated a series of consistent DMSP NTL data in Southeast Asia from 1992 to 2018 and analyzed the dynamic of resulted NTL at different scales. The evaluations based on profile curves, spatial patterns, scatter correlations, and histograms, of NTLs, indicate that our approach can achieve a good agreement between DMSP and simulated DMSP data in the same year. Our approach offers the potential for generating a time series of global DMSP NTL data from 1992 to present, which can contribute a more continuous and consistent monitoring of human activities and a better understanding of the urbanization process.

Urban cellular automata (CA) models propagate and accumulate errors during the modeling process due to the model structure or stochastic processes involved. It is feasible to assimilate real-time observations into an urban CA model to reduce model uncertainties. However, the assimilation performance is sensitive to the spatio-temporal units in the assimilation algorithm, that is, spatial block size and window length (temporal interval). In this study, we coupled an assimilation model, an ensemble Kalman filter (EnKF) and a Logistic-CA model to simulate the urban dynamic in Beijing over a period of two decades. Our results indicate that the coupled EnKF-CA model outperforms the CA-alone counterpart by about 10% in terms of the figure of merit, which reflects the agreement of modeled pixels. We also find that the assimilation performance using a finer block (1 km) is better than that using a coarser block (5 km and 10 km) because of the better depiction of spatial heterogeneity using a finer block. Moreover, the improvement of intermediate outputs using the coupled EnKF-CA model is effective for a certain period (e.g. 5 years). This implies that a high-frequency assimilation may not significantly improve the model performance. The sensitivity analyses of spatio-temporal assimilation in the EnKF-CA model provide a better understanding of the assimilation mechanism that couples with land-use change models.

The diffuse and specular components of leaf reflection are both important to determine the leaf optical properties as well as to describe the leaf bidirectional reflectance distribution function (BRDF). However, the specular component is usually ignored in practice in numerous canopy reflectance models that describe the interaction between solar light and vegetated scene components. To evaluate the impact of leaf-specular reflection on canopy bidirectional reflectance factor (BRF) characteristics, we introduce a leaf BRDF model into the radiosity-graphics combined model (RGM; a 3-D scene model) to calculate canopy BRFs with nondiffuse leaves. The modified RGM is validated by comparing simulated BRFs against in situ measured BRFs over real maize canopies. The results show that ignorance of leaf-specular reflection can result in up to 50% of relative error in the blue band (435.8 nm). A series of maize canopies with different leaf angle distributions (LADs) is reconstructed to investigate the effect of five major biophysical/geometrical parameters such as leaf area index, LAD, leaf surface property, view direction, and solar zenith angle on leaf-specular reflection contributions to the canopy BRF. It is demonstrated that increasing the incident solar zenith angle and decreasing the mean leaf angle impact the angular distribution of the canopy BRF more significantly than other factors. The cumulative hemispherical relative and absolute errors of canopy BRF caused by the leaf-specular reflection are often too large to be ignored, even for canopies with rough surface leaves. Moreover, the relative error of BRF in visible waveband shows that, in general, leaf-specular reflection has a large impact than that in near-infrared waveband. However, such impact can be sufficiently accounted for by even just consideration of the first-order leaf-specular reflection in canopy reflectance calculation, leading to a substantial improvement in simulation accuracy for most vegetation canopies.

The Defense Meteorological Satellite Program's Operational Linescan System (DMSP-OLS) nighttime lights imagery has proven to be a powerful remote sensing tool to monitor urbanization and assess socioeconomic activities at large scales. However, the existence of incompatible digital number (DN) values and geometric errors severely limit application of nighttime light image data on multiyear quantitative research. In this paper, we extend and improve previous studies on intercalibrating nighttime lights image data to obtain more compatible and reliable nighttime lights time-series (NLT) image data for China and the U.S. through four steps, namely, intercalibration, geometric correction, steady-increase adjustment, and population data correction. We then use gross domestic product (GDP) data to test the processed NLT image data indirectly and find that sum light (summed DN value of pixels in a nighttime light image) maintains apparent increase trends with relatively large GDP growth rates but does not increase or decrease with relatively small GDP growth rates. As nighttime light is a sensitive indicator for economic activity, the temporally consistent trends between sum light and GDP growth rate imply that brightness of nighttime lights on the ground is correctly represented by the processed NLT image data. Finally, through analyzing the corrected NLT image data from 1992 to 2008, we find that China experienced apparent nighttime lights development in 1992–1997 and 2001–2008, respectively, and the U.S. showed nighttime lights decay in large areas after 2001.

The accurate prediction of crop yield is of great help for grain policy making as the importance of food in human life. By assuming a homogeneous and vertical laminar structure and introducing a multilayer-two-big-leaf model, we developed a radiative transfer equation for winter wheat canopy and a model named RS-P-YEC (Remote Sensing — Photosynthesis — Yield Estimation for Crop) for winter wheat yield estimation. In this model, we converted the net primary productivity to winter wheat yield using harvest index. In this study, we estimated yield of winter wheat in North China Plain using the RS-P-YEC model. The simulated yield agrees well with observations from agro-meteorological stations and the R2reaches to 0.817. This study demonstrates that RS-P-YEC model is useful in the yield estimation of winter wheat in North China Plain with widely available remotely sensed images.

A new LAI retrieval method is developed. The algorithm borrows ideas from the principles and methods of ground LAI measurements, and adopts a new frame which differs from traditional remote sensing LAI inversion methods. The ground data acquired from two field experiments are used to validate the algorithm. In order to resolve the scale exchange problem between high resolution ground observation and low resolution remote sensing data, two high resolution remote sensing images almost having the same resolutions with ground measurements are used as transitions.

Net primary productivity (NPP) is important in the global carbon budget. The change of NPP can be a good indicator of climate variation to some extent. Therefore, it is necessary to study the relationship between climate factors and inter-annual change of NPP, which will help us to understand global change. An empirical exponential model between NPP and integrated NDVI in the Yellow River Basin in China has been established. The spatial distribution pattern and dynamic change of annual NPP from year 1982 to 1998 are analyzed by using multi-temporal 8 km resolution AVHRR-NDVI data. The results show that there exists an incline trend of mean NPP for whole basin while the rainfall decreases slightly, which demonstrates that human activity effects the vegetation cover and NPP much. Finally, in order to analyze the effect of rainfall and temperature on inter-annual change of NPP, correlation coefficient between rainfall, temperature and NPP are computed respectively. It is found that relativity between rainfall, air temperature and NPP is complicated for different climate and vegetation zone. NPP is not highly correlated with climate factors in most places, which may be caused by human activity and other factors. The effect of rainfall on NPP is significant in desert steppe region, while the effect of temperature on NPP is significant in alpine vegetation region and Qinhai-Xizang Plateau. The correlation coefficient between NPP and temperature is negative in area where NPP is positively correlated with rainfall, while it is positive in area where NPP is negatively correlated with rainfall.

We propose an iterative multistage inversion (IMI) algorithm to retrieve the land surface component temperatures for nonisothermal vegetation canopy. Our algorithm is based on a thermal emission model that can simulate the directional effects from the nonisothermal surfaces. Our IMI algorithm just inverts the most uncertain and most sensitive parameters at each step using the most sensitive observation samples, and then adjusts the initial values based on the retrieval results. This inversion process is repeated until convergence condition is satisfied. Compared with the inversion method that try to invert all of the parameters at the same time, the IMI algorithm tends to give more accurate mean values for the parameters and is more stable when the noise level is relative low.

Leaf area index (LAI) is an important biophysical parameter, and remote sensing provides the possibility for the LAI retrieval over large area. Model based inversion is one of the main LAI retrieval methods, and the multi-angle data are the important data sets. However, the general model-fitting algorithm is time consuming in LAI inversion. In this paper, we proposed a kernel-driven model and neural network based LAI inversion algorithm to speed the process. The data obtained by the new Airborne Multi-angle Thermal/Visible Imaging System (AMTIS) is synchronous and has higher resolution. Compared with the low-resolution multi-angle data such as MISR and MODIS, it has a resolution as high as 1.36 m. Using the kernel-driven model, the BRF was reconstructed from the AMTIS data. On the other hand, a 3-dimension radiative transfer model and the measured parameters were used to model the BRF. Then LAI was inversed based on the neural network. Synchronous ground-based measurements of LAI for wheat were taken in Shunyi to validate our method. Some conclusions from the study: (1) LAI can be retrieved successfully using the high-resolution multi-angle data based on neural network; (2) based on the neural network and the kernel-driven model, the inversion rate can be improved; (3) by adjusting the soil moisture classification, the inversion precision can be improved.

Algorithms for remote sensing image classification can be cataloged into classic classification, fuzzy classification, and neural network classification. For classic classification algorithm, the space distribution of data features should be assumed, and it is difficult to put in expert knowledge to remote sensing information. In the fuzzy classification algorithm, the meaning of the subordinate degree is not definite. In the neural network classification algorithm, network framework parameters are difficult to decide, training time is long, and the neural network tends to fall into a local optimization situation. A modified Fuzzy-ISODATA algorithm and a BP neural network algorithm have been developed. The integration of the two algorithms was applied to remote sensing classification. A comparison of classification accuracy, speed and practicability for each algorithm was made based on the same training sampling area. The experiment was conducted in Shunyi, Beijing, China (40/spl deg/00'-40/spl deg/18'N, 116/spl deg/28'-1161/spl deg/58'E, which covers a total area of 1021 km/sup 2/) with a TM image. The result indicates that the accuracy of integration classification algorithm increases compared with the simple fuzzy clustering algorithm and the simple neural network algorithm in the Shunyi area, but the speed should be improved.

A new method to analyze the differences of soil moisture in an area is advanced. Soil moisture is related closely to vegetation growth. An experiment was conducted in the Yellow River Basin in China. The change of vegetation cover is the result of many factors. We postulate that soil moisture in the Yellow River Basin is mainly determined by the precipitation. The delaying effect of the precipitation on the vegetation cover was analyzed in the Yellow River Basin using the cross partial correlation index between precipitation and NDVI. The cross partial correlation index has already excluded the impact of temperature on vegetation cover. Different delaying effects represent different soil moisture. The delaying interval can be adapted to measure soil moisture difference at different levels. On the basis of the above result, the difference of soil moisture in the Yellow River Basin was analyzed, and the difference map of soil moisture in the Yellow River Basin was made.