Existing convolution recurrent neural networks (ConvRNNs)-based memory cells majorly take advantage of gated structures and attention mechanisms to extract discontinuous latent associations for spatial-temporal sequence forecast (STSF) problems, which may lead to serious over-fitting and spurious relationships with correlated noise. It is a consensus that incorporating cause-effect relationships in modeling can alleviate these problems. In this paper, we propose a Causality Attention Unit (CAU) to assist ConvRNNs by complementing the causal inference ability in a plug-and-play way. Specifically, CAU serially consists of the attention module and causality module. The former is constructed by a spatial-channel attention layer, which preliminarily generates the correlated future with the correlations between historical memories and the current state. The latter borrows the idea of transfer entropy (TE) to detect the latent cause-effect relationships and precisely correct the correlated future. A space-time exchange strategy for accelerating the calculation ofTEin CAU is also designed. CAU can be easily combined with the existing ConvRNN cells, and we construct a simple general model to predict long-term spatial-temporal series, which consists of encoder/decoder and stacked CAU paralleled to stacked ConvRNN cells. After determining the optimal model structure, we carry out a series of experiments to evaluate model performance, including comparisons with other advanced models, training loss analysis, and multiple ablation and sensitivity studies. Experimental results show that our proposed model can effectively improve the performances of existing ConvRNNs to the state-of-the-are level on representative public datasets, including Moving MNIST, KTH, BAIR, and WeatherBench. The ablation and sensitivity studies verify the superiority of CAU. The learned causal maps precisely distinguish the pixel attributions and motion characteristics in sophisticated entangled scenarios.

In this paper, a Physics-Informed Red Tide forecast model (PIRT) considering causal-inferred predictors selection is proposed. Specifically, the directed acyclic graph-graph neural network (DAG-GNN) method is first applied to quantify the causality among multiple ocean-atmosphere-biology variables for selecting the most significant predictors of the red tides (or other chlorophyll variations). Then, the encoder-decoder model consisting of an Energy Attention Module (EAM) is built for daily red tide forecasting. The multi-sourced multivariate dataset during 2010-2020 covering the East China Sea serves to train and evaluate PIRT. The experimental results demonstrate that the predictors in the learned causal graph are closely related to the occurrence and decay of red tides, which exhibits high physical interpretability. PIRT has a superior forecasting skill, the predictions of which are with highly consistent spatial patterns especially in extreme events. The 7-lead-day forecast errors for chlorophyll are within 0.9 mg∙m-3, which is much better than the other models. This also indicates that PIRT can be used as a reliable tool to study the ecology of the East China Sea.

Abstract Conditional nonlinear optimal perturbation (CNOP) defines an optimization problem to study predictability and sensitivity of the oceanic and climatic events in the nonlinear system. One effective method to solve the corresponding problem is feature extraction-based intelligent algorithm (FEIA) framework. In the previous study, the mapper and the re-constructor of the framework are generally obtained by principal component analysis (PCA), but the solving performance still needs to further improve. Recently, neural network has attracted the attention of lots of researcher, and many structures of neural network can be used to construct the mapping-reconstruction structure of FEIA framework. However, the related studies applying neural network in FEIA framework are lacking. Compared with PCA, neural network might obtain a proper structure for FEIA framework with the well-directed training. Therefore, this paper suggests two ways applying neural network in FEIA framework, and the corresponding frameworks are tested to solve CNOP of double-gyre variation in Regional Ocean Modeling System (ROMS). The results show that FEIA framework with neural network can obtain the solutions with better objective function values, and the corresponding solutions have a larger probability leading to the related physical phenomenon.

The North Atlantic Oscillation (NAO) has a significant effect on the global weather and climate variation. Thus, it has widespread scientific research value to enhance prediction skill of the NAO events. The NAO is quantified by the NAO index which can be defined from sea level pressure (SLP), and has the characteristics of changeability and complexity. With the applications of deep learning approaches in the field of climatic prediction, deep neural networks provide the alternative to predict the NAO index beyond numerical models. In this paper, long short-term memory (LSTM) is used to predict the NAO index sequence. Furthermore, ConvLSTM is used to predict SLP grid data with capturing the temporal and spatial interdependencies. The predicted NAO index can be obtained by projecting output SLP on the NAO anomaly pattern. In order to improve the prediction accuracy, especially for extreme events, we adopt discrete wavelet transform (DWT) to preprocess data, thus DWT-LSTM and DWT-ConvLSTM models are proposed for the NAO index data and SLP grid data respectively. Preprocessing steps include discomposing the input data into the low-frequency and the high-frequency components using DWT along with considering the local time dependency. Observation data is selected as the benchmark, and our models are compared against multiple models, such as support vector regression (SVR), LSTM, gated recurrent unit (GRU) etc. The experimental results demonstrate that both DWT-LSTM and DWT-ConvLSTM perform better, particularly at peak values. In addition, our models are much closer to observation in multi-step forecasting compared with ensemble forecasts based on numerical models.

El Niño Southern Oscillation (ENSO) event is characterized by sea surface temperature (SST) anomalies in the tropical Pacific and mainly identified with Oceanic Niño Index (ONI). ENSO forecasting is very challenging owing to the existence of predictability barrier and chaos of climate variability. Recently, machine learning approaches have received considerable attention besides conventional numerical models for this task. However, these existing works mostly focus on investigating single ONI data, neglecting the spatial and temporal dependencies of SST data, and the skill of predictions reduce significantly beyond a lag time of 6 months. With the goal of capturing the spatial and temporal dependencies of SST simultaneously and improving the skill of prediction over longer time horizon, we propose ConvLSTM-RM model, which is a hybrid of convolutional LSTM and rolling mechanism, and use it to build an end-to-end trainable model for ENSO forecasting problem in this paper. Specifically, ENSO forecasting is formulated as a spatiotemporal sequence forecasting problem in which both the input and the output are SST sequences, and ONI can be acquired based on the output. Experiments on historical SST dataset demonstrate that ConvLSTM-RM outperforms seven well-known methods over multiple time horizons (6-, 9- and 12-month). We also apply this model to predict the latest ENSO event during 2015/2016. The results show that our model gets a relatively reliable prediction both from SST grid maps and ONI.

Intelligent algorithms have been applied extensively to solving conditional nonlinear optimal perturbation (CNOP), which is considered as an effective approach in the study of weather and climate predictability. Recent studies reveal that single particle intelligent algorithms could solve CNOP efficiently while their results are unstable. Conversely, swarm intelligent algorithms can get similar CNOP to adjoint method stably but still show lower time efficiency. For the purpose of balancing between efficiency and stability, in this paper, we introduce a linearly decreased dimension number strategy for dynamic search fireworks algorithm (dynFWA), called ld-dynFWA, and apply it to solving CNOP. Meanwhile, to accelerate the computation speed, we parallelize the ld-dynFWA with MPI. To demonstrate the validity, we take Zebiak-Cane (ZC) numerical model as a case to study ENSO event. Compared with CTS-SS and MABC, which are the latest research on single particle intelligent algorithm and swarm intelligent algorithm for solving CNOP respectively, experimental results show that the proposed parallel ld-dynFWA can obtain a better CNOP efficiently and stably.

Conditional nonlinear optimal perturbation (CNOP) is an initial perturbation evolving into the largest nonlinear evolution at the prediction time. It has become a useful tool in meteorology and oceanography. The common method for solving the CNOP is the adjoint-based method which is always referred to as the benchmark. Unfortunately, many numerical models have no corresponding adjoint models, and developing a new one is usually a huge engineering, which consequently limits the application of the CNOP. In order to avoid adjoint models, we propose a principal components-based great deluge method to solve the CNOP. Through extracting principal components, the original problem is reduced into a low-dimensional space to hunt the coordinate of the optimal CNOP with the great deluge method. A regeneration strategy is also designed for the great deluge method to break away from local optimal positions. In addition, the proposed method can be parallelized to improve the computing efficiency. To demonstrate the validity, the proposed method has been studied in the Zebiak-Cane model to solve the CNOP. Experimental results show that the proposed method can efficiently obtain a satisfactory CNOP, approximate to the one computed with the adjoint-based method, and the parallelizing version can reach the speedup of 9.7 times with 12 cores.

QoS prediction has become an important step in service recommending and selecting. Most QoS prediction approaches are using collaborative filtering as a prediction technique. But collaborative filtering may suffer from data sparsity problem which degrade the prediction accuracy. In order to alleviate the data sparsity problem of collaborative filtering, we presented a hybrid QoS prediction approach by applying clustering on web services before applying collaborative filtering (named services clustering QoS prediction, SCQP). The clustering process cluster web services in to service clusters in which services have the same physical environment. Then the similarity between users is calculated based on these service clusters instead of individual services. So that there are more information to be used when calculate the similarity and it will contribute to elevate the prediction precision. The experimental results showed that our hybrid approach could not only achieve higher prediction precision, but also reduce the computation time than other collaborative filtering based prediction methods.

Conditional nonlinear optimal perturbation (CNOP) is an extension of linear singular vector(LSV) to nonlinear optimization. Generally, CNOP is solved with such adjoint based algorithms as SPG2, SQP. Unfortunately, it is often difficult to obtain the corresponding adjoint models for some nonlinear models. In addition, for nonlinear models containing discontinuous “on-off” switches, the adjoint based methods can hardly find the correct gradient direction for solving the CNOP. These two factors restrict the application of CNOP. Intelligence algorithms are utilized to handle these problems and has made some improvements. Nevertheless, the intelligent method cannot be applied to solve CNOP of complex models due to its dimensional limitation. Therefore, the principal component analysis based genetic algorithm (PCAGA) is proposed to solve the CNOP of complex models. The PCAGA is composed of two key processes: dimension reduction and genetic optimization. To demonstrate the validity, PCAGA is applied to solve the CNOP of the Zebiak-Cane (ZC) model for studying ENSO predictability and compared with the adjoint based method. Experimental results show that PCAGA can achieve similar results to the adjoint based method in the high-dimensional space of a medium- complexity model without the adjoint models. The proposed method also can future be applied to solve variational data assimilation (VDA).

This paper focuses on modeling and parameter estimation for the electrohydraulic actuation system of an articulated forestry machine. The linear graph method is implemented in deriving mathematical models of the swing, boom and stick subsystems. Actuation dynamics are subsequently integrated with manipulator dynamics to result in a complete machine model. Identification procedures employed in estimating physical parameters are discussed. Model validation studies show good agreement between model predictions and experiments. The derived models will be used for designing a controller for coordinated endpoint motion, for prediction, and for a real-time graphical training simulator.

This paper focuses on the modeling, parameter estimation, and model validation in open and closed-loop of an experimental forestry machine manipulator. Symbolic Newton-Euler and linear graph methodologies are used in deriving mathematical models of the swing, boom and stick subsystems. Actuation dynamics are integrated with manipulator dynamics to result in a complete manipulator and actuation model. Identification procedures employed in estimating physical parameters are discussed. Model validation studies show good agreement between model predictions and experiments. The models will be used for designing a controller for coordinated end-point motion and for a real-time graphical training simulator.