Combination of Wavelet Analysis and Artificial Neural Networks Applied to Forecast of Daily Cosmic Ray Impulses

Artificial neural network modelling has proven incredibly effective in an impressively wide range of scientific disciplines. The combination of these various methods with wavelet decomposition signal processing has similarly proven to be a powerful development for statistical forecasting of a number of environmental processes. Space weather modelling and prediction has often been applied to forecasting of solar activity and that of the planetary magnetic field. However, prediction of cosmic ray impulses has seen little development in the context of neural network modelling. In the present study, a combination of wavelet neural networks was adapted from previous research in order to predict daily average values of cosmic ray impulses 30 days in advance. Additional comparison of both neural network and linear regression modelling with and without wavelet decomposition was conducted for further demonstration of increased accuracy with wavelet neural networks in a simple input-output fitting model.


INTRODUCTION
The use of advanced computational models, particularly artificial neural networks, in the long-term forecasting of various environmental factors has recently seen a surge in increased application [1]. Artificial neural networks (ANN) are nonlinear methods of artificial machine learning based on the nervous system. Rather than relying on correlative relationships, ANNs learn much the way a child does by 'observing' multiple instances of a particular outcome. This method of analysis may therefore provide useful alternative approaches to a number of statistical problems [2].
Neural networks are particularly useful for highly complex or 'noisy' datasets and advanced predictions. Input data is entered through the network in a series of artificial 'neurons' which output to a hidden layer of neurons before output convergence (prediction). In a typical input-output fitting neural network, backpropagation algorithms are often employed and constitute one of the primary foundations for ANN analyses which dynamically update statistical weights in the system; these statistical weights form the 'synaptic connections' between neurons in the network.
A primary training algorithm used in backpropagation ANN learning is the Levenberg-Marquardt algorithm, which is a standard curve fitting method often used in generic problem solving using a damped least squares process [3]. Given minute variations in statistical results upon subsequent re-training of a particular network, large numbers of individual ANNs may be trained and tested in order to average the outputs for more accurate performance results. While autoregressive functions have previously been applied to neural network analyses for prediction of various space weather phenomena [1], exploration of advance statistical prediction has recently been examined without the necessity for strictly autoregressive network feedback. Furthermore, the combination of both ANN models and wavelet decomposition methods has proven effective in other areas of environmental prediction [4].
In wavelet analysis, a given signal is decomposed into a series of time-frequency domain variables such that by summing resultant frequencies will result in a reconstruction of the original signal. A mother wavelet is applied to the original data in order to extract further information about the original variable of interest within a set of time-frequency representations for subsequent processing of a continuous-time signal. This method of decomposition allows for additional information and much greater detail of a particular process for further analysis. Wavelet decomposition has recently seen much wider use in direct association with neural network predictions by 'increasing' the available information of a single variable for further analysis [4]. Data forecasting specifically for events related to galactic cosmic rays (CR) has received surprisingly little attention in the context of neural network computations. Cosmic rays are simply high-energy particles which have been theorized to originate from a number of extragalactic sources, the most prominent of which include supernovae [5]. The composition of CRs consists predominantly of protons, with other heavy nuclei. These particles are capable of penetrating the atmosphere of the planet and, through ionization processes, are involved in weather conditions on Earth [6]. They also produce a large proportion of the ambient background radiation to which all life is exposed. Various health risks have been associated with cosmic radiation, particularly at aircraft altitudes [7] and, of course, outside of the Earth's atmosphere [8]. However, various effects of CR events have also been revealed within microelectronic systems [9]. For the present study, combination of wavelet decomposition and ANN input-output fitting procedures were adapted from Krishna et al. [4] and employed in order to predict the average daily CR impulses per minute 30 days (~1 month) in advance. Additional neural networks and linear regression models were produced both with and without wavelet decomposition for comparison of overall performance.

METHODS
Daily averages of cosmic ray activity (average CR impulses per minute) were obtained (N = 1431, µ = 9347.5, σ = 266.86) from the Moscow Neutron Monitor database (http://cr0.izmiran.rssi.ru/mosc/main.htm). Wavelet decomposition was applied to CR data using one-dimensional (1D) analysis with db2 (Daubechies-2) as the mother wavelet (Fig. 1). Three-scale decomposition was applied in order to decompose the data into one lowfrequency approximation (a 3 ) and three sets of high-frequency details (d 3 , d 2 , d 1 ). The resultant time-frequency variables (Fig. 3) were entered into a multilayer neural network as input neurons.
. Input-output neural network fitting was conducted using a multilayer backpropagation network. The standard Levenberg-Marquardt algorithm was employed with a single hidden layer and three hidden layer neurons, determined by performance a priori. Initial neuron weights were randomized, while a sigmoid (logistic) hidden layer activation function was used along with a hyperbolic tangent (linear) output layer activation function (Fig. 2). Accuracy was measured according to the root mean squared error (RMSE), while both parametric (r) and non-parametric (rho) correlation coefficients were employed for determining overall fit of each model. Original CR values were lagged by 30 days and entered as a single output neuron for prediction without autoregressive feedback. For each network analysis, a total of 50 individual neural networks were trained with a random subset of data (70 % of cases used to train each network) and each was tested on the remaining cases (N = 430 for test data). Results from each test were averaged for a more accurate approximation of overall performance. This series of procedures was conducted using data both with and without wavelet decomposition. Finally, linear regression analysis was applied, again, both with and without prior wavelet transformation, for further comparison.
Regression analyses were conducted using SPSS 17, while all other analyses were completed with Matlab 2011a software.

RESULTS
Initial analysis of untransformed data (e.g., without prior wavelet decomposition) was conducted using ANN methods previously discussed to predict CR impulses from data that had been lagged by 30 days.

Figure 4. Fit of target and predicted data from ANN analysis without wavelet decomposition
Secondary ANN modelling was employed with time-frequency variables obtained through wavelet analysis. It was determined that the wavelet neural network (WNN) adapted from previous research [4] produced results similar to the preliminary ANN with slightly greater average accuracy (RMSE = 151.03 impulses/min) and moderately improved overall fit for values predicted 30 days in advance ( Fig. 5; r = .843, rho = .853, p < .001).
Finally, a series of linear regression analyses were conducted using a random sample of lagged input data. The first employed lagged data from the original signal without wavelet decomposition. A significant model was obtained with accuracy less than that revealed for either ANN or WNN analysis (RMSE = 182.76 impulses/min). Overall fit for target and predicted values, while significant, was also less than that found for previous analyses ( Fig.  6; r = .747, rho = .801, p < .001).  Next, time-frequency variables obtained through wavelet analysis were entered into a second linear regression with full variable entry. As with the neural network models, employing wavelet decomposed input variables showed a slight increase in average accuracy (RMSE = 178.31 impulses/min) compared to analysis with standard predictor values. This was also the case for overall data fit for target and predicted values, showing an improved overall fit ( Fig. 7; r = .772, rho = .807, p < .001) compared to the previous regression model. In order to compare average fit of predicted values to target data between models, a Fisher r to z transformation was applied to correlation coefficients (r to z = 0.5[ln(1+r) -ln(1r)]) in order to facilitate z-test comparisons and difference probabilities. The WNN used revealed a significantly improved fit of target data compared to linear models used (p < .001). While the output fit for the WNN model was not significantly greater than the standard ANN model (p > .05), both models produced significantly greater data fit compared to regression analysis (z = 3.88 for WNN and 2.61 for ANN, ps < .01). Finally, while the combination of both wavelet decomposition and linear regression produced an overall fit that was not significantly reduced compared to the ANN model (p > .05), the WNN analysis outperformed the wavelet regression (z = 3.01, p < .01)

CONCLUSIONS
As found in a number of previous research protocols [4,10], the combination of both wavelet analysis and ANNs can significantly improve overall accuracy and fit of statistical predictions. Although standard analyses may be perfectly capable of achieving specific research goals, a combination of analytical methods can often improve or enhance existing results. While neural networks have previously been used for prediction in a number of areas, including space weather [1], pattern recognition [11], and medical diagnosis [12], applying signal processing techniques from wavelet analysis appears capable of introducing greater accuracy for statistical modelling in this context [4,10].
The current results both confirm and extend the use of WNN models through additional comparison with linear statistical methods. Furthermore, the specific application of wavelet decomposition in combination with regression analysis was proven more effective than standard processing when compared to neural network models. That predictive space weather modelling has seen increased use and effectiveness recently [1] suggests a further need to examine the potential use of ANNs in this specific context. Along with the current results, this research implies the need to expand data prediction to alternate statistical techniques which may improve both accuracy and overall understanding of the phenomena in question [13].
Given that CR activity has been associated with effects on a range of microelectronics, particularly satellite systems [14][15], it may be important to develop early prediction of this type of event with greater accuracy. Furthermore, these techniques should be explored for a range of space weather events for which early warning applications may be relevant. Predictive modelling of space weather phenomena could be beneficial to a diverse array of research areas including microelectronic functioning [9], solar-geomagnetic [1] and interplanetary conditions [16], and even human biology [17].