One of the key's to the discovery of the Higgs Boson by the Large Hadron Collider at CERN
was the recognition of the quantity of data required to reveal a signature of the Higgs Boson. This needed a novel approach to the analysis and storage of the mountain of data that would be generated by the Large Hardron Collider.... big data had arrived. Just as CERN was first with the world wide web it was also first with the recognition of the Big Data problem. The story started with the discovery of the neutral current (a.k.a. the Z boson) in 1973 by the Gargamelle bubble chamber. This was remarkable because three events were observed in over 1.4 million bubble chamber photographs taken over a two year period, we didn't have digital image processing back then! Now, research is massively dependent on digital image processing and it is fascinating to follow the enormous variety of research problems in the area of solar physics, these researchers make use of toolkits such as the IDL solar software library, the more recent SunPY project or even Matlab. Along with many disciplines, there are two massive problems in this research field
- Solar physics has a big data problem, how do researchers collaboratively analyse the mountain of data both historical and being generated by new satellites studying our nearest star.
- How do we ensure that the software we use continues to be fit for purpose?
The previous talk given by Xin Huang discussed a model for deep learning based solar flare forecasting. Solar flares originate from the release of the energy stored in the non-potential magnetic field of active regions, the triggering mechanism for these flares, however, are still unknown. For this reason, conventional solar flare forecasting is probabilistic and based on the statistical relationship between the characteristic parameters of active regions and solar flares. In the deep learning method, forecasting patterns can be learned from the line-of-sight magnetograms of solar active regions. It is necessary to obtain observational data with sufficient size to train the forecasting model and test its performance. Huang described how a dataset was created from the line-of-sight magnetogarms of active regions observed by SOHO/MDI and SDO/HMI from April 1996 to October 2015 along with the corresponding soft X-ray solar flares observed by GOES. The MDI data was taken as the training set and the HMI data as the testing set. The experimental result indicated that (1) the forecasting patterns can be automatically reached with the training set and these patterns can also be applied to the testing set, which is reduced to be the MDI proxy data; (2) the performance of the deep learning forecasting model is not sensitive to the given forecasting periods (6 hour, 12 hour, 24 hour or 48 hour); (3) a reasonable forecasting model is achieved for solar flares with higher importance. Huang used a deep learning package called CAFFE and used a single NVIDIA GPU (see references 6-9) below. He described how a cascade of layers in a convolutional neural network were used for feature extraction. The trick with deep learning is to exploit readily trained networks and to make use of supervised learning.
This talk was rather inspirational I've known for a long time that the Matlab package provides machine learning toolbox. At the risk of a little knowledge being dangerous I decided to try one of the matlab deep learning demos with a GPU, which is a Demonstration of Image category classification using deep learning (ref 2). This was very easy to run and I attempted a simple image classification on a set of photographs, clearly this is very powerful. But this is open to all our users on the central HPC at the university of sheffield it's possible to run the matlab deep learning demos. ShARC features a range of deep learning and machine learning software which has been well used and tested by the RSE and machine learning groups at The University of Sheffield.
A further possibility for researchers is to use the new deep learning cluster, JADE, based at Oxford ( see reference 10 ). It is fortunate that The University of Sheffield is a partner in this project making access much easier for researchers to use this powerful and increasingly used technique to meet the challenge of the big data problem (see reference 19). We can look forward to some excellent adventures exploring the great solar data mountain!
- The discovery of the weak neutral currents
- Demonstration of Image category classification using deep learning with Matlab
- Neural network toolbox for Alexnet Network with Matlab
- Neural network importer for CAFFE models
- Mathworks neural networks toolbox team
- Predicting Coronal Mass Ejections Using Machine Learning Methods
- Solar Flare Prediction Model with Three Machine-learning Algorithms using Ultraviolet Brightening and Vector Magnetograms
- Space weather research group (Bradford)
- Automated Prediction of CMEs Using Machine Learning of CME – Flare Associations
- AUTOMATIC SHORT-TERM SOLAR FLARE PREDICTION USING MACHINE LEARNING AND SUNSPOT ASSOCIATIONS
- Studying imagery from solar dynamics
- Application of Convolution Neural Network to the forecasts of flare classification and occurrence using SOHO MDI data
- Application of a deep-learning method to the forecast of daily solar flare occurrence using Convolution Neural Network
- GPU Computing Sheffield