I.J. Image, Graphics and Signal Processing, 2020, 2, 9-18
Published Online April 2020 in MECS (http://www.mecs-press.org/) DOI: 10.5815/ijigsp.2020.02.02
Exploring the Effect of Imaging Techniques Extension to PSO on Neural Networks
Anes A. Abbas
Department of Computer Science, University of Bahrain
Email: nasas.tacktechs@gmail.com
Nabil M. Hewahi
Department of Computer Science, University of Bahrain
Email: nhewahi@uob.edu.bh
Received: 30 August 2019; Accepted: 17 November 2019; Published: 08 April 2020
Abstract—In this paper we go through some very recent under control. Most of the metaheuristic techniques imaging techniques that are inspired from space depend on randomness but usually randomness might exploration. The advantages of these techniques are to lead to more cost in terms of time and resources. To help in searching space. To explore the effectiveness of reduce this problem, human expeditions move in a these imaging techniques on search spaces, we consider controlled randomness to ensure reaching to near-the Particle Swarm Optimization algorithm and extend it optimum solution consume less cost. using the imaging techniques to train multiple neural Many researchers work to either propose new networks using several datasets for the purpose of metaheuristic techniques inspired from biology or classification. The techniques were used during the develop systems that combine various metaheuristic population initialization stage and during the main search. techniques. Proposing new techniques such as Genetic The performance of the techniques has been measured Algorithm (GA)[11], Simulated Annealing (SA) [17], ant based on various experiments, these techniques have been colony [6,7], bat algorithm[31], PSO and fish swarm evaluated against each other, and against the particle [16][26], and Combining more than one technique such swarm optimization algorithm alone taking into account as PSO and GA, GA and SA, or PSO and SA the classification accuracy and training runtime. The [4][12,13][27][31]. In both the cases, proposing new results show that the use of imaging techniques produces metaheuristic or combine more than one technique, the better results. target is to better explore the search space and achieve better results (near-optimum). Index Terms—Search Space Imaging, Metaheuristics, In [22] Richards and Ventura proposed a technique to Optimization, Particle Swarm Optimization, Artificial initialize the population called centroidal Voronoi Neural Networks, Population Initialization. tessellati which starts with a population that is generated randomly then iteratively tries make the particles move
far from each other as possible to have a diversity in the initial population. In [21] authors proposed another I. INTRODUCTION
technique for population initialization where the random
The main purpose of all metaheuristic algorithms is to
particles are generated and with each generated particle
explore the search space and help in reaching to the its complement/inverse particle is also generated. Then optimum or near-optimum solution. Some of the main from these particles the initial population is generated. problems that might not help the metaheuristic techniques
In [18] Maaranen et.al proposed a technique called
to reach to optimum solutions is either because of the
quasi-random generator technique to initialize the
complexity of the problem or problem resources are
population. This technique forms repetitive patterns to
unclear or limited [15]. Most of the metaheuristic
avoid having many particles in similar/close locations.
algorithms are inspired and adopted from biology or This technique’s idea is based on generating a population nature such as chromosomes, birds, fish swarm and bats with high diversity as possible. [10].
Based on the researchers work, they try to have a
In nature, humans try to explore space through various
diverse initial population which means having diverse
tools such as telescopes, satellites and radars. Usually
particles so that they can capture the most of the good
expeditions to discover the space are sent after very deep particles. However, this might not always true because investigations through the previous mentioned tools. This this might be done regardless of the importance of these will help in limiting the scope of exploration towards the
particles. The imaging technique depend basically on
scientist’s target. The closer the expedition to the target,
exploration first then expedition, which means checking
the more effort will be done starting from the reached
first the potential particles then forming the population.
position. The general idea is to maintain randomness
Copyright ? 2020 MECS I.J. Image, Graphics and Signal Processing, 2020, 2, 9-18
10 Exploring the Effect of Imaging Techniques Extension to PSO on Neural Networks
Initialize swarm
Do until maximum iterations or minimum error criteria For each particle
Calculate location’s fitness value
If the fitness value is better than pBest Set pBest = current fitness value If pBest is better than gBest Set gBest = pBest For each particle
Calculate particle’s new velocity and location End Do until
Fig. 1. Particle swarm optimization pseudo code; pBest: Best solution
found by particle, gBest: Best solution found by swarm [28]
In this research, our objectives is to explore the effect of imaging techniques as extend to PSO in the classification of neural networks. We are not interested to compare the obtained results with other approaches, but instead we care about comparing the obtained results using extend PSO and the regular PSO. To evaluate imaging techniques, they are implemented to extend a particle swarm optimization metaheuristic used to tune the weights and biases of multiple classification neural networks in seven different datasets. Thus, the following three subsections briefly introduce the particle swarm optimization metaheuristic and the artificial neural network techniques.
A. Particle Swarm Optimization
PSO is a very well know metaheuristic search algorithm used in various applications ranging from science, business, optimization to engineering [14][27][31][32]. The idea of the algorithm has been inspired from the movements and travel of birds. The main steps of the algorithm is illustrated in Fig. 1. Each bird in the bird’s flock represents a particle and in each algorithm iteration the velocity and the location of each particle is updated. For more details about concepts of PSO, researchers can refer to [16][26]. As will be explained in the next sections PSO is selected to be our metaheuristic technique. B. The neural network
Artificial Neural Networks (ANN) are an active area of research in the fields of artificial intelligence and machine learning invented as techniques to loosely mimic the function of neurons in living beings.
Applications of the neural network such as classification require configuring a set of parameters called the weights and biases of the network so that for a given set of inputs describing some object, the neural network could produce a correct output identifying that object. Several algorithms such as the back-propagation algorithm can be used to perform the task of configuring/training the network [24]. In this paper we use imaging techniques as extension to metaheuristic technique (i.e., PSO) to explore their effectiveness in improving the accuracy of neural network classification.
Copyright ? 2020 MECS C. Metaheuristic Approaches for ANN
Backpropgation (BP) algorithm is a very well known algorithm used in feed farward ANN training. Training using BP may yield some times to what so called local minima which prevents reaching to global minma and the training does not improve [13]. Various techniques and methods have been tried to get rid of local minima, some of these are using metaheuristic techniques. Many researches have been working to adjust the neural networks weights based on methaheuristic techniques to improv the classification, some of these are based on GA and PSO as examples [12][19]. In other cases, reserchers attempt to combine more than one metaheuristic approach to utilize the capabilities of each one of them, therefore, trying to obtain better classification accuracy than using one technique alone [12,13]. In this paper we select PSO as a well defined metaheutstic technique that has been used in improving the neural network classification as a technique to be extended by imaging techniques to measure the effectiveness of those imaging techniques in improving the neural network classification
II. IMAGING TECHNIQUES
In this section we present briefly some of the imaging techniques that were proposed by Abbas and Hewahi [1]. These imaging techniques work as a telescope to gather information first about the search space then help the metaheuristic technique to direct its search. This process will be continuously performed throughout all the searching stages (during initialization or during the main search) within the metaheuristic algorithm. To understand the imaging techniques, we need first to define what we mean by image. Imaging techniques in [1] have been tested on multiple optimization functions using COCO and have shown a good potential to improve the results. A. The image
Image in imaging techniques concept means capturing a certain area in our search space. This captured area should be bounded and we refer to that by scope. The image has resolution which means the number of pixels, here those pixels are referred to be as temporary particles. When the image is taken within a certain scope, its pixels will be checked against their usefulness based on the used imaging technique, if useful, they will be within the population and if not, they will be discarded. Every pixel has a fitness value and based on that it will be preserved and added to the population or discarded [1]. The imaging technique can be merged to a metaheuristic search as shown in Fig. 2. Fig. 3 shows the extend PSO. Capturing images (pixels) in the Figures mean instantiating several particles within a certain range but without still considering them as particles. The image technique then selects some of these to create the initial population. Similar thing will be done during the main search by selecting certain pixels to be temporary particles and then based on that the population will be updated. The temporary pixels will be then discarded.
I.J. Image, Graphics and Signal Processing, 2020, 2, 9-18
Exploring the Effect of Imaging Techniques Extension to PSO on Neural Networks 11
Generally, the more is the number of temporary particles/pixels, the better is the captured area of the search space. Fig. 2. General metaheuristic imaging extension flowchart
Fig. 3. PSO imaging extension flowchart
B. Imaging Types
In this research we present various imaging techniques that can be incorporated with any metaheuristic algorithm which starts from initial population. We consider five types, out of which four are used during the initial population and one used during the main search. The four used for the initial population are Starry Night (SN),
Copyright ? 2020 MECS Fireworks (FW), Lanterns (LN) and the Grid (GRD). The first three imaging types are given in [1], whereas the fourth one is a new proposal, The fifth imaging type is used during the main search and called Connecting the Dots (CDT) [1]. ?
Starry night
This technique is applied during the generation of the initial population generation. Fig. 4 shows the SN mechanism.
Fig. 4. Starry night (SN) imaging technique
Starry night technique is quite simple in a way that one global image represents the search space and as many as possible number of pixels are generated. For every generated pixel, a fitness value is computed. Those pixels with high values are selected to form the initial population and the rest will be ignored. One major parameter in this technique is the number of generated pixels in the scope of the image [1]. ?
Fireworks
This technique is applied during the generation of the initial population generation. Fig. 5 depicts the mechanism of FW.
Fig. 5. Fireworks (FW) imaging technique
This technique can be summarized as below:
a. Follow the same procedure used in SN by having
one global image and generation of pixels.
b. Create some equal sized local images within the
scope of the global image.
c. Form the population from the best pixels in the
local images and the global image.
As done with SN technique for every pixel a fitness value is obtained and then those with the best values are selected as the initial population and the rest will be ignored. In this technique we have several parameters such as number of pixels in the scope of the global image,
I.J. Image, Graphics and Signal Processing, 2020, 2, 9-18
成像技术对粒子群优化算法的推广对神经网络的影响(IJIGSP-V12-N2-2)
