The particle swarm optimization (PSO) algorithm is an established nature-inspired population-based meta-heuristic that replicates the synchronizing movements of birds and fish. PSO is essentially an unconstrained algorithm and requires constraint handling techniques (CHTs) to solve constrained optimization problems (COPs). For this purpose, we integrate two CHTs, the superiority of feasibility (SF) and the violation constraint-handling (VCH), with a PSO. These CHTs distinguish feasible solutions from infeasible ones. Moreover, in SF, the selection of infeasible solutions is based on their degree of constraint violations, whereas in VCH, the number of constraint violations by an infeasible solution is of more importance. Therefore, a PSO is adapted for constrained optimization, yielding two constrained variants, denoted SF-PSO and VCH-PSO. Both SF-PSO and VCH-PSO are evaluated with respect to five engineering problems: the Himmelblau’s nonlinear optimization, the welded beam design, the spring design, the pressure vessel design, and the three-bar truss design. The simulation results show that both algorithms are consistent in terms of their solutions to these problems, including their different available versions. Comparison of the SF-PSO and the VCH-PSO with other existing algorithms on the tested problems shows that the proposed algorithms have lower computational cost in terms of the number of function evaluations used. We also report our disagreement with some unjust comparisons made by other researchers regarding the tested problems and their different variants.
Optimization can be simply defined as the process of searching for the best outcome. The process of optimization is not new to human beings; they have been applying it since the dawn of time. Owing to its role in solving real-world problems, optimization is imperative for research in applied mathematics. With the advent of digital computing machines in the early 1950s, population-based stochastic algorithms started to develop, paving the way for solutions to complex optimization problems that were previously considered to be unsolvable [
Without loss of generality, in the minimization sense, a COP can be defined as follows [
In problem
Nature-inspired algorithms (NIAs), as the name suggests, are inspired by successful biological and/or physical systems in nature [
PSO was developed by Kennedy and Eberhart in 1995. Owing to its ease of implementation, PSO is one of the most widely used SI-based algorithms. In the PSO algorithm, an initial population of particles (swarm members) is generated randomly in a given search space. Each particle is assigned a velocity. In subsequent iterations, the velocity of each particle is updated using appropriate formulae. This updated velocity is then used to update the position of the particle. This process of updating velocities and then positions is repeated until some stopping criteria are fulfilled. To deal with constrained problems, PSO, like other EAs and SI-based algorithms, needs modification in the sense that some CHT must be incorporated into its framework. Several such modifications for single- and multi-objective constrained optimization are detailed in [
Any parameter-free CHT is preferred over the commonly used penalty function approach for constraint handling provided it gives competitive results. The superiority of feasibility (SF) [
In [
The remainder of the paper is organized as follows. Section 2 contains a brief literature review pertaining to PSO and some constrained versions of PSO. Section 3 presents a brief introduction of the two CHTs and details of proposed algorithms. Section 4 presents and discusses experimental results. A short summary of the work and plans for the future are given in the concluding Section 5.
In this section, we first describe PSO and a number of its improved variants. Then, a number of promising constrained algorithms employing PSO as a base algorithm are discussed.
PSO is among the most popular algorithms in the field of optimization. It has been widely used in various applications including in engineering and industry. PSO was inspired by the navigation patterns and synchronized movements of bird flocks. PSO was developed in 1995 for unconstrained optimization problems. Over the years, various versions of PSO have been developed. Owing to its ease of implementation, PSO is among the most widely used SI-based algorithms. However, PSO differs from EAs, as it assumes that a member of the swarm never dies but rather flies from one place to another in a given search space. In the PSO algorithm, an initial population of particles (swarm members) is generated randomly in a given search space. Each particle is assigned a velocity, which can initially be set to zero or generated randomly [
where
where
As we can see from the inertia component of
When first introduced, the inertia weight was kept constant, but various researchers soon experimented with changing the value of the parameter
where
The parameter
The velocity given by
As far as the population size of the swarm is concerned, there is no significant difference in the results when the population size is varied [
Owing to the fast converging capability of PSO, various researchers have integrated it with CHTs for solving COPs. Some of the prominent works using PSO as a base algorithm are described in the following sections.
In FS-PSO [
In HPSO [
In HPSO, the personal best solution,
The authors of SR-PSO incorporated stochastic ranking (SR) into PSO to develop a hybrid algorithm for COPs. Although SR is a popular CHT that works quite well in most cases, there remains a user-defined parameter
In [
In [
In [
In this section, first we briefly describe the two CHTs that were used to adapt PSO for constrained optimization. Then, the two adapted algorithms, SF-PSO and VCH-PSO, are detailed.
Any parameter-free CHT will be preferred over a penalty function approach if it gives competitive results. The SF technique is one such CHT. In a penalty function approach, if competing solutions are feasible, the selection is based on objective function values; if competing solutions are infeasible, objective function values are augmented with a penalty term. The SF technique is different in the sense that if the competing solutions are infeasible, their objective function values are no longer important, and their degrees of infeasibility are considered for their ranking. Furthermore, the SF technique is a combination of two different categories. On the one hand, it separates feasible solutions from infeasible ones; on the other hand, it penalizes infeasible solutions. If competing solutions are infeasible, their constraint violations will be used to augment the objective function value of the worst feasible solution, not the competing solution’s own objective function value. This fitness function is constructed as follows:
where
Between two feasible solutions, winner will be the one with the lower objective function value.
Between a feasible and an infeasible solution, a feasible solution will always be preferred over an infeasible one.
Between two infeasible solutions, the winner will be the one with the lesser degree of constraint violations.
The pseudo-code for selection in the SF technique is given in Algorithm 1.
In VCH technique, the number of constraint violations was used as a preferred CHT. Here, selection of the winner is made in such a way that if the two competitors are infeasible, the winner will be pushed towards the feasible region. If the competitors are feasible, the quality of the solution will be improved. In any CHT, the criteria for selection of a winner among the infeasible competitors is mainly based on either their distance from the feasible region or the degrees of their constraint violations. In VCH, it is the number of constraint violations that decides the fate of infeasible competitors. VCH is a CHT that is based on the principle of “survival of the fittest.” Fortune cannot favor a solution that is not fit. On the other hand, techniques such as SR have a space for the survival of the luckiest. A solution can go to the next generation by chance even though it may be the worst with respect to the degree of constraint violations. In SR, if for a user-defined parameter
Between two feasible solutions, the winner will be the one with the lower objective function value.
Between a feasible and an infeasible solution, a feasible solution will always be preferred over an infeasible one.
Between two infeasible solutions, the winner will be the one with the smaller number of constraint violations.
Between two infeasible solutions having the same number of constraint violations, the winner will be the one with lesser degree of constraint violations.
The pseudo-code of VCH is given in Algorithm 2.
PSO in its original version, like the majority of other meta-heuristics, was developed for unconstrained problems. To deal with constrained problems, the PSO algorithm needs modification in the sense that some CHT must be incorporated into its framework. Two adapted algorithms, VCH-PSO and SF-PSO, for problem
Although the SF technique is sometimes placed into the category of CHTs that separate feasible solutions from infeasible solutions, it does not do so in a true sense. Rather, it adds the amount of infeasibility to the objective function value of the worst feasible solution, such that infeasible solutions are penalized without giving weight to the penalty term. Thus, the SF technique can be referred as a parameter-free penalty function approach. SF-PSO differs from the SF technique in the sense that in the case of infeasible solutions, the objective function value of the worst feasible solution is not added to form the new fitness function. For infeasible competitors, the degree of constraint violations is the sole decisive factor in SF-PSO.
In VCH-PSO, the decision is solely based on the number of constraint violations when comparing infeasible solutions. If the numbers of constraint violations for two competitors are equal, then the degree of violation will be the basis for selection of the winner. The focus of this research was to solve COPs of type (1). In our proposed algorithms, equality constraints present in any problem are transformed into inequalities as follows:
where
In order to evaluate the performance of the proposed algorithms, five engineering problems were selected from the available literature. All these problems were constrained and non-linear. As these problems had already been attempted by various researchers, a comparative study was used to help evaluate the performance of the proposed algorithms. The results were compiled from 20 independent runs of each algorithm on each problem. The PC configuration and parameter settings used were as follows.
The system used for the simulations ran Windows 10 with 8 GB RAM and a Core m3 1.00 GHz processor. All experiments were performed in the MATLAB R2013a environment.
In this section, the experimental results are discussed for the selected engineering problems. It can be noted from the comparison tables that different researchers used different numbers of function evaluations (
This nonlinear optimization problem was proposed by Himmelblau in 1972 [
Algorithms | Best | Worst | Mean | st. dev. | NFEs |
---|---|---|---|---|---|
−30665.5386718 | −30665.5386718 | −30665.5386718 | 350000 | ||
HPSO | −30665.539 | −30665.539 | −30665.539 | 81000 | |
SR-PSO | −30665.5386 | −30665.5364 | −30665.5386 | 100000 | |
CPSO | −30665.53 | −29846.65 | −30665.53 | NA | 240000 |
SF-PSO | −30665.5386718 | −30665.5386712 | − |
25000 | |
VCH-PSO | −30665.5386718 | −30665.5386712 | −30665.5386717 | 25000 |
Some researchers have claimed far better results than the ones as mentioned in
Algorithm | Best | Worst | Mean | st. dev. | NFEs |
---|---|---|---|---|---|
−31011.9988 | −30762.8890 | −30947.3262 | 55.86 | 50000 | |
−31025.5168 | −31012.5285 | −31023.9699 | 2.62 | 50000 | |
VCH-GA | −30998.951 | −30800.891 | −30845.422 | 48.61 | 50000 |
SF-PSO | 2.51e −4 | 25000 | |||
VCH-PSO | 25000 |
This change in the problem definition has greatly affected the results. The results for this second version attained by our algorithms VCH-PSO and SF-PSO are compared with those of
This problem was originally proposed by Rao in 1996 [
Algorithm | x1 | x2 | x3 | x4 | f (x) |
---|---|---|---|---|---|
SS-Rao | 0.2444 | 6.2177 | 8.2915 | 0.2444 | 2.381 |
SF-PSO | 0.2444 | 6.2175 | 8.2915 | 0.2444 | 2.381 |
VCH-PSO | 0.2444 | 6.2175 | 8.2915 | 0.2444 | 2.381 |
Algorithm | Best | Worst | Mean | st. dev. | NFEs |
---|---|---|---|---|---|
SR-PSO | 1.724866 | 1.725422 | 1.724899 | 1.12E-06 | 100000 |
CPSO | 1.728024 | 1.782143 | 1.748831 | 0.0129 | 240000 |
NM-PSO | 1.733339 | 1.726373 | 0.003 | 80000 | |
VCH-GA | 1.726718 | 1.728074 | 1.727529 | 0.000421 | 50000 |
SF-PSO | 1.724852 | 1.888685 | 1.73866 | 0.039532 | 25000 |
VCH-PSO | 1.724852 | 1.76926 | 1.729839 | 0.012529 | 25000 |
Algorithm | |
||||
---|---|---|---|---|---|
PSO-GA | 0.20573 | 3.25312 | 9.036624 | 0.20573 | |
BWOA | 0.205829 | 3.251922 | 9.034556 | 0.205829 | 1.69562 |
SF-PSO | 0.20573 | 3.25312 | 9.036624 | 0.20573 | |
VCH-PSO | 0.20573 | 3.253121 | 9.036624 | 0.20573 |
The spring design problem was first proposed by Arora in 1989 [
Optimal solutions obtained by VCH-PSO, SF-PSO, and some other contemporary algorithms are listed in
Algorithm | Best | Worst | Mean | st. dev. | NFEs |
---|---|---|---|---|---|
SR-PSO | 0.012668 | 0.012685 | 0.012678 | 100000 | |
NM-PSO | 0.012675 | 0.012924 | 0.01273 | 80000 | |
VCH-GA | 0.012672 | 0.012706 | 0.012693 | 50000 | |
ETEO | 0.01269 | 0.01267 | 8100 | ||
SF-PSO | 0.014808 | 0.013291 | 25000 | ||
VCH-PSO | 0.016412 | 0.013394 | 25000 |
This table indicates that VCH-PSO and SF-PSO attained the same best results with
This problem is based on the minimization of the cost of manufacturing a vessel that can be used to store air. Four decision variables are used to represent the thickness of the vessel, thickness of the head, internal radius, and length of the cylindrical section of the vessel. The problem was proposed by Sandgren in 1988 [
Algorithm | Best | Worst | Mean | st. dev. | NFEs |
---|---|---|---|---|---|
SR-PSO | 5886.198 | 6315.015 | 80.44 | 100000 | |
CPSO | 6061.078 | 6363.804 | 6147.133 | 86.45 | 240000 |
NM-PSO | 5930.314 | 5960.056 | 5946.79 | 9.16 | 80000 |
VCH-GA | 6059.792 | 6060.215 | 6060.062 | 0.12 | 50000 |
SF-PSO | 6411.586 | 6099.488 | 158.99 | 25000 | |
VCH-PSO | 5866.686 | 6554.209 | 6111.09 | 219.17 | 25000 |
Algorithm | x1 | x2 | x3 | x4 | f (x) |
---|---|---|---|---|---|
BWOA | 1.258663 | 0.621865 | 65.17912 | 10.19874 | 5347.689 |
SF-PSO | 1.258847 | 0.622249 | 65.22523 | 10 | |
VCH-PSO | 1.258847 | 0.622249 | 65.22523 | 10.00001 |
Kannan and Kramer designed this problem in 1994 [
Algorithm | Best | Worst | Mean | st. dev. | NFEs |
---|---|---|---|---|---|
BA | 263.8962 | 263.9025 | 263.9061 | 15000 | |
SR-PSO | 263.8958 | 263.908 | 263.8978 | 100000 | |
CSA | 25000 | ||||
SRIFA | 240000 | ||||
ETEO | 263.8959 | 263.8958 | 600 | ||
SF-PSO | 263.8959 | 263.8982 | 263.8964 | 25000 | |
VCH-PSO | 263.8995 | 263.8963 | 25000 |
In this work, two constrained variants of PSO, SF-PSO and VCH-PSO, are proposed. The performance of both algorithms has been evaluated on five engineering problems. Two issues regarding these problems have been identified through this work. First, different versions of the problems exist. Himmelblau’s nonlinear problem has two versions that differ in the constraints involved. The welded beam problem has three versions, which vary in constraints’ definitions and the terms involved in constraints. The pressure vessel problem has four different versions, which differ in the definition of the objective function. The second issue is regarding the bounds for decision variables. Various researchers have used different bounds. This greatly affects the results, as changing the bounds means changing the search space.
For the first version of Himmelblau’s problem, both algorithms attained the optima mentioned in the literature. For the second version of the problem, the algorithms obtained better results than some well-known recent algorithms. Three different versions of the welded beam problem were tried: In the first two versions, our algorithms produced competitive results, while for the third version, our results were better than those of the competitors. For the spring design problem, our algorithms performed better than some state-of-the-art algorithms. In the first version of the pressure vessel design problem, VCH-PSO and SF-PSO gave high mean and st. dev. values, indicating their weak performance. However, for the second version, Chen’s algorithm BWOA was surpassed by a wide margin. For the three-bar truss problem, VCH-PSO gave better results but with a narrow margin. These results are encouraging, as VCH-PSO attained the optimum with a 100% feasibility rate. The low st. dev. values in most cases indicate the consistent performance of the proposed algorithms. The experimental results show no marked difference between the proposed algorithms. It may also be noted that our proposed algorithms achieved results better than or comparable with those of competing algorithms on most of the tested problems, with fewer
In future, the SF and VCH techniques will be implemented in other SI-based algorithms such as the FA and BA to adapt them for constrained optimization.