Keywords

1 Introduction

Large scale black-box optimization problems play a very important role in many scientific and engineering applications. It can be formulated as follows [12]:

$$\begin{aligned} x^{*}=\arg min_{x \in R^{D}}f(x) \end{aligned}$$
(1)

where \(x^{*}\) is the global minimizer to be searched and \(f: R^D \rightarrow R\) is the evaluation function. In this paper, we focus on the large scale black-box optimization problem [11], i.e., the evaluation function f typically has three properties:

  • Black-box: When a candidate solution \(x \in R^{D}\) is evaluated by f, we can only obtain the function evaluation value f(x). Any other information of f, such as gradient information, is unavailable.

  • High-dimension: The dimension D of the candidate solution x is typically larger than or equal to 1000 (i.e. \(D \ge 1000\)).

  • Multimodal: The evaluation function f can have many local optima and global optima solutions.

Recently, swarm intelligence (SI) [6, 7] has been a promising technique to solve the large scale black-box optimization problems [2, 8]. Most existing SI algorithms such as particle swarm optimization (PSO) [14] and ant colony optimization (ACO) [5] are inspired by the swarm behaviours of the simple creatures in nature. However, BSO [3, 4, 15], a young and promising SI algorithm, is inspired by the brainstorming process of the high-level human being, which utilizes the clustering operation to simulate the brainstorming process and generates the new candidate solutions by learning other solutions from intercluster or intracluster randomly. In general, the existing BSO algorithms can suffer from three problems: (1) cannot obtain good performance on the high-dimension optimization problems; (2) several hyperparameters of algorithm itself need to be optimized; (3) the clustering operation (e.g. k-means [10]) is high computational cost.

To address the aforementioned problems, in this paper, we propose a novel BSO variant named BSO-CLS. It is inspired by the participators’ ideas generating process of brain storm, i.e., each new idea is generated by cooperatively learning the ideas from other participators. Thus, for each candidate solution of BSO-CLS, it iteratively updates itself by linearly combining other solutions with the weights deriving from the function evaluation values of other solutions.

The contributions of this paper are summarized as follows:

  • We introduce a novel cooperative learning strategy to simulate the new ideas generating process of brain storm. The analysis and experiment show that the novel learning strategy is effective and efficient.

  • In order to validate the effectiveness of the proposed method, we test it on 6 benchmark functions with the 1000 dimensions. The empirical results show that BSO-CLS can outperform the vanilla BSO and the other BSO variant with the learning strategy.

The rest of this paper is organized as follows. Section 2 introduces the representative BSO variants. Section 3 introduces BSO-CLS in detail, and Sect. 4 presents and discusses the experimental setting and results. The conclusions are drawn in Sect. 5.

2 Related Work

In this section, we will introduce some representative BSO variants. In order to reduce the high computational cost of clustering operations, BSO-OS [13] proposed to cluster individuals in the objective space instead of in the solution space. MBSO [17] proposed a modified BSO which introduced the simple grouping method (SGM) with the idea difference strategy (IDS). RGBSO [1] proposed to utilize the random grouping strategy in clustering.

Recently, some learning based new solutions generating strategies have been proposed. BSOLS [16] proposed a novel learning strategy after updating operator to improve the diversity of population. OLBSO [9] introduced a orthogonal learning (OL) framework and incorporated it with BSO to improve the performance.

3 Proposed Method

In this section, we will introduce the proposed method in detail. The key idea of BSO-CLS is that it utilizes a novel method to simulate the new ideas generating process of the brain storm. Specifically, as shown in Fig. 1, during the brain storm process, the participators generally go through three processes to cooperatively learn other participators’ ideas before proposing their own ideas. Taking participator P0 as an example, the first process is to listen to other participators’ ideas and select the ideas that inspire him. The second process is to analyze the selected ideas. Some of them have a greater impact on him, and others have a smaller impact on him. The third process is to come up with his own ideas based on his analysis and share them with other participators. Thus, other participators can follow these three processes to come up with their own ideas until the brain storm is finished.

Fig. 1.
figure 1

The processes of the new ideas proposed in the brain storm.

Full size image

Based on the processes introduced above, the BSO-CLS can be implemented as follows:

Step 1: Randomly initialize N candidate solutions \(X \in R^{P \times D}\) with D dimension.

Step 2: Evaluate the fitness \(Y = f(X)\) using the evaluation function f.

Step 3.1: For each candidate solution, randomly select m other candidate solutions \(x_{1},x_{2}, \cdots , x_{m}\) as the learning objects.

Step 3.2: For each learning object \(x_{i}\), calculate the learning weight \(w_{i}\) using the formula (2) by normalizing the fitness values \(f(x_{1}),f(x_{2}), \cdots , f(x_{m})\) of the learning objects.

$$\begin{aligned} w_{i} = \frac{f(x_{i})-\mu }{\sigma } \end{aligned}$$
(2)

where \(\mu \) and \(\sigma \) are the mean and standard deviation of the fitness values \(f(x_{1}),f(x_{2}), \cdots , f(x_{m})\) respectively.

Step 3.3: Generate the new candidate solution \(x_{new}\) using the formula (3) and evaluate its fitness value.

$$\begin{aligned} x_{new} = \sum _{i=1}^{n}w_{i}x_{i} \end{aligned}$$
(3)

Step 3.4: If N new solutions have not been generated, go to Step 3.1.

Step 4: If the stop condition is not met, return Step 3.1.

The procedures of BSO-CLS have been given in the Algorithm 1.

figure a

4 Experiments

In this section, we will present and discuss the experimental settings and results.

4.1 Benchmark Functions

To validate the effectiveness of the BSO-CLS, we test it on the 6 benchmark functions including the unimodal function (Sphere) and the multimodal functions (Rosenbrock, Ackley, Rastrigin, Schwefel, and Ellipsoid). Furthermore, in order to test the proposed method’s performance on the various dimensions, we vary the dimensions from 10 to 1000 for all benchmark functions. The information of the benchmark functions are summarized in Table 1.

Table 1. The information of benchmark functions
Full size table

4.2 Comparison Method

This paper aims to propose a novel learning mechanism to generate the new candidate solutions for BSO algorithms. Therefore, we choose the most representative SI algorithm PSO [14], the original BSO algorithm [15] and the other BSO variant BSOLS with learning strategy [16]. The hyperparameters setting for each algorithm is given in Table 2.

Table 2. The hyperparameters setting for each algorithm
Full size table

4.3 Experimental Setting

In order to fairly compare all algorithms, they are tested with the same population size 100 and the same maximum number of function evaluations \(1E4 \times D\) in each run for each test function, where D is the dimension. In order to reduce the statistical error, each algorithm is tested 30 times independently, and the mean and standard deviation (std) are reported for comparison.

Table 3. Solutions accuracy (mean and std) comparisons.
Full size table

4.4 Results and Discussion

The experimental results are given in Table 3, and the best results are marked with bold.

As shown in Table 3, we can clearly observe that: (1) The proposed method can obtain competitive performance on the low dimension problems against other methods. (2) The proposed method outperforms all the comparison methods on the high dimension problems. (3) With the dimension increasing, the proposed method obtains better results on the Rosenbrock test function.

5 Conclusion

In this paper, we proposed a novel BSO variant named BSO-CLS, which introduced a simple yet effective cooperative learning mechanism. The key concept is inspired by the new ideas generating process of brain storm which the participators propose their own ideas by cooperatively learning other participators’ ideas. Thus, BSO-CLS iteratively updates the candidate solutions by linearly combining other solutions with the weights deriving from the fitness values of other solutions. The effectiveness of the proposed method was validated on 6 benchmark functions for both low-dimension and high-dimension settings. The empirical results showed that BSO-CLS can outperform the vanilla BSO and the other BSO variant with the learning strategy especially for the large-scale optimization problems.

In the future, we plan to validate the proposed method on more complex problems, such as the rotation and shifted benchmark functions. We intend to explore the reasons behind the better performance on Rosenbrock function with the dimension increasing. Furthermore, we will test the influence of different normalization functions.