Data-driven sales optimization with regression and chaotic pattern search

Chaotic maps

Chaotic maps generate chaotic numbers possessing attributes such as pseudo-randomness, sensitivity, regularity, and ergodicity, thereby enhancing the Chaos Optimization Algorithm (COA). These properties of chaotic numbers aids COAs in systematically exploring the entire search space over time. Several chaotic maps (Feng et al., 2017; Fister et al., 2015; Rani, Jayan & Alatas, 2023; Rani & Jayan, 2021; Vallabhaneni et al., 2020; Varol Altay & Alatas, 2020; Bingol & Alatas, 2023) are defined in the literature like logistic map, Chebyshev map, sine map, circle map, ICMIC map, tent map, neuron map, Kent map, Gauss map, etc.

Chebyshev map: Generally, Chebyshev chaotic maps are used to deal with security issues, digital communication, neural networks and in global optimization algorithms. The iterative formula that generates chaotic sequences using Chebyshev map is as follows.

$$x_{n+1}=cos\left(\alpha co{s}^{-1}\left(x_n\right)\right),x_n\in \left(-1,1\right),\alpha \in \mathbf{R}.$$

Based on the Lyapunov exponent and scatter diagram, we choose the best value for the $\alpha$ as 8 as explained in Rani, Jayan & Alatas (2023). Here the sequence $x_n$ is the set of chaotic numbers scattered in the interval $\left(-1,1\right).$ The fixed point $x^{\ast }$ of a chaotic map $x_{n+1}=f\left(x_n\right)$ is a point at which $x^{\ast }=f\left(x^{\ast }\right)$. The chaotic nature of a map will get lost at this point and so care must be taken to avoid the usage of this point as an initial point in any algorithms where chaotic maps are used. The fixed points of Chebyshev map are $-0.9397,-0.9010,-0.5,-0.2225,0.1736,0.6235,0.7660\mathrm{a}\mathrm{n}\mathrm{d}1$.

Hybrid chaotic pattern search algorithm (HCPSA) (Rani, Jayan & Alatas, 2023)

COAs typically excel at swiftly navigating towards the vicinity of the optimum, yet they may require additional computations to precisely reach the optimum from that neighborhood. To mitigate this, hybrid algorithms like HCPSA combine COA in the initial stage to approach the function’s minimum neighborhood and employ a local search algorithm in the subsequent stage to efficiently attain the minimum, as detailed in reference (Rani, Jayan & Alatas, 2023). Rani, Jayan & Alatas (2023) describes an algorithm well-suited for optimizing functions with multiple variables. This algorithm is particularly effective for higher dimensional problems. For readers’ convenience, a description of the HCPSA algorithm from Rani, Jayan & Alatas (2023) is provided in this Section.

Stage I:

• Step 1: Input the objective function $f,$ the lower and upper bounds for the variable as $L$ and $U$. Choose the number of iterations for stage I as $N$. Set $k=1,$ initialize the dimension i and choose $fmin=+\mathrm{\infty }$.

• Step 2: Initialize the initial value of the chaotic variable $c^k$. This initial value is same for all dim in order to reduce the functional evaluations and computational time. The initial value should be selected from the defined interval for each of the chaotic maps and should not be any of the fixed points of the respective chaotic maps.

• Step 3: Map the chaotic variable $c^k$ into the optimization variable ${X_i}^{\left(k\right)}$ $i=1,2,\dots ,n$ by using the following equation. For Chebyshev map we use ${X_i}^{\left(k\right)}=$ $\left(\frac{U+L}{2}\right)+\left(\frac{U-L}{2}\right)c^k$ $i=1,2,\dots ,n$ (This transformation is done since the Chebyshev map generates pseudo random numbers between −1 and +1, but we need search points between L and U.)

• Step 4: Compute the function value $f\left({X_i}^{\left(k\right)}\right)$.

• If $f\left({X_i}^{\left(k\right)}\right)<fmin$ then $fmin$ = $f\left({X_i}^{\left(k\right)}\right)$ and the optimal solution is $Xmin={X_i}^{\left(k\right)}$, $i=1,2,\dots ,n$

• Step 5: Generate the next chaotic variable $c^{k+1}$ by applying the chaotic map.

• Step 6: $k=k+1$ If $k\le N$ go to Step 3, else go to Step 7.

The output of stage I is Xmin, which is in the neighborhood of global optima.

Stage II:

• Step 7: Choose Xmin as the initial point of stage II.

• Step 8: Use Pattern Search Algorithm to find global optima (Rani, Jayan & Alatas, 2023; Hooke & Jeeves, 1961).

Ordinary least square method of regression

Ordinary Least Squares (OLS) regression is a fundamental tool used in statistics and machine learning to understand the relationship between variables. It is particularly useful for linear regression models, to find a straight line that best fits the data points. OLS works by minimizing the squared differences between the actual values of a dependent variable and the values predicted by the equation based on independent variables. In this article OLS regression method is used to obtain the weights used in the velocity to sales function.

Prediction of next activity for a lead

The first and foremost step is to verify that the data is in ascending order based on the date column; otherwise, it needs to be arranged. The activities for customer A1 in date-wise order are provided as an example below. It can be noted that the consecutively repeated activities are removed from this data.

outboundcall email online outboundcall email outboundcall email inpersonconversation outboundcall email outboundcall.

This sequence is cleaned to get a good prediction by removing repeated activities. The repeated activity that is retained is the latest one. After cleaning the customer data of A1, we obtain:

online inpersonconversation email | outboundcall.

The above sequence is divided into two parts: the first n-1 activities shall be treated as the independent variables, and the last activity is considered the dependent variable. By this, the model shall be a supervised learning model.

After following this process for customers A1–A5, the cleaned data is depicted in Table 2.

All possible combinations of the Customer clean sequence of all customers with a minimum length of three are identified. For customer ID A1, all such combinations are listed in column 2 of Table 3.

After following this process for 20,000 customers, a frequency table can be generated based on the number of times the combination appears. Table 4 provides this frequency count for the training data that is used.

As per the given data, the probability is high for a person to take membership if an in-person conversation or an outbound call is made after direct mail and email. Based on Table 4, a suggestion of activities to be followed after a particular sequence in descending order of probability is given in Table 5.

After direct mail email, the sales/marketing team may go for an in-person conversation, an outbound call, or a webpage because they are the topmost options.

Using the probability obtained using the frequency count (as shown in Table 4), a prediction of the next activity is made for the sequences in column 3 of Table 2. If the predicted activity is the same as the one in column 4 of Table 2, the result is taken as ‘TRUE’; otherwise, it is considered as ‘FALSE’. Table 6 shows this step. Based on this Tue/False column, the accuracy of the model is calculated and is found to be 85% for the data taken.

Next, the data pertaining to the ‘FALSE’ cases are removed from the training set. Using the new training set, a model is created that will predict the next activity for every individual in the lead data set (testing set).

Prediction of optimal number of days for each activity

The previous section provides the sequence of the activities that must be followed based on the trained data. This section deals with deciding the optimal number of days to be spent for each activity, thus minimizing the velocity of sales.

Velocity of sales is $VTS=x_1+x_2+\dots +x_n$, where $x_i$ is the number of days taken to complete ${\mathrm{i}}^{\mathrm{t}\mathrm{h}}$ activity. Each activity carries a different weightage. Thus in order to find the optimal $x_i$ for each activity, we need to

(1) $$\mathrm{M}\mathrm{i}\mathrm{n}\mathrm{i}\mathrm{m}\mathrm{i}\mathrm{z}\mathrm{e}{w}_1{x}_1+w_2{x}_2+\dots +w_n{x}_n$$where $w_1,w_2,\dots ,w_n$ are the weights corresponding to the paths $x_1,x_2,\dots ,x_n$ respectively. The procedure for minimization of the given function is explained below:

Step 1: Bifurcate the group IDs whose sequence starts with a particular activity. For example, all the sequences starting with the activity “online”, all the sequences starting with the activity “e-mail”, all the sequences starting with the activity “outbound call”, etc., are separated and considered as different buckets.

The following is an example of a bucket with different group IDs, all of which are starting online.

• A1—online inpersonconversation email outboundcall.

• A2—online webpage email chat outboundcall.

• A3—online webpage chat email inroomletter inpersonconversation outboundcall.

• A4—online inroomletter outboundcall email inpersonconversation.

• …

Step 2: Tabulate each path and the number of days required to complete each activity in these paths for each customer ID in each bucket, as provided in Table 7. The total number of days taken to complete all activities of each customer is also tabulated. An example of this for a bucket in which all paths start with ‘online’ is provided in Table 7.

Step 3: Pivot the table obtained in the previous step by keeping group IDs as rows and paths as columns so that it becomes appropriate for applying machine learning models. After pivoting Table 7, we get Table 8.

Step 4: To the pivoted table of Step 3, apply the ordinary least square (OLS) model of regression (Daliya, Ramesh & Ko, 2021; Sathyadevan & Chaitra, 2015) by taking the number of days (second column of Table 8) as dependent and all paths (remaining columns of Table 8) as independent variables. The result of the OLS model gives an approximation for the coefficients of $x_{\mathrm{i}}$’s, i.e., $w_{\mathrm{i}}$’s in the objective function $w_1{x}_1+w_2{x}_2+\dots +w_n{x}_n$ along with a standard error of regression, $\epsilon$. For better accuracy of the proposed model, we consider $w_{\mathrm{i}}$’s also as unknowns with bounds as the coefficient of $x_{\mathrm{i}}$obtained using the OLS $\pm \epsilon$ for $i=1,2,\dots ,n$.

The minimum and median number of days corresponding to each path $x_{\mathrm{i}}$ in a bucket are taken as the lower and upper bounds for each $x_{\mathrm{i}}.$

Step 5: Apply HCPSA to minimize $w_1{x}_1+w_2{x}_2+\dots +w_n{x}_n$ with 2n unknowns, each with bounds, as given in Step 4. The solution of HCPSA gives optimal $x_{\mathrm{i}}$’s and $\sum x_{\mathrm{i}}$ will give the optimal velocity of sales for one particular bucket.

Step 6: Repeat Steps 2 to 5 for all the buckets and consolidate all bucket outputs in a single table. Remove the duplicates, if any. Finally, we obtain a list of distinct paths along with the optimal number of days required to complete the activity in that path.

The model in “Prediction of next activity for a lead” trains the customer data to predict the next activity for a lead and, the model in “Prediction of optimal number of days for each activity” suggests the number of days required to complete this activity based on the trained customer data. The results thus obtained from the training data are then used to decide the following activity for leads in the lead data (testing data) and to set a target (number of days to complete the prescribed activity) for the salespeople, which, if achieved shall optimize the velocity to sales.

The code for this research was written in Python for easy replicability. It was developed and run on a standard laptop with 8 GB RAM and an Intel i5 processor.

Executing this work presented few key challenges that includes data cleaning difficulties and arranging the data in the required sequence. By implementing improved data processing techniques, we can significantly enhance the proposed algorithm’s performance. This highlights the potential for further research and development.