1 Introduction

Human resource is generally the most valuable asset for an organization. Skilled human recourses enable an organization to excel and achieve its objectives efficiently [1, 2]. To classify an employee as skillful, her experience acts as a key indicator in addition to other basic credentials [3]. Organizations prefer to retain their existing skilled employees by offering multiple perks and benefits [4]. They also try to attract skilled resources using similar measures. At times, such skilled resources are attracted by the competitor organizations. This results in the issue of employee retention [5]. The issue is dependent on the country’s culture, its economic growth, the number of companies operating in public/private sectors and the availability of skilled human resource [6]. Interestingly, today`s corporate sector has seen an increased number of employees leaving the previous job to find better opportunities [7]. Organizations facing this challenge need to adopt new strategies and identify factors to motivate their skilled resources. Human resource (HR) departments maintain an employee retention policy for this task. Such policies are highly dependent on the data from their current staff, organization’s functions, and other previous experiences. Identification of key factors that influence employee retention is an important research undertaking. These factors do depend on the study domain. Previous studies have suggested multiple reasons for an employee to leave an organization. These can be low current pay, competitor offering better career opportunity, organization’s environment, organization’s culture or employees being bullied by the coworkers. On the contrary, an organization can also ask its employees to leave their job due to poor performance, attitude issues or financial crises. All this results in affecting overall health of an organization since new human resource needs to be evaluated, hired, trained, and transferred the domain knowledge. Therefore, skilled employee retention is crucial to many organizations. If an organization fails to retain its current employees, they must invest a considerable amount of money for training new employees again and over again.

Most of the organizations strive to keep their employees satisfied to reduce their turnover rate. Loosing skilled and experienced workers reduces organization's productivity and profitability. Previous studies [8, 9] show that to keep “employee happy,” organizations should consider some key factors like knowing the employee well, creating an interactive, innovative, and cultural environment that indirectly keeps reminding your employees to stay loyal to their organization, offering reward, and recognizing best performers. Providing workers with a better leadership also works well in retaining the staff [9]. Few of the rapidly growing sectors like telecom, information technology and higher education need to know the key factors specific to them that can assist in retaining skilled work force. The work presented here deals with this issue by utilizing computational techniques and the emerging concepts of data mining. The key aim of this study is to find the factors that can increase employee retention in various working sectors. This work uses frequent items mining (FIM) techniques from the domain of data mining to identify factors that commonly exist together to influence employee retention. Finding frequently occurring items in a transactional database is an active research problem. The problem is commonly known as market basket analysis. The applications of finding frequently occurring items range from core computer science problems to a range of multidisciplinary areas of research. The aim of market basket analysis is to find all items in a dataset that occur together above a certain frequency [10]. Later, these frequently occurring patterns are analyzed to find associations between various factors. This study is based on following research questions.

  • RQ1: Which factors do the computational techniques identify as crucial for retaining employees and what is the relationship between those factors across multiple sectors in the developing countries?

  • RQ2: Which demographic and organizational environmental factors influence employee retention across multiple sectors in the developing countries and how these factors rank against each other?

To address the abovementioned questions, this research uses a qualitative approach. The research questions are answered through a questionnaire in this work. A survey was distributed in the major cities of Pakistan such as Karachi, Lahore, Rawalpindi, and Islamabad. The survey questions were built based on the factors such as recognition and reward, advancement and growth, relationship with supervisors, work conditions, income, ethical behavior, organizational satisfaction and commitment, bullying and work harassment. These factors helped to identify the features and their correlation for employee retention. The data analysis was divided into six stages: These include (a) loading raw data from the survey forms to a text file, (b) analysis of data through one-way ANOVA, (c) identification of correlating factors through frequent items mining (FIM), (d) analysis of data through exploratory factor analysis (EFA), (e) analysis of data through Pearson correlation (PC), and (f) analysis of data through regression analysis (RA). The association rule mining technique, which is preceded by the FIM method, is used to analyze and interpret the data. The ARM is a tool that identifies the frequently occurring factors in the responses with other features. The Statistics Package for Social Science Software (SPSS) is also used to analyze the data. One-way ANOVA is used to see a significant difference in data, and the EFA is used to interpret the variables. The Pearson correlation is used to observe the correlation between independent and dependent variables, and regression analysis is used to study the impact of independent variables on the dependent ones. A combinational approach is applied to the data that helps in analyzing the responses.

1.1 Present work aim and motivation

The employee retention is a growing problem in today's modern world, and it needs to be solved using various retention strategies to improve the employees' turnover rate. There is a demand for skilled workers in areas such as hospitals, software industry, universities, banks, and many other emerging sectors. However, unfortunately, the number of qualified employees at times remains low. Organizations are therefore in a need to find ways to reduce their turnover rate. This study aims to determine the factors that can reduce such organizational problems. Specifically, the task here is to determine what factors are used for higher employee retention in various organizations. This study is focused to find the features that influence employee retention and the relationship between independent factors and employee retention. The findings will be useful for many organizations to enhance their retention strategies. This work is motivated by the employees’ perspective rather than the organizational point of view. Therefore, the finds of this work are based on the data collected from various mid- to early-career individuals instead of taking the decisions-makers’ perspective.

The rest of the paper is organized as follows. Section 2 contains a detailed literature review on employee retention and other important factors for the same. Section 3 lists the methodology utilized in this work. Section 4 contains the results obtained. Section 5 lists the policy implications. Finally, Sect. 6 concludes this work and also mentions a few of the further research directions.

2 Related work

This section covers the previous work on employee retention. The section is organized factorwise where the previous work on each factor that can help retain employees is described. The section also contains relevant theories that can help build a conceptual framework for this or other such studies. Previous studies have shown the following factors that influence employee motivation to continue working with the same organization, working environment, organization commitment, reward and recognition, work performance, supervisor support, and income.

2.1 Working environment

The working environment is a factor in an organization where employee tends to show their positive abilities and leadership skills. Authors in [11] suggest that a positive working environment can have good impact on the employees. They state that different organizations may have dissimilar working environment depending on the clients the organization deals with. Ritter et al. [12] suggest a working environment that includes a culture where it involves good communication between co-workers, leadership style, and professional growth. In retaining employees, one needs to have a healthy working environment. The good working environment requires an appreciation for others, a strong relationship between colleagues, and no harassment [13]. Christmas et al. [14] suggest retaining an employee with good professional skills by improving the organization’s working environment. In order to improve their working environment, organizations should facilitate their employees and provide them necessary equipment that can help the organization in better management. The work in [15] presents a study on employee engagement. Their aim is to find correlation between purpose and joy in a work environment among the managers and their employee engagement. The domain of study is medical profession, and the data are collected from nurses. The authors use Cronbach's alpha to gauge internal consistency in a population sample. The obtained results do not find any significant correlation between nurse manager meaning and joy in their work and the employee engagement. Their study is limited to a specific set of individuals, and the same method if applied to a different dataset may yield diverse results. The work in [16] presents a study on employees’ perception on formulation of human resource policies. They also cover the implementation of various human resource retention plans in their work. The focus domain of their work is the hospitality industry. The data in their work are primarily obtained through personal interviews of employees in a specific region within a few cities.

2.2 Organization commitment

Studies have shown that employees with higher commitment stay with the organization for a longer period, whereas those having low commitment leave the organization during early stages. The employees with higher commitment also desire to stay in the organization and work hard with a positive attitude. Previous work identifies that organization's commitment is related to employees’ turnover. Higher rate of commitment level of the organization results in lower turnover. Bashir et al. [17] represented three dimensions of organizational commitment. Affective commitment is the sense of attachment toward the organization and relation with employee’s characteristics, work performance, and structure of the organization [18]. For example, an employee stays in the organization because she knows their value in the organization [19]. Continuance commitment is the realization of a cost that is related to the organization [20]. For instance, employees will stay in the organization because they know if they leave, they have to face a higher risk of not getting a new job [18]. The normative commitment deals with an emotional feeling of employees [17].

2.3 Reward/recognition and work performance

The terms reward and recognition have high impact on employee retention. These factors are used by organizations to motivate their employees. A reward is given by the organizations to the employees for their best performance, which keeps them motivated. The work in Silbert et al. [21] suggests that organizations can offer reward in the form of cash, bonuses, promotion, recognition, or announcing a worker as an employee of the month, offer trip, and other benefits. According to the authors, organizations present reward to employees so that they keep giving their best performance. Such organizations believe that reward and recognition keep employees motivated for future performance. It is important that employee should think that their perceptions are valued by the organization when they are rewarded.

Work performance is another factor that has an impact on employees and the organization. It is a critical factor for retaining employees. Reviewing the performance of employees can help both the organization and the employees. Employees can be assisted by telling them where they stand in the organization and what are their strengths and weaknesses. In a few cases where employees are highly talented, an increased pay or other benefit does not motivate them; however, performance appraisal does. The organization implies factors like performance appraisal, leadership, reward and recognition, training, and development in order to keep employees motivated to work harder.

2.4 Supervisor support

Supervisor support is defined as a relationship between employees and managers, and it is a factor that has huge impact on the employee retention. The employees tend to stay in an organization when they have good communication skills and strong support from supervisors. When employees have a supportive environment that increases their ability and comfort level of working, they tend to produce excellent results. The authors mention that an organization should be a place where the employee tends to stay. For this, the supervisors should be trained so that they can build a comfortable working environment for the staff [22]. A study suggested that improved employee’s performance results in a tendency to improve the capabilities of their work [23].

2.5 Income-related benefits

The work in [24] stated that employees and supervisors are motivated to work effectively when they are paid and provided with other benefits. There are a number of reasons for employees to be dissatisfied with a job. In addition to an individual’s domestic issues, income is one of the reasons when employees feel dissatisfied [25]. To improve retention strategies, organizations should periodically increase income scales and other benefits such as good working environment, leadership skills, the workload that employee can bear, and flexible timings. Deery et al. [26] find other factors such as flexibility in work, learning, and training, provision of resources to employees and reward system to improve employee retention. Gevrek et al. [27] explore the Schadenfreude effect in employee retention. They study five different salary rises in their work. Their study is based on a dataset constructed over a period of five years by obtaining data from university employees. The obtained results suggest that a one-time, small increase in compensation does not influence employee retention. The work in [28] aims to identify the retention strategies that have an actual effect on the employee turnover. They present a procedure to build an uplift model for testing the effectiveness of the different strategies for the task at hand. Their uplift model is based on a machine learning classifier, i.e., random forest. It is used for personal treatment learning estimation.

2.6 Bullying and work harassment

Bullying is considered as one of the serious problems at the workplace. Studies conducted worldwide identify increased bullying factor in organizations [29]. There are direct negative effects of bullying. It is stated that violence in the workplace also increases the factors such as bullying, workplace harassment, and emotional abuse [30]. The work in [31] examines the correlation between workplace bullying and high-performance work practices (HPWPs). They also suggest a few possible solutions. The obtained results suggest a positive effect of HPWPs on employee well-being. They also observe that reduced role conflict has an influence of HPWPs and less bullying. A limitation of their work is reliance on single-source, self-reported data. This may have caused biased views.

2.7 Factors that improve retention

There are a few other factors that can improve employee retention. These have been identified by an assortment of research contributions. Past work states that retaining talented employees should be the organization's primary focus. In their work, health, success and safety are correlated with retaining the employees. The studies in [32] and [33] identified some strategies for retaining employees and improving employee productivity by including factors in organizations such as appreciating employee on a good performance, mentoring, management, morale, and employee development training. Work in [34] identifies factors such as leadership skills, utilization of skill, compensation, safety and security and professional success to improve employee retention. A study [35] conducted in five companies of India on hundred managers and staff concludes that the factors such as income, training possibilities and careful selection of employee improve job satisfaction and commitment. It also has an influence on retaining employees. Another study on middle managers of Nigeria concludes three factors: compensation, advancement growth and affiliation, to be the reason to stay within the organization [36]. A research on hotel employees discovered that employee tends to stay in an organization for a longer period if they are satisfied with their job and the environment of the organization. The communication has always been a factor through which one can understand the employees better. Studies have shown that poor communication between co-workers leads to a poor employee retention. The economic circumstances and market forces in the world have an impact on the employee’s decision to stay or leave an organization. The certainty of an employee leaving a job and finding another job is when economic conditions are better. A research study found that the better the economic surroundings, the higher are the chances for an employee to leave the organization. Somewhat similar work that utilizes computational methods [37,38,39,40,41] to predict customer churn can be seen in past works. Similar computational methods [42] can be utilized to predict the retention period of a particular employee in an organization. However, for this, the historical data related to the employee and the company will be required to train the model.

2.8 Employee retention factors in the developing countries

Compensation is considered to be a key factor to retain employees in the developing countries. In this context, the work in [43] presents a case study of Hong Kong and China. The data are collected from 704 respondents to identify the important compensation components by various organizations. The study also identifies the six most important compensation components from an employee perspective. In Hong Kong, these five factors are salary, merit pay, end-year bonus, annual leaves, mortgage loan, and profit sharing, whereas for China the first three factors are the same as those for the Hong Kong and the remaining three include housing provision, overtime allowance, and individual bonus. This suggests that the employee retention factors vary between various countries and economies. Lall et al. [44] evaluate the analytical framework of the globalization–employment relationship in the developing countries. The focus of their study is on the manufacturing sector employees. It is observed in the study that globalization may cause an outflux of the talent pool from the developing/underdeveloped countries to the developed nations. Lowell et al. [45] present a report on the impact of high-skilled mobility from the developing countries. The report focuses on eight countries, namely Bulgaria, South Africa, Argentina, Uruguay, Jamaica, India, Philippines, and Sri Lanka. They identify four issues yet to be researched about. First is to evaluate the particular channels of impact generated by highly skilled emigration. Second is to study the range of feedback effects on the total emigration impact. Third and fourth are how high-skilled migration increases country trade and the need for documentation. Bhatnagar et al. [27] present talent management strategies for employee retention in a developing country, i.e., India. The author finds that low factor loadings indicate low engagement scores at the beginning of the career. However, high factor loadings at intermediate stages of employment are indicative of high engagement levels. A key finding is that good engagement results in higher retention in the developing countries. The work in [46] utilizes a new Cultural Intelligence (CI) measure to empirically study the evidence on several key antecedents of CI across five countries. The measure is named as Business Cultural Intelligence Quotient (BCIQ). This or a similar measure can be adopted for employee retention.

Based on the abovementioned literature survey, the conceptual framework developed for the current study is demonstrated in Fig. 1. As evident from this literature review, a detailed study that identifies key employee retention factors and correlates them with each other using a computational technique for the developing countries is needed. This work aims to bridge this gap.

Fig. 1
figure 1

Conceptual framework of the proposed work

Full size image

3 Methodology

This section describes the methodology used to collect and analyze the data. Moreover, the section also describes the research design, area and population selected for the study, its sampling procedure/size, and the data collection procedure. This work presents a quantitative research that will answer questions asked from multiple organizations. The queries are related to factors such as work environment, work performance and motivation, organization commitment, and satisfaction, reward and recognition, income, supervisors support and bullying, and work harassment.

3.1 Research design

For the current research study, a quantitative research mechanism is carried out via questioner distribution to a targeted population. The responses were measured through the statistical instrument. Quantitative research is to be carried out for a huge number of population, and they are tested by mathematical and statistical instruments. On the contrary, qualitative research is not appropriate for this research study as qualitative research deals with data related to observation and a specific style. It does not statistically describe findings. The exploratory research answers the “why” and “how” questions, whereas descriptive research focuses on four Ws, namely “what,” “where,” “when,” and “who.” Therefore, the exploratory research methodology is also not applied here because of the close-ended nature of the questioner.

3.2 Theoretical framework

The concept of employee retention falls under the theoretical framework of leadership, motivation theory and practice. The theoretical framework of this research is specifically based on the work of Latham [47]. Latham’s theory not only provides a chronological history of motivation theory and practice, but also presents an “insider view” on leadership and motivation. He presents six distinct eras of how motivation theory and practice has evolved over the past 110 years. The first era, according to Latham, presents the birth of behavioral theory in management and motivation. Industrial and Organizational (I/O) psychologists in this era were not interested in studying inner motivations and considered money to be the primary motivator at the workplace. The second era is marked with the trend of measuring the impact of attitudes on work and employee motivation. This era placed emphasis on the decision-maker and revealed the importance of identifying variables in building theoretical frameworks. The first and second eras are deemed obsolete for the current research due to their unidimensional approach toward measuring employee motivation. However, both these eras are fed into the proposed work indirectly. In the third era, the focus turned toward assessing and forecasting factors that influence employee motivation. This era had the strongest impact on organizational practices in the developing countries. The fourth era introduced the notion of scientific theories and methods in leadership and motivation research. The present research is based on the leadership and motivation theories of the third and the fourth era due to their relevance in the developing countries. According to Latham, we are currently in the fifth era and this period is marked with putting the practitioner at the center and devising frameworks that proactively and holistically aid in taking well-informed decisions. However, the sixth era is the era of the future. Latham predicts that the future of leadership and motivation theory will take deeper roots in psychology and consider the emotions and beliefs of employees. This research aims to provide crucial lessons for practitioners in the fifth and sixth eras.

3.3 Geographical zones

This study is carried out in four major cities of Pakistan, namely Karachi, Lahore, Rawalpindi, and Islamabad. The choice of these sites is made based on their population and availability of larger number of public and private organizations. Karachi is one of the biggest business hubs and also has many other service-oriented companies. Lahore is one of the known cities of the Punjab province, the populationwise largest province of Pakistan, where people are struggling to be retained in their organization, and most of the research data were collected from this zone. Islamabad, which is the capital city of Pakistan, has many organizations, and data were also collected from here. Figure 2 shows an overview of the general research design.

Fig. 2
figure 2

Overall research design

Full size image

3.4 Population of the study

For this study, the target group was all categories of sectors where we could get a significant number of employees. This was done to analyze the factors which are generally applicable to all possible working classes instead of focusing on just any particular group. The organizations in which this study is carried out are large appliances venders, corporate sector, schools, universities, banking sector, government organizations, hotel industry, information technology companies, hospitals, professional engineers, and telecommunication sector.

3.5 Sampling size and data collection

The sample size is an illustration which tells about the targeted population in the research. To carry this research, a target of 1000 was set and 853 responses were received. However, to achieve more responses, the targeted population could have been increased. For the current study, enough samples were received, i.e., 85.3% turnout rate; therefore, the target was not further increased. Figure 3 lists an overall summary of the data collected. Both primary and secondary methods were used for data collection. It is important for the researchers to test the result of hypothesis, and it is also important to collect data through secondary methods to save time.

Fig. 3
figure 3

Summary of the collected data

Full size image

3.5.1 Primary data collection

Primary data collection is a method of collecting genuine data. Questioners are the primary data source in this research. These were developed based on existing theories on employee retention. The collected data help to analyze patterns through FIM technique and Statistical Package for the Social Sciences (SPSS). Questioners are the best way to gather data, and it is the most effective and efficient mechanism through which one can measure various factors. This study was conducted on many individuals in diverse organizations. Firstly, all forms were distributed in multiple organizations and within one-month the forms were returned. The data were recorded in a Microsoft excel sheet for further process. Moreover, data analysis was performed through FIM and SPSS.

3.5.2 Secondary data collection

Secondary data collection method was used for reviewing theories and literature from many sources such as research papers, articles, and thesis reports. These sources were used to relate the factors that influence employee retention and learn employee retention strategies.

3.6 Hypothesis

Properly formalized hypothesis enables to guide the research toward appropriate simulation and experiments in order to answer key research questions. For this study, seven initial hypotheses were formed. These are listed as follows:

  • H1: Better work environment will result in higher employee retention.

  • H2: Higher organizational commitment results in higher employee retention.

  • H3: Increase in reward and recognition system results in higher employee retention.

  • H4: Increase in the individual’s work performance results in increased employee retention.

  • H5: Higher support and supervision by managers result in higher employee retention.

  • H6: Increase in employee income results in increased employee retention.

  • H7: Higher rate of bullying and work harassment results in lower employee retention.

3.7 Research instrument

When large amount of data is needed for a study, a survey seems the most effective way to do the needful. The questionnaire for this study was designed using Google formsFootnote 1, and also a few instances were printed in the hard copy. The survey form was divided into two sections: the first section asked for the demographic information such as gender, age, experience (overall), experience (with the current organization), organization name, organization category, and monthly income range, whereas the second section asked for the factors affecting employee retention. Moreover, the second section was comprised of 54 questions and these questions were measured by a five-point Likert scale ranging from one to five, where 1 showed strongly agree, 2 showed agree, 3 indicated neutral, 4 showed disagree, and 5 showed strongly disagree. The questions contained in the survey are listed in “Appendix.”

A few constraints and problems were faced while conducting this study. Some companies refused to fill the survey because they thought that the survey was a bit lengthy and it will take their time. Few did not return the required number of forms requested from them. There were a very few people who did not understand English. For such individuals, questions translated into their local language were used.

4 Results and findings

This section presents the experiments conducted and their results. These experiments are mainly conducted using SPSS as a tool and FIM as a data mining technique. The demographic profile utilized here includes gender, age, overall experience, experience with the current employer, marital status, and income. The experiments are conducted mainly to answer the following questions.

  • Using computational techniques, which factors are crucial for retaining employees and what is the relationship between those factors across multiple sectors in developing countries?

  • Using computational techniques, what is the impact of motivation, recognition and reward, advancement and growth, commitment and satisfaction, work environment, individual’s performance, support and supervision by managers, employee income, bullying and work harassment on employee retention across multiple sectors in developing countries?

  • How do these factors improve the organization’s overall environment and increase the rate of retaining employees?

4.1 Demographic profile

Getting key information about the respondents is important before drawing conclusions about any finding. For the current study, 36% of the participants were female and 64% were male. For this study, age was categorized into five ranges: less than 20 years, between 20 and 30 years, between 30 and 40 years, between 40 and 50 years, and greater than 60 years. According to this categorization, the highest response was obtained from the 20–30-year bracket, whereas the second highest response was from 30–40 category. Based on experience, the highest number of responses came from those who had work experience of less than five years and the lowest number of responses was from individuals having work experience greater than 10 years. Among the respondents, 45.6% were single and 54.1% were married. The highest response rate, i.e., 17.5%, was from the individuals working in the higher education sector. From the salary perspective, maximum responses were from those having annual income between 4329 and 8658 USD and the lowest response rate was from those respondents who had an annual income greater than 16,500 USD.

4.2 Factor analysis

This study focuses on various factors such as working environment, organization commitment, reward and recognition, work performance, supervisor support, income, bullying, and work harassment for employee retention. Table 1 lists the mean, standard deviation, and significant difference between male and female respondents using one-way ANOVA for the employee retention factors. When both male and female were asked about the working environment in their organization, the mean for males was 1.75 and for females, this was 2.10. This indicates that male agrees on working environment to be important for employee retention, whereas females neither agree nor disagree. There was no significant difference between male and female considering organization commitment. Considering this factor, the mean for male participants is 2.52 and for female it is 2.55 indicating their disagreement. The reward and recognition factor has a significant difference. The male participants have a mean of 2.00, and females have a mean of 2.34 which lies in the agreeing range. The factor work performance has a significant difference where the mean value for males is 2.45 indicating their agreement and females have a mean of 2.90 that shows they neither agree nor disagree. When questions related to the supervisor’s administration were asked, the results indicate no significant difference between males and females. For the factor of income, there is a significant difference observed in the male and female groups, where the mean for men is 2.44 and for women it is 3.21, indicating their disagreement.

Table 1 Difference between male and female respondents
Full size table

Table 2 lists the results when considering all factors and grouping these by age. The results show the highest mean for bullying factor and work harassment, considering the age-group greater than 60. The work environment has the lowest mean, i.e., 1.41 for the age-group of 40–50, whereas the highest standard deviation of 1.672 is for the factor work performance considering participants having age less than 20 years. The lowest standard deviation is 0.840 for the factor work environment within the age-group of 40–50. A significant difference is observed for the factors of working environment, reward and recognition, supervisors support, and income within the various group of ages. As shown in Table 3, all single and married respondents have the highest mean and standard deviation in bullying and work harassment factors and the lowest mean and standard deviation in working environment factor.

Table 2 ANOVA results considering all factors and grouped by age
Full size table
Table 3 ANOVA results considering all factors and grouped by marital status
Full size table

Table 4 lists the results grouped professionwise. The factor work environment here got the highest mean in the domain of medicine, and the lowest mean is obtained for the individuals working in telecommunication sector. The highest standard deviation is for hotel industry, while the lowest standard deviation is for the vender category. The organizational commitment has the highest mean value in the education sector (schools) and the lowest mean in the banking sector. However, the highest standard deviation is noted in government employees and the lowest standard deviation is observed in employees of the professional engineering companies. The reward/recognition being the third factor has the highest mean and standard deviation in hotel industry, and that has the lowest mean and standard deviation in telecommunication sector employees. The factor work performance has the highest mean in medicine sector and professional engineering, while it has the lowest mean in the field of education sector (schools). The highest standard deviation of work performance is observed in the hotel industry, and the lowest standard deviation is in the professional engineering sector employees. The highest mean and standard deviation of supervisor support are also for the hotel industry, and the lowest mean and standard deviation are that of the telecommunication department. The highest mean of income is observed in the telecommunication department, whereas the lowest mean is observed for the individuals working in the hotel industry. The income factor has the highest standard deviation in medicine domain and the lowest standard deviation in the government employee. The highest mean of the factor bullying and work harassment is observed in the education sector (schools), and the lowest mean is observed for the hotel industry. There is no indication of a significant difference in factors except for bullying and work harassment.

Table 4 ANOVA results grouped by organization
Full size table

Table 5 lists the results grouped salarywise. The work environment factor has the highest mean between those respondents who earn more than 27,166 USD annually and the lowest mean for those respondents who earn between 8767 USD and 16,235 USD. The exploratory factor analysis is used here to uncover the underlying patterns. By applying EFA on two categories of experience, the results show that these factors can further be divided into three groups. Table 6 summarizes the results of this.

Table 5 ANOVA results grouped by income
Full size table
Table 6 EFA results
Full size table
Table 7 Correlation between employee retention and other factors
Full size table
Table 8 Regression analysis
Full size table
Table 9 FIM results
Full size table
Table 10 Case processing summary and reliability statistics
Full size table
Table 11 Quantitative comparison
Full size table

4.3 Hypothesis

To test the hypothesis formed in Sect. 1, a correlation between various employee retention factors is computed (see Table 7) and the regression analysis is performed (see Table 8). Based on these, the formed hypothesis is either accepted or rejected.

4.3.1 H1: Better work environment will result in higher employee retention

The findings for hypothesis H1 indicate that the working environment is positively correlated with employee retention, which means a better working environment in an organization results in higher employee retention. The p value is less than 0.05 which means that there is a significant relationship between working environment and employee retention. The working environment`s B value is 0.294, which means that this factor has 29.4% of an impact on employee retention. The t value also shows that it has high impact on employee retention. Based on these results, H1 is accepted.

4.3.2 H2: Higher organizational commitment results in higher employee retention

The results in Tables 7 and 8 indicate that organizational commitment is slightly correlated with employee retention. The p value for this suggests that there is no significant relationship between organizational commitment and employee retention. The organizational commitment`s B value is 0.034, which means that this factor has 3.4% influence on employee retention. The t value also shows that it has low impact on employee retention. Therefore, H2 is rejected.

4.3.3 H3: Increase in reward and recognition system results in higher employee retention

The findings in Tables 7 and 8 for hypothesis H3 indicate that reward and recognition is positively correlated with employee retention. The p value for this is less than 0.05, which means that there is a significant relationship between reward/recognition and employee retention. For this factor, B value is 0.330, which means that this factor has 33% impact on employee retention. The t value also shows that it has a significant impact on employee retention. Therefore, this results in accepting H3.

4.3.4 H4: Increase in the individual’s work performance results in increased employee retention

The findings for hypothesis H4 indicate that work performance is positively correlated with employee retention. The p value is less than 0.05, which means that there is a significant relationship between work performance and employee retention. The work performance's B value is 0.311, which means that this factor has 31% of an impact on employee retention. The t value also shows that it has significant impact on employee retention. Based on these figures, H4 is accepted.

Fig. 4
figure 4

Accepted or rejected hypothesis

Full size image

4.3.5 H5: Higher support and supervision by managers result in higher employee retention

The findings for hypothesis H5 in Tables 7 and 8 indicate that supervisor support is positively correlated with employee retention. The p value is less than 0.05, which means that there is a significant relationship between supervisor support and employee retention. The B value of supervisor support is 0.253, which means that this factor has 25.3% of an impact on employee retention. Therefore, H5 is accepted.

4.3.6 H6: Increase in employee income results in increased employee retention

The findings for hypothesis H6 indicate that income is positively correlated with employee retention. The p value of this factor is also less than 0.05, which means that there is a significant relationship between income and employee retention. The B value for income is 0.299, which means that this factor has 29.9% impact on employee retention. The t value also shows that it has high impact on employee retention. Based on these figures, H6 is accepted.

4.3.7 H7: Higher rate of bullying and work harassment results in lower employee retention

Finally, the findings for hypothesis H7 indicate that bullying and work harassment is slightly correlated with employee retention. The p value for this factor is not greater than 0.05, which means that there is no significant relationship between bullying and work harassment and employee retention. Therefore, H7 is rejected. Figure 4 pictorially represents the acceptance or rejection of the seven hypothesis.

4.4 Frequent items identification

The FIM technique [48] from the domain of data mining is utilized here to find factors that frequently occur together to influence employee retention. The FIM is used over transactional databases to find all those items that occur together above a certain frequency, known as the minimum support. In order to utilize FIM in this work, first all responses were converted in a database transaction format. Each row of the database represented all responses from a unique respondent. This formed a dataset with 853 records. Later, this dataset was partitioned into various categories to identify frequently occurring job retention factors for a specific group. This categorization was done for the following attributes: gender, marital status, overall experience, job description (organization), and income. Table 9 lists the results of this experiment. There are a number of algorithms available to extract the frequent items from a dataset. The output of all these algorithms is the same. However, they consume different amounts of execution time. From an application point of view, it does not matter which FIM algorithm is utilized as long as the dataset size is not extremely large. This work utilizes the AIMFootnote 2 (Another Itemset Miner) implementation of the FIM technique to extract patterns. For the sake of completeness, Fig. 5 shows a comparison of time consumed by AIM against another FIM algorithm, i.e., kDCI (k Direct Count and Intersect) for various minimum support (minSup) values. The figure indicates that kDCI is quicker that AIM in finding the frequent itemsets.

Fig. 5
figure 5

Performance comparison between kDCI and AIM in finding frequent items

Full size image

4.5 Reliability test

A reliability test was conducted before any other test to make sure that the data are reliable. The Cronbach’s Alpha test was performed for the reliability of the data, and results indicated that the collected data were 87% reliable. Table 10 lists the results of this. The value of Cronbach’s alpha ranges between 0 and 1. A Cronbach’s alpha value greater than 0.6 is considered reliable. As shown in Table 10, the value obtained for the collected data is 0.874 indicating the reliability of the collected data.

4.6 Comparison

This section presents a qualitative comparison between the present work and past contributions regarding the identification of factors that influence employee retention. The comparison is based on seven factors, i.e., has the work considered multiple sectors, are the data mining methods utilized, is there the use of computational methods in drawing conclusions, what is the sample size, what is the geographic location of the study, is the study employee centric or organizational centric, and does the survey contain open-ended questions. The choice of comparison methods is made here due to their closeness to the task at hand and recency. Table 11 lists the quantitative comparison. It can be observed that the present work utilizes data mining methods and covers multiple domains as compared to the past works. Additionally, the majority of the past work utilizes the computational methods to gain insights about the employee retention. The table also indicates that the use of open-ended questions is rare while collecting the data. A few works have not mentioned their sample size; therefore, this field is left blank in the table.

5 Policy implications

Labor laws in many developing countries are at a nascent stage. Debates on employee rights, such as medical cover [49, 50], provisions of sabbaticals, data protection, diversity management, investing in human resource through training and development programs, etc., are still isolated practices only functional in a handful of multinational organizations in the developing world. This research provided an in-depth understanding of the impact of demographics on employee retention across multiple sectors, which will enable policymakers to (a) develop retention strategies in the backdrop of severe competition, (b) improve organizational long-term sustainability, (c) improve organizational brand name through providing better working conditions to employees, and (d) understand the dynamics of employee retention across multiple sectors and industries.

Increased global competition has inevitably led to a severe competition in talent acquisition and retention. Organizations, today, are not only competing for customers, but also for employees. Thus, losing a resourceful human talent can be devastating for an organization. If an organization is facing quick turnover, this can adversely affect its long-term sustainability. Talented employees are not only hard to find; they exist in clusters. Therefore, if an organization loses a dissatisfied employee, a bad word-of-mouth gets spread about the specific organization, which may then find it extremely hard to attract talented employees. Conversely, if an organization has a low employee turnover, the organization shall be able to contest and survive in highly competitive markets, ensure long-term sustainability, and celebrate a good brand name.

For policymakers, this research provides the basis to understand and re-evaluate the systems and practices of motivation; recognition and reward; and advancement and growth, by placing a strong emphasis on organizational justice. Policymakers shall be able improve their decision making through this research by considering numerous variables, which may impact employee behavior, specifically retention. This work enables policymakers to systematically diagnose and comprehend organizational structures and communicational channels in light of employees’ relationship and authority dynamics with the supervisor, thus redefining organizational esprit de corps in the developing world across multiple sectors. Through this research, policymakers shall be able to decipher the complexities of work conditions and highlight aspects which contribute to or pose a challenge to employee retention. Policymakers are interested in developing customized policies for clusters of employees who have similar ethical behavior and income level. This research dived deep into how ethical behavior and income level impact employee retention and how policymakers should distinguish between employees of varying ethical behaviors and income levels. Another policy implication of this research is that it shall enable policymakers to develop policies and practices which place emphasis on organizational commitment and satisfaction. The results revealed a strong relationship between organizational commitment and satisfaction, and employee retention. Policies and practices addressing organizational commitment and satisfaction shall not only ensure that talented employees are retained in the organization, but shall also attract new and budding talent more effectively and efficiently. Finally, bullying and work harassment has become a serious concern for several organizations in the developing countries. Gender discrimination discourages several women in the developing countries to either quit or switch their workplace. Bullying, harassment and gender discrimination are not only severely unethical, but also bring the organization in the limelight for the wrong reasons. Thus, this research provides policymakers with the insight and tools to develop proactive policies to discourage bullying and work harassment and encourage fair and equal treatment of all employees.

In a nutshell, at a microlevel, this research delivers policymakers with the right variables and tools to assess the state of employee retention in an organization. At the macrolevel, however, this research provides an in-depth analysis of trends and patterns of employee retention across multiple sectors. The research sheds light on how policymakers can encourage organizations to improve employee retention through training and development programs, medical cover, sabbatical, flexible working hours, etc. Through these techniques, policymakers can benchmark best practices for employee retention. Moreover, this work highlighted which sectors are severely suffering from low employee retention, thus allowing policymakers to target specific sectors/industries on a high-priority basis.

Like any other research, there were a few limitations of this study. The aspect of training and development was not considered in this work. Another limitation was that few respondents thought that survey forms were too lengthy and even some organizations rejected to fill out these. This study was only limited to the boundaries of Pakistan. The findings may be different if applied to a different country or may vary if considered different demographic variables.

6 Conclusion

Retaining skilled employees has always been a major concern for any organization across the globe. Organizations spend a significant amount on their training and development programs for this purpose. This work presented computational methods to identify factors for employee retention using their feedback collected through a questionnaire. The focus here was to identify factors to improve employee retention strategies based on the computational methods. A survey was conducted mainly within four sectors, namely health care, business, academics, and banking sector, to collect the data. The survey was divided into two parts: the first part included demographic information and the second part contained questions pertaining to employees’ job description and their satisfaction. The questions on the second portion were based on theories such as Herzberg's duality theory, expectancy theory, social cognitive theory, self-determination theory, social bonding theory, and sociocultural theory. The findings showed that the factors such as work environment, organization commitment, reward and recognition, work performance, supervisor support, income and bullying and work harassment have an impact on demographic profile. When these factors were correlated with employee retention, the statistical tests illustrated that, except organization commitment and bullying, all variables were identified to be strongly linked with employee retention. These factors have tended to have a power through which organization can improve the working environment and facilitate not only their client but also the employees.

From the extension point of view, there are many other factors that can be used for employee retention other than those utilized here. These may include training and development, medical cover, sabbatical and paid leaves, to name a few. This research can extend to multiple countries, and the effect of various cultures on the employee retention can be studied. The survey form can also have a few open-ended questions so that the investigation can better identify what an employee feels like when given an option to mention any factor of her choice.