ERP System Selection: Criteria Sensitivity

ERP System Selection: Criteria Sensitivity Frantisek Sudzina Copenhagen Business School, Denmark fs.caict@cbs.dk Andreja Pucihar University of Maribor, Slovenia andreja.pucihar@fov.uni-mb.si Gregor Lenart University of Maribor, Slovenia gregor.lenart@fov.uni-mb.si Abstract. Enterprise resource planning (ERP) system selection process is an important step in ERP adoption, since inadequately selected ERP may affect companies’ market share and implementation time, effort and cost. Choosing appropriate ERP selection criteria is not trivial. Literature suggests that very little had been written about ERP selection criteria in academic journals. But even more complicated is attributing actual weights to the criteria. The research was therefore aimed at perceived importance of and satisfaction with ERP selection criteria. Results of a questionnaire survey conducted in Danish, Slovak and Slovenian companies are presented in the article. 1 Introduction The enterprise resource planning (ERP) system is an integrated set of programs that provides support for core business processes, such as production, input and output logistics, finance and accounting, sales and marketing, and human resources. An ERP system helps different parts of an organization to share data, information and, hopefully, also knowledge, to reduce costs, and to improve management of business processes (Aladwani, 2001). Wier et al. (2007) argue that ERP systems aim to integrate business processes and ICT into a synchronized suite of procedures, applications and metrics which goes over firms’ boundaries. ERP systems used to be a domain of large companies but there is a still increasing number of small and midsized enterprises adopting adopt them as well. There are some reasons for this trend, including a saturation of the market, as most large organizations have already implemented an ERP system, increasing possibilities and need for the integration of systems between organizations and the availability of relatively inexpensive hardware (Gable and Stewart, 1999). An interesting research question arises – what ERP system selection criteria should be included in the decision making. Evaluation of information systems investments as such is rather old; researchers, such as Frielink (1961) and Joslin (1968), began working on evaluation of information systems already in 1960s. Any kind of evaluations implicitly requries to define dimension and to create metrics for the dimensions. But according to Keil and Tiwana (2006), very little had been written about ERP system selection criteria in academic journals. This literature study tends to support the statement. 2 Literature study ERP systems per se received a lot of attention in the last years; there are many ERP systems research instances and quite a lot of reviews, e.g. Esteves and Pastor (2001), Shehab, Sharp, Supramaniam, and Spedding (2004) and Botta-Genoulaz, Millet and Grabot (2005). Regarding the latter, it does not mention ERP system selection issue at all. The literature review is based on articles from journals covered in Web of Science. Probably the most relevant articles are ones, which topics include strings “enterprise resource planning” and “selection” or “selecting”. There are 38 articles, which fulfill the criteria, out of 548 articles on enterprise resource planning. Only 17 out of these 38 articles mention selection criteria. The remaining 21 articles discussed ERP system selection in general or as a part of implementation process or terms selection and ERP were not related. According to Keil and Tiwana (2006), very little had been written about packaged software selection criteria in academic journals. They conducted a literature search of both academic papers and practitioner articles. They looked into leading information systems and computer science journals that had specifically studied the criteria that managers use in evaluating packaged enterprise software systems. They used the ABI-INFORM database and searched for strings “enterprise systems” or “ERP systems”; and “selection criteria”. Although “systems” in “ERP systems” and “criteria” in “selection criteria” may seem to be too restrictive (e.g. criteria might be described as attributes, factors, and (independent) variables), even shortened key words would lead only to the same results. They focused their preliminary search on peer-reviewed academic articles in information systems, such as Information Systems Journal, MIS Quarterly, Journal of Management Information Systems, Information Systems Research, European Journal of Information Systems, Management Science, Decision Sciences, Communications of ACM, and Decision Support Systems, and in computer science, such as IEEE Transactions on Software Engineering, IEEE Transactions on Engineering Management, IEEE Software, IEEE Computer, ACM Transactions on Information Systems. They claim that they found only three scholarly publications on the subject (Chau, 1995; Montazemi, Cameron and Gupta, 1996; Bernroider and Koch, 2001). There were more practitioner oriented articles on the subject. Keil and Tiwana (2006) synthesized criteria found in these articles into seven: (1) cost, (2) reliability, (3) functionality, (4) ease of use, (5) ease of customization, (6) ease of implementation and (7) vendor reputation. Kumar, Maheshwari and Kumar (2002) and Kumar, Maheshwari and Kumar (2003) identified following criteria: (1) functionality of the system, (2) systems reliability, (3) fit with parent/allied organization systems, (4) available business best practices in the system, (5) cross module integration, (6) system using latest technology, (7) vendor reputation, (8) availability of regular upgrades, (9) compatibility with other systems, (10) vendor’s support/service infrastructure, (11) ease in customizing the system, (12) lower costs of ownership, (13) better fit with company’s business processes. Berchet and Habchi (2005) looked deeper into ERP systems implementation. The intention of the article was not to provide a list of selection criteria but they were mentioned implicitly in the section on selection of the vendor and software. The new system was supposed to help the company to (1) increase organizational efficiency, (2) achieve transparency and real-time management, (3) tighten internal control, and (4) enhance collaboration between departments. Wei, Chien and Wang (2005) identified six system software factors and three vendor factors, which they call attributes. System software criteria are: (1) total costs, (2) implementation time, (3) functionality, (4) user friendliness, (5) flexibility, (6) reliability, and vendor criteria are: (7) reputation, (8) technical capability, (9) service. Each attribute can be split into evaluation items. They use analytic hierarchy process to estimate weights for the criteria. Ayağ and Özdemir (2007) use a similar, maybe a bit more advanced, approach – analytic network process with fuzzy numbers. Their criteria are: (1) system cost (license fee, vendor support, maintenance cost, infrastructure cost), (2) vendor support (good reputation, consulting performance, R&D capability, technicalsupport capability, training performance), (3) flexibility (upgrade ability, ease of integration, easy of in-house development), (4) functionality (module completion, function fitness, security level), (5) reliability (stability, recovery ability), (6) ease of use (easy of operations, easy of learning), (7) technology advance (standardization, integration of legacy systems, easy to maintain). They provide also calculated weights for the criteria. Another fuzzy approach can be found in (Bueno and Salmeron, 2008). Bueno and Salmeron identified 27 criteria: (related to ERP systems) (1) possibility of applying industry solutions, (2) credibility of the system, (3) the capacity to integrate the ERP with the current IS/IT, (4) trust in the ERP system, (5) modularity, (6) adaptation of the ERP to the current system needs, (7) capability of the ERP system to offer information on time, (8) intuitiveness of the ERP system, (9) software costs, (10) consultation costs, (11) maintenance costs, (12) hardware requirements, (13) specialist team requirements, (14) high average implementation time, (15) parameter complexity, (16) project planning, (17) possibility of objectively defining the concepts, (and related to organization are) (18) employee continuing education, (19) average age of the personnel, (20) continuing education of the decision-making group, (21) suggestions/recommendations made by the users, (22) traditional organizational culture, (23) complexity of the organizational structure, (24) high performance, (25) number of employees/company size, (26) traditional organizational strategy, (27) complexity of organizational processes, which they then presented in a form of a fuzzy cognitive map Han (2004) describes more the process than concrete selection criteria. Han mentiones: (1) vendor, (2) functionality, (3) scalability, (4) technology as the important initial cut but it is up to the decision maker to define more specific criteria. Nah and Delgado (2006) concentrate on ERP implementation and upgrade. They perceive ERP selection as the first and most important steps in an ERP implementation. They stress organizational fit of ERP in order to minimize customization. Moreover, they empathize that selected ERP must meet the information and functional needs of the organization, and must support the organization's current and future processes business processes. Regarding the selection process, they suggest that all functional areas are involved in ERP selection, in order to have a comprehensive view of the enterprise's requirements for the ERP system. In addition to that, they propose that some key considerations are made whether the system will be centralized or decentralized and how compatible it will be with existing systems. In spite of promising title of (Verville and Halingten, 2003), the article did not offer a normative list of selection criteria. Suggested criteria can be divided into two groups – system and vendor evaluation criteria. Examples of system criteria include: (1) improvement over current systems, (2) customization, (3) user interfaces, (4) is the platform that the organization intends for the proposed solution to operate on ideal for optimum performance, (5) is the organization’s existing DBMS compatible with the proposed solution, (6) can the proposed solution integrate into the organization’s existing hardware architecture, (7) what is the architecture of the proposed solution: client/server, two-tier, three-tier, or other, (8) what is the capacity (minimum and maximum) of the proposed solution, (9) scalability of the system, (10) training (in-house or external to the organization; does vendor conduct the training or is outsourced?), (11) performance, (12) security features, (13) implementation. Examples of vendor criteria include: (14) ability to assist the organizations with the implementation, (15) association with or the availability of third party vendor/partners, (16) vision (future plans and trends regarding the direction of the technology and or strategic positioning), (17) financial strength, (18) market share (sales volume, size), (19) annual growth rate, (20) customer support, (21) product recognition, (22) range of products, (23) ability to meet future needs, (24) ability to provide references, (25) reputation, (26) vision and/or strategic positioning of the vendor, (27) longevity of the vendor, (28) qualifications, experience, and success in delivering solutions to organizations of a similar size, complexity, and geographic scope, (29) quality of the vendor’s proposal, (30) demonstrated understanding of requirements, constraints, and concerns, (31) implementation plan that properly positions the proposed solution to achieve the maximum level of business benefits, (32) implementation services, (33) implementation strategy, (34) support services. Umble, Haft and Umble (2003) mention seven criteria: (1) price, (2) supplier support, (3) ease of implementation, (4) closeness of fit to the company‘s business, (5) flexibility when the company’s business changes, (6) technological risk, (7) value (total implementation cost versus total value to the company). Rao (2000) identified five ERP selection criteria for Indian SMEs: (1) affordability, (2) domain knowledge suppliers, (3) local support, (4) technically upgradable, (5) uses latest technology. Fisher, Fisher, Kiang et al. (2004) used data envelopment analysis to evaluate mid-level ERP packages. They utilized an extensive study of the available features and functions and performance of various mid-level packages in these features and functions published by Jones (2002). Regarding the analysis, the following criteria were used: (1) service and support, (2) training, (3) scalability, (4) implementation flexibility, (5) integration, (6) manufacturing process, (7) core financials, (8) purchasing and sales, (9) human resource process, (10) international tax support, (11) average cost of packages, (12) support fees, (13) training fees, and (14) average implementation. Lall and Teyarachakul (2006) criticize Fisher, Fisher, Kiang et al. (2004) for being restricted to the mid-level ERP software, which was designed for midsized organizations, and for using only data provided by ERP vendors. Lall and Teyarachakul (2006) used data provided by the end-user company, for which the ERP selection process was conducted. The ERP selection process consisted of three stages: initial selection stage, candidate evaluation stage and final evaluation stage. The initial selection stage was a filtering phase where filtering criteria such as operating system, data base requirements, network architecture and implementation cost were used to develop a candidate list. In the second stage, the candidate list was evaluated using product/vendor ranking models and benchmarking to reduce it to a smaller set. In the final evaluation stage, a smaller six vendors were required to demonstrate how their ERP systems met the functionalities identified by the company. Data envelopment analysis was used at the final evaluation stage to select the most efficient system out of the six. They used four ERP selection criteria: (1) complexity of implementation, (2) estimated cost of implementation, (3) functional match and (4) vendor profile. Complexity of implementation is a measure of the internal effort with respect to the time and effort involved in the configuration, documentation, training and support functions of the ERP implementation. The estimated cost of implementation includes cash flow for acquisition of hardware, software and networking devices, end user training and documentation. Functional match is a measure of the strength and capability of the ERP system to meet the business requirements of the firm. Vendor profile is an attribute based on factors such as the financial strength of the ERP system vendor, regional presence of the vendor and prompt availability of software upgrades and technical support. Bernroider and Stix (2006) combined the merits of two concepts individually applied in decision making: the utility ranking method and the data envelopment analysis. Their requirements comprised the categories (1) controlling and reporting, (2) accounting, (3) logistics, (4) purchasing, (5) needs of local divisions, (6) services and engineering, (7) sales, and (8) business management. Several attributes were defined in each group for evaluation summing up to 73 criteria. The weights and utility values were defined by a decision committee with key users from all functional departments. Although Liao, Li and Lu (2007) provide 4 ERP selection criteria: (1) function and technology, (2) strategic fitness, (3) vendor’s ability; (4) vendor’s reputation; these are to be perceived rather as examples to be used in an analytic hierarchy process model. Yang, Wu and Tsai (2007) use 10 criteria: (1) adaptability of ERP system (employed system technologies, embedded database system, system development tool and language, compatibility with old systems, system efficiency, completeness of system documentation in Chinese and English), (2) service quality of consultants (expertise about ERP implementation, ability of project manager, implementation methodology and tool, experience on similar cases), (3) system education, (4) system acceptance by end users (hardware requirements, compatibility with old hardware, hardware upgrade capability, fitness of available modules, scalability, flexibility, usability, acceptance by middle-to-high level managers, working load for end users), (5) guarantee for implementation schedule (familiarity with client, implementation schedule planning, risk of over-budget, risk of out-of-scope), (6) maintenance and customization (maintenance capability, customization capability), (7) cost (ERP system authorization cost, maintenance cost, hardware cost, consultant fee, education fee), (8) customer service quality (planning of succeeding services, construction domain knowledge, education schedule arrangement), (9) technical support from ERP system vendor and other, (10) performance on service proposal and live demo, in their case study of ERP selection for a construction firm in Taiwan. Based on the analysis of the Web of Science articles, it is possible to sum up, that there are three general categories of criteria: • system-related criteria, • vendor-related criteria, and • end-user-related criteria. System-related criteria constitute the highest proportion of the criteria analyzed; vendor-related criteria are mentioned by majority of authors but their total number is much smaller than than number of system-related criteria; and only Bueno and Salmeron (2008) mention end-user-related criteria. The same criteria are often decribed by different terms. When accounted for synonyms in a wide sense of the word, it was possible to identify 15 systemrelated criteria, mentioned in at least 3 articles; they are summarized in Table 1. Criterion Price No. of articles 9 Ease/speed of implementation 8 Functionality 7 Customization/ parametrizatio n 7 Organizational 7 Articles Ayağ and Özdemir (2007); Bueno and Salmeron (2008); Fisher, Fisher, Kiang et al. (2004); Keil and Tiwana (2006); Lall and Teyarachakul (2006); Rao (2000); Umble, Haft and Umble (2003); Wei, Chien and Wang (2005); Yang, Wu and Tsai (2007) Bueno and Salmeron (2008); Fisher, Fisher, Kiang et al. (2004); Keil and Tiwana (2006); Lall and Teyarachakul (2006); Umble, Haft and Umble (2003); Verville and Halingten (2003); Wei, Chien and Wang (2005); Yang, Wu and Tsai (2007) Ayağ and Özdemir (2007); Keil and Tiwana (2006); Kumar, Maheshwari and Kumar (2002); Kumar, Maheshwari and Kumar (2003); Liao, Li and Lu (2007); Wei, Chien and Wang (2005) Berchet and Habchi (2005); Bueno and Salmeron (2008); Keil and Tiwana (2006); Kumar, Maheshwari and Kumar (2002); Kumar, Maheshwari and Kumar (2003); Verville and Halingten (2003); Yang, Wu and Tsai (2007) Bernroider and Stix (2006); Kumar, Maheshwari fit and Kumar (2002); Kumar, Maheshwari and Kumar (2003); Lall and Teyarachakul (2006); Liao, Li and Lu (2007); Nah and Delgado (2006); Umble, Haft and Umble (2003) Reliability Ayağ and Özdemir (2007); Bueno and Salmeron 6 (2008); Keil and Tiwana (2006); Kumar, Maheshwari and Kumar (2002); Kumar, Maheshwari and Kumar (2003); Wei, Chien and Wang (2005) Ease of use Ayağ and Özdemir (2007); Bueno and Salmeron 6 (2008); Keil and Tiwana (2006); Verville and Halingten (2003); Wei, Chien and Wang (2005); Yang, Wu and Tsai (2007) Integration Bueno and Salmeron (2008); Fisher, Fisher, Kiang 5 et al. (2004); Kumar, Maheshwari and Kumar (2002); Kumar, Maheshwari and Kumar (2003); Verville and Halingten (2003) Latest Ayağ and Özdemir (2007); Kumar, Maheshwari 4 technology and Kumar (2002); Kumar, Maheshwari and Kumar (2003); Rao (2000) Flexibility Ayağ and Özdemir (2007); Umble, Haft and 3 Umble (2003); Wei, Chien and Wang (2005) Scalability Fisher, Fisher, Kiang et al. (2004); Han (2004); 3 Verville and Halingten (2003) Training 3 Fisher, Fisher, Kiang et al. (2004); Verville and Halingten (2003); Yang, Wu and Tsai (2007) Upgrades Kumar, Maheshwari and Kumar (2002); Kumar, 3 Maheshwari and Kumar (2003); Rao (2000) Modularity 3 Bueno and Salmeron (2008); Kumar, Maheshwari and Kumar (2002); Kumar, Maheshwari and Kumar (2003) Information Berchet and Habchi (2005); Bueno and Salmeron 3 needs (2008); Nah and Delgado (2006) Table 1. System-related criteria in Web of Science articles. Regarding vendor-related criteria, there are two that appear in at least 3 articles; they are presented in Table 2. Criterion No. of articles Vendor support 10 Articles Ayağ and Özdemir (2007); Fisher, Fisher, Kiang et al. (2004); Kumar, Maheshwari and Kumar (2002); Kumar, Maheshwari and Kumar (2003); Liao, Li and Lu (2007); Rao (2000); Umble, Haft and Umble (2003); Verville and Halingten (2003); Wei, Chien and Wang (2005); Yang, Wu and Tsai (2007) Vendor Keil and Tiwana (2006); Kumar, Maheshwari and 7 reputation Kumar (2002); Kumar, Maheshwari and Kumar (2003); Lall and Teyarachakul (2006); Liao, Li and Lu (2007); Verville and Halingten (2003); Wei, Chien and Wang (2005) Table 2. Vendor-related criteria in Web of Science articles. Surprisingly, none of the analyzed articles contained any empirical research, which would give insight into how important are the criteria and/or how companies are satisfied with the criteria. 3 Data and Methodology The questionnaire research was conducted in May and June 2007. Questionnaire forms accompanied by cover letters were mailed to randomly selected companies in Denmark, Slovakia and Slovenia. Denmark and Slovakia have a similar number of inhabitants (mid-year population of Denmark was 5,468 mil. and 5,448 of Slovakia in 2007), there were only 2,009 inhabitants of Slovenia. Gross domestic product per hour in 2007 EKS$ was 44,46 in Denmark, 27,90 in Slovakia and 32,53 in Slovenia. According to The Networked Readiness Index 2006–2007 rankings (World Economic Forum, 2007), Denmark is the first with a score of 5,71, Slovakia 41st with a score of 4,15, and Slovenia 30th with a score of 4,41. Lists of addresses and information about the number of employees were retrieved from CD-Direct in Denmark and from respective Statistical Bureaus in Slovakia and Slovenia. In each country, 600 questionnaires were sent to small, 300 to medium enterprises, and 300 to large companies. The number of questionnaires mailed to small companies was double the number of medium and large companies because small companies constitute the highest proportion of companies and based on our personal experience, they are less likely to respond. In total, there were 223 responses (21 from Denmark, 112 from Slovakia, and 90 from Slovenia). Since none of the Web of Science articles mentioning ERP selection criteria contained any empirical research, our questionnaire survey was based on criteria identified by Bernroider, Koch (2001), who conducted an empirical survey in Austrian companies. Y2K readiness and EURO currency conversion were left out because they were not relevant anymore. The criteria used in the research are: • software costs (licenses, maintenance, etc.), • short implementation time, • functionality of the system, • organizational fit of system, • systems reliability, • system usability, • system interoperability, • advanced technology, • system flexibility, • integrated and better quality of information, • vendor support, • vendor reputation, • market position of vendor, • reduced cycle times, • improved service levels/quality, • increased organizational flexibility, • enhanced decision making, • incorporation of business best practices, • business process improvement, • e-business enablement, • increased customer satisfaction, • improved innovation capabilities, • enabler for desired business processes, • operating system independency, • internationality of software, • availability of an industry focused solution, • enabling technology for CRM, SCM, etc., • connectivity (intra/extranet, mobile comp., ...). Twelve of these criteria correspond to twelve out of 17 criteria enumarated in Tables 1 and 2. The advantage of using these criteria is in readers being able to compare results of this research and ones presented in Bernroider, Koch (2001). Importance of and satisfaction with, i.e. two dimension of sensitivity to, these dependent variables are measured on Likert scale 1-5, where 1 is of very little importance and 5 is of very high importance in the former case, and 1 means that expectations fell short and 5 that expectations were exceeded in the latter case. Independent variables are country, company size, turnover growth, representation of the IT department on the board level (CIO) and information strategy. The questionnaire research was conducted in Denmark, Slovakia and Slovenia. Analyzed are small, midsized and large companies, where companies from 10 to 49 employees are considered to be small enterprises, companies from 50 to 249 employees are considers to be midsized enterprises, and companies with 250+ employees are considered to be large enterprises. Turnover growth over the years 2004-2006 is divided into five categories: reduction in turnover, stable turnover, turnover growth of 0-5%, turnover growth of 5-10%, and turnover growth of more than 10%. Information strategy stands for formal information strategy and representation of the IT department on the board level means that there is a CIO or alike director for IT. All analyses are analysis of variance (ANOVA), a multivariate approach is used and results are commented on confidence level α = 0,05. Tukey-Kramer multiple-comparison test is used to identify between what instances of an independent variable there are significant differences. Standardized Cronbach’s alpha is used to measure consistency of chosen selection criteria. It is suggested that it should be at least 0,7. 4 Importance of Selection Criteria Standardized Cronbach’s alpha for importance of selection criteria is 0,903, what can be interpreted that importance of the criteria are correlated. Principal component analysis with varimax rotation was used with missing data substituted by multivariate normal distribution but each criterion was identified as a factor, so no suggestion how to decrease the number of factors could be drawn. According to ANOVA, evaluation of importance depends on country (pvalue < 0,001), company size (p-value = 0,002), turnover growth (pvalue < 0,001), information strategy (p-value < 0,001) and selection criterion (pvalue < 0,001). It is not possible to rule out significance of for representation of the IT department on the board level (CIO) because relate p-value is 0,055. According to Tukey-Kramer multiple-comparison test, there are significant differences in evaluation of importance of selection criteria between all countries (average of 3,54 in Denmark, 3,82 in Slovakia and 3,96 in Slovenia), between SMEs and large companies (3,71 in large companies, 3,79 in small companies and 3,81 in midsized companies), turnover growth (Table 3), companies without information strategy (3,72) and companies with information strategy (3,82) and differences between selection criteria are presented in Table 4. Companies without a CIO rated importance of selection criteria 3,74 on average, the ones with a CIO rated importance 3,80 on average. Turnover reduction reduction not sign. stable (0%) significant 0-5% not sign. 5-10% 10%+ significan significant t stable (0%) significant not sign. not sign. significan not sign. t 0-5% not sign. not sign. not sign. significan significant t 5-10% significant significant significant not sign. not sign. 10%+ significant not sign. significant not sign. not sign. Table 3. Difference of importance of ERP selection criteria by turnover growth. Criterion Importance Operating system independency 3,10 Enabling technology for CRM, SCM, etc. 3,15 Market position of vendor 3,18 Internationality of software 3,21 E-business enablement 3,28 Improved innovation capabilities 3,30 Vendor reputation 3,43 System interoperability 3,54 Reduced cycle times 3,55 Connectivity (intra/extranet, mobile comp., ...) 3,56 Availability of a industry focused solution 3,65 Increased organizational flexibility 3,67 Short implementation time 3,68 Enabler for desired business processes 3,69 Incorporation of business best practices 3,72 Advanced technology 3,73 Software costs (licenses, maintenance, etc.) 3,90 Increased customer satisfaction 3,92 Enhanced decision making 4,02 Improved service levels/quality 4,06 System flexibility 4,16 Vendor support 4,18 System usability 4,19 Business process improvement 4,20 Organizational fit of system 4,21 Functionality of the system 4,36 Integrated and better quality of information 4,40 Systems reliability 4,51 Table 4. Importance of ERP system selection criteria. Differences of 0,36 are statistically significant. There is no significant difference between perceived importances of systems reliability, integrated and better quality of information, functionality of the system, organizational fit of system, business process improvement, system usability, vendor support, and system flexibility. These eight criteria should therefore be considered as the most important without any ranking. Although software costs were identified by authors of nine out of 17 articles reviewed in the literature survey (using terms, such as cost, cost of ownership, total costs, system cost, software costs, price affordability, average cost of packages, estimated cost of implementation), there are eight significantly more important criteria than software costs. The second most often mentioned criterion was ease or speed of implementation. But in this survey, it ranked only as the 16th most important criterion. 5 Satisfaction with Selection Criteria Standardized Cronbach’s alpha for satisfaction with selection criteria is 0,903, what can be interpreted that importance of the criteria are correlated. Principal component analysis with varimax rotation was used with missing data substituted by multivariate normal distribution but each criterion was identified as a factor, so no suggestion how to decrease the number of factors could be drawn. According to ANOVA, evaluation of satisfaction with selected ERP systems depends on country, company size, turnover growth, information strategy, representation of the IT department on the board level (CIO) and satisfaction criterion (all p-values were smaller than 0,001). According to Tukey-Kramer multiple-comparison test, there are significant differences in evaluation of satisfaction selection criteria between Slovakia (3,00) on one hand and Slovenia (3,13) and Denmark (3,21) on other hand; between midsized (3,07) and large companies (3,08) compared to small companies (3,20); between companies which experience turnover growth of 0-5% (2,97) or reduction in turnover (2,97) on one hand and companies with stable turnover (3,18), turnover growth of 5-10% (3,21) an turnover growth of more than 10% (3,26) on other hand; between companies without information strategy (3,08) and companies with information strategy (3,15); between companies with a CIO (3,06) and without one (3,17); between all importance levels (2,62 for 1, 2,85 for 2, 3,11 for 3, 3,40 for 4, 3,59 for 5) and differences between selection criteria are presented in table 5. Criterion Short implementation time Improved innovation capabilities Incorporation of business best practices Software costs (licenses, maintenance, etc.) Increased organizational flexibility Enabling technology for CRM, SCM, etc. Increased customer satisfaction Connectivity (intra/extranet, mobile comp., ...) Enabler for desired business processes System flexibility Enhanced decision making E-business enablement Satisfaction 2,88 2,93 2,94 2,97 2,98 2,99 3,01 3,05 3,08 3,10 3,10 3,12 Reduced cycle times System interoperability Organizational fit of system Operating system independency Improved service levels/quality Functionality of the system Vendor support Availability of a industry focused solution Advanced technology Business process improvement System usability Internationality of software Systems reliability Integrated and better quality of information Table 5. Satisfaction with selected ERP systems. 3,13 3,13 3,14 3,15 3,15 3,16 3,18 3,18 3,21 3,22 3,22 3,23 3,35 3,39 There is a significant difference between fulfillment of expectations in systems reliability compared to short implementation time, improved innovation capabilities, incorporation of business best practices, software costs (licenses, maintenance, etc.), increased organizational flexibility, and enabling technology for CRM, SCM, etc.; and between fulfillment of expectations in integrated and better quality of information compared to short implementation time, improved innovation capabilities, incorporation of business best practices, software costs (licenses, maintenance, etc.), increased organizational flexibility, enabling technology for CRM, SCM, etc., increased customer satisfaction. Software costs are worth mentioning, since they scored as 23rd compared to Table 4 where they scored as 9th. But even more interesting is that out of all criteria, short implementation time seems to be the cause of the biggest dissatisfation. This probably explains why ease and speed of customization ranked so high (i.e. as the second) in Table 1. 6 Conclusion The literature study, which is based on 38 articles from journals covered in Web of Science, seems to support the statement of Keil and Tiwana (2006) that very little had been written about ERP system selection criteria in academic journals. It is rather surprising that very few articles actually offer (positivistic) weights/importance of individual criteria (obviously, weights are not to be expected in articles, which suggest to use data envelopment analysis) and none of them do it in a normative manner. Investigating companies from Denmark, an old member state of European Union, Slovakia, a new member state of of European Union, and Slovenia, a new member state of European Union but one of the most economically and technically advanced out of the new member state, should make the findings generalizable for companies in all member states of European Union. According to the questionnaire survey, the eight most important ERP selection criteria are systems reliability, integrated and better quality of information, functionality of the system, organizational fit of system, business process improvement, system usability, vendor support, and system flexibility. There is no significant difference between importances of these criteria. When it comes to satisfaction with purchased ERP systems, they exceeded expectations mostly in systems reliability and integrated and better quality of information, and fell short mostly in short implementation time, improved innovation capabilities, incorporation of business best practices, software costs (licenses, maintenance, etc.), increased organizational flexibility, and enabling technology for CRM, SCM, etc., increased customer satisfaction. To sum up, there is a strong positive relation between the importance of ERP selection criteria and how well are these expectations met by actually purchased ERP systems. References Aladwani, A. M. (2001). Change management strategies for successful ERP implementation. Business Process Management Journal, vol. 7, no. 3, 266-275. Ayağ, Z. and Özdemir, R. G. (2007). An intelligent approach to ERP software selection through fuzzy ANP. International Journal of Production Research, vol. 45, no. 10, 2169-2194. Berchet, C. and Habchi, G. (2005). The implementation and deployment of an ERP system: An industrial case study. Computers in Industry, vol. 56, no. 6, 588-605. Bernroider, E. and Koch, S. (2001). ERP selection process in midsize and large organizations. Business Process Management Journal, vol. 7, no. 3, 251-257. Bernroider, E. and Stix, V. (2006). Profile distance method- a multi-attribute decision making approach for information system investments. Decision Support Systems, vol. 42, no. 2, 988-998. Botta-Genoulaz, V., Millet, P. A. and Grabot, B. (2005). A survey on the recent research literature on ERP systems. Computers in Industry, vol. 56, no. 6, 510-522. Bueno, S. and Salmeron, J. L. (2008). Fuzzy modeling Enterprise Resource Planning tool selection. Computer Standards and Interfaces, vol. 30, no. 3, 137-147. Chau, P. Y. K. (1995). Factors used in the selection of packaged software in small businesses: views of owners and managers. Information and Management, vol. 29, no. 2, 71-78. Esteves, J. and Pastor, J. (2001). Enterprise Resource Planning Systems Research: An Annotated Bibliography. Communications of AIS, vol. 7, no. 8, 1-51. Fisher, D. M., Fisher, S. A., Kiang, M. Y. et al. (2004). Evaluating mid-level ERP software. Journal of Computer Information Systems, vol. 45, no. 1, 38-46. Frielink, A. B. (1961). Auditing automatic data processing, Elsevier, Amsterdam. Gable, G. and Stewart, G. (1999). SAP R/3 implementation issues for small to medium enterprises, in Proceedings of the Fifth America’s Conference on Information Systems, pp 779-781, Milwaukee, WI, USA. Han, S. W. (2004). ERP-Enterprise resource planning: A cost-based business case and implementation assessment. Human Factors and Ergonomics in Manufacturing, vol. 14, no. 3, 239-256. Jones, R. A. (2002). Spotlight on Midlevel ERP Software. Journal of Accountancy, vol. 193, no. 5, 24-47. Joslin, E. O. (1968). Computer selection. Addison-Wesley, London. Keil, M. and Tiwana, A. (2006). Relative importance of evaluation criteria for enterprise systems: a conjoint study. Information Systems Journal, vol. 16, no. 3, 237-262. Kumar, V., Maheshwari, B. and Kumar, U. (2002). Enterprise resource planning systems adoption process: a survey of Canadian organizations. International Journal of Production Research, vol. 40, no. 3, 509-523. Kumar, V., Maheshwari, B. and Kumar, U. (2003). An investigation of critical management issues in ERP implementation: emperical evidence from Canadian organizations. Technovation, vol. 23, no. 10, 793-807. Lall, V. and Teyarachakul, S. (2006). Enterprise Resource Planning (ERP) System selection: A Data Envelopment Analysis (DEA) approach. Journal of Computer Information Systems, vol. 47, no. 1, 123-127. Liao, X. W., Li, Y. and Lu, B. (2007). A model for selecting an ERP system based on linguistic information processing. Information Systems, vol. 32, no. 7, 1005-1017. Montazemi, A. R., Cameron, D. A. and Gupta, K. M. (1996). An empirical study of factors affecting software package selection. Journal of Management Information Systems, vol. 13, no. 1, 89-105. Nah, F. F. H. and Delgado, S. (2006). Critical success factors for enterprise resource planning implementation and upgrade. Journal of Computer Information Systems, vol. 46, special issue, 99-113. Rao, S. S. (2000). Enterprise resource planning: business needs and technologies. Industrial Management and Data Systems, vol. 100, no. 1-2, 81-88. Scott, J. A. and Kaindl, L. (2000). Enhancing functionality in an enterprise software package. Information and Management, vol. 37, no. 3, 111-122. Shehab, E. M., Sharp, M. W., Supramaniam, L. and Spedding, T. A. (2004). Enterprise resource planning: An integrative review. Business Process Management Journal, vol. 10, no. 4, 359-386. Umble, E. J., Haft, R. R. and Umble, M. M. (2003). Enterprise resource planning: Implementation procedures and critical success factors. European Journal of Operational Research, vol. 146, no. 2, 241-257. Verville, J. and Halingten, A. (2003). A six-stage model of the buying process for ERP software. Industrial Marketing Management, vol. 32, no. 7, 585-594. Wei, C. C., Chien, C. F. and Wang, M. J. J. (2005). An AHP-based approach to ERP system selection. International Journal of Production Economics, vol. 96, no. 1, 4762. Wier, B., Hunton, J. and HassabElnaby, H. R. (2007). Enterprise resource planning systems and non-financial performance incentives: The joint impact on corporate performance. International Journal of Accounting Information Systems, vol. 8, no. 3, 165-190. World Economic Forum (2007). The Global Information Technology Report 2006-2007, accessed on 10 June 2008, available at http://www.greaterzuricharea.ch/content/05/downloads/2007_nri_wef.pdf Yang, J. B., Wu, C. T. and Tsai, C. H. (2007). Selection of an ERP system for a construction firm in Taiwan: A case study. Automation in Construction, vol. 16, no. 6, 787-796.