RESEARCH

UNDERSTANDING THE PERCEIVED VALUE OF

USING BIM FOR ENERGY SIMULATION

Anderson M. Lewis1, Rodolfo Valdes-Vasquez, Ph.D.2, and Caroline Clevenger, Ph.D.3

ABSTRACT

Downloaded from http://meridian.allenpress.com/jgb/article-pdf/14/1/79/1769796/1943-4618_14_1_79.pdf by Indonesia user on 22 October 2023

Building Information Modeling (BIM) and Energy Simulation have both become
increasingly important tools in the building sector. The potential exists to achieve a
synergistic relationship between BIM and energy simulation software programs by
leveraging information stored within BIM models to inform energy models to save
time and improve a design’s performance. This study seeks to understand the per-
ceptions of green design stakeholders with regard to the value of BIM-based energy
simulation as well as associated barriers and benefits. To do so, an e-survey was used
to collect data. Data analysis consisted of descriptive statistics, Cronbach’s Alpha
coefficients, Spearman correlations, and Mann-Whitney U tests. Results suggest that
little to no correlation exists between green design stakeholders’ perceptions regarding
the value of using information from BIMs to inform energy simulation, and their
engagement level with BIM and/or energy simulation. While this study has been
limited by its sample size and location of participants, the results help identify differ-
ent user groups’ perceptions and receptiveness to using BIM-based energy simulation
tools, which are advancing and transforming the AEC industry. Studies, such as this,
emphasize the need for further research on understanding the modeling processes
related to energy simulation and BIM models.

KEYWORDS
BIM, energy modeling, building performance, design stakeholder, perceptions

1. INTRODUCTION
An increased awareness of climate change, increasingly stringent building codes, and rising
energy costs are leading to a surge in global demand for better performing buildings, which
has invited designers to pay more attention to building performance. Applying sustainable
design principles can improve the overall performance of a building. However, difficulty exists
in knowing the exact implications of design changes on overall building performance or in
identifying interactive effects of such changes on individual building systems. In order to model

1. BIM Program & Project Technology Specialist, Procon Consulting, Arlington, VA 22201; email: alewis@proconconsulting.com
2. Associate Professor, Dept. of Construction Management, Colorado State University, Fort Collins, CO 80523 (corresponding author);
email: rvaldes@colostate.edu
3. Associate Professor & Assistant Director, Construction Engineering and Management, University of Colorado-Denver, Denver, CO
80204; email: caroline.clevenger@ucdenver.edu

Journal of Green Building 79

potential impacts of design changes on building performance, designers often rely on energy
simulations. Building Information Modeling (BIM) is a process that utilizes 3-D, parametric
modeling software capable of storing project information that can be updated and extracted by
various users throughout a building’s life cycle. BIMs serve as repositories for storing and updat-
ing data that can be extracted and analyzed, and therefore, can be used to effectively inform the
decision-making process during design (Azhar et al. 2009). As information repositories, BIMs
provide a significant opportunity to leverage such information and inform energy simulations.
Architectural, Engineering, and Construction (AEC) industries are quickly adopting BIM
as project management and design tools. A report from McGraw-Hill (2012) indicates BIM
adoption increased from 28% in 2007 to 71% in 2012. Multiple studies have identified the

Downloaded from http://meridian.allenpress.com/jgb/article-pdf/14/1/79/1769796/1943-4618_14_1_79.pdf by Indonesia user on 22 October 2023

benefits, trends, risks, challenges and perceptions of BIM related to such adoption (Azhar 2011,
Bryde et al. 2013, McGraw-Hill 2012). Likewise, green design stakeholders are increasingly
using energy simulation tools to inform or validate the decision-making process and to validate
previous design decisions on projects where building performance is of high importance. Sparse
literature exists that identifies green design stakeholders’ perceptions on using BIM to aid in
generating energy models and managing their simulations. In addition, less literature exists
that identifies the main benefits and barriers perceived by green design stakeholders regarding
using BIM models, which provide geometric relationships and material properties, to create
energy simulations.
The purpose of this research is to identify the main perceived barriers and benefits asso-
ciated with using BIM models for energy simulations. In addition, this study seeks to deter-
mine green design stakeholders’ overall perceptions of the value associated with using BIM for
energy simulations and to investigate how BIM and energy simulation engagement levels may
impact their perceptions of value. The results presented help readers understand how to better
implement BIM-based energy simulation while mitigating barriers and optimizing benefits.
Additionally, examining discrepancies between user groups can lead to the identification and
improvement of shortfalls in current BIM-based energy simulation processes. Finally, under-
standing how perceptions and engagement levels differ among different software user groups
may help in developing strategies for implementing BIM-based energy simulation tailored to
specific groups.

2. POINT OF DEPARTURE
Building Information Modeling (BIM) is a process that utilizes a 3-D, parametric design soft-
ware that is capable of storing project information that can be updated and extracted by various
users. BIM is capable of demonstrating the entire life cycle of a building virtually. Building
Information Models (BIMs) are the virtual representations of a project that result from the BIM
process, which can act as a communication platform between project stakeholders (Mowata
and Carter 2013). BIMs are capable of representing the geometry, spatial relationships, geo-
graphic information, quantities and properties of building elements, and can be used to create
cost estimates, material inventories, and project schedules (Azhar 2011). Unlike drafting tools
such as AutoCAD, BIM allows for drawings to be completed more efficiently since it utilizes
parametric change technology and can have robust information embedded within the model
(Azhar et al. 2009). Parametric change technology maintains model consistency by allowing
users to create a single model, that when updated, automatically reflects the changes made in all
applicable model views. BIM also improves documentation reliability by providing a platform

80 Volume 14, Number 1

for multiple stakeholders to access and update information. In addition, fixing a problem in
a computer model costs only a fraction of what it would cost to fix the mistake in the field
(Smith and Tardif 2012).
BIM adoption rates among the Architectural, Engineering, and Construction (AEC)
industry more than tripled between 2007 and 2012 because of the potential to increase pro-
ductivity during a project’s life cycle (McGraw-Hill 2012). For instance, owners can use BIM
to recognize project needs, design teams can analyze several project scopes, general contractors
can enhance coordination with vendors and suppliers, and facility managers can analyze the per-
formance of the facility during the operation and decommissioning phases (Bryde et al. 2013).
Additionally, indirect benefits, such as increased safety, enhanced quality, reduced schedules,

Downloaded from http://meridian.allenpress.com/jgb/article-pdf/14/1/79/1769796/1943-4618_14_1_79.pdf by Indonesia user on 22 October 2023

cost savings, lower labor costs, and waste reduction may be realized from BIM implementation
(Russell et al. 2014).
In general, the AEC industry holds a very positive perception of integrating BIM into
workflows and those that adopt BIM perceive it to have a very positive impact on their company
(McGraw-Hill 2008). Firms that track their Return on Investment (ROI) when using BIM on
projects, showed initial ROIs of 300 to 500%. Also, such firms indicated higher value for using
BIM on projects than those who did not track their ROI (McGraw-Hill 2008). In addition,
as users gain experience with BIM, their view of its impact improves significantly (McGraw-
Hill 2008).
Energy Simulation refers to the process of predicting a building’s energy performance
through software analysis, while an energy model refers to a computerized representation of a
building and its properties that are used to perform energy simulation calculations (ASHRAE
2005). Building energy simulation programs are capable of evaluating energy impacts of mul-
tiple designs across dynamic interrelated systems in a rapid manner, which make them a valu-
able decision-making tool for design and construction professionals who seek high levels of
building performance (Azhar et al. 2012). Iterative building performance analysis during the
design process leads to more comprehensive feedback on the performance implications of design
variations, which can lead to more energy efficient designs. Early design and preconstruction
stages of a project are the critical phases for making design decisions on that impact energy
performance (Azhar et al. 2012). Similar thinking is also emphasized by Attia and De Herde
(2011) who argue that 20% of the design decisions taken earlier influence 80% of all subse-
quently design decision.
Despite the fact energy simulation software has become increasingly user-friendly and
time-efficient to use, these software programs still require considerable time and expertise to
complete with high levels of accuracy. A large number of parameters needed to run an energy
simulation for a whole building and decision uncertainty early in the design process can yield
a vast, under-determined parameter space (Raftery et al. 2011). Assumptions are necessary for
under-determined parameter space. Inaccurate assumptions can lead to inaccuracies in building
performance simulations, which ultimately leads to unreliable energy simulation results to base
decisions on (Hong, Kim & Kwak 2011). In addition, buildings are often designed by multiple
stakeholders who are in charge of designing distinct subsystems. Ineffective communication
channels among stakeholders can result in incomplete information or delay in the transfer of
information needed to run an energy simulation.
Energy modeling tends to be an involved and error-prone process, as it requires identi-
fying and entering numerous inputs (Tang et al. 2010). However, leveraging geometries and
information embedded in BIM models can greatly hasten and simplify the energy simulation

Journal of Green Building 81

input process (Kim et al. 2013). Since BIM programs give users the ability to store and update
data about a building that can be periodically extracted and analyzed throughout the design
process, an ideal opportunity exists to leverage this information to inform energy models and
to ultimately improve the decision-making process (Azhar et al. 2009). The use of BIM-based
energy simulation tools can simplify the burdensome, arduous process of running simulations
(Azhar & Brown 2009) and can help users avoid the time-consuming, error-prone process
of re-entering information pertaining to building components, geometry, etc. (Stumpf et al.
2009). BIM software used in conjunction with energy simulation software allow for building
performance analysis of a structure to be performed more frequently. This iterative analysis of a
project throughout the design phase ensures that performance criteria are maximized (Azhar et

Downloaded from http://meridian.allenpress.com/jgb/article-pdf/14/1/79/1769796/1943-4618_14_1_79.pdf by Indonesia user on 22 October 2023

al. 2011). Since BIM-based energy simulation allows users to rapidly predict the performance
of different designs, it can help designers choose a building design that maximizes functional-
ity, cost and performance (Schade et al. 2011). For example, a study by Shoubi et al. (2015)
demonstrated that by modeling, then running simulations on, different building configurations
on a Malaysian based bungalow, they were able to achieve significant energy savings. BIM can
reduce costs associated with traditional energy analysis by making information needed for the
energy analysis process routinely available as a byproduct of the standard design process (Azhar
et al. 2009). In addition, existing information databases can help inform assumptions about
energy simulation in order to make them more accurately represent an actual building’s opera-
tion and potentially improve the accuracy of a BIM (GSA 2012).
A number of barriers currently prevent the universal adoption of BIM-based energy simu-
lation in the AEC industry. According to Azhar et al. (2010, p. 221) risks and challenges include:
“lack of interoperability between various BIM-based applications, the relative slowness of the
mechanical design community in adopting BIM, and lack of BIM-based analysis applications
certified by the California Energy Commission.” Additionally, project contract types and lack
of proficiency with BIM and/or energy simulation tools act as challenges to levering informa-
tion from BIMs to inform energy simulations to its fullest extent. Information that is needed
to compile a complete BIM model is often fragmented, being created by various stakeholders
throughout the lifecycle of a building (Mowata and Carter 2013). However, the integration of
this information is essential in producing accurate energy simulation results. Although BIM-
based energy simulation is far from perfect in predicting a building’s energy usage, its use is
growing in the AEC industries. Despite this growth, many construction companies still do not
perceive an immediate need to use BIM for environmental/ sustainability analysis, however,
they believe that it will become increasingly important in the near and far future (Ku and
Taiebat 2011).

3. RESEARCH APPROACH
3.1 Survey Instrument
An e-survey was developed based on past studies to determine respondents’ perceptions regard-
ing leveraging BIM for energy simulation while observing demographic data about respondents.
This survey allowed researchers to identify the factors that impact how green design stakehold-
ers’ engagement levels with BIM and/or energy simulation impacts their overall perceptions
of leveraging BIM for energy simulation. An extensive literature review and interviews with
design and construction professionals helped guide the development of the survey. During the

82 Volume 14, Number 1

development of the survey, open-ended feedback was provided from industry professionals and
academics during two rounds of pilot testing.
The e-survey, consisting of four sections, includes: Demographics, BIM Aptitude, Energy
Simulation Aptitude, and How BIM and Energy Simulation Work Together. The Demographics
section gauges firm type, position title, company size, company zip code, and the breakdown
of work type for each respondent. Demographic data is essential to provide meaningful context
for survey results due to the high degree of variability across companies and professional practice
throughout the AEC industry. Size ranges for this question were used directly from McGraw-
Hill (2012), which distinguishes company sizes by annual revenue and firm type. Table 1 shows
the categorization of firm sizes by firm type (construction related or design related) and net

Downloaded from http://meridian.allenpress.com/jgb/article-pdf/14/1/79/1769796/1943-4618_14_1_79.pdf by Indonesia user on 22 October 2023

revenue ranges that were used in the survey instrument.
The BIM Aptitude and Energy Simulation Aptitude sections in the survey were included
to better understand which distinct software products respondents used and to measure respon-
dents’ engagement levels. An index was adapted from a McGraw-Hill (2012) study to measure
respondent engagement with BIM and energy simulation. This study was selected as a baseline
since trade research often has a compelling impact on perceptions and is widely read, especially
as it relates to technology application in the AEC industry. The engagement index is out of
27 points, where 27 indicates very high engagement scores, 19–26 indicates high engagement
scores, 11–18 indicates a medium engagement score and 3–10 indicates a low engagement
score. This engagement index is comprised of three categories: user experience, user expertise
and firm implementation levels. These three categories are all self-reported by respondents.
Experience measures the number of years a respondent has been using BIM. Expertise indicates
the level each respondent selected as best representing his or her personal skills with BIM and
energy simulation respectively. Implementation measures the percentage of projects being done
in BIM and by the respondents’ firm. For example, if one respondent indicated that he or she
had 3 years of experience with BIM (3 points), had an advanced level of expertise with BIM
(6 points), and was very heavy on implementation (8 points), therefore this respondent had an
overall BIM engagement score of 17. Tables 2 and 3 show the breakdown of the engagement
index point structure as well as firm classification respondent classification level.

TABLE 1. Firm sizes by firm type and net revenue.

Firm Type
Design Related
Construction Related (Architects, Engineers, Energy
(Construction Managers, General Modelers, Energy Consultant,
Firm Size Contractors) Owner & Other)
Small firms Less than $25 million Less than $500,000
Small to medium firms $25 million to less than $100 million $500,000 to less than $5 million

Medium to large firms $100 million to less than $500 $5 million to less than $10 million
million
Large firms $500 million or more $10 million or more

Journal of Green Building 83

TABLE 2. BIM and Energy Simulation Engagement index point structure (Adapted from
McGraw-Hill 2012).

Experience Expertise Implementation

1 year 1 point Beginner 1 point Light (<15%) 1 point
2 years 2 points Moderate 3 points Moderate (15%–30%) 3 points
3 years 3 points Advanced 6 points Heavy (31%–60%) 5 points
4 years 4 points Expert 10 points Very heavy (Over 60%) 8 points

Downloaded from http://meridian.allenpress.com/jgb/article-pdf/14/1/79/1769796/1943-4618_14_1_79.pdf by Indonesia user on 22 October 2023

5 years 5 points
> 5 years 9 points

In addition, the BIM Aptitude and the Energy Simulation Aptitude sections determine
which software products are used by each respondent’s firm. BIM and energy simulation soft-
ware programs were identified through an exhaustive literature review and through web searches.
The Energy Simulation Aptitude section goes a step further to gauge respondents’ perceptions
of the accuracy of their energy simulation results at predicting actual building operation usage.
The last section, How BIM and Energy Simulation Work Together, determines if respon-
dents use BIM models to inform their energy simulation(s) and if so measures respondents’
overall perceptions of using BIMs to inform energy simulations using a seven-point Likert scale.
Additionally, this survey section investigates respondents’ overall perceptions of the benefits
and barriers associated with leveraging BIM for energy simulation by asking them to what
degree they agree/disagree with a series of statements that pertain to the benefits and barriers
associated with using BIM for energy simulation. At the end of this section, respondents were
provided with a text box, so they may qualitatively describe additional barriers that may have
unintentionally been excluded of the survey instrument.
The authors acknowledge that the survey instrument has some limitations. The survey ques-
tions do not capture the description of how the BIM and energy modelers actually used BIM
for simulation. For instance, the survey did not ask which specific aspects of a BIM model were
used for the energy simulations (geometry only, data only, or both) or if any partial data transfers
were experienced due to interoperability issues. Thus, we can only infer if the respondents were

TABLE 3. Engagement classification level (Adapted from McGraw-Hill 2012).

Tier of BIM and Energy simulation

engagement (Each-Level) Range of scores for each E-Level
Very High 27
High 19 to 26
Medium 11 to 18
Low 3 to 10

84 Volume 14, Number 1

thinking about the model or the data. In addition, the participants were not asked about which
phase(s) energy simulation was performed (conceptual , schematic, design development, etc.),
which might impact their perception of the BIM’s usefulness. Continued study of the actual
BIM-based energy simulation process represents an opportunity for further research.

3.2 Data Analysis Methods

Nominal, ordinal, and ratio data were collected from this survey instrument, leading the research-
ers to draw from a range of descriptive statistics to examine frequencies, mean, median, and stan-
dard deviation, which show dispersion of the results around the mean. Cronbach’s Alpha tests
were run to determine the reliability of both the BIM and energy simulation engagement score

Downloaded from http://meridian.allenpress.com/jgb/article-pdf/14/1/79/1769796/1943-4618_14_1_79.pdf by Indonesia user on 22 October 2023

indexes. These two indexes have three categories (experience, expertise, and implementation as
shown in Table 2) and the Cronbach’s Alpha test allows researchers to measure the strength of
that consistency among items in these categories. In addition, bivariate Spearman correlations
were performed so that the relationship between energy simulation/BIM engagement scores and
respondents’ perceptions of the value associated with using BIMs to inform simulation could
be observed. The Spearman’s correlation test was used because of the nature of the ordinal scales
that were used to create the engagement indices. Furthermore, Mann–Whitney U (MWU) tests
were run to determine the significance levels in the differences in mean values between differ-
ent user groups overall perceptions of the value associated with using information from BIMs
to inform energy simulation and their perceptions of how accurate the energy simulation is at
predicting an actual building’s performance.

4. RESULTS
4.1 Sample
The sample population consists of green design stakeholders located in the U.S. that use BIM
and/or energy simulation software as a part of their job. For the purpose of this paper a green
design stakeholder is a person who holds an interest in the design of a project that is slated to
achieve greater levels of energy efficiency, produce less carbon and/or minimize environmental
impact more than an average building. The sample started as a convenience sample that was
then allowed to “snowball” to the contacts of those in the initial convenience sample. The initial
convenience sample (consisting of approximately 210) was gathered through a variety of chan-
nels including: academics, ENR’s Top Green Contractors list from 2011 and 2013, a contact at
BIMforum.org and a local sustainability consulting firm. Additionally, respondents were asked
to forward this survey along to any of their contacts who met the green design stakeholder
criteria. Including forwarded surveys, the number of respondents who received the survey is
estimated to be approximately 270.
A total of 85 responses were generated. However, a total of 34 respondent results were
removed from analysis because they were either insufficiently complete or because respondents
did not meet the criteria of green design stakeholders. The final survey sample size analyzed is
51. Based on a sample size of 51, which came from an estimated sample population of 270, the
response rate is estimated at approximately 19%. The breakdown of respondent firm types was
comprised of 35% engineering firms, 27% architectural firms, 20% general contracting firms,
8% other, 6% energy consulting firms and 4% construction management firms. As a whole,
respondents indicated that 43% of their work was comprised of institutional projects, 39%
commercial, 10% residential, 7% industrial and 1% other. Although respondents were located

Journal of Green Building 85

TABLE 4. Respondents’ firm size breakdown by type.

Firm Type
Firm Size Construction Related Design Related
Small firms 1 4
Small to medium firms 0 18
Medium to large firms 2 13
Large firms 9 4

Downloaded from http://meridian.allenpress.com/jgb/article-pdf/14/1/79/1769796/1943-4618_14_1_79.pdf by Indonesia user on 22 October 2023

all over the U.S., the majority were located in Colorado (n = 38), which is a limitation of this
study and prevents generalization of the results. The breakdown of firm sizes is outlined in Table
4, showing more of the respondents are professionals from design related firms.
In terms of their professions, respondents held a wide range of roles such as mechani-
cal engineers, project managers, preconstruction, design manager, electrical engineers, and
architects. However, the most prevalent responses were from those engaged in the architecture
profession. The breakdown of BIM respondent experience was: Less than or equal to 1 year (n =
4, 8%), Greater than 1 year to less than or equal to 2 years (n = 4, 8%), Greater than 2 years to
less than or equal to 3 years (n = 10, 19%), Greater than 3 years to less than or equal to 4 years
(n = 10, 19%), Greater than 4 years to less than or equal to 5 years (n = 6, 11%), Greater than 5
years (n = 19, 36%). In addition, survey respondents had mixed backgrounds with the software
programs that they use. Autodesk Revit and Autodesk Navisworks were the two most widely
used BIM software programs by respondents’ firms, whereas eQuest and Trane TRACE were
the two most widely used energy simulation software programs by respondents’ firms. Although
respondents who did not use either BIM or energy simulation programs were omitted from
the data analysis, the remainder of respondents engaged with BIM, energy simulation or both
software programs to some degree. BIM-only comprised the largest segment of respondents,
with 24 of 51 (or 47% of respondents). BIM and energy simulation users comprised the second
highest group of users with 22 of 51 (43% of respondents) using it, while energy simulation-
only users comprised the smallest group of people with only 5 of 51 (or 10% of respondents).

4.2 Analyses
A Cronbach’s Alpha test was run on both the BIM and energy simulation engagement indices
(broken down in Table 2) to determine how closely related the scale’s items (experience, exper-
tise, and implementation) were effectively determining the reliability of each scale. Both scales
were determined to be reliable, with a 0.638 and 0.765 Chronbach’s Alpha value for the BIM
and energy simulation engagement indexes, respectively. Engagement scores are measured on a
3–27 point scale (shown in Table 3). Of the 46 respondents that use BIM, engagement scores
varied widely with a mean score of 15.04 and a Standard Deviation (SD) of 6.13. Of all BIM
users, eleven fell into the low engagement level group, twenty-four medium, seven high and four
were very high. Similarly, of the 27 respondents that used energy simulation, the mean engage-
ment score was 15.85 with an SD of 7.66. Of all energy simulation users, eight fell into the low
engagement level group, seven medium, nine high and three were very high.

86 Volume 14, Number 1

 When asked the question “How accurate do you perceive energy simulation in predicting a
building’s actual operating energy usage?” on a seven-point Likert scale (one being most negative,
four being neutral, and seven being most positive), the average of all responses for this question
(n = 50) was approaching somewhat accurate (with a mean of 4.66 and SD of 1.12). However,
when looked at in distinct user groups (BIM-only, BIM and energy simulation, and energy
simulation-only) different trends began to emerge. For example, BIM-only users perceived
energy simulation accuracy at predicting actual performance to be the highest with a mean
response of 4.87 (SD of 1.014) while BIM and energy simulation users had a mean response of
4.45 (SD of 1.057). However, a Mann–Whitney U test of these means did not show a statisti-
cal level of significance (p 0.267), indicating that there was no significant difference between

Downloaded from http://meridian.allenpress.com/jgb/article-pdf/14/1/79/1769796/1943-4618_14_1_79.pdf by Indonesia user on 22 October 2023

these two user group averages. Despite the lack of statistical significance, this difference in
means begins to suggest a trend that BIM-only users might be more optimistic in capabilities
of energy simulation programs.
When asked their level of agreement with the statement “there is significant room for
improvement in the process by which stakeholders provide me with information pertaining to the
creation of an Energy Model,” respondents (n = 51) were in the range of slightly agree to agree with
an average score of 5.37 for the response group. Similar to the previous question, BIM users
had the highest agreement level with this statement with a mean value of 5.59 (SD of 0.959)
while BIM and energy simulation users and energy simulation-only users had mean values of
5.33 (SD of 1.167) and 4.6 (SD of 1.949), respectively. Even though the discrepancy between
BIM-only users and energy simulation is large, no meaningful results can be drawn because the
sample of energy simulation-only users is too small (n = 5).
Additionally, when asked to indicate their overall Perception of the Value (PoV) associated
with using information from BIMs to inform energy simulation (on a seven-point Likert scale
where 1 is a very low value, 4 is neither a high nor low value and 7 is a very high value), the mean
score of the respondent group (n = 51) was 4.39 (SD of 1.662). Again, when broken down
into distinct user groups, BIM-only users had the highest perception of the value associated with
using information from BIMs to inform energy simulation with a mean of 4.88 (SD of 1.484),
energy simulation-only users (n = 5) had a mean value of 3 (SD of 1.871) and BIM and energy
simulation users had a mean of 4.18 (SD of 1.651). This also shows that survey respondents
who were BIM-only users appear to have the most positive perceptions of the value associated
with BIM-based energy simulation.
Initially, the researchers hypothesized that a positive correlation would exist between both
respondents’ BIM and energy simulation engagement scores and their PoV associated with
using information from BIMs to inform energy simulation. After running Spearman correlation
tests between the variables, the researchers found a weak negative correlation (r = –0.3034)
existed between energy simulation engagement and PoV, and a weak positive correlation (r =
0.1498) existed between BIM engagement scores and PoV. By normal standards, the associa-
tions between these two variables are not considered statistically significant (p 0.1698 and p
0.3205, respectively). However, when broken down into distinct user groups, the BIM-only
user group (n = 24) had a positive correlation (r = 0.4377) and showed statistical significance
(p 0.0324) between BIM engagement and PoV. This positive correlation implies that BIM-only
users are more likely to have a higher overall perception of the value associated with using BIMs
to inform energy simulation as their engagement levels with BIM increase. This positive cor-
relation may also be because BIM-only users become more confident in their ability to produce
complete and accurate models that contain all parameters necessary for energy simulations. As

Journal of Green Building 87

TABLE 5. Breakdown of user group Spearman’s correlation between BIM/energy simulation
engagement and PoV.

Energy Simulation
Engagement Score BIM Engagement Score
Perception of value BIM & energy –0.3034 (p 0.1698) 0.1498 (p 0.3205)
(PoV) simulation (n = 22)
BIM-only (n = 24) — 0.4377 (p 0.0324)*

*Correlation is significant at the .05 level

Downloaded from http://meridian.allenpress.com/jgb/article-pdf/14/1/79/1769796/1943-4618_14_1_79.pdf by Indonesia user on 22 October 2023

a summary, Table 5 shows the breakdown of different user group correlations between BIM/
energy simulation engagement and PoV.

4.3 Perceptions of Benefits and Barriers

Lastly, the researchers investigated green design stakeholders’ perception of the benefits and
barriers with using BIM-based energy simulation. All respondents identified facilitates com-
munication, reduced process-related costs, and ability to examine more design options as the main
benefits of BIM-based energy simulation. Table 6 shows the breakdown of user group perception
of benefits items related to BIM-based energy simulation.
The energy simulation-only user group was purposely omitted from Table 6 due to their
small sample size of 5. Significance difference at p < .10 was found in two of the benefit items,
indicating discrepancies of perceptions between these user groups. Interestingly, BIM-only
had much higher levels of agreement that time savings is a benefit of BIM-based energy simula-
tion than did BIM and energy simulation users. This may be because BIM-only users are only
responsible for creating the BIM models, but BIM and energy simulation users have to ensure

TABLE 6. Breakdown of user group perception of Benefit items related to BIM-based energy
simulation.

BIM & energy Significance

simulation users BIM-only users (Mann–
(n = 22) (n = 24) Whitney U)
Benefit Items Mean Mean p
Facilitates communication 5.45 5.08 0.3125
Reduced Process-Related Costs 5.36 4.88 0.0891*
Ability to Examine More Design Options 4.91 5.29 0.6384
Time Savings 4.05 5.04 0.0854*
Increased Accuracy 4.00 4.75 0.1118
Technical Ease 3.64 4.08 0.5687

*Significant at the .10 level (at least 90 percent certainty)

88 Volume 14, Number 1

that all the correct information is embedded within the model before using it to inform an
energy simulation. On average, BIM and energy simulation users perceived that there is greater
potential for BIM-based energy simulations to reduce process-related costs than the BIM-only
users. This difference may reflect the fact that creating BIM models may be a more streamlined,
less iterative process than performing energy simulations. However, the iterate and exploratory
nature of performing energy simulation may be seen as an effective way of generating better
and more cost efficient design decisions.
Generally, all respondents identified Lack of BIM standards for model integration with multi-
disciplinary teams, Learning curve, and Software functionality as the main barriers of BIM-based
energy simulation. On average respondents agreed to some extent that all the items listed in

Downloaded from http://meridian.allenpress.com/jgb/article-pdf/14/1/79/1769796/1943-4618_14_1_79.pdf by Indonesia user on 22 October 2023

Table 7 (aside from “hardware costs”) acted as barriers to implementing BIM-based energy
simulation, the highest level of agreement was still in-between somewhat agree and agree. Table
7 also shows the breakdown of user group perception of barrier items related to BIM-based
energy simulation. Again, the energy simulation-only user group was purposely omitted from
Table 7 due to their small sample size of 5.
Compared to BIM and energy simulation users, BIM-only users have a much lower per-
ception of additional time needed to build a model, with mean values of 5.32 (above slightly
agree) and 4.5 (between neither agree nor disagree), respectively with a significance of p 0.0096.

TABLE 7. Breakdown of user group perception of Barrier items related to BIM-based energy
simulation.

BIM & energy BIM- Significance

simulation users only users (Mann–
(n = 22) (n = 24) Whitney U)
Benefit Items Mean Mean p
Lack of BIM standards for model integration 5.5 5.21 0.2891
with multi-disciplinary teams
Learning Curve 5.32 4.92 0.1141
Software Functionality 5.05 4.79 0.3125
Additional time needed to build the model 5.32 4.50 0.0096**
Interoperability issues 4.86 4.67 0.6455
Training Cost 4.95 4.63 0.2713
Lack of others’ capability to collaborate on a 4.91 4.75 0.7039
BIM model
Lack of management buy in 4.14 4.96 0.1527
Software cost 4.64 4.29 0.4473
Lack of motivation to change current processes 4.14 4.54 0.4654
Hardware cost 3.68 3.88 0.7114

**Significant at the .01 level (at least 99 percent certainty)

Journal of Green Building 89

This could be due to the fact that BIM-only users are not completely familiar with all of the
information that must be included in a BIM model to create a complete energy model.
Lack of BIM standards for model integration with multidisciplinary teams, learning curve and
software functionality were the only other barrier items that were at or above or at somewhat agree
for perceived barriers to using BIM for energy simulation, but no statistical significance was
found between groups. Despite the literature review mentioning interoperability as a barrier
when leveraging BIM for energy simulation, responses, on average, were below slightly agree
(4.82) that interoperability is a barrier to BIM-based energy simulation (Hitchcock & Wong,
2011).
Overall, respondents had the highest level of agreement with the statement “Integrating

Downloaded from http://meridian.allenpress.com/jgb/article-pdf/14/1/79/1769796/1943-4618_14_1_79.pdf by Indonesia user on 22 October 2023

BIM with energy simulation tools facilitates greater levels of communication among design stakehold-
ers” as a benefit associated with using BIM-based energy simulation. In addition, respondents
highly agreed that “reduced process-related costs” and “the ability to examine more design
options” were all benefits associated with using BIM for energy simulation.

5. CONCLUSIONS
The overall goal of this exploratory study was to determine what green design stakeholders
perceive as the main barriers and benefits to leveraging BIM for energy simulation and to deter-
mine how BIM and energy simulation engagement scores impact green design stakeholders’
overall perceptions of the value associated with using BIM for energy simulation. Correlating
green design stakeholder perceptions on the value associated with BIM-based energy simula-
tion and BIM and energy simulation engagement scores allows researchers to observe and test
if engagement with either (or both) tool is likely to increase their perceptions of BIM-based
energy simulation. Specifically, a positive correlation of 0.4377 was found between the overall
perception of the value associated with using BIMs to inform energy simulation and their
BIM engagement scores for BIM-only users. This correlation might indicate that as BIM-only
users become more familiar with using BIM they may perceive higher levels of value associated
with using information from BIMs to inform energy simulation, which may be because they
become more confident in their ability to construct better models. However, BIM-only users
may not know exactly what information energy modelers require from BIM models and may
be overly confident in their model’s usefulness for energy simulation purposes. Greater training
and education along with creating a feedback loop with downstream energy simulators may
help alleviate issues that arise from modeling over confidence.
Based on the responses, green design stakeholders’ overall perceptions of the value associ-
ated with using information from BIMs to inform energy simulation were between neither low
nor high value and somewhat high-value with a mean score of 4.39. Although not statistically
significant, when comparing distinct user groups within the respondent pool, BIM-only users
had the highest average perception of value associated with using BIM to inform energy simu-
lations (mean of 4.88), while those who only used energy simulation had the lowest (mean
of 3). BIM-only users also have the highest perceptions of an energy simulation program’s
ability to accurately predict building performance with a mean value of 4.87. This comparison
suggests that BIM-only users may have overly optimistic expectations of the capabilities of
energy simulation.
It was also found that different user groups’ perceptions of the greatest benefits and bar-
riers associated with using BIM-based energy simulation varied considerably for some items.

90 Volume 14, Number 1

Interestingly, BIM-only users had a much a more positive perception of the benefits of time-
savings (5.04) and technical ease (4.75) compared with BIM and energy simulation users at
4.05 and 3.64, respectively. This helps reinforce that BIM-only users may be overly optimistic
in their ability to create models that are usable for energy simulators or that energy simulators
may only have incomplete or inaccurate models to work with. Overconfidence on the behalf
of the BIM-only users may be due to their lack of experience running energy simulations
first hand or knowledge of the specific parameters required to run accurate energy simula-
tions. In the field of psychology, this aligns with a cognitive bias referred to as the Dunning-
Krueger Effect.
The results from this cross-sectional and exploratory study point out that a need for

Downloaded from http://meridian.allenpress.com/jgb/article-pdf/14/1/79/1769796/1943-4618_14_1_79.pdf by Indonesia user on 22 October 2023

additional research on this topic exists. The authors acknowledge that a small sample size is a
limitation of our study, and while the findings are meaningful, they may not be generalizable.
In future studies, the sample size should be larger and more evenly distributed throughout
the U.S. and should include more energy modelers so that more significant conclusions can
be drawn from their responses. Opportunities also exist to examine demographic data about
respondents (such as sex and age) and their working environment (such as type of company,
project type, contract type, and level of building performance targeted) in more detail to iden-
tify trends that develop from different demographic groups and determine if these factors are
correlated to their perceptions.
Although the Cronbach’s Alpha test determined the engagement indexes were reliable, new
engagement indices could be created to represent engagement levels. Further research on this
topic should investigate education level and training experience of the green design stakeholders
who use BIM for energy simulation. Additional research could focus on different strategies to get
distinct user groups to implement BIM for energy simulation. Lastly, examining in greater detail
how those who already use BIM for energy simulation actually utilize information stored in
BIM models to inform their energy simulations and the challenges associated with this process
could also help develop a richer context for developing a guiding framework for AEC firms to
conduct BIM-based energy simulations.

REFERENCES
ASHRAE (American Society of Heating, Refrigerating, and Air-Conditioning Engineers). (2005) “Handbook—
Fundamentals, Chapter 32—Energy estimating and modeling methods.” ASHRAE, Atlanta, GA.
Attia, S. and De Herde, A. (2011). “Early design simulation tools for net zero energy buildings: a comparison of
ten tools.” Paper presented at the Proceedings of Building Simulation.
Azhar, S. (2011). “Building Information Modeling (BIM): Trends, Benefits, Risks, and Challenges for the AEC
Industry.” Leadership and Management in Engineering, 11(3), 241–252.
Azhar, S., Brown, J., and Farooqui, R. (2009). “BIM-based sustainability analysis: An evaluation of building
performance analysis software.” Paper presented at the Proceedings of the 45th ASC Annual Conference.
Azhar, S., Carlton, W., Olsen, D., and Ahmad, I. (2011). “Building information modeling for sustainable design
and LEED rating analysis.” Automation in Construction, 20(2), 217–224.
Azhar, S., Khalfan, M., and Maqsood, T. (2012). “Building information modelling (BIM): now and beyond.”
Australasian Journal of Construction Economics and Building, 12(4), 15–28.
Bryde, D., Broquetas, M., and Volm, J. M. (2013). “The project benefits of Building Information Modeling
(BIM).” International Journal of Project Management, 31(7), 971–980.
GSA. (2012). GSA BIM guide for energy performance.
Hitchcock, R., and Wong, J. (2011). “Transforming ifc architectural view bims for energy simulation: 2011.”
Proceedings of Building Simulation 2011: 12th Conference of International Building Performance Simulation
Association, 1089–1095.

Journal of Green Building 91

Hong, T., Kim, H., and Kwak, T. (2011). “Energy-saving techniques for reducing CO2 emissions in elementary
schools.” Journal of Management in Engineering, 28(1), 39–50.
Kim, H., Anderson, K., Lee, S., and Hildreth, J. (2013). “Generating construction schedules through automatic
data extraction using open BIM (building information modeling) technology.” Automation in Construction,
35, 285–295.
Ku, K., and Taiebat, M. (2011). “BIM Experiences and Expectations: The Constructors’ Perspective.” International
Journal of Construction Education and Research, 7(3), 175–197. doi: 10.1080/15578771.2010.544155
McGraw-Hill Construction (2008). “Building information modeling: Transforming design and construction to
achieve greater industry productivity.” Smart Market Report. New York: McGraw-Hill. <http://www.aia.org/
aiaucmp/groups/aia/documents/pdf/aias077483.pdf> (Feb. 12, 2014).
McGraw-Hill Construction (2012). “The business value of BIM in North America: multi-year trend analysis and
user ratings (2007–2012): Smart Market Report.” New York: McGraw-Hill. <http://bimforum.org/wp-

Downloaded from http://meridian.allenpress.com/jgb/article-pdf/14/1/79/1769796/1943-4618_14_1_79.pdf by Indonesia user on 22 October 2023

content/uploads/2012/12/MHC-Business-Value-of-BIM-in-North-America-2007-2012-SMR.pdf> (Feb.
12, 2014).
Mowata, I., and Carter, K., (2013). Sustainable BIM-Based evaluation of buildings.” Procedia-Social and
Behavioral Sciences.
Raftery, P., Keane, M., and O’Donnell, J. (2011). “Calibrating whole building energy models: An evidence-based
methodology.” Energy and Buildings, 43(9), 2356–2364.
Russell, D., Cho, Y. and Cylwik, E. (2014). ”Learning Opportunities and Career Implications of Experience with
BIM/VDC.” Pract. Period. Struct. Des. Constr. 19, Special Issue: Construction Engineering: Leveraging
Project and Career Success, 111–121
Schade, J., Olofsson, T., and Schreyer, M. (2011). “Decision-making in a model-based design process. Construction
Management & Economics,” 29(4), 371–382.
Shoubi, M. V., Bagchi, A., and Barough, A. S. (2015). Reducing the operational energy demand in buildings
using building information modeling tools and sustainability approaches. Ain Shams Engineering Journal,
6(1), 41–55.
Smith, D. and Tardif, M. (2012). “Building information modeling: a strategic implementation guide for architects,
engineers, constructors, and real estate asset managers.” John Wiley & Sons.
Stumpf, A., Kim, H. and Jenicek, E. (2009) “Early design energy analysis using BIMs (building information
models).” Construction Research Congress.
Tang, P., Huber, D., Akinci, B., Lipman, R., and Lytle, A. (2010). “Automatic reconstruction of as-built build-
ing information models from laser-scanned point clouds: A review of related techniques.” Automation in
Construction, 19(7), 829–843.

92 Volume 14, Number 1