Tata Motors
Tata Motors
Tata Motors
Case Analysis: Eyes of Janus: Evaluating Learning & Development at Tata Motors
This case details a revamping effort of the L&D function at Tata Motors.
1) What is your assessment of the way Tata Motors Academy (TMA) was
built and the way the programs are being delivered? Is there a better
way of doing this? (30%)
Ans. 1
Organization Context
The possible way to assess the TMA and also the delivery of the programs which are being
delivered is to understand how TMA performed on the guiding principles while highlighting,
what they did on short term and also in the long term to ensure sustainability.
4. Training effectiveness
● L&D teams used five standard criteria (delivery quality, knowledge gain, application to
job, business impact and value) to measure effectiveness.
● Business impact data were the KPI linked to the desired outcomes of investing linked to
L&D.
5. L&D Investment
● Though the outcome of training investment could be seen at the organizational level, as
the change was felt by one and all.
● Senior management appeared to be more interested in quantifiable outcomes and metrics.
OD is a long term change effort that requires as compared to short term training, which has a
very different focus but also requires an evaluation component.
● It is mentioned that L&D team was in the initial stages of developing metrics to measure
the impact of training programs on business outcomes and the way the team was
approaching was linking participants' collective opinions of the value of the program and
its predictive impact on job performance. So it were the participants who were guiding
and being autonomous of the intended achievement of the outcome of the program
instead of the senior management whose say was not mentioned, how they wanted to be
accomplished in clear stated terms. If L&D and LAC had developed the right metrics to
capture what measurable business outcome, senior management wanted to achieve
throughout this whole change process, it would have given more intended result.
● Almost 50% of the training was done internally despite having other parties during the
whole process which help us to draw an assumption that the program would have
achieved a higher success rate if the mix of both internal and externally were linked to
overall program charter.
● Also organization appeared to be weaker on the feedback side. Despite the success of
‘Iteach’ and other programs, there does not seem to making a forward step to add new
material and also new behavioural interventions after achieving this impressive success
of the program.
● The gap between the investment made and investment needed in L&D for the effective
delivery of the TMA charter and to secure the additional investment during the tough
business environment faced by TML justify the lack of acknowledging the investment
which has already been made by the organization because of not linking the direct
performance outcomes which are important to deliver the real time growth which is
evident from the profit and loss statement. From March03(12) to March12(12), there is
an increase in total income and also there is a net increase in profit also but there was no
significant increase in the total profit post introduction of the program considering the
increasing cost of raw material and overall employee cost.
● The metric developed by Bector only evaluated the preliminary level of factors. The
organization needed an advanced level of metrics to evaluate training effectiveness at
individual level and group level. The performance dashboard would have been more
advanced by including other factors e.g. which functional/ technical/behavioural training
were covered most and which gave the direct result related to business outcome. Also
what were the training areas that needed the more significant interventions? For example:
Pantnagar’s consumption of annual budget was 96% but turn out rate for the functional/
technical training is 56%.
Though organization seemed to be at the forefront while devising the strategy and understanding
the importance of the program, the scope of improvement could be identified from the below
table also.
It is evident from the case, Tata Motors faced many ups and downs in its given tenure and
despite incurring losses during economic slowdown, the organisation managed and survived in
the automobile industry. Tata motors wanted to differentiate themselves from the automobile
industry. They setup the Tata Motors Academy to train their employees. They worked on their
capabilities and also with a close focus on customer centricity and innovation to sustain and
grow in the market, they setup the Tata Motors academy to train their employees and to make
them future ready.
Currently the Learning & Development teams use the following criteria to measure efficiency in
training delivery:
i.
The number of training programs delivered
ii. No of participants
There are centrally driven programs focused on global leadership management development and
other induction programs.
Learning & Development teams used five standard criteria to measure effectiveness:
i.
Delivery quality
They were also in the initial stages of developing metrics to measure impact training
programs on business outcomes; they were also incorporating the participant’s feedback (Exhibit
8) and internal metrics which was developed by Bector (Exhibit 10) as a practice of evaluation.
They struggled to develop metrics that could evaluate training effectiveness at the group and
individual level. As finally they need to prove it to senior management in terms of quantified
outcomes and metrics which are further than these. Their current practices seem very basic. They
could use the Kirkpatrick model to have an in depth evaluation. They seem to apply maybe just 1
part of the model … its opportunity apply (Exhibit 8).
Evaluation of a training event should be carefully planned and include conversations that
establish the time frames for evaluation, type of data needed to complete the evaluation, and
what participants will be included in the sample. Another critical consideration would be to
define the training outcomes and then consider what evaluation metrics will ascertain if those
objectives have been met.
Another useful frame- work for developing a training evaluation is the Philips ROI. It is best to
conduct a summative evaluation at the end of the program to assess the four levels of evaluation,
including Phillips’ fifth level, and return on investment (ROI) (Head & Sorensen, 2005;
Rothwell et al., 2010). ROI seeks to connect interventions with outcome metrics that makes
improvement clearly definable.
The complexity of the task or skill will also influence the type of evaluation used. In addition,
the evaluation of a training event will focus only on the training activities and does not consider
wider organizational change efforts or other variables that fall outside the time frame or episode.
Usually, a training evaluation falls into the short-term category and is focused on the one
intervention/event.
ROI calculation (Adapted from Phillips, J. J. (1997). Handbook of Training Evaluation and
Measurement Methods, 3rd ed. Houston, TX: Gulf Publishing Company.)
Overall there is a need at TMA to measure the effectiveness of TMA training programmes. This
is a daunting task due to the large number of courses being offered, the multi-site specific
training, and the dealer training. The challenge is to see what is working, what is contributing to
the increase in profits, and how best to utilize the limited HR resources. Also finding relevant
metrics to demonstrate and measure the impact of L&D at the bottom line of the company and
how TMA has impacted talent management of TML and related businesses is also a task for
TMA now. It is better to follow a model which is all inclusive and measures performance as well
as business impact which have been suggested above.
Ans. 3
Firstly let’s understand the various initiatives taken by Tata Motors Ltd. w.r.t Tata Motors
Academy which were a response to the emerging business dynamics, these were as follows:
1. New Human Capital Strategy – A renewed focus on developing and retaining talent by
TMA was created to take care of the learning needs of all employees and other stakeholders
(Contract employees, vendors & dealers).
· This was compromised as one of the guiding principles of TMA i.e. the
· Management skills
· Operator training
· Commercial training
· Dealer training
· Corporate training
· Behavioural training
They were further divided into sub-centres. Learning support services (LSS) dept. provided
common services to the COEs. COEs and LSS operated in a matrix structure. This could also
have been a potential pain area considering the complexity that was built into structure, function
and processes making it difficult to create a robust evaluation design.
3. Program designs
capabilities & designed & customized to local L&D needs by the local L&D teams.
· Management and Global programs were designed centrally for the entire
organisation.
· All behavioural training programs were aligned to the leadership and Tata Leadership
What seemed to work for programs designed and conducted by TMA was the evolution of
process of program design and revisions followed by TMA
· Its shift from conventional launch to a judicious mix of e learning, classroom learning
· The important feature of this new design a 4-6 day module on cross domain
orientation allowed participants to spend time with departments other than their own.
· The net effect of all these reforms – course dealt with issues of Individual specific,
Company specific and practice oriented and also providing understanding of other
departments.
4. Therefore, 5 features of TMAs internal reviews did bring success and broad based
acceptance at TML but the features lacked an external metric to get a holistic view of the
impact created by the programs & training. Therefore, was the vision of TMA truly
inclusive?
5. The training and variety of groups, content and regions involved made it massive and
complex operation. The task was to quantify the magnitude of the coverage required by
TMA with nearly 49,000 employees and 2000 dealers having thousands of salesmen. To
ensure uniform implementation of BU aligned learning programs for operation of large
magnitude while catering to local needs – involved 3 criteria of evaluation &
improvement:
· Efficiency
· Effectiveness
· Business outcomes
This was a weak evaluation design considering the scale they were operating on.
(Paradigm shift was created with the governance structure driven through Learning
Advisory Council i.e. training was to be driven by business rather than HR & L&D –
changing their focus to L&D team’s performance metrics and were being evolved.
The question was what would be the reference for new metrics)
(The use of 200 Iteach reward points weren’t the primary drivers rather was the
recognition accorded to the Iteach trainers as it was both an aspirational and
motivational factor for employees. Therefore, rewards system used weren’t relevant
and the question of retention of this tacit knowledge)
(Brought in efficiency data and yet the functional and technical programs with
60-70% of total offerings were conducted locally at BU level with LACs approval did
not take into account cross collaboration opportunity needs)
· Training effectiveness
(The learning impact calculator was still in the making thus the programs were
conducted in the absence of a robust evaluation design & metrics)
(Outcome and Impact seen and felt at organisational level but could not account for
group and individual level)
These guidelines did help in uncovering a number of gaps and shortcoming of TMA & the L&D
team. However, they could not ensure timely corrective measures being taken at TML.
Alternative ways to measure the effectiveness of Training program can be used from OD theory
of evaluating any change process.
1) Formative evaluation: In this part of evaluation, the implementation is in the middle of
intervention. This type of evaluation is a dipstick which is done to measure if the process going
are aligned with the required outcome. Data gathering and the feedback process for evaluation
can serve as a communication during the implementation to ensure that participants are
constantly engaged and learning are aligned with the Business outcome.
2) Summative evaluation: This type of evaluation is conducted immediately after training to
ensure that the training session offered values, merit, or worth on the levels of individuals, teams,
or organizations.
OD change evaluation theory also explains various advantages of taking an external evaluator or
an external evaluator.
Instructions: Place a check on the left column for all the areas accomplished to guide you in
determining the OD evaluation criteria (to be completed by evaluator).
Questions/Statements for Consideration
3. Have the stakeholders and/or the senior management given feedback about what
outcomes they expect from the Training evaluation?
4. Have the stakeholders and/or senior management given feedback how the results
of this evaluation will be used in the future?
6. Has a specific time period been identified for providing a summary of feedback
and results based on the required format established?
8. Are the criteria focused on the identified problem and these interventions, rather
than on the effectiveness of the overall Training & Development effort? (to ensure
validity and evaluation of different levels of the OD process)
9. Are the evaluation criteria in alignment with the Business outcome and goals?
○ Consider the low and high rigor options their focus entailed – Knowledge,
Behaviour & Attitude. However, did not clearly focus on their rigour w.r.t Impact
and Return.
○ The process variables and outcome variables w.r.t to Measurement of Training
Outcomes had developed metrics that focused and could evaluate outcomes of
training effectiveness at the organizational level and had undermined/ struggled to
develop metrics that could evaluate at the group and individual level. The strategy
alignment did focus on Primary/external metrics but struggled with Advanced/
internal metrics for measuring outcomes. Another area of concern was the
frequency of change in the existing performance metrics.
○ The strategy Alignment w.r.t strategy and organizational competence and capacity
in fragmentation phase considering the explanation of financial return on
investment. Due to lack of connect between strategy and performance in finite
and measurable terms to track the outcomes.
○ The data used for evaluation were mostly- professional data, soft data, energy
data, readiness and capability data; and lacked hard data, political data,
competency & competitor data.
○ Running too many programs at TMA led to an inability of evaluators to
demonstrate the benefit of participation of the programs run by them. A
possible reason could be the experience of the evaluators in driving such a
large scale change thus making it difficult to make evaluation decisions
and comprising the impact of an evaluators role.
Important Instructions:
When analysing and responding to the case questions please remember that you are
not simply being assessed on how well you understood the case. You are also graded
on how well you use and incorporate the course material discussed in class regarding
evaluation. A high-quality report will meaningfully relate the case situations with the
course material.
Please remember that this case was not developed exclusively for a course on Change
Evaluation. You will in some cases need to be creative to see how the concepts
discussed in class may apply.
This is a team assignment. There will be one submission per team. You will need to
submit your team’s report in a Dropbox created in Moodle.
-------------------------------------------------------------------------------------------------------------------------------
Tata Motors
This case details a revamping effort of the L&D function at Tata Motors.
1) What is your assessment of the way Tata Motors Academy (TMA) was built and
the way the programs are being delivered? Is there a better way of doing this? (30%)
Organization Context
centres.
● Enthusiastic in introducing locally innovative products to cater to the emerging
the human capital strategy. The university was established in 2011 in Mumbai.
● TMA was created to cater to the learning needs of all employees and other key
stakeholders such as dealers, vendors and contract employees.
● In TMA, 60-70% of the training was focused on developing the functional and
technical capabilities, both designed and customized by L&D to meet local needs
but the management and global programs designed centrally for the entire
organization.
● Guiding principle of TMA
1. Alignment of business strategy
2. Leadership involvement in TMA
3. Efficiency in training delivery
4. Training effectiveness
5. Appropriate L&D investment
6. Broad based learning opportunities
7. Competency of the L&D
8. Measurement of training outcomes
● Governance structure through a learning advisory council to design and create critical
training programs
● Emphasis on company specific and practice oriented issues
4. Training effectiveness
● L&D teams used five standard criteria (delivery quality, knowledge gain, application to
job, business impact and value) to measure effectiveness.
● Business impact data were the KPI linked to the desired outcomes of investing linked to
L&D.
5. L&D Investment
● Though the outcome of training investment could be seen at the organizational level, as
the change was felt by one and all.
● Senior management appeared to be more interested in quantifiable outcomes and metrics.
OD is a long term change effort that requires as compared to short term training, which has a
very different focus but also requires an evaluation component.
● It is mentioned that L&D team was in the initial stages of developing metrics to measure
the impact of training programs on business outcomes and the way the team was
approaching was linking participants' collective opinions of the value of the program and
its predictive impact on job performance. So it were the participants who were guiding
and being autonomous of the intended achievement of the outcome of the program
instead of the senior management whose say was not mentioned, how they wanted to be
accomplished in clear stated terms. If L&D and LAC had developed the right metrics to
capture what measurable business outcome, senior management wanted to achieve
throughout this whole change process, it would have given more intended result.
● Almost 50% of the training was done internally despite having other parties during the
whole process which help us to draw an assumption that the program would have
achieved a higher success rate if the mix of both internal and externally were linked to
overall program charter.
● Also organization appeared to be weaker on the feedback side. Despite the success of
‘iteach’ and other programs, there does not seem to making a forward step to add new
material and also new behavioral interventions after achieving this impressive success of
the program.
● The gap between the investment made and investment needed in L&D for the effective
delivery of the TMA charter and to secure the additional investment during the tough
business environment faced by TML justify the lack of acknowledging the investment
which has already been made by the organization because of not linking the direct
performance outcomes which are important to deliver the real time growth which is
evident from the profit and loss statement. From March03(12) to March12(12), there is
an increase in total income and also there is a net increase in profit also but there was no
significant increase in the total profit post introduction of the program considering the
increasing cost of raw material and overall employee cost.
● The metric developed by Bector only evaluated the preliminary level of factors. The
organization needed an advanced level of metrics to evaluate training effectiveness at
individual level and group level. The performance dashboard would have been more
advanced by including other factors e.g which functional/ technical/behavioral training
were covered most and which gave the direct result related to business outcome. Also
what were the training areas that needed the more significant interventions? For example:
Pant-nagar’s consumption of annual budget was 96% but turn out rate for the functional/
technical training is 56%.
· Though organization seemed to be at the forefront while devising the strategy and
understanding the importance of the program, the scope of improvement could be identified
from the below table also.
Organization power and political situations Opportunity costs (e.g., deferred decision)
It is evident from the case, Tata Motors faced many ups and downs in its given
tenure and despite incurring losses during economic slowdown, the organisation
managed and survived in the automobile industry.Tata motors wanted to differentiate
themselves from the automobile industry. They setup the Tata Motors Academy to train
their employees. They worked on their capabilities and also with a close focus on
customer centricity and innovation to sustain and grow in the market, they setup the
Tata Motors academy to train their employees and to make them future ready.
Currently the Learning & Development teams uses the following criteria to
measure efficiency in training delivery:
i. The number of training programs delivered
ii. No of participants
iii. Cost per participant etc.
There are centrally driven programs focused on global leadership management development
and other induction programs.
They were also in the initial stages of developing metrics to impact of training
programs on business outcomes, they were also incorporating the participants
feedback(Exhibit 8) and internal metrics which was developed by Bector (Exhibit 10)
as a practice of evaluation. They struggled to develop metrics that could evaluate
training effectiveness at the group and individual level. As finally they need to prove it to
senior management in terms of quantified outcomes and metrics which are further than
these. Their current practices seem very basic. They could use the Kirkpatrick model to
have an in depth evaluation. They seem to apply maybe just 1 part of the model … its
opportunity apply (Exhibit 8).
The complexity of the task or skill will also influence the type of evaluation used. In
addition, the evaluation of a training event will focus only on the training activities and
does not consider wider organizational change efforts or other variables that fall outside
the time frame or episode. Usually, a training evaluation falls into the short-term
category and is focused on the one intervention/event.
ROI calculation. (Adapted from Phillips, J. J. (1997). Handbook of Training Evaluation and
Measurement Methods, 3rd ed. Houston, TX: Gulf Publishing Company.)
3)How do you measure the effectiveness of the various initiatives taken by TMA?(35%)
Ans . 3
Firstly lets understand the various initiatives taken by Tata Motors Ltds. w.r.t Tata Motors
Academy which were as follows:
● TMA a corporate university to realise the Human Capital Strategy.
● Overarching objective was to transform TML into a world class destination for top
talent.Catered to the "learning" needs of employees, shareholders, and dealers.
● CHROs 3 criteria for
Alternative ways to measure the effectiveness of Training program can be used from OD theory
of evaluating any change process.
OD change evaluation theory also explains various advantages of taking an external evaluator
or an external evaluator.
Instructions: Place a check on the left column for all the areas accomplished to guide you in
determining the OD evaluation criteria (to be completed by evaluator).
Questions/Statements for Consideration
This is a team assignment. There will be one submission per team. You will need to submit your
team’s report in a Dropbox created in Moodle.