Update on Virginias growth

measure
Deborah L. Jonas, Ph.D.
Executive Director for Research and Strategic Planning
Virginia Department of Education
April 2011

Federal Requirement: Student-level growth

data(b)(2)
As part of conditions of accepting money from the State Fiscal Stabilization Fund (SFSF),
Virginia is required to:
Develop a growth measure; and
Provide growth data, at minimum, to teachers of reading/language arts and mathematics in
tested grades.
Data must be provided to two groups of teachers:
Teachers who taught students previously:
Example: growth data from tests administered in spring 2011 must be provided to teachers who taught
students in the 2010-2011 school year.

Teachers who are currently teaching students

Growth data from tests administered in spring 2011 must be provided to teachers working with these
students in the fall of 2011.

The state must be capable of providing student-level data to teachers no later than September
30, 2011.

Federal Requirement: Teacher impact data,

indicator (b)(3)
As part of SFSF requirements Virginia is required to:
Provide teachers of reading/language arts and mathematics with
reports of individual teacher impact on student achievement on
state assessments.
Virginias approach
Use the student-level growth data to provide teachers with
context for understanding their impact on student achievement
as measured by statewide assessments.
As multiple years of data become available, VDOE will create
reports that enable teachers to consider trends in student growth
data.
When tests measuring the same SOL are available over multiple
years, trend data can be used in aggregate.

Student Growth Percentiles (SGPs)

SGPs provide a measure that quantifies the progress each
student made on Standards of Learning (SOL) tests compared
to other Virginia students statewide who had similar prior
achievement histories on previous SOL tests.
SGPs are calculated using a statistical methodology that
accounts for student prior achievement to determine each
students progress.
SGPs provide an understanding of how much progress
students made based on where they startedregardless of
whether they started as low, moderate, or high achieving
students.

How are SGPs quantified, and how can they

be interpreted?
SGPs range from 1 to 99, and represent the percent of
students who had similar prior academic achievement
(based on SOL tests) and who earned lower scores on the
SOL test.
Example: a student who earns an SGP of 65 scored
better than 65 percent of students who had the same
prior achievement as measured by SOL tests.
Example: a student who earns an SGP of 15 scored
better than 15 percent of students who had the same
prior achievement as measured by SOL tests.

Three students with the same scaled scores,

grade 6 reading

Same three studentsin grade 7

SGP = 18

SGP=48
SGP=61

Example only. Note that SGPs account for as much historical data as are available.

Growth can be measured for high and low

achieving students
Low achieving students can demonstrate high
growth.
High achieving students can demonstrate low
growth.

Growth Measure Availability

VDOE finalized statistical models for student growth
percentiles (SGPs) working with Dr. Damian Betebenner from
the Center for Assessment.
SGPs will be available for most students who participate in the
following SOL (not alternative) assessments:
Mathematics, grades 4-8 and EOC Algebra I through ninth
grade;
Reading, grades 4-8.

VDOE will provide student-level data by September 30, 2011;

aggregate data will be provided in the future.

10

Growth levels
To support interpretation of growth data, VDOE
will define categories/levels of student growth.
Growth level

Low

Student growth percentile

range
1-34

Moderate

35-65

High

66-99

Growth levels defined as of April 2011.

11

Some students will not have student growth

percentiles calculated
Some students who have taken the grades 4-8 reading tests and/or the
mathematics tests for grades 4-8 or Algebra I will not have growth
percentiles.
There are three primary groups who will not have student growth
percentiles available from the state:
Students with only one year of assessment data available.
Students who participated in Virginias alternative assessment programs (VGLA,
VSEP, VAAP) in the year of, or year prior to, the reporting year.
Students who participated in the traditional SOL assessment for two consecutive
years but for whom no valid growth measure is available.
This group consists primarily of students who achieve at the highest levels of
Virginias assessments; growth for these students cannot be adequately captured
with current statewide assessments.

As Virginias assessments change, VDOE anticipates that more students will

have student growth percentiles available in reading and mathematics.

12

Note about students earning high scores on

Virginias tests
Student growth percentiles, at their most basic level,
describe the likelihood of a student with a particular
score history earning a particular score on the next state
assessment.
If a student scores at or near 600 in one year, the SGP
for next year is dependent on the likelihood that
students who earn perfect scores earn them multiple
years in a row.
In Virginia, the likelihood of a student earning two or
more of the highest scores in a row varies by content
area and grade level.

13

Results of VDOEs SGP Analysis of SOL Scores

and SGPs
SPGs meet statistical and subject matter expert requirements for
accurately measuring relative growth on statewide assessments
for most students.
Student growth cannot be accurately determined for some
students who earn advanced proficient scores two or more years
in a row.
For some of these students, growth as calculated by the SGP is
clearly high, but how high is not known. These students will be
reported as having high growth but will not be assigned
an SGP.
For other students, the growth category cannot be determined
accurately with current assessments. These students will have
neither an SGP or a growth category reported.

14

VDOEs approach to reporting SGPs for high

achieving students
VDOE is establishing business rules for
determining which students will have growth
data reported.
Business rules may differ in mathematics and
reading.
VDOE is striving for similar structure in these
rules between content areas.

15

Example of business rules for SGP data

Two
consecutive
scaled scores
< 550

Two
consecutive
scaled scores
> 550 and
SGP > 65

Two
consecutive
scaled scores
> 550 and
SGP < 65

Quantitative SGP
(1-99)

yes

no

no

Descriptive
growth category
(low, moderate,
high)

yes

yes

no

Growth data to
be provided

16

Providing Growth Data to School Divisions

VDOE may provide student-level growth data before
September 2011 via SSWS. Data would be from test
years 2009-2010 and earlier.
If these data can be provided prior to September 2011,
they will not be connected to teachers and will not be
aggregated.
Data would be made available to central office
personnel, and VDOE would provide a webinar to
explain the data and answer questions as they arise.

17

How will teachers receive data?

VDOE will provide data to the school divisions central offices, via
the SSWS system (2011-2012 only; changes may take place in
future years).
To comply with SFSF, school divisions are responsible for
providing student-level data to teachers consistent with
requirements of indicator (b)(2):
Teachers who previously taught students
Teachers who currently teach students

Teacher impact data (b)(3) will be comprised of student level data

and, when available and appropriate, aggregate and trend data.

18

Training on Growth Percentile Data and its

Uses
VDOE is establishing an agreement with a team
comprised of statistical experts and professional
development providers to offer school division teams
training on growth percentile data use and
interpretation.
Plans are being developed to provide introductory
training opportunities in-person and via webinars.
More information about training opportunities will be
provided as it becomes available.

19

Potential Uses of Student Growth Measures

in Virginia

School improvement
Program evaluation
Communications with parents, teachers, staff
One component of comprehensive performance evaluation
Consistent with the Code of Virginia requirement to incorporate
measures of student academic progress in evaluations ( 22.1-295).
Information on appropriate use is included in draft guidance
documents to be considered by the Virginia Board of Education,
March 2011.

Federal accountability
To be explored after Virginia meets current federal requirements for
linking students to teachers and developing growth measure;
Cannot be implemented without two years of data from new tests.

20

Implications for Teacher Performance

Evaluation

21

Background: Code of Virginia

22.1-253.13:5 Standard 5. Quality of classroom
instruction and educational leadership.

B. Consistent with the finding that leadership is

essential for the advancement of public education in
the Commonwealth, teacher, administrator, and
superintendent evaluations shall be consistent with the
performance objectives included in the Guidelines for
Uniform Performance Standards and Evaluation
Criteria for Teachers, Administrators, and
Superintendents.

22

Background: Code of Virginia

22.1-295. Employment of teachers.

C. School boards shall develop a procedure for use by

division superintendents and principals in evaluating
instructional personnel that is appropriate to the tasks
performed and addresses, among other things,
student academic progress and the skills and
knowledge of instructional personnel, including, but
not limited to, instructional methodology, classroom
management, and subject matter knowledge.

23

Teacher Evaluation Measuring Progress in

Student Achievement
In March 2011, the Virginia Board of Education
accepted for first review revised Uniform Performance
Standards and Evaluation Criteria for Teachers.*
These documents recommend that student
academic progress account for a significant
component of the evaluation (recommendation is 40
percent; six other performance standards comprise the
remaining 60 percent).
Recommends that determinations of student academic
progress be assessed with multiple measures.
*see: http://www.doe.virginia.gov/boe/meetings/2011/03_mar/agenda_items/item_l.pdf

24

SGPs as One Measure of Progress in Teacher

Performance Evaluation
Growth percentiles may form the basis of one of
multiple measures of student progress when:
Data from at least 40 students are available; and
Students with growth data are representative of
students taught; and
Growth data for at least two years are available;
three years should be reviewed whenever possible.

Aggregate growth percentile data is not

recommended when these conditions are not met.

25

Use of SGPs in other states

26

How Can Teachers Use This Data?

Reflect on student needs quickly and efficiently
Is the student growing appropriately to meeting state standards ?
*
Is the student growing as much in reading as math?
Did the student grow as much this year as last year?

Spot trends in a students learning so they can encourage

more and/or faster growth or act to stop a downward drift.
Identify students scoring low but who have grown
dramatically in the past requiring different strategies than
students with low, flat scores.
*Predictive targets will not be available in Virginia until there are two years of data available from the new
mathematics and reading assessments.
Source: Presentation by Rebecca Gau, November 2010, to Virginia stakeholders. Full presentation is
available at: http://www.cit.org/programs/cit-connect/vdoe-gm.

27

28

How Can Leaders Use this data?

Teacher professional development
Plan resources for intervention and remediation

Source: Presentation by Rebecca Gau, November 2010, to Virginia stakeholders.

29

Do students at certain grade levels grow

faster than others?
Math

% of students
% of students with % of students with
with high
low growth
typical growth
growth

Whole School

23%

30%

47%

grade 3

31%

39%

31%

grade 4

26%

25%

49%

grade 5

11%

25%

64%

Source: Presentation by Rebecca Gau, November 2010, to Virginia stakeholders,

example from Arizona data.

30

Are the low scoring students growing as

much as the high scoring students?
% of students % of students % of students
with low
with typical
with high
3rd grade
growth
growth
growth
Falls Far
Below

55%

35%

10%

Approaches

41%

34%

24%

Meets

23%

46%

31%

Exceeds

0%

10%

90%

Source: Presentation by Rebecca Gau, November 2010, to Virginia stakeholders, example from
Arizona data.

31

Growth, achievement, and economically

disadvantaged status in Colorado
School achievement is correlated
with economic status (R=-.759)

Median growth percentile is not correlated

with economic status (R=-.124)

100

650

600

550

500

25

Medan Growth
Percentile
P
MedSG

Mean
Scale
Score
ore
caleSc
MeanS

75

50

25

0
50

75

100

ForR_Pctdisadvantaged
Percent economically

25

50

75

100

ForR_Pctdisadvantaged
Percent economically

Adapted from: Schmitz, D. (2010). Utility of Growth Percentiles. Presented November 2010, Virginia
stakeholders. Full presentation is available at: http://www.cit.org/programs/cit-connect/vdoe-gm.

32

Contact information
Assessment and questions about growth calculations: s
tudent_assessment@doe.virginia.gov
Teacher performance evaluations: licensure@doe.virginia.gov
Data/master schedule collection:
resultshelp@doe.virginia.gov