Employ

Productive
Research
Approaches
Webinar Series: Be A More Productive Scholar
 Growth Marketing Manager

ROBERT P. ORMROD Lumivero

kgravesfloss@lumivero.com
KRISTIANA GRAVES FLOSS
Kenneth A. Kiewra is a professor of
educational psychology at the University of
Nebraska - Lincoln and studies teaching,
learning, and talent development. Recent
achievements include his listing among the
To p 2 % o f t h e M o s t C i t e d R e s e a r c h e r s
W o r l d w i d e , r e c e i v i n g h i s u n i v e r s i t y s y s t e m ’s
highest honor for teaching, publishing his
fifth book, SOAR to College Success and
Beyond (Cognella, 2022), and being featured
i n C o n t e m p o r a r y P i o n e e r s i n Te a c h i n g a n d
Learning. His forthcoming book is The
Productive Scholar: How the Experts Do It,
with Cambridge University Press. Kiewra has KENNETH A. KIEWRA, PhD
made more than 500 presentations to Professor of Educational Psychology
university and community groups. His website University of Nebraska- Lincoln
is: https://cehs.unl.edu/kiewra/
 Be A More
Productive Scholar Series

Employ Productive
Research Approaches
Citavi Webinar
October 19, 2023

Dr. Kenneth A. Kiewra

University of Nebraska,
Lincoln
Graduate Student Award Winners in Educational Psychology:
What Made Them Successful?
Uncover The Hidden
Curriculum: The Insider
Knowledge of How to Be
Successful In Academia

You might wonder, “Do I do

it this way or that way? I
guess I must try it this way.”
Then if it doesn’t work, you
must go back and do it the
other way. Rebecca Collie
 Advance Organizer
Collaborate on
Prioritize Research Collaborate
Multiple Projects

Be a Master Be a Systematizer Be an Extender

Formulate Good Design Feasible Handle the Review

Research Questions Studies Process
Prioritize Research
• I’m always thinking about my research. I
dream grants. I dream articles. I dream
talks. That’s essentially what’s always on
my mind. Jacquelynne Eccles
• I love what I do and become immersed in
it. Sabina Neugebauer
• I work all the time. But is that a bad thing? I
don’t think so. It’s energizing. It’s
wonderful. I love it. I love what I do. Doug
Lombardi
Prioritize Research
• You can’t be a jack of all trades. To be a successful
researcher, you must prioritize research over all else.
You must accept that you’ll be less good at other
things. I don’t ever want to be bad at anything,
especially teaching, but once I meet a threshold of
good enough, I accept that. Erika Patall
• I don't think John was the least bit interested in service.
He ducked committees as hard and fast as he possibly
could, because they took away from the important
scholarly work that he wanted to do. If he was on a
committee, he gave it minimal attention because that's
not what he was here to do. Barbara Plake on John
Glover
Prioritize
Research
Research prioritization
is also the pathway to
tenure and promotion
for faculty with research
apportionments.
Collaborate
Top scholars
collaborated on
88% of publications
in past 5 years with
average of 3.6
collaborators per
publication.
 Collaborate
Rebecca Collie Logan Fiorella Doug Lombardi Sabina Ming-Te Wang
Neugebauer
Collaborative 94% 98% 98% 90% 99%
Publications

• Throughout my career, every time I have done something important, it’s been with
somebody else. The breadth and depth of your thoughts and certainly your
personal fulfillment is going to be a lot greater if you spend your life doing
important things with other people. Michael Pressley

• I can’t speak highly enough of my collaborators. They’ve been fantastic in helping

me be productive. I learn so much from them. Rebecca Collie

• I choose my collaborators for two reasons. Number 1: I like them. Number 2: They
have ideas and skills that may well be better than mine. They are quite likely to
make me look good! David Berliner
 Collaborate
Ivar Braten is a poster scholar for collaboration.
• Among his 300-plus publications and conference presentations, only two were sole
authored.
• Braten has collaborated with 80 different scholars from 13 countries.
These collaborations are the sine qua non of my research, and I gain at least as much
as I give by them. We usually plan all aspects of the studies, including designing and
creating materials, in close collaboration [and] I’m nearly always taking part in data
collection. Typically, we decide who will be the main person responsible for writing a
particular paper or we will split up the writing, but sometimes I have also been
sitting together with doctoral students co-writing a manuscript from the first to the
last word, trying to externalize all my thinking and reflection during the writing
process. Ivar Braten
Marshall Student Collaborators
• If you’re going to be a productive
researcher, you need to get your new
people enculturated, get them motivated,
and get them to understand what they’re
supposed to do. It’s good for them; it keeps
up the productivity of the whole group and
the senior professor. Richard Anderson
• My student collaborators are really doing
the research in terms of data collection,
data analysis, and typically writing the draft
of the paper. Frank Fischer
 Collaborate On (And Stagger)
Multiple Projects
Heinz Mandl: 3-7 Ongoing Projects
Hans Gruber: 3-7 Ongoing Projects
Alexander Renkl: 10 Ongoing Projects
Frank Fischer: 15 Ongoing Projects
• We’re working on two projects where we are writing the proposal.
We have six to seven projects that are running and two more that we
are trying to get accepted for publication. Alexander Renkl
• …many projects going on in different phases within the lifecycle of a
project. Sabina Neugebauer
 Collaborate on (And Stagger) Multiple
Projects: Here’s Why
Boost Productivity. Many hands make light work. The more people working together on
more things, the more and faster things get done.

Spread Your Bets. Not every project is a winner. Not every article is Champions League
material. With multiple projects going, the failure of one is less traumatic or impactful.

Gap Filling. In the lifecycle of a project, there are gaps when not much can be done, such as when
waiting for IRB approval or waiting eight months while your manuscript is under review. These and
other gaps are easily filled by working on other projects in varying phases of completion.

Avoid Burn Out. Working for a long time on a single project can send you down a fiery
rabbit hole. It’s good to keep fresh by spreading your attention over various projects.
 Collaborate on (And Stagger) Multiple
Projects: Here’s How
• Don’t try to captain every ship. That requires too much effort, time, and
responsibility. Take the helm on some projects; add some power from the stern
on others.
• Mix in some conceptual projects. Work on both empirical studies and conceptual
pieces. Both are important. Conceptual pieces usually require fewer resources
and less work.
• Don’t leave things sitting too long as you move among projects. “When you put
something aside for too long, it’s really difficult to find the thread and pick back
up.” Ming-Te Wang
• Keep good records. Have a paper or electronic folder for each research project.
Each time you work on a project, note what was accomplished and what you
plan to do next. That will save a lot of “Where was I?” time wasting.
 Be A Master
Article A Article B Article C Article D Article E
Lecture:
Topic: Educ. Meas. Operant Learning Wildcats Trees Geometry
Length: 35 min 50 min 10 min 8 min 50 min

Rate: 120 wpm 150 wpm 100 wpm 100 wpm 175 wpm
Review: None
Placement: Following lecture Following lecture Before test Following lecture and
before test

Length: 15 min 5 min 5 min 20 min

Test:
Type: Fact recall & Fact and concept Free recall Fact recognition Problem solving
recognition

Placement: Immediate & delayed Immediate Immediate 2-day delay 1-week delay

Notes Recorded: 35% 25% 40% 45% 25%

Be A Systematizer

Cognitive Processes

Attention Organization Integration

Learner
Activities
Instructional
Methods
Be A
Systematizer
Be An Extender

Prose Illustrations Animation Multimedia Games

Be An Extender

Research
Program

Teaching and Talent

Learning Development

Graphic Parents' Productive Wisdom

Note Taking SOAR
Organizers Roles Scholars Years
Formulate Good
Research Questions
What’s your research question? My what?
• The most important thing in doing good
research is raising good research questions
that are personally interesting, have
educational and theoretical relevance, and
are feasible to carry out. Rich Mayer
• Students often try to answer the world. Their
questions are too big, too unmanageable.
Ask and answer manageable questions.
Patricia Alexander
• John was not interested in collecting
bunches of data, he was interested in
collecting specific pieces of data that were
going to answer highly specific questions.
Alice Corkill on John Glover
 Formulate Good Research Questions

• Poor: Are behavioral objectives effective?

• Good: What effect do behavioral objectives have on
learning objective-directed and nonobjective-directed
content?

• Poor: Does a graphic organizer aid learning?

• Good: Is a graphic organizer more effective than an
outline for increasing fact and relationship learning?
Some Good Research Questions
• Do students achieve more when they study notes in
paragraph, list, or matrix form?
• Does inserting organizational cues in lectures boost
note taking and achievement compared to not
inserting cues?
• Which note-taking medium, laptop or longhand,
results in more notes and higher achievement?
Design Feasible Studies
Feasibility is a big problem for most students. Sometimes they have a great
research question but their plan for answering the question is not feasible. So,
sometimes you might need to give up a little bit to do a study. Find a research
method that allows you to answer your question but is also as simple and as
straightforward as possible. A lot of students make the methodology more
complicated than it needs to be. They put in too many factors and explore
unnecessary side issues. Keep things simple and clean. Rich Mayer
• Design studies that focus on one clear question rather than many
questions.
• Design a simple first study that compares the most extreme versions of
just two groups.
• If results show an advantage, then explore more revealing research
designs.
Design Feasible Studies
• Do students achieve more when they study notes in
paragraph, list, or matrix form? Paragraph v. List v.
Matrix
• Does inserting organizational cues in lectures boost
note taking and achievement compared to not
inserting cues? Cues v. No cues
• Which note-taking medium, laptop or longhand, results
in more notes and higher achievement? Laptop v.
Longhand
 Handle The Review Process: Understand That
It’s Flawed
Reviewers Are Brutal
• Reviewers provide “12 pages of technically incompetent and hopelessly unsympathetic
criticisms from three mutually inconsistent reviews.”
• Advice to reviewers: “The human tendency toward criticizing others dwells so strongly in
our constitutions that we find it much easier to recognize flaws than to discover virtues.
Allow yourself the pleasure of saying something nice or paying a compliment occasionally.”
Editors Are Uninvolved Managers
“I did an independent review of your manuscript and agree with the reviewers…”
Really?
• How can you agree with erroneous statements reviewers made due to careless reading?
• How can you agree with irrelevant or incomprehensible ideas?
• How can you agree with reviewers who disagree?
Handle The Review Process:
Manuscript Rejected
Join the club: Rejection rate for top-tier journals is 90%.
• I’ve probably had more papers rejected from the
Journal of Educational Psychology than anyone in the
history of that journal. That’s one thing about this field,
you must learn how to take rejection and how to take
criticism. Rich Mayer
• All academics must get used to rejection, and young
scholars must realize that rejection is something all
scholars must live with and learn from no matter how
senior. Receiving feedback is sometimes painful, but
you just need to get over it. [Young scholars would
better understand that no scholars are immune from
rejection] if we all published a CV of rejections. Ming-
Te Wang
• You can always find a home for a paper in a second-tier
journal. Ming-Te Wang
 Manuscript Rejected: Frame Failure
• Abraham Lincoln failed in business, defeated for The Legislature, failed in business, and
defeated for Elector, Congress, Congress, Senate, Vice President, and Senate.
• Ty Cobb, greatest hitter in baseball history, had a career batting average of .367.
• F. Scott Fitzgerald once had more than 100 rejection letters pinned to his walls.

• Although rejection hurts, I try not to take it

personally. Reviewers have valid comments that can
help, so I try to apply their input and move on,
because the important thing is continuing to pursue
my research questions and to disseminate the
findings. I continue to believe in my acquired abilities
and tell myself to keep trying. Rich Mayer
• Learn to live with failure. Just don’t live with it too
long. Hal Fletcher
Handle The Review Process: Revise
and Resubmit

1. Celebrate
2. Foot in Door
3. Revise: Address Every Point
4. Craft Powerful Response Letter
Response Letter
1. Be polite and respectful.
Dear Sir, Madame, or Other:
Enclosed is our latest version of Ms. #1996-02-22-
RRRRR, that is the re-re-re-revised revision of our
paper. Choke on it. We have again rewritten the
entire manuscript from start to finish. We even
changed the g-d-running head! Hopefully, we have
suffered enough now to satisfy even you and the
bloodthirsty reviewers.
• Accept the reviewers’ criticisms gracefully and
behave as if every word of their comments
offered a pearl of wisdom… Cultivate a posture of
humility.
 • Act in good faith by trying to fix all
aspects of the paper with which
Response Letter reviewers have found fault. Forget
your righteous indignation over
what seems like the incompetence,
insensitivity, and mutual
2. Respond to all points raised. inconsistency of the reviewers’
One perplexing problem was dealing with suggestions comments.
13-28 by Reviewer B. As you may recall (that is, if you • I always spend time with the
even bother reading the reviews before sending your reviews and try to figure out what
decision letter), that reviewer listed 16 works that
he/she felt we should cite in this paper. These were on they are telling me. Every one of
a variety of different topics, none of which had any those points is always telling me
relevance to our work that we could see. The only something, and we try to react to
common thread was that all 16 were by the same that and respond to it. Michael
author, presumably someone whom Reviewer B greatly Pressley
admires and feels should be more widely cited. To
handle this, we have modified the Introduction and
added, after the Review of the Relevant Literature, a
subsection entitled “Review of Irrelevant Literature''
that discusses these articles.
Response
Letter
3. Accept the blame.
Authors should state their
hypotheses.
Although hypotheses appeared
in the Present Study section on p.
7-8, I see that they are easily
overlooked among the many
ideas in this section. The revised
manuscript now divides the
Present Study section into three
subsections, each with a
descriptive heading. Hypotheses
now appear under the p. 8
subheading “Hypotheses.”
Response Letter
4. Make each response complete and self
contained.

Reviewer 3, Comment 7: Provide a fuller

description of the lecture characteristics such
as lecture length and rate.

Response: Additional information about

lecture characteristics: length, rate, and idea
units appears on p. 9, lines 17-19 in the
Materials subsection of the revised
manuscript as follows: “The lecture contained
121 idea units, lasted 45 minutes, and was
presented at an average rate of 150 words per
minute.”
Response
Letter
5. Be thankful and positive.

Assuming you accept this paper, we would also like to

add a footnote acknowledging your help with this
manuscript and to point out that we liked the paper
We are thankful to you and to the
much better the way we originally submitted it, but you reviewers for the insightful and
held the editorial shotgun to our heads and forced us to helpful comments and
chop, reshuffle, hedge, expand, shorten, and in general suggestions that have made this a
convert a meaty paper into stir-fried vegetables. We much stronger manuscript.
could not—or would not—have done it without your
input.
Be a More Productive
Scholar: Employ
Productive Research
Approaches
Upcoming Events
Register Today
Lumivero Community: Continue the Discussion

Share and Learn

• Research Webinars
• Podcast: Between the Data

Connect
• Product User Groups
• Research Groups

Funding Opportunities Join Now

• Early Career Researcher Grant - 2024
The Data Landscape
The most complete collection of research, data
and decision-making tools available anywhere