Brain-Training Pessimism, but Applied-Memory Optimism

Psychological Science in the

Public Interest

2016, Vol. 17(3) 187 –191

Reprints and permissions:

sagepub.com/journalsPermissions.nav

DOI: 10.1177/1529100616664716

pspi.sagepub.com

As is convincingly demonstrated in the target article

(Simons et al., 2016, this issue), despite the numerous

forms of brain training that have been tested and touted

in the past 15 years, there’s little to no evidence that cur-

rently existing programs produce lasting, meaningful

change in the performance of cognitive tasks that differ

from the trained tasks. As detailed by Simons et al.,

numerous methodological issues cloud the interpretation

of many studies claiming successful far transfer. These

limitations include small sample sizes, passive control

groups, single tests of outcomes, unblinded informant-

and self-report measures of functioning, and hypothesis-

inconsistent significant effects. (However, note that, with

older adults, a successful result of the intervention could

be to prevent decline in the training group, such that they

stay at their pretest level while the control group

declines.) These issues are separate from problems

related to publication bias, selective reporting of signifi-

cant and nonsignificant outcomes, use of unjustified

one-tailed t tests, and failure to explicitly note shared

data across publications. So, considering that the litera-

ture contains such potential false-positive publications

(Simmons, Nelson, & Simonsohn, 2011), it may be sur-

prising and disheartening to many that some descriptive

reviews (Chacko et al., 2013; Salthouse, 2006; Simons

etal., 2016) and meta-analyses (Melby-Lervåg, Redick, &

Hulme, 2016; Rapport, Orban, Kofler, & Friedman, 2013)

have concluded that existing cognitive-training methods

are relatively ineffective, despite their popularity and

increasing market share.

For example, a recent working-memory-training meta-

analysis (Melby-Lervåg etal., 2016) evaluated 87 studies

examining transfer to working memory, intelligence, and

various educationally relevant outcomes (e.g., reading

comprehension, math, word decoding). The studies var-

ied considerably in terms of the sample composition

(age; typical vs. atypical functioning) and the nature of

the working-memory training (verbal, nonverbal, or both

verbal and nonverbal stimuli; n-back vs. span task

methodology; few vs. many training sessions). Despite

the diversity in the design and administration of the train-

ing, the results were quite clear. Following training, there

were reliable improvements in performance on verbal

and nonverbal working-memory tasks identical or similar

to the trained tasks. However, in terms of far transfer,

there was no convincing evidence of improvements,

especially when working-memory training was com-

pared to an active-control condition. The meta-analysis

also demonstrated that, in the working-memory-training

literature, the largest nonverbal-intelligence far-transfer

effects are statistically more likely to come from studies

with small sample sizes and passive control groups. This

finding is not particularly surprising, given other work

showing that most working-memory training studies are

dramatically underpowered (Bogg & Lasecki, 2015) and

that underpowered studies with small sample sizes are

more likely to produce inflated effect sizes (Button etal.,

2013). In addition, small samples are predominantly the

reason irregular pretest-posttest patterns have been

observed in the control groups in various working-mem-

ory and video-game intervention studies (for review, see

Redick, 2015; Redick & Webster, 2014). In these studies,

inferential statistics and effect-size metrics provide evi-

dence that the training “worked,” but investigation of the

descriptive statistics tells a different story. Specifically, a

number of studies with children and young adult sam-

ples have examined intelligence or other academic

achievement outcomes before and after training. Closer

inspection indicates that training “improved” intelligence

or academic achievement relative to the control condi-

tion because the control group declined from pretest to

posttest—that is, the training group did not significantly

change from pretest to posttest.

664716PSI

XXX10.1177/1529100616664716McCabe et al.Brain-Training Pessimism

research-article2016

Corresponding Author:

Randall W. Engle, School of Psychology, Georgia Institute of

Technology, 654 Cherry St., Atlanta, GA 30332

E-mail: [email protected]

Brain-Training Pessimism, but Applied-

Memory Optimism

Jennifer A. McCabe

, Thomas S. Redick

, and

Randall W. Engle

Center for Psychology, Goucher College;

Department of Psychological Sciences, Purdue University; and

School of Psychology, Georgia Institute of Technology

188 McCabe et al.

However, if the brain-training methods outlined in

Simons etal. (2016) have little efficacy outside of the nar-

row range of the tasks that are repeatedly practiced, are

there interventions that might be more effective? Cer-

tainly, we think that cognitive functioning can be altered,

although most research has not been able to determine

whether a particular intervention produces temporary,

state-like change or enduring, trait-like modification

(Ilkowska & Engle, 2010). Whereas others have reviewed

non-cognitive interventions such as aerobic exercise

(Erickson, Hillman, & Kramer, 2015), meditation training

(Diamond & Lee, 2011), and pharmaceutical use (M. E.

Smith & Farah, 2011), we will focus on cognitive tech-

niques based on over 100 years of research that individu-

als can implement with relatively little cost and essentially

no risks or negative side effects. However, as will be

seen, each technique requires an intentional, active effort

on the part of the learner that is likely viewed as less

enjoyable than playing a video game.

Learning Strategies From Applied

Memory Research

Given the lack of consistent evidence for working-

memory training programs, many of which are designed

and implemented at great cost, we propose a reinvigo-

rated and reimagined implementation of lower-cost

learning-strategy “training programs.” Applied memory

research, particularly in the interdisciplinary space wed-

ding cognitive and educational psychology, has consis-

tently shown benefits of several core groups of related

learning strategies. Indeed, the translation of basic

memory research to educationally relevant contexts has

been a focus of recent major publications by leaders in

the field (e.g., Dunlosky, Rawson, Marsh, Nathan, &

Willingham, 2013; Roediger & Pyc, 2012). These learning

strategies have been mainly examined in the context of

studies with clear applications to the traditional class-

room. However, the existing strong evidence in favor of

these strategies suggests an opportunity to further expand

their implementation in various training environments.

In this section, we describe several core memory strat-

egies as alternatives to costly interventions such as work-

ing-memory training programs. Echoing the organizational

schemes from Dunlosky etal. (2013) and Roediger and

Pyc (2012), below we summarize three categories of

strategies established by memory research as effective for

long-term memory: elaboration, testing, and spacing.

After describing each strategy, we discuss the conclusions

regarding translation to educational contexts in recent

research.

The first major category, meaning-based elaboration,

stems from Craik and Lockhart’s (1972; see also Craik

& Tulving, 1975) evidence for the theory that deep

processing of information at the encoding stage, through

semantic connections, is superior to more shallow (sur-

face) types of processing. There are several subcategories

of strategies based on the elaboration principle. For

example, use of the self-schema during encoding is based

on the self-reference effect, which suggests that memory

will be stronger for information related to oneself (e.g.,

Klein & Kihlstrom, 1986; Rogers, Kuiper, & Kirker, 1977).

That is, learners who actively make connections to their

own lives should be more successful in remembering the

content of educational materials compared to learners

who use relatively shallow strategies such as rereading or

highlighting the text. This variety of elaboration has not

been a focus of recent translational research, however.

Two related meaningful elaboration strategies are

elaborative interrogation, which involves asking oneself

questions during learning (e.g., B. L. Smith, Holliday, &

Austin, 2010), and self-explanation, which involves

describing to oneself why a particular strategy choice

was made during problem-solving (e.g., Wong, Lawson,

& Keeves, 2002). Both Dunlosky et al. (2013) and

Roediger and Pyc (2012) have argued that these are

promising strategies in the educational domain but have

also called for more research in applied contexts. Toward

this goal, a recent review presented evidence that train-

ing in self-explanation strategies led to more connected

and coherent knowledge as learners developed expertise

(Richey & Nokes-Malach, 2015).

Next, the use of imagery enables elaboration by utiliz-

ing multiple modalities for encoding. Supplementing the

rehearsal of language-based information with mental

images greatly benefits memory. According to Paivio’s

(1986) dual-coding theory, using verbal and image-based

encoding enhances the number of routes for retrieval

(see also Bower & Winzenz, 1970; Paivio, Smythe, &

Yuille, 1968). Though basic memory research has estab-

lished the use of imagery as a consistently and highly

impactful strategy across a variety of materials and situa-

tions, Dunlosky etal. (2013) categorized the strategy of

imagery for text as potentially limited to specific text-

learning situations (e.g., when students have stronger

existing domain knowledge from which to generate

images) and therefore in need of further research to

establish boundary and transfer conditions.

The third category of elaboration is the use of mne-

monics, which can be verbal or visual but are often a

combination of both. Mnemonic devices help to impose

an organizational or chunking scheme onto to-be-learned

material, which makes the material more meaningful to

the learner (though the degree to which the mnemonics

involve meaningful elaboration does vary) and ultimately

more memorable at a later time (e.g., Bellezza, 1996).

Common examples of mnemonic techniques include first-

letter mnemonics (e.g., acronyms, acrostics), keyword

Brain-Training Pessimism 189

mnemonics, the pegword method, the method of loci,

and the use of songs, rhymes, and stories. Dunlosky etal.

(2013) focused their review of the extant literature on the

keyword technique, which involves creating a similar-

sounding keyword for a to-be-learned term, then con-

necting the keyword to the term’s definition via a mental

image. They concluded that keyword mnemonics may be

most helpful for foreign language learning and expressed

concerns about the shorter-term nature of the learning.

Yet some research has in fact demonstrated long-term

learning, with transfer to applied assessments, for the key-

word technique (e.g., Carney & Levin, 2008). Additional

studies are needed to determine the best fit of specific

mnemonic strategies to different types of content and

learning environments.

The next major category of empirically supported

memory strategy is testing (or retrieval-based learning).

Research has consistently demonstrated that practicing

retrieval of information from long-term memory is a

potent learning event in and of itself. That is, compared to

simply rereading a text, engaging in effortful retrieval of

text information (i.e., taking a test) results in far better

memory outcomes (e.g., Karpicke, 2012; McDaniel,

Howard, & Einstein, 2009; Roediger & Karpicke, 2006).

Dunlosky et al. (2013) concluded from their extensive

review that practice testing is extremely effective for lon-

ger-term learning in a variety of learning situations. We

also note that testing is one example of a broader cate-

gory of memory strategy called generation, which is

based on the idea that memory is better for learner-cre-

ated materials than instructor/trainer-created ones (e.g.,

Slamecka & Graf, 1978; for more on generation in relation

to educationally relevant materials, see McCabe, 2015).

Turning now to spacing (or distributed processing) as

a third memory principle with great potential for transla-

tion to training programs, research has repeatedly shown

a large benefit from taking breaks between periods of

study, as compared to massing or cramming studying

into one session, even with total study time held constant

(e.g., Cepeda, Pashler, Vul, Wixted, & Rohrer, 2006;

Kornell & Bjork, 2007; S. M. Smith & Rothkopf, 1984).

One reason why spacing is a slightly different type of

learning strategy is that it refers more to the schedule of

study than to a specific encoding technique. Thus, a

combination of strategies from the first two categories

(elaboration and testing) implemented on a schedule to

spread out learning sessions over time, with breaks in

between, should be even more effective. Spacing and

testing have garnered the most support as promising

memory principles to be applied to real-world contexts.

These strategies were enthusiastically endorsed by both

Dunlosky etal. (2013) and Roediger and Pyc (2012).

Almost three decades ago, Dempster (1988) lamented

the lack of application of the well-documented spacing

effect to educational contexts. After all, some of these

strategies, including spacing, have been known to be

effective since Ebbinghaus (1885/1913). Those in charge

of research on education and training programs should

consider whether we have come any further in applying

spacing and other memory principles established by

decades of research in cognitive psychology. Although

the memory strategies discussed above have been mainly

explored in traditional educational contexts (i.e., K-12

and higher education), there is no reason why they could

not be expanded to training situations in areas such as

the military, work training, and older adult education.

Teachers and trainers, along with researchers, have a

responsibility to learners. They should understand and

apply the best practices for learning and durable mem-

ory, representing the current state of knowledge and con-

sensus in the field.

In general, though, people are likely not aware of the

benefits of these strategies, given that when college stu-

dents were asked about the most effective learning strate-

gies, either directly through self-report (Kornell & Bjork,

2007) or indirectly through the evaluation of learning

scenarios (McCabe, 2011), they showed low metacogni-

tive awareness that these strategies (e.g., testing, spacing)

should be most helpful for learning. This speaks to a

major challenge in implementing these strategies in

applied learning contexts—namely, to convince the peo-

ple in charge of the “training,” whether these are teachers

designing classroom activities or students deciding how

to study, that these strategies support durable learning

and long-term retention.

In fact, some of the strategies discussed above are not

obvious and are even quite counterintuitive, having been

dubbed “desirable difficulties” by Bjork (1994). Desirably

difficult learning situations are those that the learner may

perceive as slow, effortful, and error-prone, yet which

demonstrate substantial long-term memory benefits. In

the present learning moment, it may not feel like signifi-

cant learning is actually taking place; thus, learners (and

teachers) may avoid these strategies and may even opt

for less effective but more intuitively appealing and

“easy” strategies such as highlighting or rereading.

Those studies that have examined the impact of learn-

ing-strategy “training” in higher education have shown

promising results, at least with regard to improved student

knowledge about the empirically supported study strat-

egy choices (e.g., McCabe, 2011) and increased meta-

cognition and subsequent academic performance (e.g.,

Azevedo & Cromley, 2004; Fleming, 2002; Tuckman, 2003;

see McCabe, 2014, for an instructional resource about

incorporating learning- and memory-strategy demonstra-

tions in the classroom). We call for additional controlled

research to determine the effectiveness and generalizabil-

ity of learning-strategy training in real-world educational

190 McCabe et al.

situations. In particular, researchers and educators should

examine the relative impact of these strategies when

directly compared to each other in similar learning envi-

ronments, and also the combined impact when the strate-

gies are used in tandem (e.g., testing or self-explanation

or imagery implemented on different spaced/distributed

schedules). And now that we know there is a metacogni-

tive disconnect with regard to what learners believe is the

best way to learn, we need to explore how to train them

to use the empirically based strategies; in an ideal world,

learners (some of whom will become teachers and train-

ers) would develop a tool kit of these effective habits of

mind when encountering to-be-learned information.

Another obstacle to implementation is that even if

those in charge of training know about the strategies,

they may feel more swayed by intuitively appealing yet

unsubstantiated ideas, including ideas about learning

styles (Pashler, McDaniel, Rohrer, & Bjork, 2008) and, as

discussed extensively in the target article, working-mem-

ory training (Rabipour & Davidson, 2015). It is also pos-

sible they are not convinced that the memory principles

will be realistically helpful in their specific classroom or

learning context (see Daniel & Poole, 2009, for a critique

of the memory-first approach). And, of course, it does

take motivation to change and/or implement a brand-

new training program. We recommend the popular press

book Make It Stick: The Science of Successful Learning

(Brown, Roediger, & McDaniel, 2014) for teachers and

trainers as an effective means of translating many of the

memory principles discussed here.

Conclusion

Working-memory training as currently implemented

does not work. One hundred years of research on basic

memory phenomena has discovered many procedures

that do!

Declaration of Conflicting Interests

The authors declared that they had no conflicts of interest with

respect to their authorship or the publication of this article.

References

Azevedo, R., & Cromley, J. (2004). Does training on self-

regulated learning facilitate students’ learning with hyper-

media? Journal of Educational Psychology, 96, 523–535.

doi:10.1037/0022-0663.96.3.523

Bellezza, F. (1996). Mnemonic methods to enhance storage

and retrieval. In E. L. Bjork & R. A. Bjork (Eds.), Memory:

Handbook of perception and cognition (pp. 345–380). San

Diego, CA: Academic Press. doi:10.1016/B978-012102570-

0/50012-4

Bjork, R. A. (1994). Memory and metamemory considerations in

the training of human beings. In J. Metcalfe & A. Shimamura

(Eds.), Metacognition: Knowing about knowing (pp. 185–

205). Cambridge, MA: MIT Press.

Bogg, T., & Lasecki, L. (2015). Reliable gains? Evidence

for substantially underpowered designs in studies of

working memory training transfer to fluid intelligence.

Frontiers of Psychology, 5, Article 1589. doi:10.3389/

fpsyg.2014.01589

Bower, G. H., & Winzenz, D. (1970). Comparison of associa-

tive learning strategies. Psychonomic Science, 20, 119–120.

doi:10.3758/BF03335632

Brown, P. C., Roediger, H. L., & McDaniel, M. A. (2014). Make

it stick: The science of successful learning. Cambridge, MA:

The Belknap Press of Harvard University.

Button, K. S., Ioannidis, J. P. A., Mokrysz, C., Nosek, B. A.,

Flint, J., Robinson, E. S. J., & Munafò, M. R. (2013). Power

failure: Why small sample size undermines the reliability of

neuroscience. Nature Reviews Neuroscience, 14, 365–376.

doi:10.1038/nrn3475

Carney, R. N., & Levin, J. R. (2008). Conquering mnemono-

phobia, with help from three practical measures of mem-

ory and application. Teaching of Psychology, 35, 176–183.

doi:10.1080/00986280802186151

Cepeda, N., Pashler, H., Vul, E., Wixted, J. T., & Rohrer, D.

(2006). Distributed practice in verbal recall tasks: A review

and quantitative synthesis. Psychological Bulletin, 132,

354–380. doi:10.1037/0033-2909.132.3.354

Chacko, A., Feirsen, N., Bedard, A., Marks, D., Uderman, J. Z.,

& Chimiklis, A. (2013). Cogmed working memory training

for youth with ADHD: A closer examination of efficacy uti-

lizing evidence-based criteria. Journal of Clinical Child &

Adolescent Psychology, 42, 769–783. doi:10.1080/15374416

.2013.787622

Craik, F. I. M., & Lockhart, R. S. (1972). Levels of process-

ing: A framework for memory research. Journal of Verbal

Learning and Verbal Behavior, 11, 671–684. doi:10.1016/

S0022-5371(72)80001-X

Craik, F. I. M., & Tulving, E. (1975). Depth of processing

and the retention of words in episodic memory. Journal

of Experimental Psychology: General, 104, 268–294.

doi:10.1037/0096-3445.104.3.268

Daniel, D. B., & Poole, D. A. (2009). Learning for life: An eco-

logical approach to pedagogical research. Perspectives on

Psychological Science, 4, 91–96.

Dempster, F. N. (1988). The spacing effect: A case study in

the failure to apply the results of psychological research.

American Psychologist, 43, 627–634. doi:10.1037/0003-

066X.43.8.627

Diamond, A., & Lee, K. (2011). Interventions shown to aid

executive function development in children 4 to 12 years

old. Science, 333, 959–964. doi:10.1126/science.1204529

Dunlosky, J., Rawson, K. A., Marsh, E. J., Nathan, M. J., &

Willingham, D. T. (2013). Improving students’ learning

with effective learning techniques: Promising directions

from cognitive and educational psychology. Psychological

Science in the Public Interest, 14, 4–58. doi:10.1177/

1529100612453266

Ebbinghaus, H. (1913). Memory (H. A. Ruger & C. E. Bussenius,

Trans.). New York, NY: Teachers College. (Original work

published 1885; paperback ed., New York: Dover, 1964)

Brain-Training Pessimism 191

Erickson, K. I., Hillman, C., & Kramer, A. F. (2015). Physical

activity, brain, and cognition. Current Opinion in Behavioral

Sciences, 4, 27–32. doi:10.1016/j.cobeha.2015.01.005

Fleming, V. (2002). Improving students’ exam performance by

introducing study strategies and goal setting. Teaching of

Psychology, 29, 115–119. doi:10.1207/S15328023TOP2902_07

Ilkowska, M., & Engle, R. W. (2010). Trait and state differences

in working memory capacity. In A. Gruszka, G. Matthews,

& B. Szymura (Eds.), Handbook of individual differences in

cognition: Attention, memory, and executive control (pp.

295–320). New York, NY: Springer.

Karpicke, J. D. (2012). Retrieval-based learning: Active retrieval

promotes meaningful learning. Current Directions in Psycho-

logical Science, 21, 157–163. doi:10.1177/0963721412443552

Klein, S. B., & Kihlstrom, J. F. (1986). Elaboration, organiza-

tion, and the self-reference effect in memory. Journal of

Experimental Psychology: General, 115, 26–38. doi:10

.1037/0096-3445.115.1.26

Kornell, N., & Bjork, R. A. (2007). The promise and perils of

self-regulated study. Psychonomic Bulletin & Review, 14,

219–224. doi:10.3758/BF03194055

McCabe, J. (2011). Metacognitive awareness of learning strate-

gies in undergraduates. Memory & Cognition, 39, 462–476.

doi:10.3758/s13421-010-0035-2

McCabe, J. A. (2014). Learning and memory demonstrations for

the psychology classroom. Retrieved from Office of Teaching

Resources in Psychology, Society for the Teaching of

Psychology website: http://teachpsych.org/page-1603066

McCabe, J. A. (2015). Learning the brain in Introductory

Psychology: Examining the generation effect for mnemon-

ics and examples. Teaching of Psychology, 42, 203–210.

doi:10.1177/0098628315587617

McDaniel, M. A., Howard, D. C., & Einstein, G. O. (2009). The

Read-Recite-Review study strategy: Effective and portable.

Psychological Science, 20, 516–522. doi:10.1111/j.1467-

9280.2009.02325.x

Melby-Lervåg, M., Redick, T. S., & Hulme, C. (2016). Working

memory training does not improve performance on mea-

sures of intelligence or other measures of “far transfer”:

Evidence from a meta-analytic review. Perspectives on

Psychological Science, 11, 512–534.

Paivio, A. (1986). Mental representations: A dual coding

approach. Oxford, England: Oxford University Press.

Paivio, A., Smythe, P. C., & Yuille, J. C. (1968). Imagery versus mean-

ingfulness of nouns in paired-associate learning. Canadian

Journal of Psychology, 22, 427–441. doi:10.1037/h0082782

Pashler, H., McDaniel, M., Rohrer, D., & Bjork, R. (2008).

Learning styles: Concepts and evidence. Psychological

Science in the Public Interest, 9, 105–119. doi:10.1111/

j.1539-6053.2009.01038.x

Rabipour, S., & Davidson, P. R. (2015). Do you believe in brain

training? A questionnaire about expectations of computer-

ised cognitive training. Behavioural Brain Research, 295,

64–70. doi:10.1016/j.bbr.2015.01.002

Rapport, M. D., Orban, S. A., Kofler, M. J., & Friedman, L. M.

(2013). Do programs designed to train working memory,

other executive functions, and attention benefit children

with ADHD? A meta-analytic review of cognitive, academic,

and behavioral outcomes. Clinical Psychology Review, 33,

1237–1252. doi:10.1016/j.cpr.2013.08.005

Redick, T. S. (2015). Working memory training and interpreting

interactions in intelligence interventions. Intelligence, 50,

14–20.

Redick, T. S., & Webster, S. B. (2014). Videogame inter-

ventions and spatial ability interactions. Frontiers in

Human Neuroscience, 8, Article 183. doi:10.3389/

fnhum.2014.00183

Richey, J. E., & Nokes-Malach, T. J. (2015). Comparing four

instructional techniques for promoting robust knowledge.

Educational Psychology Review, 27, 181–218. doi:10.1007/

s10648-014-9268-0

Roediger, H. L., & Karpicke, J. D. (2006). Test-enhanced learn-

ing: Taking memory tests improves long-term retention.

Psychological Science, 17, 249–255. doi:10.1111/j.1467-

9280.2006.01693.x

Roediger, H. L., & Pyc, M. A. (2012). Inexpensive techniques

to improve education: Applying cognitive psychology to

enhance educational practice. Journal of Applied Research

in Memory and Cognition, 1, 242–248. doi:10.1016/

j.jarmac.2012.09.002

Rogers, T. B., Kuiper, N. A., & Kirker, W. S. (1977). Self-

reference and the encoding of personal information.

Journal of Personality and Social Psychology, 35, 677–688.

doi:10.1037/0022-3514.35.9.677

Salthouse, T. A. (2006). Mental exercise and mental aging.

Perspectives on Psychological Science, 1, 68–87. doi:10.1111/

j.1745-6916.2006.00005.x

Simmons, J. P., Nelson, L. D., & Simonsohn, U. (2011).

False-positive psychology: Undisclosed flexibility in

data collection and analysis allows presenting anything

as significant. Psychological Science, 22, 1359–1366.

doi:10.1177/0956797611417632

Simons, D. J., Boot, W. R., Charness, N., Gathercole, S. E.,

Chabris, C. F., Hambrick, D. Z., & Stine-Morrow, E. A. L.

(2016). Do “brain training” programs work? Psychological

Science in the Public Interest, 18, xxx–xxx.

Slamecka, N. M., & Graf, P. (1978). The generation effect:

Delineation of a phenomenon. Journal of Experimental

Psychology: Human Learning and Memory, 4, 592–604.

doi:10.1037/0278-7393.4.6.592

Smith, B. L., Holliday, W. G., & Austin, H. W. (2010). Students’

comprehension of science textbooks using a question-

based reading strategy. Journal of Research in Science

Teaching, 47, 363–379. doi:10.1002/tea.20378

Smith, M. E., & Farah, M. J. (2011). Are prescription stimu-

lants “smart pills”? The epidemiology and cognitive

neuroscience of prescription stimulant use by normal

healthy individuals. Psychological Bulletin, 137, 717–741.

doi:10.1037/a0023825

Smith, S. M., & Rothkopf, E. Z. (1984). Contextual enrichment

and distribution of practice in the classroom. Cognition and

Instruction, 1, 341–358. doi:10.1207/s1532690xci0103_4

Tuckman, B. (2003). The effect of learning and motivation strat-

egies training on college students’ achievement. Journal of

College Student Development, 44, 430–437. doi:10.1353/

csd.2003.0034

Wong, R. M. F., Lawson, M. J., & Keeves, J. (2002). The effects

of self-explanation training on students’ problem solving

in high-school mathematics. Learning and Instruction, 12,

233–262. doi:10.1016/S0959-4752(01)00027-5