Metacognitive
Concerns in Designs and Evaluations of Computer Aided Education and Training:
Are We Misleading Ourselves About Measures of Success?
Bob
Jensen
Trinity University
715 Stadium Drive
San Antonio, TX 78212
Phone: 210-736-7347 Fax: 210-736-8134
email: rjensen@trinity.edu
Request for a Favor: This document is the first of a
sequence of research papers that are related to my Working
Paper 255 and Working Paper 260. All are
extensions of the Jensen and Sandlin online book. The document that follows
is a very rough draft. I would appreciate any feedback that
you can provide by email, phone, fax, or letter. I want to improve this
paper for my scheduled future workshops.
Table of Contents
Update Messages
Introduction
Metamemory and Metacognition:
The Metalevel Activities of the Brain
Making Learning More
Easy, Fun and Collaborative: Are We Taking Things Too Far in Virtual Learning Worlds?
The BAM Pedagogy at the University of
Virginia
Differences Between Traditional Group Learning and
Cooperative Learning
Warnings:
Suggestions for Future Research and Designs for Asynchronous Learning Networks
Conclusion
From Villanova University: BAM
for Managerial Accounting Education
Accounting Professors in Support
of Online Testing That, Among Other Things, Reduces Cheating
Acknowledgement of Paula Hertel
Appendix 1
Why Aren't Stories Good Food?
Email Messages About Evaluation Criteria and Processes
Appendix 2
The Differences Between Traditional Group Learning and Cooperative Learning
Appendix 3
The Emperor's Naked as He Can Be
Appendix 4
Update Message from Robert Bjork
MIT
Encyclopedia of the Cognitive Sciences (shown
in a new window)
Appendix 5
"
Assessing the Impact of Instructional Technology on Student
Achievement," by Lorraine Sherry, Shelley Billig, Daniel Jesse, and Deborah
Watson-Acosta.
Appendix 6
February 2003 Updates on Accounting Education Pedagogy
Update Messages
"The Case Against Case Studies: How Columbia's B-school is teaching
MBAs to make decisions based on incomplete data," by Geoff Gloeckler,
Business Week, January 24, 2008 ---
http://www.businessweek.com/magazine/content/08_05/b4069066093267.htm?link_position=link1
Shortly after R. Glenn Hubbard took over as
dean of Columbia Business School in 2004, he began hearing rumblings
from executives about the quality of MBA graduates. They were
undoubtedly smart but often unprepared to handle the most crucial of
managerial responsibilities: quickly solving problems with less than
perfect information. Among those wanting more from new hires is Henry
Kravis, co-founder of the private equity firm Kohlberg Kravis Roberts.
"I want to see MBAs who can jump in and make decisions, not jump in and
learn to make decisions," he says.
Hubbard made his own executive decision. He
devised a new twist on the case study—the teaching format invented by
Harvard Business School almost a century ago and used by most B-schools.
Hubbard's so-called decision brief offers less information about a
situation than the case study, and it doesn't present the solution until
students have grappled with the issues on their own. "We want our
students to be used to dealing with incomplete data," Hubbard says.
"They should be able to make decisions out of uncertainty."
Even Michael J. Roberts, the executive director
of the Arthur Rock Center for Entrepreneurship at Harvard and author of
more than 100 HBS case studies, acknowledges the potential benefits of
Hubbard's approach, which was introduced to Columbia students last fall.
"Framing problems and finding the data to analyze those problems is a
skill that MBAs need and that the classic case doesn't fully exploit,"
Roberts says. Hubbard expects such endorsements, as well as those of
companies, will encourage other business schools to make room one day
for Columbia's decision briefs in their curriculums. Hubbard, at least
initially, doesn't plan to sell the decision briefs but to use them to
tap into faculty research.
Hubbard isn't giving up on the traditional case
study altogether. As part of an initiative called CaseWorks, Columbia
will produce cases designed to reflect contemporary issues (which other
schools do already), while also creating decision briefs that do away
with the Harvard formula (which no one else has done). To help guide the
program, Hubbard has turned to two people familiar with the deficits of
the old methods: Stephen P. Zeldes, who has been at Columbia for more
than a decade and is now chairman of the economics department at the
B-school, and former Harvard case writer Elizabeth Gordon.
TOO MUCH INFORMATION The stock case study
presents a tidy narrative arc, with a protagonist and a clear story
line. One of the more widely used HBS cases focuses on Intel's (INTC)
former marketing vice-president, Pamela Pollace, as she decides whether
Intel should extend the "Intel Inside" branding campaign to products
other than computers. In 24 pages, students are provided with
information on Intel and the history of microprocessors, as well as
details about market share and segmentation. Pollace's major concern,
they learn, is brand dilution; the potential reward is likely worth the
risk. In effect, the students are guided along the decision-making
process.
If this case were a Columbia decision brief,
students might see a video interview in which Pollace describes the
challenge. They would also be given a few documents on the background of
the campaign itself—the same data a manager at the company would have,
but no more. Then, students would discuss possible solutions. Afterward,
the group would see a second video of Pollace explaining how she handled
the issue before debating whether or not she made the right decision.
So far, Columbia has produced six briefs that
take on of-the-moment business challenges: Among them is one that
focuses on General Electric's (GE) business-process-outsourcing division
in India. Given increased competition, the company needed to consider a
bigger investment, as well as the possibility of serving non-GE
customers. With just a little more information than that, students are
asked to come up with various strategies. "The idea is to try to
simulate what it will be like in a real workplace," says Gordon. "There
is uncertainty, things aren't predigested, all the information won't be
there."
The first field test for the new teaching
technique will be this summer, when the MBAs head out to their
internships. At Goldman Sachs (GS), which hires more Columbia interns
than any other company, the co-head of campus recruiting, Janet Raiffa,
hopes to encounter students who are more independent thinkers. As for
Kravis, his firm doesn't employ summer interns.
In some ways the pedagogy proposed by Columbia is a shorter and
cheaper version of the BAM approach first proposed by Catanach, Croll, and
Grinacker. The BAM approach uses a year-long case and students can seek out
data in virtually every way they will do it later on while on the job
(including paying for data if necessary) --- See Below
Bob Jensen's threads on Learning at Research Schools Versus "Teaching
Schools" Versus "Happiness" With a Side Track into Substance Abuse ---
http://www.trinity.edu/rjensen/Theory01.htm#Happiness
February 17, 2005 message from Bob Jensen
I call your attention to Page 4 of the Spring 2005 newsletter called “The
Accounting Educator” from the Teaching and Curriculum Section of the
American Accounting Association --- http://aaahq.org/TeachCurr/newsletters/index.htm
The current Chair (Tomas Calderon) has a piece about “reflection”
which is nice to reflect upon. There are abstracts of papers in other
journals that relate to education, and an assortment of teaching cases.
Marinus Bouman has a nice piece entitled “Using
Technology To Integrate Accounting Into The Business Curriculum.”
Interestingly, the Sam M. Walton College of Business at the University of
Arkansas no longer has courses in Principles of Accounting (or Marketing or
Finance). You should read Bouman’s article to find out what took the place
of these principles courses in a daring curriculum experiment.
Since I teach accounting theory, I found Bob Clusky’s paper “Where’s
Accounting Theory) quite interesting. Even more than AIS, “Accounting
Theory” is a phrase still in search of a definition.
Tim Fogerty has a piece on Distance Education. It is somewhat negative in
tone, but Tim seems to sigh that the march forward is inevitable and the
current boundries of education from one program or one institution will
evaporate as students seek courses and modules from anywhere in the world. I
might take issue with some of his conclusions such as testing and/or
assessment, but this is not the time or the place for that. See http://www.trinity.edu/rjensen/assess.htm
There is much, much more of interest in this 32 page newsletter. Go to http://aaahq.org/TeachCurr/newsletters/index.htm
Page 4 describes a forum to be headed up by Tim Fogarty (Case Western)
and Call for Papers in which the last paragraph reads as follows:
*******************************
Issues in Accounting Education, in conjunction with the Teaching &
Curriculum Section of the AAA has asked me to edit a dedicated forum with
an expected publication date of Spring 2006. I would like to extend an
opportunity to accounting educators to submit essays for this issue.
Proposed pieces for inclusion should be 25 pages (double spaced) or less.
Submissions will be peer-reviewed with an emphasis on clarity and strength
of ideas. The deadline for the first drafts is March 1, 2005. There would
also be an opportunity to discuss these ideas in a CPE session at the AAA
meeting in San Francisco.
*****************************
Bravo to Thomas, Tim, and other volunteers who are continuing the
momentum of this essential section of the AAA! This is the lifeblood of why
we are in this profession.
Bob Jensen
February 17, 2005 message from Bob Jensen
Note
the following paragraph that I wrote in my previous message:
*********************
Marinus Bouman has a
nice piece entitled “Using Technology To Integrate
Accounting Into The Business Curriculum.”
Interestingly, the
Sam
M.
Walton
College
of Business at the
University
of
Arkansas
no longer has courses in Principles of Accounting (or Marketing or Finance).
You should read Bouman’s article to find
out what took the place of these principles courses in a daring curriculum
experiment.
*******************
The simulation pedagogy used in the "Business Foundations"
course at the Walton College seems to be quite related to the BAM pedagogy ---
http://www.trinity.edu/rjensen/265wp.htm#UVA000
The
more I think about this, the more I think that the
Walton
College
as evolved, I assume quite independently, into something quite similar to what
Jack Wilson (a physicist) pioneered at
Rensselaer
well over a decade ago. Core
courses (such as physics) in the general curriculum at
Rensselaer
were taken from the curriculum and replaced with technology-based “studio”
learning. The
University
of
Arkansas
is doing something similar in relying upon technology when taking the core
courses, such as Principles of Accounting, out of business education.
The
following is taken from http://www.trinity.edu/~rjensen/245glosf.htm#Studio1
Studio
classroom= An application of computer technology
pioneered by Jack M. Wilson at Rensselaer Polytechnical
Institute for replacing large lecture courses with students working in pairs
in front of computer screens where they interactively tackle problems and
issues rather than listen to or passively watch lectures in front of a mass
lecture section. The only lecture comes at the beginning and end of class
where the instructor commences or wraps up the learning session. The
"studio" is a combination lab and electronic classroom.
Dr.
Wilson now serves as the President of the
University
of
Massachusetts
system. He had been serving as the Vice President for Academic Affairs
of the
University
of
Massachusetts System
and is the founding Chief Executive Officer (CEO) of UMassOnline,
the University of
Massachusetts Virtual University. As Vice president he was
responsible for the coordination of the academic programs in research and
teaching throughout the five campus system. As CEO of UMassOnline
he worked with the five physical campuses,
Amherst
,
Lowell
,
Boston
,
Worcester
, and
Dartmouth
to provide online access to the programs of the
University
of
Massachusetts
.
Jack
Wilson was one of the early pioneers in education technologies and learning.
He is now CEO and founder of UMass Online
.
Dr.
Wilson, also known as an entrepreneur, was the Founder (along with Degerhan
Usluel and Mark Bernstein), first President, and
only Chairman of LearnLinc
Corporation (now Mentergy),
a supplier of software systems for corporate training to Fortune 1000
Corporations. In early 2000. LearnLinc
merged with Gilat
Communications, (GICOF)
which also acquired Allen Communication
from the Times Mirror group. The Gilat-Allen-LearnLinc
combination forms a powerful "one stop shopping" resource for
E*Learning that is now the Mentergy unit of Gilat
Communications. (The LearnLinc
Story).
Dr
Wilson was the J. Erik Jonsson '22 Distinguished
Professor of Physics, Engineering Science, Information Technology, and
Management and the Co-director of the
Severino
Center
for Technological
Entrepreneurship at
Rensselaer
. After coming to
Rensselaer
in 1990, he served as the
·
Dean
of Undergraduate Education,
·
Dean
of Professional and Continuing Education,
·
Interim
Provost,
·
Interim
Dean of Faculty, and as the
·
Founding
Director of the
Anderson
Center
for Innovation in
Undergraduate Education.
In
these roles, Wilson led a campus wide process of interactive learning and
restructuring of the educational program, known for the design of the Studio
Classrooms, the growth of the Distributed Learning Program, the creation of
the Faculty of Information Technology, and the initiation of the student
mobile computing (universal networked laptop) initiative
The
Studio Classrooms at
Rensselaer
replaced large sized core courses taught by traditional lecture pedagogy with
student teams responsible largely for teaching themselves using computer-aided
and interactive course materials --- http://www.rpi.edu/dept/NewsComm/WNCTW/ad7.html
Welcome
To Interactive Learning
Roll up your sleeves and take a seat in the
Rensselaer
studio classroom. Classes
of about 60 students are engaged at wired workstations - utilizing cutting
edge tools like Web-based technologies, full-motion video, computer
simulation, and other laboratory resources. An
instructor and teaching assistant move from workstation to workstation
observing and coaching. Notes are taken with a simple mouse click, as
students download files and class materials onto their required laptops.
It's an innovative blend of discussion and skill-building, high-tech inquiry
and problem-solving - preparing scholars to succeed in the new business
world. It's all part of Interactive Learning at
Rensselaer
.
More
Studios Than
Hollywood
Interactive Learning is more than just a concept at
Rensselaer
; it's a working reality.
The approach has been infused throughout all of our undergraduate
disciplines in more than 25 studio classrooms with more being built all the
time. In the LITEC studio classroom, students build remote-controlled cars
in a project-based, team environment. In the Circuits Studio, students
develop and test their own circuits. The Collaborative
Classroom, funded by the National Science Foundation, serves as a testbed
for using computer technology to collaborate on design projects. At
Rensselaer
, knowledge and application
are seamlessly intertwined.
Teaching
How We Teach
Rensselaer
's revolutionary model for
education has been talked about, honored, and emulated. We earned the first
Pew Charitable Trust Award for the Renewal of Undergraduate Education and
the first Boeing Outstanding Educator Award, among others. Last year, we
were named to administer an $8.8 million Pew-funded program to bring
educational innovation to other universities in this country: The
Center for Academic Transformation. Literally hundreds of institutions
have visited
Rensselaer
to learn how we teach.
No
Stopping Now
Of course, the very thinking that enabled
Rensselaer
to initiate Interactive
Learning is the same mindset that keeps us pressing forward.
Rensselaer
's Anderson
Center for Innovation in Undergraduate Education was founded 11 years
ago with the continuing mission of making
Rensselaer
a leader in innovative
pedagogy. More recently, the
Rensselaer
Academy
of Electronic Media has become the spawning ground for highly creative
visualization software that enables students to learn scientific and
engineering principles in ways never before possible. We continue to look
for new and better methods to evolve education - meeting the present and
future needs of our students, professors, and global businesses. Because
solving real-world challenges is our mission and
our passion.
For a
summary short summary see http://www.bc.edu/bc_org/avp/soe/cihe/newsletter/News15/text4.html
. (See also Electronic
classroom)
Random Thoughts (about learning from a retired professor of
engineering) ---
http://www4.ncsu.edu/unity/lockers/users/f/felder/public/Columns.html
Dr. Felder's column in Chemical Engineering Education
Focus is heavily upon active learning and group learning.
Bob Jensen's threads on learning are in the following links:
http://www.trinity.edu/rjensen/assess.htm
http://www.trinity.edu/rjensen/255wp.htm
http://www.trinity.edu/rjensen/265wp.htm
Metacognitive Concerns
in Designs and Evaluations
of Computer Aided Education and Training: Are We
Misleading Ourselves About Measures of Success?
I've gotten to the point where I really don't lecture
anymore, so if I look awkward up here talking to you, it's because I haven't lectured in
four years. And the second point is that I've lost total faith in lecturing. Try telling
your students something important in class, and then finding out how many heard it to
where they can remember it the next day.
David Croll, University of Virginia
Transcribed from a videotape recording
Annual Meetings of the American Accounting Association
Dallas, Texas, August 20, 1997
Caveat: . I am grateful to
Professors Croll and Catanach for allowing me to videotape their inspiring presentation.
The quotations from Professors Croll and Catanach that appear at various points in this
document have never been edited by those professors or modified from a transcript of a
presentation that I videotaped at a conference. My videotape was transcribed by my
secretary, Debbie Bowling. The transcription
was modified by me only when Debbie failed to understand certain terminology. I
prefer to minimize changes in the transcription so that what is read remains as close as
possible to what the audience listened to at the conference. None of us speak with the
formalized vocabulary and grammar used in our writing. Also we cannot edit what we said in
the same manner that we can edit what we wrote. Hence, transcriptions should not be judged
as writing. In August 1998, Tony Catanach moved to Villanova University in
Villanova, Pennsylvania.
Introduction
R. Jensen (1998a) documents
the exponential increase in asynchronous learning where students take on greater
responsibility for teaching themselves and set their own learning paces. Evidence is
mounting that student performance thereby improves. Some of our finest and oldest
universities in the world are experimentally replacing lectures with asynchronous learning
networks where students no longer attend scheduled classes. The early success of the Sloan
Foundation funded experiments, where traditional lectures are replaced by asynchronous
learning networks (ALNs), are making all institutions take notice even for full-time
resident students. Examples of these successes and concerns about these successes are
documented in R. Jensen (1998a).
In addition, traditional (synchronous) lectures and cases are taking place in electronic
classrooms where live Internet connections and presentation multimedia are replacing chalk
and flip charts. Students in electronic classrooms or in front of their own laptops at
home can be taken into virtual learning worlds and electronic chat rooms that make
learning easier, faster, more collaborative, and more fun. Virtual worlds make learning
more contextual. Biology students work with virtual organs and organisms, chemistry and
physic students enter virtual laboratories, medical students diagnose and heal virtual
patients, sociology students colonize virtual societies, finance students choose
portfolios in virtual markets, etc.
Across many years and nations, educators have long known that interactive learning
tends ceteris paribus to be better than passive learning. Computer and networking
technologies make interactive learning more effective and efficient. Students having
keyboard, mouse, joystick, microphone, web camera, and other input controls have greater
powers of interaction with instructors, other students, libraries, experts, and objects
around the world. Students not only can listen to Beethoven and Chopin, they can be
transported back in time to the virtual lives and times of master composers. The great
libraries of online documents, major university library holdings, and government archives
are at their fingertips.
Jensen and Sandlin (1998, Chapter 2) list many advantages of
newer learning technologies. R. Jensen (1998a) outlines various
worries and concerns. The main purposes of this paper are as follows:
- to point out several "metalevel" concerns about newer learning technologies;
- to discuss learning material design considerations for newer technologies that may
improve long-term memory and performance of students. In particular, we face the challenge
of designing greater ambiguities and learning
frustrations into computer-aided learning materials that tend to reduce rather than
increase ambiguities into learning tasks.
- To caution about metacognitive risks (possibly paradoxes) in evaluating programs,
courses, and learning modules prior to graduation and even earliest years of post-graduate
work and study.
"The Great Debate: Effectiveness of Technology in Education," by
Patricia Deubel, T.H.E. Journal, November 2007 ---
http://www.thejournal.com/articles/21544
According to Robert Kuhn (2000), an expert in brain
research, few people understand the complexity of that change. Technology is
creating new thinking that is "at once creative and innovative, volatile and
turbulent" and "nothing less than a shift in worldview." The change in
mental process has been brought about because "(1) information is freely
available, and therefore interdisciplinary ideas and cross-cultural
communication are widely accessible; (2) time is compressed, and therefore
reflection is condensed and decision-making is compacted; (3) individuals
are empowered, and therefore private choice and reach are strengthened and
one person can have the presence of an institution" (sec: Concluding
Remarks).
If we consider thinking as both individual
(internal) and social (external), as Rupert Wegerif (2000) suggests, then "[t]echnology,
in various forms from language to the internet, carries the external form of
thinking. Technology therefore has a role to play through supporting
improved social thinking (e.g. providing systems to mediate decision making
and collective reasoning) and also through providing tools to help
individuals externalize their thinking and so to shape their own social
worlds" (p. 15).
The new tools for communication that have become
part of the 21st century no doubt contribute to thinking. Thus, in a debate
on effectiveness or on implementation of a particular tool, we must also
consider the potential for creativity, innovation, volatility, and
turbulence that Kuhn (2000) indicates.
Continued in article
Bob Jensen's threads on education technology are at
http://www.trinity.edu/rjensen/000aaa/0000start.htm
Bob Jensen's threads on assessment are at
http://www.trinity.edu/rjensen/assess.htm
Metamemory and Metacognition: The Metalevel
Activities of the Brain
The brain stores long-term memory in the outer cerebral cortex with complex
recollection systems among memory patterns of billions of neurons. Most things stored
therein cannot be instantly recalled, but ways in which knowledge becomes embedded and
patterns of use over time affect if and when recall takes place. Much depends upon
associations between stored knowledge. For example, I probably would never recall the name
of Don Quixote’s horse had this not been a memorable question (for me) on a 1961 GRE
examination. That trivia question "sticks" in my mind all these years because of
the humor of being asked about the only thing I remembered from that book (last read while
I was a high school farm boy who liked horses).
There are also important distinctions between visual versus verbal memory, and these
distinctions are crucial in designs of asynchronous learning materials. In fact, virtually
all of Howard Gardner's theories on seven types of intelligence are important to take into
account in the design of learning materials. These are described as follows by The Gardner
School at http://www.swopnet.com/ed/TAG/7_Intelligences.html
:
1. Linguistic
Children with this kind of intelligence enjoy writing, reading, telling stories or
doing crossword puzzles.
2. Logical-Mathematical
Children with lots of logical intelligence are interested in patterns, categories and
relationships. They are drawn to arithmetic problems, strategy games and experiments.
3. Bodily-kinesthetic
These kids process knowledge through bodily sensations. They are often athletic,
dancers or good at crafts such as sewing or woodworking.
4. Spatial
These children think in images and pictures. They may be fascinated with mazes or
jigsaw puzzles, or spend free time drawing, building with Legos or daydreaming.
5. Musical
Musical children are always singing or drumming to themselves. They are usually quite
aware of sounds others may miss. These kids are often discriminating listeners.
6. Interpersonal
Children who are leaders among their peers, who are good at communicating and who seem
to understand others’ feelings and motives possess interpersonal intelligence.
7. Intrapersonal
These children may be shy. They are very aware of their own feelings and are
self-motivated.
However, designing multimedia with the above types of intelligence in mind does not in
and of itself aid to metacognition and metamemory. The mysterious process of
"deciding" (it’s an unconscious process) what gets stored versus what gets
lost or discarded takes place in a deeply embedded part of the brain known as the
hippocampus. That organ is analogous to a computer’s file manager when it stimulates
neurons to form an episodic network of neurons. Two important criteria affecting this
process are emotional significance and association significance (associations with
episodes already embedded in neuron structures). Information overload of the brain is not
so much an issue of the number of neurons as it is with the input rate and complexity of
processing incoming perception vis-à-vis emotions and episodic associations. Short-term
memory "junk" such as a seldom-used phone number never clutters up our long-term
memory episodic networks. Rote memorization of the phone number may last for a day but is
not likely to last for a decade or even a week after it is used. When the hippocampus is
damaged, a person might never store incoming knowledge and live only with recordings made
when the hippocampus functioned properly. Alzheimer’s disease attacks the hippocampus
region of the brain. Aaron, Benjamin, and Bjork (1998, p. 55) define metamemory as follows:
There has been a surge of interest in metamemory --- the study of
what people know and understand about their own memory and memorial processes. From a
theoretical standpoint, there has been a particular effort to explain why certain
metamnemonic measures, such as the feelings of knowing . . . are accurate or
inaccurate under various conditions.
The term "cognition" refers to the entire process of knowing --- a process
comprised of perception, concentration, memory, judgment, and awareness. A subset of
cognition known as metacognition is set apart from most cognitive activities and takes
place in the frontal lobes. Metacognition is little understood even though we do know that
it is impacted by episodic association formations in the hippocampus and that our
well-developed metacognition is what sets human beings apart from other species.
Psychologists refer metacognition as the monitoring process superimposed upon the control
process of cognition. This monitoring can alter the control processes. Metacognition gives
rise to what is commonly referred to as "Feeling of Knowing" or FOK judgments in
cognition literature. It also gives rise to what is known as "novelty" versus
"familiarity" awareness of perceptions.
Thiamine deficiency in the frontal lobes often accompanies severe alcohol abuse and is
known in medicine as the Korsakoff Syndrome. Korsakoff amnesia thereby differs from
amnesia that does not so dramatically affect the frontal lobes. Shimamura
and Squire (1986) report that patients diagnosed with Korsakoff Syndrome have impaired
FOK judgments. The theory is that frontal lobes processes modulate the hippocampus
processes of forming episodic associations. Cognitive scientists make a distinction
between basic memory and metamemory. Metamemory helps to make distinctions between stored
memories. Weingradt, Leonesio, and Loftus (1994) note that a witness to an
automobile accident may store perceptions at the scene of the accident and knowledge from
what he or she read about or heard subsequent to the event. On Page 183 they elaborate as
follows:
Consider a typical situation in which an eyewitness is asked to report
what he or she has seen. Whether the witness is asked to identify a perpetrator, describe
the events leading up to an automobile accident, or discriminate between information
actually witnessed in an event or read about in a newspaper article, he or she is required
to make judgments about information in memory. In metacognitive terms then, eyewitnesses
are often required to make metamemory monitoring judgments. As should be evident from our
discussion of the misinformation effect as a metacognitive error, the literature on
metamemory monitoring can provide valuable insights into the reasons for poor eyewitness
memory performance.
These metalevel processes enable humans to reflect upon thoughts and behaviors. Nelson and Narens
(1994) refer to "metalevel" processes that are set apart from the
lower-order "object-level" memory and cognition processes. Metalevel processes
monitor perceptions to assess their novelty/familiarity and to direct attentiveness and
feelings. Janet Metcalfe (1996,
pp. 381-382) quotes a passage from Plato in which Socrates refers to an
"inner voice." She then goes on to write the following passage:
Socrates' inner voice is the earliest well-known example of a
self-reflective consciousness --- an internal monitor and control system. A
self-monitoring consciousness figures in the musings of Descartes, emerging as "the
thinker" whose existence cannot be doubted, even though all the content of the
thought is open to question."
Later on Page 382 she continues as follows:
An accumulating body of evidence points to the prefrontal cortex as a
brain region of critical importance for these [metalevel] functions. This region is
distinct from those areas responsible for the knowledge and operations upon that knowledge
that are the object of reflection, though, of course, they are interconnected. As Nelson and Narens
(1994) have pointed out, object-level functions of cognition, such as memory storage,
retrieval, and applying the operations of problem solving, may be separable from metalevel
functions that monitor and control the object-level functions. In the normal person these
two levels, cognition and metacognition, interact in a complex manner. This interaction is
critical not only for problem solving, planning, and for memory, but arguably for
sensitive assessments of one’s own appropriate social behavior.
Metacognition is essential to what earlier writers have called introspection. Nelson and Lorens
(1994, p. 17) quote William James in 1890 as follows:
Introspective Observation is what we have to rely on first and
foremost and always. . . . I regard this belief as the most fundamental of all postulates
of Psychology.
Object-level versus metalevel distinctions become important to educators for
self-evident reasons. The implications are immense. For example, it may suffice to make it
as easy as possible (e.g., with visual aids) for students to memorize multiplication
tables, but we are not so certain that it is best to make it as easy as possible to
understand number theory theorems. Quite possibly students lose introspective powers if
they do not re-discover some aspects of number theory on their own rather than start at
the top of the heap of known theory. Some aids to making it easier to understand the
theorems may be detrimental to long-term memory and performance.
Making Learning More Easy, Fun
and Collaborative: Are We
Taking Things Too Far in Virtual Learning Worlds?
I have been making hypermedia CD-ROMs for my students since 1992. Until recently, my
goals have been to make learning more easy, fun, inspirational, realistic, collaborative,
and efficient. I take my students into worlds of virtual financial contracting and
reporting. When seeking answers, my students can watch and listen to world experts explain
complex transactions (e.g., financial instruments derivatives contracts and mathematical
models for analyzing investment risks) and issues and theories about accounting for those
transactions. I have videotaped experts at numerous conferences and workshops. Then I
digitized the experts’ audio/video excerpts onto CD-ROMs and organized the material
for pedagogic ease and efficiency. I also transcribed hundreds of hours of audio into text
that can be searched using key words and is presented in a variety of formats (including
Jeopardy-type games that make learning more fun). Readers can view some of my course
materials from links at my web site.
I also made things easier by obtaining rights to share commercial literature databases
and CPA Examination review courses on CD-ROMs. For example, by using the Price-Waterhouse
Researcher CD-ROM, the accounting and auditing standards of the major developed nations
are literally at the tips of student fingers on a keyboard. I can make my assignments
exceedingly tough and realistic. Students can find the answers using my CD-ROMs, network
server files, the Internet, textbooks, and photocopy handouts. I created a learning world
in which my students can quickly find answers and do not have to conduct those slow,
frustrating, and serendipitous library searches. When computer coding became involved for
students (e.g., to create Microsoft Access databases or to code JavaScript web documents),
I wrote tutorials that made learning much easier.
My goal was to take students to heights that were not possible in my courses before
computing and networking hypermedia technologies. By making the hunt for answers easier
and fun and collaborative, I could cover more material and more difficult material. My
courses were not necessarily easier since I added more material. However, my courses were
more efficient since students learned faster and spend less time searching for answers and
completing projects. Learning became easier and in some respects more fun since student
could see and hear experts whose names appear in the literature.
My own research, however, is far more serendipitous than research than I require from
my students. Once in a while I pull myself away from the Internet and wander amongst the
shelved books in the library. I peruse tables of contents and take a glance at sections of
books and journal contents that capture my eye (I like to think of it as metacognitive
success in identifying something novel that is not familiar to me). An article that I
actually stumbled upon in the manner described above has had a profound impact upon my
teaching and research. That article is "Memory and Metamemory Considerations in
the Training of Human Beings" by Bjork (1994). Dr. Bjork convinced me that I was setting some wrong
goals when creating asynchronous learning materials for my courses. In addition, he
convinced me that many of the things that I had written about education technology are
wrong.
I do not go so far as to say that the goals of using new technologies for making
learning more easy, fun, inspirational, collaborative, and efficient are wrong per se.
I do, however, suspect that there are dangers in making these goals too sacrosanct in
preparing learning materials for lectures and asynchronous learning outside the classroom.
By way of illustration, let me focus on a passage from Bjork (1994, p. 197-198).
The process of accessing stored information given
certain cues also does not correspond to the "playback" of a typical recording
device. The retrieval of stored information is a fallible, probabilistic process that is
more inferential and reconstructive than literal. . . . The information in our long-term
memories that is, and is not, accessible at a given point in time is heavily dependent on
the cues available to us, not only on cues that explicitly guide the search for the
information in question, but also on environmental interpersonal, mood-state, and
body-state cues.
In years past, I created cases on the computer that outlined financial contracting
situations. For example, a typical case would entail an interest rate and/or foreign
currency risk exposure that a company hedged with circus swaps (combinations of interest
rate and currency swap financial instruments derivatives contracts). In this virtual world
of contracting, I then gave my students links to my own learning materials and Internet
links that provided the information needed to assess and account for risks. They had to
find the solutions and then be prepared to explain those solutions.
After pondering Bjork (1994)
and related references on metacognition, I decided that my students were too dependent
upon virtual worlds (cases) that I created and solutions that I provided in the course
materials. Now I make my students write their own cases and propose their own solutions.
Extra credit is given for attacking issues for with there are no known solutions. Credit
is given to finding sources that I do not provide and, in many instances, have never
encountered. Extra credit is given for searches outside the fields of accounting and
finance. My goal is to multiply to encode the learning and retrieval process. Bjork (1994, p. 189) writes the
following:
On the encoding side, we would like the learner to achieve, for
lack of a better word, an understanding of the knowledge in question, defined as an
encoding that is part of a broader framework of interrelated concepts and ideas. Critical
information needs to be multiply encoded, not bound to single sets of semantic or
situational cues. . . . Similar to the argument for multiple encoding, it is also
desirable to induce successful access to knowledge and procedures in a variety of
situations that differ in cues they do and do not provide.
Whether learning contexts are defined as "cases" or "virtual
simulations" or "virtual games," the important point to consider in their
designs is that they may be too context specific and/or lack variety to the detriment of
long-term memory and performance. Our students may spend too much time mastering solutions
that we make available to them albeit in some form of treasure hunts. Our cues may be
too singular and too situational in the computer worlds that we present to our students.
The BAM Pedagogy at the University of
Virginia: Is BAM's
Success Due to Metacognition and Metamemory Phenomena?
Year 2004 Update
Congratulations
to Tony Catanach and Noel Barsky from Villanova University
Tony and Noel were
awarded the 2004 American Accounting Association Innovation in Accounting
Education Award in Orlando on August 11, 2004. The award is for their
development of a simulation model for teaching management accounting. The
model is called the Business Planning Model and is available with the textbook Management
Accounting , A Business Planning Approach (Houghton Mifflin) --- http://snipurl.com/CatanachBPM
Designed for use in introductory or graduate-level
managerial accounting courses, this text applies an objective-based approach
to managerial accounting topics. Unlike traditional cost-accounting texts, Management
Accounting emphasizes the critical role that information plays in decision
making, strategy execution, and overall enhancement of a firm's value. This
text meets the growing demand for an integrated, "survey of
business" approach to managerial accounting.
Through problem-based learning and the business
planning model (BPM), Management Accounting develops in students those
competencies expected of today's business professionals. This innovative
pedagogical approach stresses the understanding and application of the basic
business process; risk assessment and its relation to business strategy;
critical thinking, reasoning, and analysis; oral and written communication
skills; and techniques for team building. Real-world business problems and
simulations place students in the role of business consultant.
- Each chapter includes mini-cases, exercises, and
problems built around the challenges faced by managers today. An
integrated text-long continuing example of a fictitious service business, C&F
Enterprises, simulates the actual strategic planning process, as well
as resource management and performance measurement activities commonly
found in business practice.
- Business Beacons boxed features provide a
bridge from chapter concepts to contemporary business examples by
highlighting actual companies and linking managerial accounting topics to
real-world examples.
- Net Gains sections at the end of each
chapter direct students to Internet resources that enrich chapter
material. Students can use after-class time to link to tools that perform
many common accounting computations, while spending more in-class time on
discussion and development of analytical skills.
- Each chapter concludes with a Business Planning
Application, an optional module that allows students to engage in a
semester-long business planning simulation.
- A detailed and flexible teaching guide offers
instructors additional resources, tips, and tools for implementing the
innovative Management Accounting approach.
- A comprehensive appendix--Preparing and
Presenting the Business Plan--integrates and expands upon the business
planning applications and concepts from the text. It provides students
with a detailed guide on how to prepare and present a business plan
through exercises and additional resources.
The authors
presented a CPE workshop on the BPM in Orlando --- http://aaahq.org/AM2004/cpe/cpe27.htm
The Business
Planning Model is an innovative extension of the Business Activity Model
approach to teaching Intermediate Accounting without lectures (the students must
learn on their own). I wrote a paper about the BAM model at http://www.trinity.edu/rjensen/265wp.htm#UVA000
The BAM pedagogy was developed at the University of Virginia (when Tony was on
the faculty at Virginia) as one of the Accounting Education Change Commission
funded projects.
On August 20, 1997, at the Annual Meetings of the American Accounting Association in
Dallas, I videotaped an extraordinary presentation by Professors David Croll and Anthony
Catanach from the McIntire School of Commerce at the University of Virginia. The topic was
the Business Activity Model (BAM) revolutionary and award-winning revision of the
Intermediate Accounting sequence in which all lectures and textbook assignments were
replaced by a two-semester focus on accounting for a business from its inception through
seven years of operation. Development of the BAM case was initially funded with $50,000
from the Accounting Education Change Commission (AECC). The AECC was set up to inspire
innovation and allocate funds given by the largest eight accounting firms to foster change
in accounting education pedagogy. In 1997, the BAM experiment won the prestigious $5,000
Innovation in Accounting Education Award administered by the American Accounting
Association. The main innovation of the BAM pedagogy is that
students teach themselves in a discovery learning pedagogy. The main purpose
of BAM is make students constantly confront ambiguity. They
are assigned to teams, but teams are not graded. Although spreadsheets and some other
learning materials are on line, this is not a high technology ALN application with chat
rooms, etc.
In my viewpoint this paper also has relevance to the findings of another AECC-funded
experiment called the Project Discovery (PD) project at the University of Illinois at
Urbana-Champaign. That study took a slightly different approach across multiple
accounting courses to study the effects of the importance of having students learn from
"complex, ill-structured, ambiguous problems and cases similar to those found in
practice" as reported by Stone and Shelly (1997, p. 26). Stone and Shelly repeatedly
stress that their research notes the impact of such factors on learning but make no
attempt to attribute causality. I speculate that some of the causal factors are
metacognitive. Even though my paper focuses on the BAM Program at the University of
Virginia, I think my conclusions extend to the PD Program at the University of Illinois.
The BAM pedagogy is a "success" in terms of a number of important criteria.
Students on internships and recent graduates in full-time jobs report that the BAM
learning context is much more like "what really happens" on the job. In
particular, they feel better prepared for dealing with ambiguities
of real life. Secondly, long-term memory seems to be enhanced by the BAM pedagogy. More
than a year passes between completion of Intermediate Accounting and sitting for the
national CPA examination. Performance on the CPA examination improved markedly using the
BAM pedagogy. Without having the traditional lecture/drill pedagogy students remember
better with the BAM learn-it-yourself pedagogy. Instructors using the BAM pedagogy claim
students do better. They also claim that weaker students that tend to have more trouble
with examinations are given more opportunities to show that they know more than is
measured on examinations. In the BAM pedagogy, students make presentations in front of
fellow team members and in front of the entire class. Students have more realistic and
memorable interactions in the BAM pedagogy. David Croll relates the following:
. . . afterwards she felt better about it, and
participated a lot more and seemed to be much stronger at it. In student groups we have
finance majors as well as accounting majors in this course, and some of the finance majors
are great at speaking out. And so I get one young lady who's going off to work for a
Big-Six CPA firm, and she had three finance majors, three real talkers, in her group. And
on this accounting issue, the three men were adamant that they had this correct answer and
she was wrong. It turns out that she really had the right answer --- and I felt great! She
stuck with it. She argued them down.
And it went on for about ten minutes. We stood back and let her
fight it out with them. She took them on --- all three. And she's gonna have to do that in
her professional career. Go out there and say, this sounds good. This is the correct
accounting. And afterwards, I think she had a lot more confidence and interest in going
out and doing her job.
David Croll, University of Virginia
Transcribed from a videotape recording
Annual Meetings of the American Accounting Association
Dallas, Texas, August 20, 1997
The BAM pedagogy across two semesters is revolutionary accounting education.
Intermediate Accounting in virtually all colleges is a textbook driven set of two or three
courses aimed at mastering rules for accounting for financial transactions. Since business
contracting and transactions have become so complex in the past three decades, the
complexity and volume of the rules and practices have burgeoned. Intermediate Accounting
classes are traditionally synchronous in terms of lectures and/or case discussions.
Learning under the BAM pedagogy is asynchronous both inside and outside the classroom. More importantly, the BAM approach makes students search for answers on
their own or in teams such that "they teach themselves."
I made a CD-ROM of the BAM presentation and feature the BAM pedagogy and outcomes
assessments in my education technology
workshops that I conduct on college campuses around the world. After
stumbling belatedly upon the Bjork
(1994) paper (subsequent to giving several workshops featuring the BAM pedagogy), it
dawned on me that reasons why faculty at the University of Virginia feel that the BAM
pedagogy is a resounding success relative to traditional pedagogy lie in metacognitive and
metamemory phenomena. In this paper I will try to make the case that some reasons for the
success of the BAM pedagogy lie in metalevel processes in the brains of students.
Please keep in mind that the success of the BAM experiment may be heavily due to the
skills and dedication of the faculty and the high quality of the students participating in
the experiments. It is not at all certain that other faculty can carry this off at other
places and times. It is not clear that these same faculty (Croll and Catanach) can
maintain the intensity and motivation year after year that were present in the first four
years of experimentation. The BAM pedagogy takes students and faculty into many issues for
which there are no known solutions and requires that faculty and students tolerate far
more ambiguity throughout the entire year of learning. Ambiguity creates stress for students and faculty.
According to David Croll, ambiguity is the reason Robert
Grinaker (original BAM case author who is now emeritus), David Croll, and Anthony Catanach
proposed making such a dramatic change in pedagogy for Intermediate Accounting. Graduates
prior to the BAM pedagogy graduated with an air of confidence that they had mastered
accounting rules and pat solutions for the real world. In the first year on the job they
discovered that these pat answers seldom applied in practice. They were not prepared for
the ambiguities and complexities of real world careers. Many of them changed careers or
returned to law schools according to Professor Croll. The problem is pervasive and is in
no way unique to accounting graduates from the University of Virginia. The accounting
firms are not happy because of the turnover in new employees, and the new employees are
not happy because they were not prepared to deal with unresolved problems and ambiguities
of their work assignments. Furthermore, newly minted graduates prior to the BAM pedagogy
were not trained and educated to deal with ambiguities and introspective talents needed
for complex business transactions. These are the major reasons the largest firms attempted
to foster accounting education change with AECC grants to colleges and universities.
Hence, one goal of the BAM pedagogy is to deal with more ambiguous issues prior
to graduation. These are invariably viewed as harder hurdles for students. Without knowing
it, the BAM developers were in fact conforming to what Robert Bjork claims is most
important for long-term performance. Bjork (1994, pp. 189-193) called this point "The Need to
Introduce Difficulties for the Learner." In particular, Bjork stresses the need to
introduce "variation and/or unpredictability," "contextual
interference," distributing the learning of topics over extended intervals of time,
reducing feedback, "using tests as learning events," and being willing take
risks of lowered evaluations of instructors.
UVA010
The Business Activity Model at the University of Virginia:
The Need to Introduce Difficulties for the Learner
Whatever the exact mixture of manipulations that might turn out
to be optimal, however, one general characteristic of that mixture seems clear: It would
introduce many more difficulties and challenges for the learner . . . . Recent surveys of
the relevant research literature . . . leave no doubt that many of the most effective
manipulations of training --- in terms of post-training retention and transfer --- share
the property that they introduce difficulties for the learner.
Bjork (1994, p. 189)
Metacognition: Knowing about knowing
Now one of the problems is Bob wrote the case
without thinking about the accounting. If I ever wrote a case, I'd write a case about what
I knew. He wrote a case about a business he knew. And so there were times when he got us
into places that we didn't know how to get out of. And Bob would say, "well Dave,
what do you think about how we ought to book this?" And I'd say, "gee I don't
know. What do you think?" And so we were where the two of us really didn't know how
to book it. And it was coming down the line the students were going to get up and gonna
ask for this. And my goodness, we were gonna to be in trouble. So Bob said, "here's
how I think," and he did it that way. And then he said, " the FASB has a hot
line. We'll call them up."
So he called them up. And he told them the
problem, and then there was a pause. And then the voice said, how are you guys gonna book
it? And Bob told them, and they said "sounds good to us." So that's how we book
it.
So what we're talking about is not easy It is extremely
difficult; Clearly it's difficult for the students. They're in there, but when we talk about ambiguity we really mean it. I mean, they're out there no matter how bright they are.
David Croll, University of Virginia
Transcribed from a videotape recording
Annual Meetings of the American Accounting Association
Dallas, Texas, August 20, 1997
UVA020
The Business Activity Model at the University of Virginia:
Distributing Practice on a Given Task
In general, compared to distributing practice sessions on a given
task over time, massing practice or study sessions on the to-be-learned procedures or
information produces better short-term performance or recall of that procedure or
information, but markedly inferior long-term performance or recall.
Bjork (1994, p. 190)
Metacognition: Knowing about knowing
With blocked scheduling, each task is learned separately, and learning one task is
completed before the trainee moves to another task. With random scheduling, the tasks are
intermixed during acquisition . . . .Blocked scheduling always produced a faster rate of
learning during acquisition, but, regardless of the type of text scheduling, random
scheduling during acquisition resulted in the best performance at retention testing.
Healy and Sinclair (1996, p. 531)
Memory: Handbook of perception and cognition
You've got to understand, this was very
innovative. We don't try to cover a topic in one sitting, or three days in a row. We've
got seven years of this case. We're going to cover taxes for seven years. We don't have to
teach all the tax chapter in the first year . . . we have to teach enough to get them
through with how to book a loss. And then the company does turn it around in about the
third year --- it's just a little early. But now they are going to have to do deal with
those tax issues. So it's like peeling an onion. We go deeper and deeper into these taxes.
And I had the pleasure one of the first years I taught it of having one of our very bright
students come up to me in about the fifth year and say, you guys are going to make us do
taxes all seven years aren't you?
And I got to smile and tell them, you're going to do taxes all of
your life. Yes we did, all seven years.
David Croll, University of Virginia
Transcribed from a videotape recording
Annual Meetings of the American Accounting Association
Dallas, Texas, August 20, 1997
At this point let me say that a lot of our students don't listen
to us about documenting their work and organizing their work. And so when they come back .
. . this is great, I love this part! I go back in September when they return from
internships when we introduce year 4. We don't go back and redo everything for them, and
they need all the stuff from the first three years of the case. And they didn't organize
it, and they're dead, ok? They learn very quickly by year 5 and 6 how to organize a set of
work papers to support the financial statements.
Anthony Catanach, University of Virginia
Transcribed from a videotape recording
Annual Meetings of the American Accounting Association
Dallas, Texas, August 20, 1997
Whereas traditional Intermediate Accounting courses take up one text book topic at a
time and then move on to the next topic over the entire two or three semester course
coverage, the BAM model takes up nearly all topics repeatedly across two semesters. In
each of the seven years of accounting for the business, the same problems arise each year
in adjusting for accruals, estimating bad debts, valuing inventories, filing corporate tax
returns, etc.
UVA030
The Business Activity Model at the University of Virginia:
Varying the Conditions of Practice
It has now been demonstrated in a variety of ways, and with a
variety of motor, verbal, and problem-solving tasks, that introducing variation and/or
unpredictability in the training environment causes difficulty for the learner but
enhances long-term performance --- particularly the ability to transfer training to novel
but related task environments.
Bjork (1994, p. 189)
Metacognition: Knowing about knowing
How's this thing delivered? Ok, and I forgot to
answer your other question. I'll answer the other one . . . how the students react? First
semester, remember, you're destroying their model, their learning model, with a sledge
hammer. Ok? And so for the first two thirds of the semester, cuz I'm Satan! I'm a little
bit worse than Dave --- my students sit there like this . . . great body language . . .
for two thirds of the semester. Year 1, it's like pulling teeth.
Year 2: they finally realize that it's not going away. Ok? And
that you mean it, and that they have to play, and they start playing. Now where the real, that
real, hook comes in is when they go away for internships --- that's the summer between
the Intermediate 1 and Intermediate 2 semesters --- cuz that's where we place internships
at Virginia. After their internships, students come back and say "this is what really
happens!" Afterwards, in Intermediate 2, they've been revitalized. And they've bought
into BAM. And after that it's no problem
.
Anthony Catanach, University of
Virginia
Transcribed from a videotape recording
Annual Meetings of the American Accounting Association
Dallas, Texas, August 20, 1997
UVA040
The Business Activity Model at the University of Virginia:
Providing Contextual Interference
In Mannes and Kintsch’s (1987) experiment, for example, subjects had
to learn the content of a technical article (on industrial uses of microbes) after having
first studied an outline that was either consistent with the organization of the article
or inconsistent with that organization (but provided the same information in either case).
The inconsistent condition impaired subjects’ verbatim recall and recognition of the
article’s content (compared to the consistent condition), but facilitated performance
on tests that required subjects to infer answers or solve problems based on their general
understanding of the article’s content.
Bjork (1994, p. 190)
Metacognition: Knowing about knowing
And so they're seeing old and new, and I think
that's a pretty powerful teaching tool as well. A student discovered a problem in our
case. And what had happened is this, we had been taking a tax deduction for write-offs to
the bad debt. One of our students noted this --- it happened in my class. It happened in
my class naturally, the new guy on the faculty. And he said, Dr. Catanach, we just got out
of Ms. Jones class and she said that you couldn't take a deduction for that until it was
actually written off, you couldn't take it, you know, because you are using the allowance
method. I'm like, yeah, --- that sounds good to me. And then I said: "Well . .
. we'll get back to this later on."
And so I ran into my colleagues --- David and I and Bob --- we
had this debriefing and they went, yeah. Yes, so how are we going to do this, how are we
going to fix this? And so we went back to the class the next day; we said, don't worry
about the bad debt tax write-off. That's our position. We took a really weak defensive
position, and this is like year 2 of the case when this student pointed this out. And so
we let the mistake lie until year 6 of the case in the second semester. And in year 6 what
we did is we injected an IRS audit which would allow the IRS to come in and find the tax
error in year 2, and then students had to deal with a correction of an error issue. We
didn't have that in the original case. So then we put correction of an error assignment in
there. And if that wasn't enough, we had a stock split as well
.
Anthony
Catanach, University of Virginia
Transcribed from a videotape recording
Annual Meetings of the American Accounting Association
Dallas, Texas, August 20, 1997
UVA050
The Business Activity Model at the University of Virginia:
Reducing Feedback to the Learner
. . . reducing the frequency of feedback makes life more
difficult for the learner during training, but can enhance post-training performance.
Bjork (1994, p. 191-192)
Metacognition: Knowing about knowing
. . . performance assessment during the acquisition phase of training is not a
reliable indicator of training efficiency and skill learning. Only post-training retention
and transfer testing, after an appropriate retention interval, can provide a true measure
of learning and performance and the effectiveness of any experimental manipulation
designed to enhance training efficiency.
Healy and Sinclair (1996, p. 530)
Memory: Handbook of perception and cognition
. . . . described a study by Schooler and Anderson (1990) who showed that reducing the
number of feedback trials during the learning of a programming language decreased
acquisition performance but facilitated retention performance . . .
Healy and Sinclair (1996, p. 530)
Memory: Handbook of perception and cognition
Professors Croll and Catanach made no mention of reduced feedback in Intermediate
Accounting before verus after adopting the BAM self-learning pedagogy. In both instances,
examinations were given at relatively frequent intervals and students receive feedback
about grade patterns to date in the course. Examinations were used to assess grades and
student study teams are not graded as teams. Also examination grading was not very
subjective on the types of examinations given in the courses. Frequent feedback and
relatively objective grading are probably major reasons why students did not give the
instructors low evaluations in the courses.
When attempts are made to improve long-term memory and performance by reducing feedback
frequency and more subjective grading near the end of each course, it is highly likely
that instructors will pay a heavy price in course evaluations by students. Also, keep in
mind that when students are taking multiple courses, they tend to concentrate on courses
more when tests are imminent. Frequency of examinations tends to increase attention given
to a course even though it may impair long-term performance where students must be graded
based upon final performance at the end of the course. It should be noted that if every
course based 100% of the course grade on the final examination and/or the end-of-term
project, it might unduly stress students at the end of the semester. There are of course
other alternatives. Examinations and project deadlines can be scheduled throughout the
course with the holding back of grading feedback until the entire set of things to be
graded are evaluated by course instructors. Students, however, despise delayed feedback
for a variety of reasons. Much remains to be researched on the tradeoff between feedback
in courses and long-term memory and performance. Another problem is that impacts of
feedback vary a great deal between students. Eric Jensen writes the following:
Does all this mean that external rewards are also good for the brain? The answer
is no. That’s because the brain’s internal reward system varies from one student
to the next. You’d never be able to have a fair system. How students respond depends
on genetics, their particular brain chemistry, and life experiences that have wired their
brains in a unique way. Rewards work as a complex system of neurotransmitters binding to
receptor sites on neurons. . . . Most teachers have found that the same external reward is
received differently by two different students.
E. Jensen (1998,
p. 65)
Teaching with the brain in mind
UVA060
The Business Activity Model at the University of Virginia:
Using Tests as Learning Events
And Landauer and Bjork (1978; see also Rea &
Modigliani, 1995) found that "expanding retrieval practice," in which
successive recall test are made progressively more difficult by increasing the time and
intervening events prior to each next test of some target information, facilitates
long-term recall substantially --- compared to the same number of tests administered at
constant (and easier) delays.
Bjork (1994, p. 192)
Metacognition: Knowing about knowing
Now, let me cover a couple things quickly here
and then I'll go back; one thing is this, I'm not really grading these students in terms
of their class participation and the group work. How I'm going to grade them is I'm going
to grade them the same way as I graded them before. I'm going to give them exams using old
CPA problems, but I changed some things slightly so that I agreed with the answers. But
for learning to learn, over two semesters, I'm going to give them six take home exams. I'm
fortunate at UVA --- we have an honor system, and it works. So I thought well, maybe some
schools won't have an honor system it may not work for them.
So I've tried take home exams in the summer school, where in
effect, they don't take it home they simply work on it in a three hour block while they
are in class. And I can watch them. And it works. The take home exams do not cover
material I've covered. They cover material we're going to cover. But it's fresh and new.
We give them the exam, they've got to go out, in some fashion, either with, well with
both, with the official pronouncements and with the intermediate text and find out the
answer to this. And sometimes to be nasty, we give them a problem having to do with a
forthcoming statement that hasn't been issued yet, so of course there's nothing in the
intermediate text so they have to read the pronouncement to figure out how to deal with
it.
David Croll, University of Virginia
Transcribed from a videotape recording
Annual Meetings of the American Accounting Association
Dallas, Texas, August 20, 1997
UVA070
The Business Activity Model at the University of Virginia:
Impact on Teaching Evaluations
The tendency for instructors to be pushed toward training
programs that maximize the performance or evaluative reaction of their trainees during
is exacerbated by certain institutional characteristics that are common in real-world
organizations. First, those responsible for training are often themselves evaluated in
terms of the performance and satisfaction of their trainees during the training, or at the
end of the training. Second, individuals with the day-to-day responsibility for training
often do not get a chance to observe the post-training performance of the people they have
trained; a trainee’s later successes and failures tend to occur in settings that are
far removed from the original training environment, and from the trainer himself or
herself. It is also rarely the case that systematic measurements of the post-training
on-the-job performance are even collected, let alone provided to a trainer as a guide . .
.
Bjork (1994, p. 193)
Metacognition: Knowing about knowing
. . . performance assessment during the acquisition phase of training is not a reliable
indicator of training efficiency and skill learning. Only post-training retention and
transfer testing, after an appropriate retention interval, can provide a true measure of
learning and performance and the effectiveness of any experimental manipulation designed
to enhance training efficiency.
Healy and Sinclair (1996, p. 530)
Memory: Handbook of perception and cognition
A pedagogy that forces students to teach themselves or each other while relying less on
instructors may lead to lower teaching evaluations than a pedagogy where the teacher lives
up to student expectations of what a teacher should be doing inside and outside the
classroom. The BAM experiment is an exception. Professors Croll and Catanach had high
evaluations under traditional and under BAM experiment where they eliminated lectures and
forced students to learn on their own. It is not clear just why their evaluations did not
plummet. A major factor is that students had frequent feedback about grades and seemed to
be pleased that they were actually learning. A second factor is that students were allowed
to seek help in learning from most any source and were assigned to teams that seemed to
aid in the learning process. During class periods, the instructors "hovered"
close enough to team deliberations and most likely did more teaching than they care to
admit (e.g., when students really got hung up and appeared to run into a blank wall or
proceeded down wrong pathways). Outside the classroom, the instructors were available for
help. Classes were relatively small (around 35 students) such that instructors could
provide help to individuals and groups of students. Croll and Catanach attribute part of
the success of their teaching evaluations to "working toward solutions
together."
So what we're talking about is not easy. It is extremely
difficult --- clearly it's difficult for the students. They're in there, but when we
talked about ambiguity we really meant it. I mean, they're in deep water no matter how
bright they are. They're out there swimming. Now that may be why we got good reviews ---
we were swimming with them, and we all got through this thing together.
David Croll, University of Virginia
Transcribed from a videotape recording
Annual Meetings of the American Accounting Association
Dallas, Texas, August 20, 1997
A recent message from a biology professor at Trinity University reads as
follows:
The July 24, 1998 Chronicle of Higher Education
has a very interesting Point of View by Paul Trout of Montana State. To quote from the
article:
Administrators can rebut that calumny by using evaluation forms that are more than
student-satisfaction surveys. Instead of asking students to rate the professor’s
"stimulation of interest," "concern for students," and impartiality in
grading" - categories that allow disgruntled students to make pinatas of their
professors - evaluation forms should ask whether the course was demanding, whether
performance standards were high,, whether the workload was challenging, whether the
grading was tough, whether the student learned a lot. Those kinds of questions make it a
little harder for students who resent a heavy workload or low grades to give spiteful
responses.
It would seem that the Trinity Course Evaluation form as a little of each sentiment:
satisfaction and "tough." When the current form was being designed, these two
"sentiments" where not on my mind. It might be curious to rewrite the evaluation
form from these two perspectives and see what we get.
The Trout article was about incivility in the classroom and what he called
"education lite." I found it an interesting piece.
Blystone in Texas
Robert V. Blystone, Ph.D. <RBLYSTON@Trinity.edu>
Professor of Biology
Trinity University
San Antonio, Texas 78212
210.736-7243 210.736-7229 FAX
UVA080
The Business Activity Model at the University of Virginia:
Lessons From the Dead Poets Society
The movie Dead Poets Society showed examples of why students recalled so much of
their learning. There were changes in location, circumstances, use of emotions, movement,
and novel classroom positions. We know that learners remember much more when the learning
is connected to a field trip, music, a disaster, a guest speaker, or a novel learning
location. Follow up with a discussion, journal writing, a project, or peer teaching.
E. Jensen (1998,
p. 110)
Teaching with the brain in mind
Just like in regular business, Jerry Loose (the
error-prone Controller of the company in the BAM case) doesn't write notes. You write his
notes for him; it's his notes. But you end up writing his notes for him to do his
{function--not sure of this word}. And so, after the third year is completed, what's done
is we've got correct balance sheets, correct income statements, correct cash flow
statements, correct notes. Now how I teach that on the third day is, the students come in,
in their groups and sit down. I'll take a group, I'll take the spokesperson for the group,
send them to the board and say; write up the balance sheet, you are there. Write up the
income statement, we're getting them to share, you get cash flow and you just sit there
and we'll talk about notes when I get there. And off they go.
Now again, chances are the person I picked wasn't the one that
wrote this. There is so much work in this. I mean, we are not talking about small case.
When Bob got into it, he got into it. And this is the case. It can't be done by any single
individual --- you have to all share in doing this thing. And then you have to help each
other work your way through it. And so the person that's up doing a presentation is
getting coached from the sidelines by possibly the person that wrote it. Put this here,
put that here, here's my sheet. Do this. And once they're all up, I have whomever I called
on explain it all. Again with normally, with short coaching, they can explain the
important parts and with help from their team they can get all the parts of it.
David Croll, University of Virginia
Transcribed from a videotape recording
Annual Meetings of the American Accounting Association
Dallas, Texas, August 20, 1997
UVA090
The Business Activity Model at the University of Virginia:
Find the Answers Where You May
Stated more broadly, the conditions of training need to be
constructed to reveal to the subject what knowledge and procedures are, and are not, truly
accessible under the types of conditions that can be expected to prevail in the
post-training environment. Some of the best ways to achieve that goal involve making life
seem more difficult for the learner.
Bjork (1994, p. 201)
Metacognition: Knowing about knowing
That's basically what we're doing. We're asking
peo