Metacognitive Concerns in Designs and Evaluations of Computer Aided Education and Training: 
Are We Misleading Ourselves About Measures of Success
?

Bob Jensen
Trinity University
715 Stadium Drive
San Antonio, TX 78212
Phone:  210-736-7347  Fax:  210-736-8134
email:  rjensen@trinity.edu

Request for a Favor: This document is the first of a sequence of research papers that are related to my Working Paper 255 and Working Paper 260.  All are extensions of  the Jensen and Sandlin online book.   The document that follows is a very rough draft.  I would appreciate any feedback that you can provide by email, phone, fax,  or letter.  I want to improve this paper for my scheduled future workshops.

Hline.jpg (568 bytes)

Table of Contents

Update Messages  

Introduction

Metamemory and Metacognition: The Metalevel Activities of the Brain

Making Learning More Easy, Fun and Collaborative: Are We Taking Things Too Far in Virtual Learning Worlds?

The BAM Pedagogy at the University of Virginia


Differences Between Traditional Group Learning and Cooperative Learning

Warnings: Suggestions for Future Research and Designs for Asynchronous Learning Networks

Conclusion

From Villanova University:  BAM for Managerial Accounting Education 

The Where's My Professor Game at Brigham Young University

Accounting Professors in Support of Online Testing That, Among Other Things, Reduces Cheating

Acknowledgement of Paula Hertel

Appendix 1
Why Aren't Stories Good Food?

Email Messages About Evaluation Criteria and Processes
 

Appendix 2
The Differences Between Traditional Group Learning and Cooperative Learning

Appendix 3
The Emperor's Naked as He Can Be

Appendix 4
Update Message from Robert Bjork

MIT Encyclopedia of the Cognitive Sciences (shown in a new window)

Appendix 5 "
Assessing the Impact of Instructional Technology on Student Achievement," by Lorraine Sherry, Shelley Billig, Daniel Jesse, and Deborah Watson-Acosta.

Appendix 6
February 2003 Updates on Accounting Education Pedagogy

Appendix 7
How the Brain Deals With Information Overload

Appendix 8
The Importance of Paying Attention for Longer Periods of Time

Appendix 9
How to Train the Aging Brain
 

Appendix 10
That Placebo Effect in Research: Dan Ariely on Tennis Shoes and Toilet Paper

Appendix 11
Computer Trained Yet Deeply Intuitive 

Appendix 12
College Degrees Without Instructors

Appendix 13 Updates

Yeah Right!
"Forget What You Know About Good Study Habits," by Guy Spier & Peter Rothman, Simoleon Sense, September 8, 2010 ---
http://www.simoleonsense.com/forget-what-you-know-about-good-study-habits/


Update Messages

Education is an admirable thing, but it is well to remember from time to time that nothing that is worth learning can be taught.
Oscar Wilde

"The Objective of Education is Learning, Not Teaching (audio version available)," University of Pennsylvania's Knowledge@Wharton, August 20, 2008 --- http://knowledge.wharton.upenn.edu/article.cfm;jsessionid=9a30b5674a8d333e4d18?articleid=2032

If everyone is thinking alike, then somebody isn't thinking.
George S. Patton

"The Secret to Learning Anything: Albert Einstein's Advice to His Son," by Maria Popova, Brain Pickings, June ---
http://www.brainpickings.org/index.php/2013/06/14/einstein-letter-to-son/


From Emory University
Study Skills Tip Sheets & Advice --- http://www.college.emory.edu/home/academic/learning/studyskillsconsultations/tips.html

Bob Jensen's threads on Tools and Tricks of the Trade ---
http://www.trinity.edu/rjensen/000aaa/thetools.htm


"Why Floundering Is Good Trying and failing leads to faster learning," by Annie Murphy Paul, Psychology Today, July 11, 2012 ---
http://www.psychologytoday.com/blog/how-be-brilliant/201207/why-floundering-is-good

Call it the “learning paradox”: the more you struggle and even fail while you’re trying to master new information, the better you’re likely to recall and apply that information later.

The learning paradox is at the heart of “productive failure,” a phenomenon identified by Manu Kapur, a researcher at the Learning Sciences Lab at the National Institute of Education of Singapore. Kapur points out that while the model adopted by many teachers and employers when introducing others to new knowledge — providing lots of structure and guidance early on, until the students or workers show that they can do it on their own — makes intuitive sense, it may not be the best way to promote learning. Rather, it’s better to let the neophytes wrestle with the material on their own for a while, refraining from giving them any assistance at the start. In a paper published earlier this year in the Journal of the Learning Sciences, Kapur and a co-author, Katerine Bielaczyc, applied the principle of productive failure to mathematical problem solving in three schools in Singapore.

With one group of students, the teacher provided strong “scaffolding” — instructional support — and feedback. With the teacher’s help, these pupils were able to find the answers to their set of problems. Meanwhile, a second group was directed to solve the same problems by collaborating with one another, absent any prompts from their instructor. These students weren’t able to complete the problems correctly. But in the course of trying to do so, they generated a lot of ideas about the nature of the problems and about what potential solutions would look like. And when the two groups were tested on what they’d learned, the second group “significantly outperformed” the first.

The apparent struggles of the floundering group have what Kapur calls a “hidden efficacy”: they lead people to understand the deep structure of problems, not simply their correct solutions. When these students encounter a new problem of the same type on a test, they’re able to transfer the knowledge they’ve gathered more effectively than those who were the passive recipients of someone else’s expertise.

In the real world, problems rarely come neatly packaged, so being able to discern their deep structure is key. But, Kapur notes, none of us like to fail, no matter how often Silicon Valley entrepreneurs praise the salutary effects of an idea that flops or a start-up that crashes and burns. So, he says, we need to “design for productive failure” by building it into the learning process. Kapur has identified three conditions that promote this kind of beneficial struggle. First, choose problems to work on that “challenge but do not frustrate.” Second, provide learners with opportunities to explain and elaborate on what they’re doing. Third, give learners the chance to compare and contrast good and bad solutions to the problems. And to those students and workers who protest this tough-love teaching style: you’ll thank me later.

Competency-Based Programs (where instructors do not assign the grades) Can Work Well But Do Not Always Work Well

A Research Report
"Competency-Based Degree Programs in the U.S. Postsecondary Credentials for Measurable Student Learning and Performance," Council on Adult and Experiential Learning," 2012 ---
http://www.cael.org/pdfs/2012_CompetencyBasedPrograms

Executive Summary
As our economy evolves, there is growing recognition of the importance of an educated workforce. A key challenge is how to help more people, particularly adults, succeed at the postsecondary level and earn degrees. However, promoting degree completion is not our only challenge. Today our higher education system is facing a crisis regarding its perceived quality. One model for improving quality is competency-based education, in which an institution clearly defines the specific competencies expected of its graduates. This paper examines the current state of competency-based postsecondary education in the U.S., profiling the various types of competency-based, or competency-focused, models that currently exist, the extent to which these programs assess for student competencies or learning outcomes, and the extent to which these programs operate outside of a credit-based system. These programs can help inform other institutions interested in developing a stronger focus on competencies, whether by demonstrating the possibilities of high quality programs or by facilitating the recognition of learning.

Jensen Comment
The good news is that competency-based grades virtually put an end to games played by students to influence their grades from their instructors. Instead they may be more demanding on their instructors to do a better job on content rather than being their buddies. Competency-based grading goes a long way to leveling the playing field.

However, a competency-based system can be dysfunctional to motivation and self-esteem. One of my old girl friends at the University of Denver was called in by her physical chemistry professor who made a deal with her. If she would change her major from chemistry he agreed to give her a C grade. I honestly think an F grade would've discouraged her to a point where she dropped out of college. Instead she changed to DU's nursing school and flourished with a 3.3 gpa. Purportedly she became an outstanding nurse in a long and very satisfying career that didn't require much aptitude for physical chemistry. For some reason she was better in organic chemistry.

I can't imagine teaching a case course in the Harvard Business School where the course grades are entirely based on a final examination that depends zero upon what the course instructor feels was "class participation." There's not much incentive to participate in class discussions if the those discussions impact some way upon grades and instructor evaluations (such as evaluations for graduate school and employment).

Much of what is learned in a course or an entire college curriculum cannot be measured in test grades and term paper grading (where the readers of the term papers are not the instructors).

In spite of all the worries about competency-based grading and student evaluations, there are circumstances where competency-based education inspires terrrific learning experiences.

Bob Jensen's threads on competency-based assessment ---
http://www.trinity.edu/rjensen/Assess.htm#ConceptKnowledge


Center for Research on Learning and Teaching --- http://www.engin.umich.edu/teaching/crltengin/researchscholarship/index.html


"More Faculty Members Adopt 'Student Centered' Teaching," Chronicle of Higher Education, October 18, 2009 ---
http://chronicle.com/article/Chart-More-Faculty-Members/48848/

Professors are warming to new methods of teaching and testing that experts say are more likely to engage students, a UCLA survey found last year. Below are percentages of faculty members who said they used these approaches in all or most of the courses they taught. Those trends may continue, UCLA says, as full professors retire. Assistant professors were much more likely, for example, to structure teaching around small groups of students, while full professors were more likely to lecture extensively.
  2005 2008
Selected teaching methods
Cooperative learning (small groups of students) 48% 59%
Using real-life problems* n/a 56%
Group projects 33% 36%
Multiple drafts of written work 25% 25%
Student evaluations of one another’s work 16% 24%
Reflective writing/journaling 18% 22%
Electronic quizzes with immediate feedback in class* n/a 7%
Extensive lecturing (not student-centered) 55% 46%
Selected examination methods
Short-answer exams 37% 46%
Term and research papers 35% 44%
Multiple-choice exams 32% 33%
Grading on a curve 19% 17%
* Not asked in the 2005 survey
Note: The figures are based on survey responses of 22,562 faculty members at 372 four-year colleges and universities nationwide. The survey was conducted in the fall and winter of 2007-8 and covered full-time faculty members who spent at least part of their time teaching undergraduates. The figures were statistically adjusted to represent the total population of full-time faculty members at four-year institutions. Percentages are rounded.
Source: "The American College Teacher: National Norms for the 2007-8 HERI Faculty Survey," University of California at Los Angeles Higher Education Research Institute

Downfall of Lecturing ---
http://www.trinity.edu/rjensen/assess.htm#DownfallOfLecturing

Bob Jensen's threads on higher education are at
http://www.trinity.edu/rjensen/HigherEdControversies.htm


The problem is that our students choose very bland, low nourishment diets in our modern day smorgasbord curricula. Their concern is with their grade averages rather than their education. And why not? Grades for students and turf for faculty have become the keys to the kingdom!
Bob Jensen

"Are Undergraduates Actually Learning Anything?" by Richard Arum and Josipa Roksa. Chronicle of Higher Education, January 18, 2011 ---
http://chronicle.com/article/Are-Undergraduates-Actually/125979/

Drawing on survey responses, transcript data, and results from the Collegiate Learning Assessment (a standardized test taken by students in their first semester and at the end of their second year), Richard Arum and Josipa Roksa concluded that a significant percentage of undergraduates are failing to develop the broad-based skills and knowledge they should be expected to master. Here is an excerpt from Academically Adrift: Limited Learning on College Campuses (University of Chicago Press), their new book based on those findings.

Continued in article

Our Compassless Colleges: What are students really not learning?
http://www.trinity.edu/rjensen/HigherEdControversies.htm#Berkowitz


"The Case Against Case Studies:  How Columbia's B-school is teaching MBAs to make decisions based on incomplete data," by Geoff Gloeckler, Business Week, January 24, 2008 --- http://www.businessweek.com/magazine/content/08_05/b4069066093267.htm?link_position=link1

Shortly after R. Glenn Hubbard took over as dean of Columbia Business School in 2004, he began hearing rumblings from executives about the quality of MBA graduates. They were undoubtedly smart but often unprepared to handle the most crucial of managerial responsibilities: quickly solving problems with less than perfect information. Among those wanting more from new hires is Henry Kravis, co-founder of the private equity firm Kohlberg Kravis Roberts. "I want to see MBAs who can jump in and make decisions, not jump in and learn to make decisions," he says.

Hubbard made his own executive decision. He devised a new twist on the case study—the teaching format invented by Harvard Business School almost a century ago and used by most B-schools. Hubbard's so-called decision brief offers less information about a situation than the case study, and it doesn't present the solution until students have grappled with the issues on their own. "We want our students to be used to dealing with incomplete data," Hubbard says. "They should be able to make decisions out of uncertainty."

Even Michael J. Roberts, the executive director of the Arthur Rock Center for Entrepreneurship at Harvard and author of more than 100 HBS case studies, acknowledges the potential benefits of Hubbard's approach, which was introduced to Columbia students last fall. "Framing problems and finding the data to analyze those problems is a skill that MBAs need and that the classic case doesn't fully exploit," Roberts says. Hubbard expects such endorsements, as well as those of companies, will encourage other business schools to make room one day for Columbia's decision briefs in their curriculums. Hubbard, at least initially, doesn't plan to sell the decision briefs but to use them to tap into faculty research.

Hubbard isn't giving up on the traditional case study altogether. As part of an initiative called CaseWorks, Columbia will produce cases designed to reflect contemporary issues (which other schools do already), while also creating decision briefs that do away with the Harvard formula (which no one else has done). To help guide the program, Hubbard has turned to two people familiar with the deficits of the old methods: Stephen P. Zeldes, who has been at Columbia for more than a decade and is now chairman of the economics department at the B-school, and former Harvard case writer Elizabeth Gordon.

TOO MUCH INFORMATION The stock case study presents a tidy narrative arc, with a protagonist and a clear story line. One of the more widely used HBS cases focuses on Intel's (INTC) former marketing vice-president, Pamela Pollace, as she decides whether Intel should extend the "Intel Inside" branding campaign to products other than computers. In 24 pages, students are provided with information on Intel and the history of microprocessors, as well as details about market share and segmentation. Pollace's major concern, they learn, is brand dilution; the potential reward is likely worth the risk. In effect, the students are guided along the decision-making process.

If this case were a Columbia decision brief, students might see a video interview in which Pollace describes the challenge. They would also be given a few documents on the background of the campaign itself—the same data a manager at the company would have, but no more. Then, students would discuss possible solutions. Afterward, the group would see a second video of Pollace explaining how she handled the issue before debating whether or not she made the right decision.

So far, Columbia has produced six briefs that take on of-the-moment business challenges: Among them is one that focuses on General Electric's (GE) business-process-outsourcing division in India. Given increased competition, the company needed to consider a bigger investment, as well as the possibility of serving non-GE customers. With just a little more information than that, students are asked to come up with various strategies. "The idea is to try to simulate what it will be like in a real workplace," says Gordon. "There is uncertainty, things aren't predigested, all the information won't be there."

The first field test for the new teaching technique will be this summer, when the MBAs head out to their internships. At Goldman Sachs (GS), which hires more Columbia interns than any other company, the co-head of campus recruiting, Janet Raiffa, hopes to encounter students who are more independent thinkers. As for Kravis, his firm doesn't employ summer interns.

In some ways the pedagogy proposed by Columbia is a shorter and cheaper version of the BAM approach first proposed by Catanach, Croll, and Grinacker. The BAM approach uses a year-long case and students can seek out data in virtually every way they will do it later on while on the job (including paying for data if necessary) --- See Below

Bob Jensen's threads on Learning at Research Schools Versus "Teaching Schools" Versus "Happiness" With a Side Track into Substance Abuse --- http://www.trinity.edu/rjensen/Theory01.htm#Happiness 


Learning Styles Sites

January 1, 2009 message from Pat Wyman [raisingsmarterchildren@gmail.com]

Hello Bob,

Happy New Year! Your name came up through a google alert, attached to my website and the complimentary learning styles inventory at http://www.howtolearn.com 

It is on your page, from the community at http://www.elearninglearning.com/learning-styles/microsoft/&query=www.howtolearn.com 

I want to thank you for this is and if there is any way I can contribute to your blog and yours to mine, articles, interviews, etc. I'd love to connect with you.

You're doing wonderful work!

Warmly,
Pat Wyman, M.A.

-- Pat Wyman Best selling author, Learning vs. Testing Co-Author,
Book Of The Year In the Medicine Category, The Official Autism 101 Manual
University Instructor of Continuing Education, California State University,
East Bay Founder,
http://www.HowToLearn.com  and http://wwwRaisingSmarterChildren.com 
Winner, James Patterson PageTurner Award Get your copy of Learning vs. Testing with complimentary materials at http://www.learningvstesting4.html

Get Tips For Raising A Smarter Child at http://www.RaisingSmarterChildren.com 

"There are two ways you can live your life - one as if nothing is a miracle, and the other as if everything is a miracle." Albert Einstein

Bob Jensen's threads on assessment and learning --- http://www.trinity.edu/rjensen/assess.htm

Bob Jensen's threads on metacognitive learning --- http://www.trinity.edu/rjensen/265wp.htm

Bob Jensen's threads on asynchronous learning --- http://www.trinity.edu/rjensen/255wp.htm

Bob Jensen's threads on education technology --- http://www.trinity.edu/rjensen/000aaa/0000start.htm

 


February 17, 2005 message from Bob Jensen

I call your attention to Page 4 of the Spring 2005 newsletter called “The Accounting Educator” from the Teaching and Curriculum Section of the American Accounting Association --- http://aaahq.org/TeachCurr/newsletters/index.htm 

The current Chair (Tomas Calderon) has a piece about “reflection” which is nice to reflect upon. There are abstracts of papers in other journals that relate to education, and an assortment of teaching cases.

Marinus Bouman has a nice piece entitled “Using Technology To Integrate Accounting Into The Business Curriculum.” Interestingly, the Sam M. Walton College of Business at the University of Arkansas no longer has courses in Principles of Accounting (or Marketing or Finance). You should read Bouman’s article to find out what took the place of these principles courses in a daring curriculum experiment.

Since I teach accounting theory, I found Bob Clusky’s paper “Where’s Accounting Theory) quite interesting. Even more than AIS, “Accounting Theory” is a phrase still in search of a definition.

Tim Fogerty has a piece on Distance Education. It is somewhat negative in tone, but Tim seems to sigh that the march forward is inevitable and the current boundries of education from one program or one institution will evaporate as students seek courses and modules from anywhere in the world. I might take issue with some of his conclusions such as testing and/or assessment, but this is not the time or the place for that. See http://www.trinity.edu/rjensen/assess.htm 

There is much, much more of interest in this 32 page newsletter. Go to http://aaahq.org/TeachCurr/newsletters/index.htm 

Page 4 describes a forum to be headed up by Tim Fogarty (Case Western) and Call for Papers in which the last paragraph reads as follows:

******************************* 
Issues in Accounting Education, in conjunction with the Teaching & Curriculum Section of the AAA has asked me to edit a dedicated forum with an expected publication date of Spring 2006. I would like to extend an opportunity to accounting educators to submit essays for this issue. Proposed pieces for inclusion should be 25 pages (double spaced) or less. Submissions will be peer-reviewed with an emphasis on clarity and strength of ideas. The deadline for the first drafts is March 1, 2005. There would also be an opportunity to discuss these ideas in a CPE session at the AAA meeting in San Francisco. 
*****************************

Bravo to Thomas, Tim, and other volunteers who are continuing the momentum of this essential section of the AAA! This is the lifeblood of why we are in this profession.

Bob Jensen

February 17, 2005 message from Bob Jensen

Note the following paragraph that I wrote in my previous message:

*********************
Marinus Bouman has a nice piece entitled “Using Technology To Integrate Accounting Into The Business Curriculum.”  Interestingly, the
Sam M. Walton College of Business at the University of Arkansas no longer has courses in Principles of Accounting (or Marketing or Finance).  You should read Bouman’s article to find out what took the place of these principles courses in a daring curriculum experiment.
*******************

 The simulation pedagogy used in the "Business Foundations" course at the Walton College seems to be quite related to the BAM pedagogy --- http://www.trinity.edu/rjensen/265wp.htm#UVA000

The more I think about this, the more I think that the Walton College as evolved, I assume quite independently, into something quite similar to what Jack Wilson (a physicist) pioneered at Rensselaer well over a decade ago.  Core courses (such as physics) in the general curriculum at Rensselaer were taken from the curriculum and replaced with technology-based “studio” learning.  The University of Arkansas is doing something similar in relying upon technology when taking the core courses, such as Principles of Accounting, out of business education. 

The following is taken from http://www.trinity.edu/~rjensen/245glosf.htm#Studio1

Studio classroom= An application of computer technology pioneered by Jack M. Wilson at Rensselaer Polytechnical Institute for replacing large lecture courses with students working in pairs in front of computer screens where they interactively tackle problems and issues rather than listen to or passively watch lectures in front of a mass lecture section. The only lecture comes at the beginning and end of class where the instructor commences or wraps up the learning session. The "studio" is a combination lab and electronic classroom. 

Dr. Wilson now serves as the President of the University of Massachusetts system.  He had been serving as the Vice President for Academic Affairs of the University of Massachusetts System and is the founding Chief Executive Officer (CEO) of UMassOnline,  the University of Massachusetts Virtual University.  As Vice president he was responsible for the coordination of the academic programs in research and teaching throughout the five campus system. As CEO of UMassOnline he worked with the five physical campuses, Amherst , Lowell , Boston , Worcester , and Dartmouth to provide online access to the programs of the University of Massachusetts

Jack Wilson was one of the early pioneers in education technologies and learning.  He is now CEO and founder of UMass Online .

Dr. Wilson, also known as an entrepreneur, was the Founder (along with Degerhan Usluel and Mark Bernstein), first President, and only Chairman of LearnLinc Corporation (now Mentergy), a supplier of software systems for corporate training to Fortune 1000 Corporations.  In early 2000. LearnLinc merged with Gilat Communications, (GICOF) which also acquired Allen Communication from the Times Mirror group.  The Gilat-Allen-LearnLinc combination forms a powerful "one stop shopping" resource for E*Learning that is now the Mentergy unit of Gilat Communications.  (The LearnLinc Story).

Dr Wilson was the J. Erik Jonsson '22 Distinguished Professor of Physics, Engineering Science, Information Technology, and Management and the Co-director of the Severino Center for Technological Entrepreneurship at Rensselaer .  After coming to Rensselaer in 1990, he served as the 

·    Dean of Undergraduate Education, 

·    Dean of Professional and Continuing Education, 

·    Interim Provost, 

·    Interim Dean of Faculty, and as the 

·    Founding Director of the Anderson Center for Innovation in Undergraduate Education.  

In these roles, Wilson led a campus wide process of interactive learning and restructuring of the educational program, known for the design of the Studio Classrooms, the growth of the Distributed Learning Program, the creation of the Faculty of Information Technology, and the initiation of the student mobile computing (universal networked laptop) initiative

The Studio Classrooms at Rensselaer replaced large sized core courses taught by traditional lecture pedagogy with student teams responsible largely for teaching themselves using computer-aided and interactive course materials --- http://www.rpi.edu/dept/NewsComm/WNCTW/ad7.html 

Welcome To Interactive Learning
Roll up your sleeves and take a seat in the
Rensselaer studio classroom. Classes of about 60 students are engaged at wired workstations - utilizing cutting edge tools like Web-based technologies, full-motion video, computer simulation, and other laboratory resources. An instructor and teaching assistant move from workstation to workstation observing and coaching. Notes are taken with a simple mouse click, as students download files and class materials onto their required laptops. It's an innovative blend of discussion and skill-building, high-tech inquiry and problem-solving - preparing scholars to succeed in the new business world. It's all part of Interactive Learning at Rensselaer .

More Studios Than Hollywood
Interactive Learning is more than just a concept at
Rensselaer ; it's a working reality. The approach has been infused throughout all of our undergraduate disciplines in more than 25 studio classrooms with more being built all the time. In the LITEC studio classroom, students build remote-controlled cars in a project-based, team environment. In the Circuits Studio, students develop and test their own circuits. The Collaborative Classroom, funded by the National Science Foundation, serves as a testbed for using computer technology to collaborate on design projects. At Rensselaer , knowledge and application are seamlessly intertwined.

Teaching How We Teach
Rensselaer 's revolutionary model for education has been talked about, honored, and emulated. We earned the first Pew Charitable Trust Award for the Renewal of Undergraduate Education and the first Boeing Outstanding Educator Award, among others. Last year, we were named to administer an $8.8 million Pew-funded program to bring educational innovation to other universities in this country: The Center for Academic Transformation. Literally hundreds of institutions have visited Rensselaer to learn how we teach.

No Stopping Now
Of course, the very thinking that enabled
Rensselaer to initiate Interactive Learning is the same mindset that keeps us pressing forward. Rensselaer 's Anderson Center for Innovation in Undergraduate Education was founded 11 years ago with the continuing mission of making Rensselaer a leader in innovative pedagogy. More recently, the Rensselaer Academy of Electronic Media has become the spawning ground for highly creative visualization software that enables students to learn scientific and engineering principles in ways never before possible. We continue to look for new and better methods to evolve education - meeting the present and future needs of our students, professors, and global businesses. Because solving real-world challenges is our mission and our passion.

For a summary short summary see http://www.bc.edu/bc_org/avp/soe/cihe/newsletter/News15/text4.html . (See also Electronic classroom)

Random Thoughts (about learning from a retired professor of engineering) ---  http://www4.ncsu.edu/unity/lockers/users/f/felder/public/Columns.html

Dr. Felder's column in Chemical Engineering Education

Focus is heavily upon active learning and group learning.

Bob Jensen's threads on learning are in the following links:

http://www.trinity.edu/rjensen/assess.htm

http://www.trinity.edu/rjensen/255wp.htm

http://www.trinity.edu/rjensen/265wp.htm

 

Bob Jensen's Other Documents Asynchronous Learning References Table of Contents

Hline.jpg (568 bytes)

 

Metacognitive Concerns in Designs and Evaluations
of Computer Aided Education and Training: Are We
Misleading Ourselves About Measures of Success?

I've gotten to the point where I really don't lecture anymore, so if I look awkward up here talking to you, it's because I haven't lectured in four years. And the second point is that I've lost total faith in lecturing. Try telling your students something important in class, and then finding out how many heard it to where they can remember it the next day.

David Croll, University of Virginia
Transcribed from a videotape recording
Annual Meetings of the American Accounting Association
Dallas, Texas, August 20, 1997

Caveat: . I am grateful to Professors Croll and Catanach for allowing me to videotape their inspiring presentation. The quotations from Professors Croll and Catanach that appear at various points in this document have never been edited by those professors or modified from a transcript of a presentation that I videotaped at a conference. My videotape was transcribed by my secretary, Debbie Bowling. The transcription was modified by me only when Debbie failed to understand certain terminology.  I prefer to minimize changes in the transcription so that what is read remains as close as possible to what the audience listened to at the conference. None of us speak with the formalized vocabulary and grammar used in our writing. Also we cannot edit what we said in the same manner that we can edit what we wrote. Hence, transcriptions should not be judged as writing.   In August 1998, Tony Catanach moved to Villanova University in Villanova, Pennsylvania.

 

Introduction

R. Jensen (1998a) documents the exponential increase in asynchronous learning where students take on greater responsibility for teaching themselves and set their own learning paces. Evidence is mounting that student performance thereby improves. Some of our finest and oldest universities in the world are experimentally replacing lectures with asynchronous learning networks where students no longer attend scheduled classes. The early success of the Sloan Foundation funded experiments, where traditional lectures are replaced by asynchronous learning networks (ALNs), are making all institutions take notice even for full-time resident students. Examples of these successes and concerns about these successes are documented in R. Jensen (1998a). In addition, traditional (synchronous) lectures and cases are taking place in electronic classrooms where live Internet connections and presentation multimedia are replacing chalk and flip charts. Students in electronic classrooms or in front of their own laptops at home can be taken into virtual learning worlds and electronic chat rooms that make learning easier, faster, more collaborative, and more fun. Virtual worlds make learning more contextual. Biology students work with virtual organs and organisms, chemistry and physic students enter virtual laboratories, medical students diagnose and heal virtual patients, sociology students colonize virtual societies, finance students choose portfolios in virtual markets, etc.

Across many years and nations, educators have long known that interactive learning tends ceteris paribus to be better than passive learning. Computer and networking technologies make interactive learning more effective and efficient. Students having keyboard, mouse, joystick, microphone, web camera, and other input controls have greater powers of interaction with instructors, other students, libraries, experts, and objects around the world. Students not only can listen to Beethoven and Chopin, they can be transported back in time to the virtual lives and times of master composers. The great libraries of online documents, major university library holdings, and government archives are at their fingertips.

Jensen and Sandlin (1998, Chapter 2) list many advantages of newer learning technologies. R. Jensen (1998a) outlines various worries and concerns. The main purposes of this paper are as follows:

 


"The Great Debate: Effectiveness of Technology in Education," by Patricia Deubel, T.H.E. Journal, November 2007 ---
http://www.thejournal.com/articles/21544

According to Robert Kuhn (2000), an expert in brain research, few people understand the complexity of that change. Technology is creating new thinking that is "at once creative and innovative, volatile and turbulent" and "nothing less than a shift in worldview." The change in mental process has been brought about because "(1) information is freely available, and therefore interdisciplinary ideas and cross-cultural communication are widely accessible; (2) time is compressed, and therefore reflection is condensed and decision-making is compacted; (3) individuals are empowered, and therefore private choice and reach are strengthened and one person can have the presence of an institution" (sec: Concluding Remarks).

If we consider thinking as both individual (internal) and social (external), as Rupert Wegerif (2000) suggests, then "[t]echnology, in various forms from language to the internet, carries the external form of thinking. Technology therefore has a role to play through supporting improved social thinking (e.g. providing systems to mediate decision making and collective reasoning) and also through providing tools to help individuals externalize their thinking and so to shape their own social worlds" (p. 15).

The new tools for communication that have become part of the 21st century no doubt contribute to thinking. Thus, in a debate on effectiveness or on implementation of a particular tool, we must also consider the potential for creativity, innovation, volatility, and turbulence that Kuhn (2000) indicates.

Continued in article

Bob Jensen's threads on education technology are at http://www.trinity.edu/rjensen/000aaa/0000start.htm

Bob Jensen's threads on assessment are at http://www.trinity.edu/rjensen/assess.htm


Bob Jensen's Other Documents Asynchronous Learning References Table of Contents

 

Metamemory and Metacognition: The Metalevel Activities of the Brain

The brain stores long-term memory in the outer cerebral cortex with complex recollection systems among memory patterns of billions of neurons. Most things stored therein cannot be instantly recalled, but ways in which knowledge becomes embedded and patterns of use over time affect if and when recall takes place. Much depends upon associations between stored knowledge. For example, I probably would never recall the name of Don Quixote’s horse had this not been a memorable question (for me) on a 1961 GRE examination. That trivia question "sticks" in my mind all these years because of the humor of being asked about the only thing I remembered from that book (last read while I was a high school farm boy who liked horses).

There are also important distinctions between visual versus verbal memory, and these distinctions are crucial in designs of asynchronous learning materials. In fact, virtually all of Howard Gardner's theories on seven types of intelligence are important to take into account in the design of learning materials. These are described as follows by The Gardner School at http://www.swopnet.com/ed/TAG/7_Intelligences.html

:

1. Linguistic

Children with this kind of intelligence enjoy writing, reading, telling stories or doing crossword puzzles.

2. Logical-Mathematical

Children with lots of logical intelligence are interested in patterns, categories and relationships. They are drawn to arithmetic problems, strategy games and experiments.

3. Bodily-kinesthetic

These kids process knowledge through bodily sensations. They are often athletic, dancers or good at crafts such as sewing or woodworking.

4. Spatial

These children think in images and pictures. They may be fascinated with mazes or jigsaw puzzles, or spend free time drawing, building with Legos or daydreaming.

5. Musical

Musical children are always singing or drumming to themselves. They are usually quite aware of sounds others may miss. These kids are often discriminating listeners.

6. Interpersonal

Children who are leaders among their peers, who are good at communicating and who seem to understand others’ feelings and motives possess interpersonal intelligence.

7. Intrapersonal

These children may be shy. They are very aware of their own feelings and are self-motivated.

However, designing multimedia with the above types of intelligence in mind does not in and of itself aid to metacognition and metamemory. The mysterious process of "deciding" (it’s an unconscious process) what gets stored versus what gets lost or discarded takes place in a deeply embedded part of the brain known as the hippocampus. That organ is analogous to a computer’s file manager when it stimulates neurons to form an episodic network of neurons. Two important criteria affecting this process are emotional significance and association significance (associations with episodes already embedded in neuron structures). Information overload of the brain is not so much an issue of the number of neurons as it is with the input rate and complexity of processing incoming perception vis-à-vis emotions and episodic associations. Short-term memory "junk" such as a seldom-used phone number never clutters up our long-term memory episodic networks. Rote memorization of the phone number may last for a day but is not likely to last for a decade or even a week after it is used. When the hippocampus is damaged, a person might never store incoming knowledge and live only with recordings made when the hippocampus functioned properly. Alzheimer’s disease attacks the hippocampus region of the brain. Aaron, Benjamin, and Bjork (1998, p. 55) define metamemory as follows:

There has been a surge of interest in metamemory --- the study of what people know and understand about their own memory and memorial processes. From a theoretical standpoint, there has been a particular effort to explain why certain metamnemonic measures, such as the feelings of knowing . . . are accurate or inaccurate under various conditions.

The term "cognition" refers to the entire process of knowing --- a process comprised of perception, concentration, memory, judgment, and awareness. A subset of cognition known as metacognition is set apart from most cognitive activities and takes place in the frontal lobes. Metacognition is little understood even though we do know that it is impacted by episodic association formations in the hippocampus and that our well-developed metacognition is what sets human beings apart from other species. Psychologists refer metacognition as the monitoring process superimposed upon the control process of cognition. This monitoring can alter the control processes. Metacognition gives rise to what is commonly referred to as "Feeling of Knowing" or FOK judgments in cognition literature. It also gives rise to what is known as "novelty" versus "familiarity" awareness of perceptions.

Thiamine deficiency in the frontal lobes often accompanies severe alcohol abuse and is known in medicine as the Korsakoff Syndrome. Korsakoff amnesia thereby differs from amnesia that does not so dramatically affect the frontal lobes. Shimamura and Squire (1986) report that patients diagnosed with Korsakoff Syndrome have impaired FOK judgments. The theory is that frontal lobes processes modulate the hippocampus processes of forming episodic associations. Cognitive scientists make a distinction between basic memory and metamemory. Metamemory helps to make distinctions between stored memories. Weingradt, Leonesio, and Loftus (1994) note that a witness to an automobile accident may store perceptions at the scene of the accident and knowledge from what he or she read about or heard subsequent to the event. On Page 183 they elaborate as follows:

Consider a typical situation in which an eyewitness is asked to report what he or she has seen. Whether the witness is asked to identify a perpetrator, describe the events leading up to an automobile accident, or discriminate between information actually witnessed in an event or read about in a newspaper article, he or she is required to make judgments about information in memory. In metacognitive terms then, eyewitnesses are often required to make metamemory monitoring judgments. As should be evident from our discussion of the misinformation effect as a metacognitive error, the literature on metamemory monitoring can provide valuable insights into the reasons for poor eyewitness memory performance.

These metalevel processes enable humans to reflect upon thoughts and behaviors. Nelson and Narens (1994) refer to "metalevel" processes that are set apart from the lower-order "object-level" memory and cognition processes. Metalevel processes monitor perceptions to assess their novelty/familiarity and to direct attentiveness and feelings. Janet Metcalfe (1996, pp. 381-382) quotes a passage from Plato in which Socrates refers to an "inner voice." She then goes on to write the following passage:

Socrates' inner voice is the earliest well-known example of a self-reflective consciousness --- an internal monitor and control system. A self-monitoring consciousness figures in the musings of Descartes, emerging as "the thinker" whose existence cannot be doubted, even though all the content of the thought is open to question."

Later on Page 382 she continues as follows:

An accumulating body of evidence points to the prefrontal cortex as a brain region of critical importance for these [metalevel] functions. This region is distinct from those areas responsible for the knowledge and operations upon that knowledge that are the object of reflection, though, of course, they are interconnected. As Nelson and Narens (1994) have pointed out, object-level functions of cognition, such as memory storage, retrieval, and applying the operations of problem solving, may be separable from metalevel functions that monitor and control the object-level functions. In the normal person these two levels, cognition and metacognition, interact in a complex manner. This interaction is critical not only for problem solving, planning, and for memory, but arguably for sensitive assessments of one’s own appropriate social behavior.

Metacognition is essential to what earlier writers have called introspection. Nelson and Lorens (1994, p. 17) quote William James in 1890 as follows:

Introspective Observation is what we have to rely on first and foremost and always. . . . I regard this belief as the most fundamental of all postulates of Psychology.

Object-level versus metalevel distinctions become important to educators for self-evident reasons. The implications are immense. For example, it may suffice to make it as easy as possible (e.g., with visual aids) for students to memorize multiplication tables, but we are not so certain that it is best to make it as easy as possible to understand number theory theorems. Quite possibly students lose introspective powers if they do not re-discover some aspects of number theory on their own rather than start at the top of the heap of known theory. Some aids to making it easier to understand the theorems may be detrimental to long-term memory and performance.

 

Bob Jensen's Other Documents Asynchronous Learning References Table of Contents

 

Making Learning More Easy, Fun and Collaborative: Are We

Taking Things Too Far in Virtual Learning Worlds?

I have been making hypermedia CD-ROMs for my students since 1992. Until recently, my goals have been to make learning more easy, fun, inspirational, realistic, collaborative, and efficient. I take my students into worlds of virtual financial contracting and reporting. When seeking answers, my students can watch and listen to world experts explain complex transactions (e.g., financial instruments derivatives contracts and mathematical models for analyzing investment risks) and issues and theories about accounting for those transactions. I have videotaped experts at numerous conferences and workshops. Then I digitized the experts’ audio/video excerpts onto CD-ROMs and organized the material for pedagogic ease and efficiency. I also transcribed hundreds of hours of audio into text that can be searched using key words and is presented in a variety of formats (including Jeopardy-type games that make learning more fun). Readers can view some of my course materials from links at my web site.

I also made things easier by obtaining rights to share commercial literature databases and CPA Examination review courses on CD-ROMs. For example, by using the Price-Waterhouse Researcher CD-ROM, the accounting and auditing standards of the major developed nations are literally at the tips of student fingers on a keyboard. I can make my assignments exceedingly tough and realistic. Students can find the answers using my CD-ROMs, network server files, the Internet, textbooks, and photocopy handouts. I created a learning world in which my students can quickly find answers and do not have to conduct those slow, frustrating, and serendipitous library searches. When computer coding became involved for students (e.g., to create Microsoft Access databases or to code JavaScript web documents), I wrote tutorials that made learning much easier.

My goal was to take students to heights that were not possible in my courses before computing and networking hypermedia technologies. By making the hunt for answers easier and fun and collaborative, I could cover more material and more difficult material. My courses were not necessarily easier since I added more material. However, my courses were more efficient since students learned faster and spend less time searching for answers and completing projects. Learning became easier and in some respects more fun since student could see and hear experts whose names appear in the literature.

My own research, however, is far more serendipitous than research than I require from my students. Once in a while I pull myself away from the Internet and wander amongst the shelved books in the library. I peruse tables of contents and take a glance at sections of books and journal contents that capture my eye (I like to think of it as metacognitive success in identifying something novel that is not familiar to me). An article that I actually stumbled upon in the manner described above has had a profound impact upon my teaching and research. That article is "Memory and Metamemory Considerations in the Training of Human Beings" by Bjork (1994). Dr. Bjork convinced me that I was setting some wrong goals when creating asynchronous learning materials for my courses. In addition, he convinced me that many of the things that I had written about education technology are wrong.

I do not go so far as to say that the goals of using new technologies for making learning more easy, fun, inspirational, collaborative, and efficient are wrong per se. I do, however, suspect that there are dangers in making these goals too sacrosanct in preparing learning materials for lectures and asynchronous learning outside the classroom. By way of illustration, let me focus on a passage from Bjork (1994, p. 197-198).

The process of accessing stored information given certain cues also does not correspond to the "playback" of a typical recording device. The retrieval of stored information is a fallible, probabilistic process that is more inferential and reconstructive than literal. . . . The information in our long-term memories that is, and is not, accessible at a given point in time is heavily dependent on the cues available to us, not only on cues that explicitly guide the search for the information in question, but also on environmental interpersonal, mood-state, and body-state cues.

In years past, I created cases on the computer that outlined financial contracting situations. For example, a typical case would entail an interest rate and/or foreign currency risk exposure that a company hedged with circus swaps (combinations of interest rate and currency swap financial instruments derivatives contracts). In this virtual world of contracting, I then gave my students links to my own learning materials and Internet links that provided the information needed to assess and account for risks. They had to find the solutions and then be prepared to explain those solutions.

After pondering Bjork (1994) and related references on metacognition, I decided that my students were too dependent upon virtual worlds (cases) that I created and solutions that I provided in the course materials. Now I make my students write their own cases and propose their own solutions. Extra credit is given for attacking issues for with there are no known solutions. Credit is given to finding sources that I do not provide and, in many instances, have never encountered. Extra credit is given for searches outside the fields of accounting and finance. My goal is to multiply to encode the learning and retrieval process.  Bjork (1994, p. 189) writes the following:

On the encoding side, we would like the learner to achieve, for lack of a better word, an understanding of the knowledge in question, defined as an encoding that is part of a broader framework of interrelated concepts and ideas. Critical information needs to be multiply encoded, not bound to single sets of semantic or situational cues. . . . Similar to the argument for multiple encoding, it is also desirable to induce successful access to knowledge and procedures in a variety of situations that differ in cues they do and do not provide.

Whether learning contexts are defined as "cases" or "virtual simulations" or "virtual games," the important point to consider in their designs is that they may be too context specific and/or lack variety to the detriment of long-term memory and performance. Our students may spend too much time mastering solutions that we make available to them albeit in some form of treasure hunts. Our cues may be too singular and too situational in the computer worlds that we present to our students.

 

 

The BAM Pedagogy at the University of Virginia: Is BAM's
Success Due to Metacognition and Metamemory Phenomena?

 


Year 2004 Update

Congratulations to Tony Catanach and Noel Barsky from Villanova University

Tony and Noel were awarded the 2004 American Accounting Association Innovation in Accounting Education Award in Orlando on August 11, 2004.  The award is for their development of a simulation model for teaching management accounting.  The model is called the Business Planning Model and is available with the textbook Management Accounting , A Business Planning Approach (Houghton Mifflin) --- http://snipurl.com/CatanachBPM 

Designed for use in introductory or graduate-level managerial accounting courses, this text applies an objective-based approach to managerial accounting topics. Unlike traditional cost-accounting texts, Management Accounting emphasizes the critical role that information plays in decision making, strategy execution, and overall enhancement of a firm's value. This text meets the growing demand for an integrated, "survey of business" approach to managerial accounting.

Through problem-based learning and the business planning model (BPM), Management Accounting develops in students those competencies expected of today's business professionals. This innovative pedagogical approach stresses the understanding and application of the basic business process; risk assessment and its relation to business strategy; critical thinking, reasoning, and analysis; oral and written communication skills; and techniques for team building. Real-world business problems and simulations place students in the role of business consultant.

  • Each chapter includes mini-cases, exercises, and problems built around the challenges faced by managers today. An integrated text-long continuing example of a fictitious service business, C&F Enterprises, simulates the actual strategic planning process, as well as resource management and performance measurement activities commonly found in business practice.
  • Business Beacons boxed features provide a bridge from chapter concepts to contemporary business examples by highlighting actual companies and linking managerial accounting topics to real-world examples.
  • Net Gains sections at the end of each chapter direct students to Internet resources that enrich chapter material. Students can use after-class time to link to tools that perform many common accounting computations, while spending more in-class time on discussion and development of analytical skills.
  • Each chapter concludes with a Business Planning Application, an optional module that allows students to engage in a semester-long business planning simulation.
  • A detailed and flexible teaching guide offers instructors additional resources, tips, and tools for implementing the innovative Management Accounting approach.
  • A comprehensive appendix--Preparing and Presenting the Business Plan--integrates and expands upon the business planning applications and concepts from the text. It provides students with a detailed guide on how to prepare and present a business plan through exercises and additional resources.

The authors presented a CPE workshop on the BPM in Orlando --- http://aaahq.org/AM2004/cpe/cpe27.htm 

The Business Planning Model is an innovative extension of the Business Activity Model approach to teaching Intermediate Accounting without lectures (the students must learn on their own).  I wrote a paper about the BAM model at http://www.trinity.edu/rjensen/265wp.htm#UVA000 
The BAM pedagogy was developed at the University of Virginia (when Tony was on the faculty at Virginia) as one of the Accounting Education Change Commission funded projects.


On August 20, 1997, at the Annual Meetings of the American Accounting Association in Dallas, I videotaped an extraordinary presentation by Professors David Croll and Anthony Catanach from the McIntire School of Commerce at the University of Virginia. The topic was the Business Activity Model (BAM) revolutionary and award-winning revision of the Intermediate Accounting sequence in which all lectures and textbook assignments were replaced by a two-semester focus on accounting for a business from its inception through seven years of operation. Development of the BAM case was initially funded with $50,000 from the Accounting Education Change Commission (AECC). The AECC was set up to inspire innovation and allocate funds given by the largest eight accounting firms to foster change in accounting education pedagogy. In 1997, the BAM experiment won the prestigious $5,000 Innovation in Accounting Education Award administered by the American Accounting Association. The main innovation of the BAM pedagogy is that students teach themselves in a discovery learning pedagogy.  The main purpose of BAM is make students constantly confront ambiguity. They are assigned to teams, but teams are not graded. Although spreadsheets and some other learning materials are on line, this is not a high technology ALN application with chat rooms, etc.

In my viewpoint this paper also has relevance to the findings of another AECC-funded experiment called the Project Discovery (PD) project at the University of Illinois at Urbana-Champaign.  That study took a slightly different approach across multiple accounting courses to study the effects of the importance of having students learn from "complex, ill-structured, ambiguous problems and cases similar to those found in practice" as reported by Stone and Shelly (1997, p. 26).  Stone and Shelly repeatedly stress that their research notes the impact of such factors on learning but make no attempt to attribute causality.  I speculate that some of the causal factors are metacognitive.  Even though my paper focuses on the BAM Program at the University of Virginia, I think my conclusions extend to the PD Program at the University of Illinois.

The BAM pedagogy is a "success" in terms of a number of important criteria. Students on internships and recent graduates in full-time jobs report that the BAM learning context is much more like "what really happens" on the job. In particular, they feel better prepared for dealing with ambiguities of real life. Secondly, long-term memory seems to be enhanced by the BAM pedagogy. More than a year passes between completion of Intermediate Accounting and sitting for the national CPA examination. Performance on the CPA examination improved markedly using the BAM pedagogy. Without having the traditional lecture/drill pedagogy students remember better with the BAM learn-it-yourself pedagogy. Instructors using the BAM pedagogy claim students do better. They also claim that weaker students that tend to have more trouble with examinations are given more opportunities to show that they know more than is measured on examinations. In the BAM pedagogy, students make presentations in front of fellow team members and in front of the entire class. Students have more realistic and memorable interactions in the BAM pedagogy. David Croll relates the following:

. . . afterwards she felt better about it, and participated a lot more and seemed to be much stronger at it. In student groups we have finance majors as well as accounting majors in this course, and some of the finance majors are great at speaking out. And so I get one young lady who's going off to work for a Big-Six CPA firm, and she had three finance majors, three real talkers, in her group. And on this accounting issue, the three men were adamant that they had this correct answer and she was wrong. It turns out that she really had the right answer --- and I felt great! She stuck with it. She argued them down.

And it went on for about ten minutes. We stood back and let her fight it out with them. She took them on --- all three. And she's gonna have to do that in her professional career. Go out there and say, this sounds good. This is the correct accounting. And afterwards, I think she had a lot more confidence and interest in going out and doing her job.

David Croll, University of Virginia
Transcribed from a videotape recording
Annual Meetings of the American Accounting Association
Dallas, Texas, August 20, 1997

 

The BAM pedagogy across two semesters is revolutionary accounting education. Intermediate Accounting in virtually all colleges is a textbook driven set of two or three courses aimed at mastering rules for accounting for financial transactions. Since business contracting and transactions have become so complex in the past three decades, the complexity and volume of the rules and practices have burgeoned. Intermediate Accounting classes are traditionally synchronous in terms of lectures and/or case discussions. Learning under the BAM pedagogy is asynchronous both inside and outside the classroom. More importantly, the BAM approach makes students search for answers on their own or in teams such that "they teach themselves."

I made a CD-ROM of the BAM presentation and feature the BAM pedagogy and outcomes assessments in my education technology workshops that I conduct on college campuses around the world. After stumbling belatedly upon the Bjork (1994) paper (subsequent to giving several workshops featuring the BAM pedagogy), it dawned on me that reasons why faculty at the University of Virginia feel that the BAM pedagogy is a resounding success relative to traditional pedagogy lie in metacognitive and metamemory phenomena. In this paper I will try to make the case that some reasons for the success of the BAM pedagogy lie in metalevel processes in the brains of students.

Please keep in mind that the success of the BAM experiment may be heavily due to the skills and dedication of the faculty and the high quality of the students participating in the experiments. It is not at all certain that other faculty can carry this off at other places and times. It is not clear that these same faculty (Croll and Catanach) can maintain the intensity and motivation year after year that were present in the first four years of experimentation. The BAM pedagogy takes students and faculty into many issues for which there are no known solutions and requires that faculty and students tolerate far more ambiguity throughout the entire year of learning. Ambiguity creates stress for students and faculty.

According to David Croll, ambiguity is the reason Robert Grinaker (original BAM case author who is now emeritus), David Croll, and Anthony Catanach proposed making such a dramatic change in pedagogy for Intermediate Accounting. Graduates prior to the BAM pedagogy graduated with an air of confidence that they had mastered accounting rules and pat solutions for the real world. In the first year on the job they discovered that these pat answers seldom applied in practice. They were not prepared for the ambiguities and complexities of real world careers. Many of them changed careers or returned to law schools according to Professor Croll. The problem is pervasive and is in no way unique to accounting graduates from the University of Virginia. The accounting firms are not happy because of the turnover in new employees, and the new employees are not happy because they were not prepared to deal with unresolved problems and ambiguities of their work assignments. Furthermore, newly minted graduates prior to the BAM pedagogy were not trained and educated to deal with ambiguities and introspective talents needed for complex business transactions. These are the major reasons the largest firms attempted to foster accounting education change with AECC grants to colleges and universities.

Hence,  one goal of the BAM pedagogy is to deal with more ambiguous issues prior to graduation. These are invariably viewed as harder hurdles for students. Without knowing it, the BAM developers were in fact conforming to what Robert Bjork claims is most important for long-term performance.  Bjork (1994, pp. 189-193) called this point "The Need to Introduce Difficulties for the Learner." In particular, Bjork stresses the need to introduce "variation and/or unpredictability," "contextual interference," distributing the learning of topics over extended intervals of time, reducing feedback, "using tests as learning events," and being willing take risks of lowered evaluations of instructors.

 

 

 Blue.gif (84 bytes)UVA010

The Business Activity Model at the University of Virginia:
The Need to Introduce Difficulties for the Learner

Whatever the exact mixture of manipulations that might turn out to be optimal, however, one general characteristic of that mixture seems clear: It would introduce many more difficulties and challenges for the learner . . . . Recent surveys of the relevant research literature . . . leave no doubt that many of the most effective manipulations of training --- in terms of post-training retention and transfer --- share the property that they introduce difficulties for the learner.

Bjork (1994, p. 189)
Metacognition: Knowing about knowing

 

Now one of the problems is Bob wrote the case without thinking about the accounting. If I ever wrote a case, I'd write a case about what I knew. He wrote a case about a business he knew. And so there were times when he got us into places that we didn't know how to get out of. And Bob would say, "well Dave, what do you think about how we ought to book this?" And I'd say, "gee I don't know. What do you think?" And so we were where the two of us really didn't know how to book it. And it was coming down the line the students were going to get up and gonna ask for this. And my goodness, we were gonna to be in trouble. So Bob said, "here's how I think," and he did it that way. And then he said, " the FASB has a hot line. We'll call them up."

So he called them up. And he told them the problem, and then there was a pause. And then the voice said, how are you guys gonna book it? And Bob told them, and they said "sounds good to us." So that's how we book it.

So what we're talking about is not easy It is extremely difficult; Clearly it's difficult for the students. They're in there, but when we talk about ambiguity we really mean it. I mean, they're out there no matter how bright they are.

David Croll, University of Virginia
Transcribed from a videotape recording
Annual Meetings of the American Accounting Association
Dallas, Texas, August 20, 1997

 

 

Blue.gif (84 bytes)UVA020

The Business Activity Model at the University of Virginia:
Distributing Practice on a Given Task

In general, compared to distributing practice sessions on a given task over time, massing practice or study sessions on the to-be-learned procedures or information produces better short-term performance or recall of that procedure or information, but markedly inferior long-term performance or recall.

Bjork (1994, p. 190)
Metacognition: Knowing about knowing

With blocked scheduling, each task is learned separately, and learning one task is completed before the trainee moves to another task. With random scheduling, the tasks are intermixed during acquisition . . . .Blocked scheduling always produced a faster rate of learning during acquisition, but, regardless of the type of text scheduling, random scheduling during acquisition resulted in the best performance at retention testing.

Healy and Sinclair (1996, p. 531)
Memory: Handbook of perception and cognition

 

You've got to understand, this was very innovative. We don't try to cover a topic in one sitting, or three days in a row. We've got seven years of this case. We're going to cover taxes for seven years. We don't have to teach all the tax chapter in the first year . . . we have to teach enough to get them through with how to book a loss. And then the company does turn it around in about the third year --- it's just a little early. But now they are going to have to do deal with those tax issues. So it's like peeling an onion. We go deeper and deeper into these taxes. And I had the pleasure one of the first years I taught it of having one of our very bright students come up to me in about the fifth year and say, you guys are going to make us do taxes all seven years aren't you?

And I got to smile and tell them, you're going to do taxes all of your life. Yes we did, all seven years.

David Croll, University of Virginia
Transcribed from a videotape recording
Annual Meetings of the American Accounting Association
Dallas, Texas, August 20, 1997

 

At this point let me say that a lot of our students don't listen to us about documenting their work and organizing their work. And so when they come back . . . this is great, I love this part! I go back in September when they return from internships when we introduce year 4. We don't go back and redo everything for them, and they need all the stuff from the first three years of the case. And they didn't organize it, and they're dead, ok? They learn very quickly by year 5 and 6 how to organize a set of work papers to support the financial statements.

Anthony Catanach, University of Virginia
Transcribed from a videotape recording
Annual Meetings of the American Accounting Association
Dallas, Texas, August 20, 1997

Whereas traditional Intermediate Accounting courses take up one text book topic at a time and then move on to the next topic over the entire two or three semester course coverage, the BAM model takes up nearly all topics repeatedly across two semesters. In each of the seven years of accounting for the business, the same problems arise each year in adjusting for accruals, estimating bad debts, valuing inventories, filing corporate tax returns, etc.

 

 

Blue.gif (84 bytes)UVA030

The Business Activity Model at the University of Virginia:
Varying the Conditions of Practice

It has now been demonstrated in a variety of ways, and with a variety of motor, verbal, and problem-solving tasks, that introducing variation and/or unpredictability in the training environment causes difficulty for the learner but enhances long-term performance --- particularly the ability to transfer training to novel but related task environments.

Bjork (1994, p. 189)
Metacognition: Knowing about knowing

 

How's this thing delivered? Ok, and I forgot to answer your other question. I'll answer the other one . . . how the students react? First semester, remember, you're destroying their model, their learning model, with a sledge hammer. Ok? And so for the first two thirds of the semester, cuz I'm Satan! I'm a little bit worse than Dave --- my students sit there like this . . . great body language . . . for two thirds of the semester. Year 1, it's like pulling teeth.

Year 2: they finally realize that it's not going away. Ok? And that you mean it, and that they have to play, and they start playing. Now where the real, that real, hook comes in is when they go away for internships --- that's the summer between the Intermediate 1 and Intermediate 2 semesters --- cuz that's where we place internships at Virginia. After their internships, students come back and say "this is what really happens!" Afterwards, in Intermediate 2, they've been revitalized. And they've bought into BAM. And after that it's no problem.

Anthony Catanach, University of Virginia
Transcribed from a videotape recording
Annual Meetings of the American Accounting Association
Dallas, Texas, August 20, 1997

 

 

Blue.gif (84 bytes)UVA040

The Business Activity Model at the University of Virginia:
Providing Contextual Interference

In Mannes and Kintsch’s (1987) experiment, for example, subjects had to learn the content of a technical article (on industrial uses of microbes) after having first studied an outline that was either consistent with the organization of the article or inconsistent with that organization (but provided the same information in either case). The inconsistent condition impaired subjects’ verbatim recall and recognition of the article’s content (compared to the consistent condition), but facilitated performance on tests that required subjects to infer answers or solve problems based on their general understanding of the article’s content.

Bjork (1994, p. 190)
Metacognition: Knowing about knowing

 

And so they're seeing old and new, and I think that's a pretty powerful teaching tool as well. A student discovered a problem in our case. And what had happened is this, we had been taking a tax deduction for write-offs to the bad debt. One of our students noted this --- it happened in my class. It happened in my class naturally, the new guy on the faculty. And he said, Dr. Catanach, we just got out of Ms. Jones class and she said that you couldn't take a deduction for that until it was actually written off, you couldn't take it, you know, because you are using the allowance method. I'm like, yeah, --- that sounds good to me. And then I said:  "Well . . . we'll get back to this later on."

And so I ran into my colleagues --- David and I and Bob --- we had this debriefing and they went, yeah. Yes, so how are we going to do this, how are we going to fix this? And so we went back to the class the next day; we said, don't worry about the bad debt tax write-off. That's our position. We took a really weak defensive position, and this is like year 2 of the case when this student pointed this out. And so we let the mistake lie until year 6 of the case in the second semester. And in year 6 what we did is we injected an IRS audit which would allow the IRS to come in and find the tax error in year 2, and then students had to deal with a correction of an error issue. We didn't have that in the original case. So then we put correction of an error assignment in there. And if that wasn't enough, we had a stock split as well.

Anthony Catanach, University of Virginia
Transcribed from a videotape recording
Annual Meetings of the American Accounting Association
Dallas, Texas, August 20, 1997

 

 

Blue.gif (84 bytes)UVA050

The Business Activity Model at the University of Virginia:
Reducing Feedback to the Learner

. . . reducing the frequency of feedback makes life more difficult for the learner during training, but can enhance post-training performance.

Bjork (1994, p. 191-192)
Metacognition: Knowing about knowing

. . . performance assessment during the acquisition phase of training is not a reliable indicator of training efficiency and skill learning. Only post-training retention and transfer testing, after an appropriate retention interval, can provide a true measure of learning and performance and the effectiveness of any experimental manipulation designed to enhance training efficiency.

Healy and Sinclair (1996, p. 530)
Memory: Handbook of perception and cognition

. . . . described a study by Schooler and Anderson (1990) who showed that reducing the number of feedback trials during the learning of a programming language decreased acquisition performance but facilitated retention performance . . .

Healy and Sinclair (1996, p. 530)
Memory: Handbook of perception and cognition

 

Professors Croll and Catanach made no mention of reduced feedback in Intermediate Accounting before verus after adopting the BAM self-learning pedagogy. In both instances, examinations were given at relatively frequent intervals and students receive feedback about grade patterns to date in the course. Examinations were used to assess grades and student study teams are not graded as teams. Also examination grading was not very subjective on the types of examinations given in the courses. Frequent feedback and relatively objective grading are probably major reasons why students did not give the instructors low evaluations in the courses.

When attempts are made to improve long-term memory and performance by reducing feedback frequency and more subjective grading near the end of each course, it is highly likely that instructors will pay a heavy price in course evaluations by students. Also, keep in mind that when students are taking multiple courses, they tend to concentrate on courses more when tests are imminent. Frequency of examinations tends to increase attention given to a course even though it may impair long-term performance where students must be graded based upon final performance at the end of the course. It should be noted that if every course based 100% of the course grade on the final examination and/or the end-of-term project, it might unduly stress students at the end of the semester. There are of course other alternatives. Examinations and project deadlines can be scheduled throughout the course with the holding back of grading feedback until the entire set of things to be graded are evaluated by course instructors. Students, however, despise delayed feedback for a variety of reasons. Much remains to be researched on the tradeoff between feedback in courses and long-term memory and performance. Another problem is that impacts of feedback vary a great deal between students. Eric Jensen writes the following:

Does all this mean that external rewards are also good for the brain? The answer is no. That’s because the brain’s internal reward system varies from one student to the next. You’d never be able to have a fair system. How students respond depends on genetics, their particular brain chemistry, and life experiences that have wired their brains in a unique way. Rewards work as a complex system of neurotransmitters binding to receptor sites on neurons. . . . Most teachers have found that the same external reward is received differently by two different students.

E. Jensen (1998, p. 65)
Teaching with the brain in mind

 

 

Blue.gif (84 bytes)UVA060

The Business Activity Model at the University of Virginia:
Using Tests as Learning Events

And Landauer and Bjork (1978; see also Rea & Modigliani, 1995) found that "expanding retrieval practice," in which successive recall test are made progressively more difficult by increasing the time and intervening events prior to each next test of some target information, facilitates long-term recall substantially --- compared to the same number of tests administered at constant (and easier) delays.

Bjork (1994, p. 192)
Metacognition: Knowing about knowing

 

Now, let me cover a couple things quickly here and then I'll go back; one thing is this, I'm not really grading these students in terms of their class participation and the group work. How I'm going to grade them is I'm going to grade them the same way as I graded them before. I'm going to give them exams using old CPA problems, but I changed some things slightly so that I agreed with the answers. But for learning to learn, over two semesters, I'm going to give them six take home exams. I'm fortunate at UVA --- we have an honor system, and it works. So I thought well, maybe some schools won't have an honor system it may not work for them.

So I've tried take home exams in the summer school, where in effect, they don't take it home they simply work on it in a three hour block while they are in class. And I can watch them. And it works. The take home exams do not cover material I've covered. They cover material we're going to cover. But it's fresh and new. We give them the exam, they've got to go out, in some fashion, either with, well with both, with the official pronouncements and with the intermediate text and find out the answer to this. And sometimes to be nasty, we give them a problem having to do with a forthcoming statement that hasn't been issued yet, so of course there's nothing in the intermediate text so they have to read the pronouncement to figure out how to deal with it.

David Croll, University of Virginia
Transcribed from a videotape recording
Annual Meetings of the American Accounting Association
Dallas, Texas, August 20, 1997

 

 

Blue.gif (84 bytes)UVA070

The Business Activity Model at the University of Virginia:
Impact on Teaching Evaluations

The tendency for instructors to be pushed toward training programs that maximize the performance or evaluative reaction of their trainees during is exacerbated by certain institutional characteristics that are common in real-world organizations. First, those responsible for training are often themselves evaluated in terms of the performance and satisfaction of their trainees during the training, or at the end of the training. Second, individuals with the day-to-day responsibility for training often do not get a chance to observe the post-training performance of the people they have trained; a trainee’s later successes and failures tend to occur in settings that are far removed from the original training environment, and from the trainer himself or herself. It is also rarely the case that systematic measurements of the post-training on-the-job performance are even collected, let alone provided to a trainer as a guide . . .

Bjork (1994, p. 193)
Metacognition: Knowing about knowing

 

. . . performance assessment during the acquisition phase of training is not a reliable indicator of training efficiency and skill learning. Only post-training retention and transfer testing, after an appropriate retention interval, can provide a true measure of learning and performance and the effectiveness of any experimental manipulation designed to enhance training efficiency.

Healy and Sinclair (1996, p. 530)
Memory: Handbook of perception and cognition

 

A pedagogy that forces students to teach themselves or each other while relying less on instructors may lead to lower teaching evaluations than a pedagogy where the teacher lives up to student expectations of what a teacher should be doing inside and outside the classroom. The BAM experiment is an exception. Professors Croll and Catanach had high evaluations under traditional and under BAM experiment where they eliminated lectures and forced students to learn on their own. It is not clear just why their evaluations did not plummet. A major factor is that students had frequent feedback about grades and seemed to be pleased that they were actually learning. A second factor is that students were allowed to seek help in learning from most any source and were assigned to teams that seemed to aid in the learning process. During class periods, the instructors "hovered" close enough to team deliberations and most likely did more teaching than they care to admit (e.g., when students really got hung up and appeared to run into a blank wall or proceeded down wrong pathways). Outside the classroom, the instructors were available for help. Classes were relatively small (around 35 students) such that instructors could provide help to individuals and groups of students. Croll and Catanach attribute part of the success of their teaching evaluations to "working toward solutions together."

So what we're talking about is not easy. It is extremely difficult --- clearly it's difficult for the students. They're in there, but when we talked about ambiguity we really meant it. I mean, they're in deep water no matter how bright they are. They're out there swimming. Now that may be why we got good reviews --- we were swimming with them, and we all got through this thing together.

David Croll, University of Virginia
Transcribed from a videotape recording
Annual Meetings of the American Accounting Association
Dallas, Texas, August 20, 1997

A recent message from a biology professor at Trinity University reads as follows:

The July 24, 1998 Chronicle of Higher Education has a very interesting Point of View by Paul Trout of Montana State. To quote from the article:

Administrators can rebut that calumny by using evaluation forms that are more than student-satisfaction surveys. Instead of asking students to rate the professor’s "stimulation of interest," "concern for students," and impartiality in grading" - categories that allow disgruntled students to make pinatas of their professors - evaluation forms should ask whether the course was demanding, whether performance standards were high,, whether the workload was challenging, whether the grading was tough, whether the student learned a lot. Those kinds of questions make it a little harder for students who resent a heavy workload or low grades to give spiteful responses.

It would seem that the Trinity Course Evaluation form as a little of each sentiment: satisfaction and "tough." When the current form was being designed, these two "sentiments" where not on my mind. It might be curious to rewrite the evaluation form from these two perspectives and see what we get.

The Trout article was about incivility in the classroom and what he called "education lite." I found it an interesting piece.

Blystone in Texas

Robert V. Blystone, Ph.D. <RBLYSTON@Trinity.edu>
Professor of Biology
Trinity University
San Antonio, Texas 78212
210.736-7243 210.736-7229 FAX

 

 

Blue.gif (84 bytes)UVA080

The Business Activity Model at the University of Virginia:
Lessons From the Dead Poets Society

The movie Dead Poets Society showed examples of why students recalled so much of their learning. There were changes in location, circumstances, use of emotions, movement, and novel classroom positions. We know that learners remember much more when the learning is connected to a field trip, music, a disaster, a guest speaker, or a novel learning location. Follow up with a discussion, journal writing, a project, or peer teaching.

E. Jensen (1998, p. 110)
Teaching with the brain in mind

 

Just like in regular business, Jerry Loose (the error-prone Controller of the company in the BAM case) doesn't write notes. You write his notes for him; it's his notes. But you end up writing his notes for him to do his {function--not sure of this word}. And so, after the third year is completed, what's done is we've got correct balance sheets, correct income statements, correct cash flow statements, correct notes. Now how I teach that on the third day is, the students come in, in their groups and sit down. I'll take a group, I'll take the spokesperson for the group, send them to the board and say; write up the balance sheet, you are there. Write up the income statement, we're getting them to share, you get cash flow and you just sit there and we'll talk about notes when I get there. And off they go.

Now again, chances are the person I picked wasn't the one that wrote this. There is so much work in this. I mean, we are not talking about small case. When Bob got into it, he got into it. And this is the case. It can't be done by any single individual --- you have to all share in doing this thing. And then you have to help each other work your way through it. And so the person that's up doing a presentation is getting coached from the sidelines by possibly the person that wrote it. Put this here, put that here, here's my sheet. Do this. And once they're all up, I have whomever I called on explain it all. Again with normally, with short coaching, they can explain the important parts and with help from their team they can get all the parts of it.

David Croll, University of Virginia
Transcribed from a videotape recording
Annual Meetings of the American Accounting Association
Dallas, Texas, August 20, 1997

 

 

Blue.gif (84 bytes)UVA090

The Business Activity Model at the University of Virginia:
Find the Answers Where You May

Stated more broadly, the conditions of training need to be constructed to reveal to the subject what knowledge and procedures are, and are not, truly accessible under the types of conditions that can be expected to prevail in the post-training environment. Some of the best ways to achieve that goal involve making life seem more difficult for the learner.

Bjork (1994, p. 201)
Metacognition: Knowing about knowing

 

That's basically what we're doing. We're asking people questions, they give us answers. Now, since it's a case, since we've been using it for years now, one of the constant questions I get has to do with the fact that they say, well, gee, my students would get the answers to this case and from the class before and it would be useless. Well, you don't understand quite what we're doing here. I don't care whether they get the answers from the class before them. In fact, we're telling these people, you're working as a team, you're partners. Get the answers from your partner. All the teams I have in class work for the same firm. Your group doesn't know the answer, ask another group. See, as an academic I can't get over this. I won't ask questions of anybody, it's cheating.

As an academic, for years I thought, gee, you're supposed to do it all yourself. I got out into business, the more successful people asked others; that knew this information. They picked up their phone, they didn't just make luncheon appointments. They actually asked people about things. Why don't we have our students do that? What am I doing then in class when we come to these questions?

David Croll, University of Virginia
Transcribed from a videotape recording
Annual Meetings of the American Accounting Association
Dallas, Texas, August 20, 1997

 

 

Blue.gif (84 bytes)UVA100

The Business Activity Model at the University of Virginia:
Importance of Context

It is in such special circumstances that the risk of having contextualized training may be greatest. If we want people to respond optimally to unanticipated novel conditions, such as emergencies and/or unique conditions of some other type, the evidence summarized in this chapter suggests that we do not want to have trained those people under fixed conditions.

Bjork (1994, p. 204)
Metacognition: Knowing about knowing

I want student groups to go away for a while and then come back. What I want from you are questions. I want professional questions. I want a good professional question from you, from each group, and I give them some hint as to how may questions there are --- somewhere around ten questions. I also tell them an on-going hint that Jerry has more problems as does normal business with errors of omission rather that errors of commission. And so, take a look at it. You've had a review, you have had a beginning accounting group. By then students are protesting that they don't know anything.

David Croll, University of Virginia
Transcribed from a videotape recording
Annual Meetings of the American Accounting Association
Dallas, Texas, August 20, 1997

 

 

 

Differences Between Traditional Group
Learning and Cooperative Learning

In Appendix 2, I have reproduced an excerpt that lists criteria for transforming group learning into cooperative learning.  It would appear that the BAM group learning is not technically cooperative learning in terms of some of the Appendix 2 criteria.

Consider the Appendix 2 criteria listed below:

  1. Cooperative learning groups are based on  positive interdependence among group members, where goals are structured so that students need to be concerned about performance of  all group members as well as their own.

  2. In cooperative learning groups, there is a clear individual accountability where every student's mastery of the assigned material is assessed, each student is given feedback on his or her progress, and the group is given feedback on how each member is progressing so that the other groups' members know who to help and encourage.  In traditional learning groups, individual students are not often held individually accountable for providing their share of the group's work and, occasionally, students will "hitchhike" on the work of others.

Meeting the above two criteria is clearly difficult if this implies revealing each others' examination performances and interim grades.  Also it is not clear that most groups work that way in the real world.  At the University of Virginia, the BAM instructors wander about the classroom observing groups in action.  However, grading criteria in the course apply only to individual performance and not upon group performance or performances of other members of a group.

It is not at all clear how metacognitive processes are affected along a spectrum of cooperation versus competition in learning.  Clearly, colleges and universities tend to be competitive in terms of grades.  Virtually all courses have some type of "curved" grading outcomes.  Any course that is not somewhat competitive may lose student motivation.  For example, if everyone gets a A in Course A and 50% are destined to get Cs in Course C, students most likely will strive harder in Course C.   The point here is that it is very difficult to disentangle cooperative/group learning from reward systems.

From a metacognitive standpoint, long-term recall and feelings of knowing are more likely to be greatly impacted by degree of effort and motivation, including competitively inspired motivation.  Also top students feel better about being distinguished as being near the top of a class where not everyone in the class is at the top.   However, competition can also be destructive to aspirations and self confidence.   Broad generalizations about such matters are clearly hazardous.

 

 

Warnings: Suggestions for Future Research and

Designs for Asynchronous Learning Networks

Blue.gif (84 bytes)Warnings010

Fish Out of Water Risks

Years ago, R. Jensen (1970) warned that relying on experiments from the behavioral sciences is analogous to living with "fish out of water." These warnings apply to the experimental conclusions of virtually all metalevel studies. In nearly every instance, the experiments have dealt with memory and learning tasks far less complicated than coverage of an entire year of a Intermediate Accounting. Many of the experiments focus on remembering names and places rather than complex processes such as accounting for pensions, bad debt allowances, depreciation, inventories, etc. Given the suspected success of BAM-type pedagogy, the time is ripe for metacognitive and metamemory research in more complicated learning settings.

 

 

Blue.gif (84 bytes)Warnings020

Contextualization Risks in Virtual Learning Worlds

One of the real problems of the BAM experiment is that it is confined mostly to Intermediate Accounting topics. In the real world, advanced topics and complex contracting issues surface that are well beyond the scope of intermediate accounting students. Although the entire basis of the BAM pedagogy is to contextualize the learning environment and force students to raise questions and find answers on their own, it is difficult to prepare them for issues that are beyond the scope of the course or, for than matter, beyond the scope of the entire college curriculum. The BAM pedagogy does, however, add some topics not found in traditional Intermediate Accounting courses. Without ever having a corporate tax course, students are required to fill out corporate tax forms across all seven years of the business case. Secondly, they encounter some auditing issues prior to having had an auditing course. By making students encounter and deal with issues for which they have no prior background, the BAM pedagogy takes a giant stride away from excess contextualization of traditional pedagogy where assignments typically are drills in applying previously-learned concepts and procedures. However, only a small portion of the plethora of novel real-world issues can be built into any case.

 

 

Blue.gif (84 bytes)Warnings030

Misleading Program, Course, and Learning Module Evaluation Risks

I recently attended the Consortium of Liberal Arts College (CLAC) 1998 annual meetings. This organization is comprised mainly of directors of information technology on liberal arts campuses. These are the professionals in charge of assisting faculty in the design and evaluation of newer learning technologies for synchronous and asynchronous programs, courses, and learning modules. One of the most valuable sessions was entitled "How to Evaluate Instructional Technology and How Technology Can be Used in Evaluating Instruction." What struck me is the virtual consensus (among panelists and the audience) that surveys and measurements of performance are untrustworthy except for tabulations that are relatively easy to verify --- such as how many students own their own computers, how many students use the Windows operating system, etc. Other statistics such as the frequency of student visits to a web site or the number of messages in an email listserv or chat room are viewed suspiciously since these statistics can be artificially inflated in many ways.

In their opinions, anecdotal evidence and "stories" from faculty and students were far more influential in administrative evaluations of programs, courses, and learning modules than were formalized evaluations and measurement instruments. Obviously, such presenters as Eleanor Lonske (Wellesley College), Diane Balestri (Vassar College), Phil Harriman (The College of Wooster), Charles Christison (Beloit College), and Michael Westfort (Connecticut College) admit that anecdotal evidence can be one-sided and misleading, but far more dangerous in their eyes were quantitative studies that are subject to too many intervening variables such as "times of day classes meet," "wordings of questions," "times and conditions of administering the evaluation forms," and the "personalities of combinations of students in a given course." In double blind studies (e.g., having the same instructor teach one class using traditional materials and another class using newer technology aids) there is seldom, if ever, a "clean experiment." Too many other uncontrolled variables enter into such experiments. Also technologies change so fast that it is not clear that extrapolations apply even one semester into the future. Meetings at the CLAC conference did not mention Hawthorne effects. Hawthorne effects refer to distortions and possibly non-sustaining effects of a treatment just because its newness captures more of an individual's attentiveness. In double blind studies of the impact of technologies upon learning, Hawthorne effects are particularly troublesome. Students are more apt to be more attentive to newer technologies simply because they are "new" curiosities. Positive results on learning impacts may not be sustaining, however, after the novelty and curiosity factors decline with repeated use of the technology over time.

For more discussion of this topic, see Appendix 1.

 

 Blue.gif (84 bytes)Warnings040

Metalevel Oversights in Learning Material Designs and Newer Pedagogy Risks

In addition to the above concerns, I would add some metacognitive and metamemory concerns for learning material designs. Students and faculty may have misleading feeling of knowing (FOK) and satisfaction. Most formalized evaluations have a short-term performance evaluations (e.g., examinations given during and at the end of a course). Students tend to give higher evaluations to courses that did the following in their eyes:

    • Made learning of complex material easier;
    • Made it easier to find reference material for projects;
    • Made learning more fun;
    • Had new pedagogy (e.g., technologies) that captured attention because of its newness (Hawthorne Effects);
    • Had high frequency feedback so that students could adjust their study intensities accordingly;
    • Allowed greater topic coverage due to increased efficiencies of learning;
    • Fostered cooperative learning;
    • Made learning less frustrating and ambiguous;

Metalevel researchers in cognitive science view each of the above criteria with high levels of suspicion. The problem is that the above criteria may lead to the following:

    • Misleading metacognitive feeling of knowing (FOK) that does not carry over into real world contextual variations and contextual complexity;
    • Impaired long-term memory that would have been enhanced by more sweat, ambiguity, frustration, and anxieties in the learning process;
    • Impaired ability to adapt to changing contexts and creatively deal with variations in challenges;
    • Impaired abilities to face ambiguities;
    • Impaired abilities to face life's ultimate competitive challenges;
    • Impaired introspection;

In this paper, it is proposed that we may be designing our computer aided training and education modules with the wrong criteria in mind. Perhaps it is wrong to always seek to make learning easier, less frustrating, more collaborative, and more fun. In computer aided learning, it is common to author in "rites of passage" that will not allow students to move on to the next modules without demonstrating mastery of prior modules. Perhaps rights of passage are not always ideal since reduced feedback sometimes leads to improved metacognitive performance and long-term memory. Perhaps certain types of frustrations aid learning. I see little value in the frustrations of computer failure and slow Internet performance, but having to physically search among the stacks for hard copy journals and books and having to locate references without the aid of keyword searches on a computer may improve metacognitive performance.

Metalevel research findings have grave implications for our using emerging technologies to create virtual learning worlds. The trend is toward making learning contextual. The goal is to immerse students into simulations of reality so that learning is more meaningful in the context to which it applies. However, any virtual world is a simplified copy of a more complex world. Students who with strong feeling of knowing (FOK) in a virtual world may be impaired by misleading feelings of knowing. Educators may be setting them up for failure by enhancing FOK to a point where such feelings are dysfunctional later on in life.

As mentioned already, the learner may be fooled by his or her own successes during training. Manipulations such as blocking practice by subtask, providing continuous feedback during training, and fixing the conditions of practice act like crutches that artificially support performance during training. When those crutches are absent in the post-training environment, performance collapses. The learner, however, will typically lack the perspective and experience to realize that he or she has not yet achieved the level of learning demanded by the post-training environment

Bjork (1994, p. 196)
Metacognition: Knowing about knowing

 

Blue.gif (84 bytes)Warnings050

Excessive Collaboration Risks

Newer technologies such as email, electronic chat rooms, webcams, web documents, telephony on the Internet, etc. make it increasingly easy, efficient, collaborative, and effective to collaborate in training, education, and research. However, in increased collaboration there may be too much of a good thing. To the extent that collaborations reduce learning sweat and frustration there may be metacognitive impairments. There may also be misleading feelings of knowing arising from such collaborations. Which parties in the collaborations really had the "knowing" part of it? Collaborations may be taking some important subjective experience out of the learning process.

Individuals who have illusions of comprehension or competence pose a greater hazard to themselves and others than do individuals who correctly assess that they lack some requisite information or skill. The reading we take of our own state of knowledge determines whether we seek further study or practice, whether we volunteer for certain jobs, whether we instill confidence in others, and so forth. In general, then, as argued by Jacoby, Bjork, and Kelly (1993), it is as important to educate subjective experience as it is to educate objective experience.

Bjork (1994, p. 194)
Metacognition: Knowing about knowing

 

Conclusion

My colleagues and students will be surprised that I wrote this document. I am viewed as a long-time advocate of computer aided learning technologies. However, this document is entirely consistent with my emerging "gut feel" that many of the things I have tried are not the best for my students. Across the years of experimenting with newer technologies, I discovered the hard way that my masses of electronic transparencies, slide shows, and similar lecture aids were likely impairing rather than aiding my lectures. As I drifted away from the lecture pedagogy toward greater reliance upon an asynchronous online pedagogy, I began to suspect that I was making asynchronous learning too easy by giving my students instant access to experts (via audio, video, and searchable transcriptions) and digitized literature databases on CD-ROMs and in server files. My one feeling of satisfaction in the past year is that I no longer seek to make my courses easier for students. I assign tougher term projects that reduce rather than enhance feedback frequency by making students be more introspective and creative. To their consternation, my grading is very subjective. In my latest technology experimentation, my students express greater frustrations with course difficulty. Perhaps I wrote this metacognition document to justify my gut feel that I am doing some things better when my course evaluations decline.

My "discovery" of metacognitive and metamemory research strengthens my worries and concerns about how we are designing our computer aided training and education materials. It is terribly frustrating since these research findings destroy some of the comparative advantages of emerging educational technologies. Computers can aid in virtually all of Gardner's types of learning. For example, there are real comparative advantages in creating contextualized virtual learning worlds, showing animated and sequences of graphical images, increasing feedback frequency, making learning easier, making learning more fun, making learning more collaborative, networking students with experts, and making literature searches easier and more efficient. I am not promoting reduced experimenting and implementing paces in computer and networking technologies in education.  What I am advocating is that the metacognitive principles be programmed into these technologies even though doing so may weaken short-term learner satisfaction with the program, course, and/or learning modules.

My "discovery" of the metacognitive and metamemory research strengthens my worries and concerns about how we evaluate our programs, courses, and learning modules. The frustrating part of metacognitive and metamemory criteria, when applied in higher education, is that learning performance cannot be evaluated prior to graduation or even in the earliest years of post-graduate work and study. Evaluating too soon may serve to put too much weight on the wrong criteria (e.g., satisfaction) that are short sighted from a metacognition standpoint. Our students may, thereby, be graduating with that misleading FOK that impairs metacognitive performance in their longer-term futures. But then how many of us in retrospect, years after graduation, have increased our regards for our toughest and most frustrating professors? Most likely these are the cussed professors buried deepest in metamemory.

In my viewpoint the above findings also have relevance to the findings of another AECC-funded experiment called the Project Discovery (PD) project at the University of Illinois at Urbana-Champaign.  That study took a slightly different approach across multiple accounting courses to study the effects of the importance of having students learn from "complex, ill-structured, ambiguous problems and cases similar to those found in practice" as reported by Stone and Shelly (1997, p. 26).  Stone and Shelly repeatedly stress that their research notes the impact of such factors on learning but make no attempt to attribute causality.  I speculate that some of the causal factors are metacognitive.  Even though my paper focused on the BAM Program at the University of Virginia, I think my conclusions extend to the PD Program at the University of Illinois.


The Where's My Professor Game at Brigham Young University

Question
Why can this innovation probably be used only once for the full effect?
Hint:  In subsequent terms the basic approach can be used although students will by then know that the person in front is being paid to guide learning.

"The Antiprofessor Speaks Out," by Kerry Soper, Chronicle of Higher Education, The Chronicle Review (from the Chronicle of Higher Education), December 5, 2008 --- http://chronicle.com/weekly/v55/i15/15b02001.htm?utm_source=at&utm_medium=en

Now that the old teaching model of "sage on the stage" has finally given way to the more progressive "guide at the side," academe is ready for another paradigm shift. In my classes, I have adopted a philosophy that I call "peer at the rear." Here's how I break down and then rebuild students' expectations of a healthy student-instructor relationship.

I start the first day of the semester at the back of the classroom — literally. Students have no idea that I'm the professor. I pull this off by wearing clothes from Old Navy, sporting a backward baseball cap, and texting nonsense on a cellphone. If anyone tries to talk to me, I am prepared with gossip about misbehaving celebrities or new alternative-music bands.

When the real prof (the old me) doesn't show up after about 15 minutes, I — as the pretend student — go ballistic. I loudly "diss" the administration, the academic system, and myself (the outdated teacher-me) and make a big show of riffling through the textbook and calling it a load of you-know-what before tossing it into the garbage can.

After I cool down a bit, some of the students usually start to leave. That's when I casually mention that I saw some "cruddy syllabus or something" up at the front of the class. Of course they dutifully grab the paper before heading off to their next old-fashioned lecture. To further undermine the credibility of the phantom, old-school professor in their eyes, I intentionally make this syllabus as obtuse, incomplete, and condescending as possible.

At the start of the next class (while still disguised as a hip, slouching student), I call out something like, "Well, dudes, it looks like Señor Soper's not gonna show; guess we'll just have to teach ourselves!" That's when the students really begin to take charge of their own education. Granted, they mostly just read the newspaper or talk about what they're going to eat when they get out of class, but I can see in the way they carry themselves a new sense of ownership over their ideas and lunch plans.

In the middle of this second class period, I crank up some techno music on the sound system while doing a popping and locking dance routine. In the middle of it, I write in giant letters on the board — "Psych! I'm your instructor!" — and take off my baseball cap, revealing my receding hairline. I can tell that some of the students are relieved, but I keep them off balance by donning iPod earbuds, resuming my dance, and pretending that I can't hear what they're saying.

If students get so frustrated that they start to leave, I tone things down a bit and reveal the details of my peer-at-the-rear philosophy. That includes doing an imitation of what my old teaching persona might have done, had he been there. After getting a taste of that pedagogical nerd, they seem to chill out a bit.

I lay the ground rules: They have to treat me as an equal, not an authority figure or even a knowledgeable mentor. This includes calling me by my first name (or a cool nickname like "Kerr Dawg" or "Super Soper") and greeting me with some kind of groovester handshake or laid-back fist bump. When that's settled, I throw up my hands, say, "Dudes, the class is yours!," and watch as the magic unfolds.

Eventually some of the more alert students will reluctantly organize themselves into study groups. This is a move in the right direction; they're no longer relying on a self-inflated "professor" to show them the way. But they're still full of predictably boring ideas, and so I do my best to disrupt their discussions with postmodern Socratic methods: walking around making annoying sounds; loudly interjecting Zen-like non sequiturs into their conversations ("he who dealt it, smelt it"); or standing behind someone while mouthing their words and mimicking their posture.

To get things going on especially slow days, I do have to facilitate a bit, but I like to keep it loose and open-ended. I might show some music-videoclips and maybe a segment from The Colbert Report, and then I'll just shrug my shoulders and say in a bored voice, "Wassup?" This may irritate students who are still addicted to oppressive educational methods, but that's my intention. Students need to be goaded to confront the fallacies of the industrial/pseudo-educational complex, such as "grades matter" and "professors know more than we do."

But educational misconceptions are so deeply embedded that a large number of my young friends get frustrated with my progressive methods; sometimes they even mount a campaign for a new instructor or to get me fired. Right on! At least they're passionate about an idea or cause — they're no longer passive robots.

Ultimately, though, my core objective is to become students' buddy — their "homey," as it were. I try to achieve that rapport by first turning their animosity toward deserving targets — anal parents, stuffy professors, and faceless administrators. Then I build my own egalitarian friendship with them in a number of relentlessly methodical ways: following them around after class, texting them weird gossip about my colleagues, forwarding them hilarious YouTube clips; and showing up at their apartments to eat snacks and "crank some Halo" on the Xbox.

Many of the students resist those overtures, probably because truly progressive changes always feel a little uncomfortable at first. But by midsemester I usually manage to convert even the most stalwart holdouts when I start undermining the university's "bogus" grading system. If students insist on handing in essays, I mock traditional evaluative judgments by writing nonsensical Beat poetry in the margins or by marking every third sentence with a shiny kindergarten star. Sometimes I even plagiarize random feedback from Wikipedia or SparkNotes, just to make a point.

If students insist on taking a test, I adopt Dadaistic strategies, like making them solve the kids' word jumble from the Sunday comics page in a ridiculously short amount of time ("Go! You've got 12 seconds!") or assigning draconian grades based only on penmanship. The latter traumatizes them until I make a big show of ripping up my grade book and feeding it through a portable shredder. In the end, they get to choose their own grade, of course. (Yes, it's usually an A, but as if they even care at that point.)

Surprisingly, my student evaluations aren't as stellar as you would expect. I get the occasional "You rock, dude," but for the most part, my new peers seem too anxious to fully embrace my antipedagogical persona. My colleagues, too, have been lukewarm about my methods, some of them even holding special meetings to gripe about the way I dress and the "chaos" I supposedly spread into their classrooms. And then, of course, there are the ninnies in the administration building who don't like the "grades" I give, the complaints they get from parents, and the way I rap and beat-box loudly across the campus.

But I'm confident that eventually my new paradigm will take hold and everyone will acknowledge that I was simply ahead of the curve. I do hope I'll be vindicated before the special-action committee at my university succeeds in firing me. But if I am fired, so be it. I'll be remembered as the Galileo of my time, the "antiprofessor" bold enough to give power back to the students, where it belongs. When our movement is large enough, no one will be able to take us down (or, rather, make us stand up). So as united peers, let us sit down in the rear and rock this old school!n

Kerry Soper is an associate professor in the department of humanities, classics, and comparative literature at Brigham Young University.

Jensen Comment
Perhaps without realizing it, perhaps Professor Soper's success with this is due to metacognition that comes with self-learning tasks. A somewhat similar and controversial pedagogy is used the the BAM approach to intermediate accounting ---
http://chronicle.com/weekly/v55/i15/15b02001.htm?utm_source=at&utm_medium=en
Years of experience with the BAM pedagogy shows that for some instructors it is a disaster despised by students, although other instructors can like Anthony Catanach can pull it off with award-winning success. It's very hard for teachers not to teach!

 


Accounting Professors in Support of Online Testing That, Among Other Things, Reduces Cheating
These same professors became widely known for their advocacy of self-learning in place of lecturing

"In Support of the E-Test," by Elia Powers, Inside Higher Ed, August 29, 2007 --- http://www.insidehighered.com/news/2007/08/29/e_test

Critics of testing through the computer often argue that it’s difficult to tell if students are doing their own work. It’s also unclear to some professors whether using the technology is worth their while. A new study makes the argument that giving electronic tests can actually reduce cheating and save faculty time.

Anthony Catanach Jr. and Noah Barsky, both associate professors of accounting at the Villanova School of Business, came to that conclusion after speaking with faculty members and analyzing the responses of more than 100 students at Villanova and Philadelphia University. Both Catanach and Barsky teach a course called Principles of Managerial Accounting that utilizes the WebCT Vista e-learning platform. The professors also surveyed undergraduates at Philadelphia who took tests electronically.

The Villanova course follows a pattern of Monday lecture, Wednesday case assignment, Friday assessment. The first two days require in-person attendance, while students can check in Friday from wherever they are.

“It never used to make sense to me why at business schools you have Friday classes,” Catanach said. “As an instructor it’s frustrating because 30 percent of the class won’t show up, so you have to redo material. We said, how can we make that day not lose its effectiveness?”

The answer, he and Barsky determined, was to make all electronically submitted group work due on Fridays and have that be electronic quiz day. That’s where academic integrity came into play. Since the professors weren’t requiring students to be present to take the exams, they wanted to deter cheating. Catanach said programs like the one he uses mitigate the effectiveness of looking up answers or consulting friends.

In electronic form, questions are given to students in random order so that copying is difficult. Professors can change variables within a problem to make sure that each test is unique while also ensuring a uniform level of difficulty. The programs also measure how much time a student spends on each question, which could signal to an instructor that a student might have slowed to use outside resources. Backtracking on questions generally is not permitted. Catanach said he doesn’t pay much attention to time spent on individual questions. And since he gives his students a narrow time limit to finish their electronic quizzes, consulting outside sources would only lead students to be rushed by the end of the exam, he added.

Forty-five percent of students who took part in the study reported that the electronic testing system reduced the likelihood of their cheating during the course.

Stephen Satris, director of the Center for Academic Integrity at Clemson University, said he applauds the use of technology to deter academic dishonesty. Students who take these courses might think twice about copying or plagiarizing on other exams, he said.

“It’s good to see this program working,” Satris said. “It does an end run around cheating.”

The report also makes the case that both faculty and students save time with e-testing. Catanach is up front about the initial time investment: For instructors to make best use of the testing programs, they need to create a “bank” of exam questions and code them by topic, learning objectives and level of difficulty. That way, the program knows how to distribute questions. (He said instructors should budget roughly 10 extra hours per week during the course for this task.)

The payoff, he said, comes later in the term. In the study, professors reported recouping an average of 80 hours by using the e-exams. Faculty don’t have to hand-grade tests (that often being a deterrent for the Friday test, Catanach notes), and graduate students or administrative staff can help prepare the test banks, the report points out.

Since tests are taken from afar, class time can be used for other purposes. Students are less likely to ask about test results during sessions, the study says, because the computer program gives them immediate results and points to pages where they can find out why their answers were incorrect. Satris said this type of system likely dissuades students from grade groveling, because the explanations are all there on the computer. He said it also make sense in other ways.

“I like that professors can truly say, ‘I don’t know what’s going to be on the test. There’s a question bank; it’s out of my control,’ ” he said.

And then there’s the common argument about administrative efficiency: An institution can keep a permanent electronic record of its students.

Survey results showed that Villanova students, who Catanach said were more likely to have their own laptop computers and be familiar with e-technology, responded better to the electronic testing system than did students at Philadelphia, who weren’t as tech savvy. Both Catanach and Satris said the e-testing programs are not likely to excite English and philosophy professors, whose disciplines call for essay questions rather than computer-graded content.

From a testing perspective, Catanach said the programs can be most helpful for faculty with large classes who need to save time on grading. That’s why the programs have proven popular at community colleges in some of the larger states, he said.

“It works for almost anyone who wants to have periodic assessment,” he said. “How much does the midterm and final motivate students to keep up with material? It doesn’t. It motivates cramming. This is a tool to help students keep up with the material.”

August 29, 2007 reply from Stokes, Len [stokes@SIENA.EDU]

I am also a strong proponent of active learning strategies. I have the luxury of a small class size. Usually fewer than 30 so I can adapt my classes to student interaction and can have periodic assessment opportunities as it fits the flow of materials rather than the calendar. I still think a push toward smaller classes with more faculty face time is better than computer tests. One lecture and one case day does not mean active learning. It is better than no case days but it is still a lecture day. I don’t have real lecture days every day involves some interactive material from the students.

While I admit I can’t pick up all trends in grading the tests, but I do pick up a lot of things so I have tendency to have a high proportion of essays and small problems. I then try to address common errors in class and also can look at my approach to teaching the material.

Len

Bob Jensen attempts to make a case that self learning is more effective for metacognitive reasons --- http://www.trinity.edu/rjensen/265wp.htm
This document features the research of Tony Catanach, David Croll, Bob Grinaker, and  Noah Barsky.

Bob Jensen's threads on "Online Education Effectiveness and Testing" are at http://www.trinity.edu/rjensen/Assess.htm#OnlineOffCampus

Bob Jensen's threads on the myths of online education are at http://www.trinity.edu/rjensen/000aaa/thetools.htm#Myths


Acknowledgement:
I want to thank Paula Hertel, Professor and Chair of the Department of Psychology, Trinity University. Dr. Hertel's specialty is memory and cognition research. She remembered to give me some good leads for this document. She also reviewed the first draft. All errors and shortcomings that remain are mine.

 

Appendix 1
Why Aren't Stories Good Food?

Email Messages About Evaluation Criteria and Processes

Email message from Bob Jensen to the AECM on June 28, 1998

My question is why aren't education researchers, journal editors, and referees listening to the experts about evaluating programs, courses, and learning modules? Why do we put up such hurdles for publication of "stories?"

This weekend I attended the Consortium of Liberal Arts College (CLAC) 1998 annual meetings held on the Trinity University Campus. This organization is comprised mainly of directors of information technology on liberal arts campuses. These are the professionals in charge of assisting faculty in the design and evaluation of newer learning technologies for synchronous and asynchronous programs, courses, and learning modules. One of the most valuable sessions was entitled "How to Evaluate Instructional Technology and How Technology can be Used in Evaluating Instruction." What struck me is the virtual consensus (among panelists and the audience) that surveys and measurements of performance are untrustworthy except for tabulations that are relatively easy to verify such as how many students own their own computers, how many students use the Windows operating system, etc. Other statistics such as the frequency of student visits to a web site or the number of messages in an email listserv or chat room are viewed suspiciously since these statistics can be artificially inflated in so many ways and say nothing about quality and content.

In their opinions, anecdotal evidence and "stories" from faculty and students were far more influential in administrative evaluations of programs, courses, and learning modules than were formalized evaluations and measurement instruments. Obviously, such presenters as Eleanor Lonske (Wellesley College), Diane Balestri (Vassar College), Phil Harriman (The College of Wooster), Charles Christison (Beloit College), and Michael Westfort (Connecticut College) admit that anecdotal evidence can be one-sided and misleading, but far more dangerous in their eyes were quantitative studies that are subject to too many intervening variables such as "times of day classes meet," "wordings of questions," "times and conditions of administering the evaluations forms," and the "personalities of combinations of students in a given course."

In double blind studies (e.g., having the same instructor teach one class using traditional materials and another class using newer technology aids) there is seldom, if ever, a "clean experiment." Too many other uncontrolled variables enter into such experiments. Also technologies change so fast that it is not clear that extrapolations apply even one semester into the future. Meetings at the CLAC conference did not mention Hawthorne effects, but these also tend to distort evaluations of programs, courses, and learning modules. Hawthorne effects are discussed in http://www.trinity.edu/~rjensen/245glosf.htm#Hawthorne Hawthorne effects tend to bias student evaluations upward in favor of newer technologies.

Maybe I am venting my weariness of having to referee my 23rd paper focused upon quantitative comparisons of distance education versus on-site courses. I'd rather compare the stories from the instructors and students. Are we imposing the statistical measurement criteria upon authors merely to satisfy some questionable biases of our refereeing process? Are our referees listening to the experts?

The other message from the CLAC experts is that evaluations should always be focused upon programs and not technologies per se.

Bob

Professor Robert E. Jensen (Bob) http://www.trinity.edu/~rjensen
Jesse H. Jones Distinguished Professor of Business Administration
Trinity University, San Antonio, TX 78212-7200
Voice: 210-736-7347 Fax: 210-736-8134
Email rensen@trinity.edu

 


Reply from Charles Walton [ cwalton@usa.net ]

Bob,

Yesterday, Saturday, I finished a year long engagement with a Board of Trustees of a college developing and implementing a new Board Governance system.

The major time drain in developing new budgeting and fiscal control system, was not restructuring Board budgeting, but was related dealing with their problems in instructional and non-instruction IS functions. The Board has invested a great deal of the taxpayers and students’ money in information technology with little data to support that investment. There was no significant difference in student satisfaction after implementation of technology and no significant difference in learning outcomes between sections with and without the technology. The amounts requested were staggering and those underrepresented or didn’t include maintenance.

On the administrative side, the systems are so poor that the IS vendor holds the school hostage. The costs for a Datatel or PeopleSoft extrication are totally out of proportion with the rest of the budget. The costs of salaries in phasing out the old vendor, the new vendor transition project manager, instructional computing head, and new CIO will be more than twice the costs of the President, VP of Instruction, VP for Student Affairs and VP for Business Affairs for at least the next two years.

I really feel for the Trustee who has to deal with these IS requests. I shouldn’t have to vent, it made for an interesting sabbatical with a lot of billable hours.

As I prepare to return from sabbatical, I must admit that I avoid IS issues at my college. My need to manage IS was satisfied years ago as an IS head.

Bob, Trustees are desperate for good food. Mostly, what college administrators feed them gives them heartburn. A word of praise for non paid Trustees who really attempt to do what is best for the college and the fanatic Trustees who work very hard with little support to make it happen.

Charles Walton, Ph.D., CPA  [ cwalton@usa.net ]
Gettysburg College

If you aren’t familiar with CMM and ISO 9000 you are an IS dinosaur.


Reply from Ailsa H. S. Nicholson [ ctiafm@uea.ac.uk ]

I would like to endorse Bob’s sentiments as regards telling the story.

Here in the UK the CTI (Computers in Teaching Initiative) have been encouraging academics to ‘tell the story’ for the last nine years. Here at the CTI Centre for Accounting Finance and Management (CTI-AFM) we have been encouraging lecturers in accounting & business education to write ‘accounts’ of their use of technology with classes for publlication on our journal ACCOUNT.

Those who do write for our journal and who contribute papers for our conference are often very frank and reflective about the experience and share with others the downs as well as the ups. Such reports are of course invaluable as it can prevent new adopters from reinventing wheels or making avoidable errors.

‘Real evaluation’ is very long term and as you point out full of pitfalls and unsurmountable problems. Also technology and software packages change or are updated so often that any result of a long term study can be unhelpful in that they may refer to products which have in the meantime been improved or changed. Telling the story is more valuable to the practicioner .

Unfortunately as you point out it is difficult to get such reports published in mainstream journals and certainly here in the UK and probably with you in the states academics are strongly discouraged from any activity which does not lead to publication in a mainstream journal. So it is an uphill struggle to ensure a good supply of reflectve accounts - or stories - on the use of technology in teaching.

But CTI-AFM continues to seek these out and to persuade academics to write for our journal or to contribute conference papers for our annual conference in April. Details of both are on our www site - address below and we will shortly be putting ACCOUNT our journal o’ Why Aren’t Stories Good Food? n-line so that colleagues in the US will be able to view the experience of their UK colleagues.

Ailsa Nicholson

Ailsa H. S. Nicholson
CTI Accounting Finance and Management
School of Management
University of East Anglia
Norwich NR4 7TJ

Tel: 01603 592312
Fax: 01603 593343

E-mail: ctiafm@uea.ac.uk

http://www.mgt.uea.ac.uk/cti


Reply from Bob Dumouchel [ owl@callamerica.net  ]

Hi Bob,

I just love this debate and as a vendor of self-study products I am anything but impartial. These studies all seem to boil down to the same result.

"Format Does Not Make a Difference". Without conducting a study I would contend that the difference when it is reported can be traced back to an improved student perception of the course.

If schools want to increase the effectiveness of their courses they should market their courses better and improve the student perception of the course after all "Perception is more important that reality". If a student starts a course believing in it, they will learn more. The strange thing about this is that at the university level the Profs credentials are usually impressive to say the least. Yet if you read the average college catalog you would never know it. When we have a product produced by someone with impressive credentials you can believe that we make a big deal out of it.

Student perception needs to be managed, courses need to be marketed. If students believe they need the course, if you create the want rather then stressing the need, the results will follow. Professional quality instruction is of course an assumption in all of this.

Good luck on your next study.

Bob Dumouchel
On With Learning Inc.
Phone (800) 272-0887 or (805) 481-0118
Fax (800) 508-0487 or (805) 481-0252

http://videoed.com

owl@callamerica.net

 


Appendix 3

The Emperor's Naked as He Can Be

Prior to one of my technology in education workshops, Bob Anthony sent me the following questions that he expects me to answer in the workshop. Perhaps, more than me, you can help him with some of these answers. Bob does not subscribe the aecm listserv, but you can contact him at Robert N. Anthony at  RNAnthony@valley.net 

He's deadly serious about these questions. Bob is one of our most successful book authors and has been retired from Harvard for many years.  Nevertheless, he still has a keen mind and his honest bluntness trademark. Years ago in Denmark, Bob was the little boy on the streets who yelled out" "The Emperor's not wearing any clothes."  

These questions are fundamental and we only have a limited amount of time in my workshops to debate such issues. If those of you attending the workshops will prepare in advance, we will try to take up these questions as time permits.

Some of the issues he raises taken up in the above document. Other issues are taken up in its much larger companion piece at http://WWW.Trinity.edu/~rjensen/255wp.htm

Bob's message is repeated below below:

**************************************************************************************************************************************************

Bob,

I read with great interest the material you sent on July 31 and look forward to your session on August 16. The following comment relates to the "self directed learning" version of "distance learning," as distinguished from "asynchronous learning" within a school. The issues relating to each of these topics are somewhat different.

First, there is in implication in much of the material you quote that there is something new about "self directed learning". Actually, the descriptions I have seen aren’t much different from those of correspondence courses that have been offered for many years. Some correspondence schools prosper, which indicates that they are doing something that customers want. The inclusion of sound is new, but what else?

It seems clear that interactive courses will not succeed. The effort involved in responding to student queries is too great, especially in view of the difficulty of obtaining money for these responses. Do you agree? (Of course, the publisher must be prepared to correct errors.)

Many of the people you quote do not appreciate the difficulty of identifying worthwhile; programs. This is an important function provided by the successful textbook companies. I was consulting accounting editor for Richard D. Irwin for many years, and read all the accounting manuscripts submitted to them. Only a small fraction were worth publishing. There is a difference between the great teacher and the great author. Bill Paton was not a leading textbook author. Although the Irwin people were pessimistic, they published a text by [XXXXX} as a favor to me; it didn’t succeed.

Therefore, the self-directed learning programs published by universities are unlikely to do well. The university editors are amateurs. Moreover, the value of a university "brand name" is not high in accounting textbooks, and I don’t see why it should be higher on an Internet product. Many mediocre courses will nevertheless appear on the Internet because the cost is so low. (There is a company today that will scan a manuscript and publish it on the Internet, without even commenting on its quality, for $500.) .

You mention corporate programs. My impression is that corporations develop successful training programs, such as how to process information in a new purchase / accounts-payable system; but training is quite different from education. Colleges and universities focus on education.

I do believe that a few companies will publish good programs and therefore they will before long dominate the market. Probably they will be conventional publishers because of their editorial and marketing expertise. Maybe there will be a few new companies focusing exclusively on the Internet.. You mention Knowledge Universe, a company started by Milliken. I am familiar with UOL publishing. It’s too early to tell who will emerge as the leaders, corresponding to the publishers of conventional texts.

Arriving at sound conclusions to issues like the above is important to me!

I am trying to make a version of my Essentials of Accounting succeed on the Internet, on CD-ROM, or both. I hope some of the above topics will be discussed in the seminar.

. Bob Anthony.


"Critical Thinking:  Why It's So Hard to Teach," by Daniel T. Willingham ---
http://www.aft.org/pubs-reports/american_educator/issues/summer07/Crit_Thinking.pdf

Also see Simorleon Sense --- http://www.simoleonsense.com/critical-thinking-why-is-it-so-hard-to-teach/

“Critical thinking is not a set of skills that can be deployed at any time, in any context. It is a type of thought that even 3-year-olds can engage in—and even trained scientists can fail in.”

“Knowing that one should think critically is not the same as being able to do so. That requires domain knowledge and practice.”

So,  Why Is Thinking Critically So Hard?
Educators have long noted that school attendance and even academic success are no guarantee that a student will graduate an effective thinker in all situations. There is an odd tendency for rigorous thinking to cling to particular examples or types of problems. Thus, a student may have learned to estimate the answer to a math problem before beginning calculations as a way of checking the accuracy of his answer, but in the chemistry lab, the same student calculates the components of a compound without noticing that his estimates sum to more than 100 percent. And a student who has learned to thoughtfully discuss the causes of the American Revolution from both the British and American perspectives doesn’t even think to question how the Germans viewed World War II. Why are students able to think critically in one situation, but not in another? The brief answer is: Thought processes are intertwined with what is being thought about. Let’s explore this in depth by looking at a particular kind of critical thinking that has been studied extensively: problem solving.

Imagine a seventh-grade math class immersed in word problems. How is it that students will be able to answer one problem, but not the next, even though mathematically both word problems are the same, that is, they rely on the same mathematical knowledge? Typically, the students are focusing on the scenario that the word problem describes (its surface structure) instead of on the mathematics required to solve it (its deep structure). So even though students have been taught how to solve a particular type of word problem, when the teacher or textbook changes the scenario, students still struggle to apply the solution because they don’t recognize that the problems are mathematically the same.

Thinking Tends to Focus on a Problem’s “Surface Structure”
To understand why the surface structure of a problem is so distracting and, as a result, why it’s so hard to apply familiar solutions to problems that appear new, let’s first consider how you understand what’s being asked when you are given a problem. Anything you hear or read is automatically interpreted in light of what you already know about similar subjects. For example, suppose you read these two sentences: “After years of pressure from the film and television industry, the President has filed a formal complaint with China over what U.S. firms say is copyright infringement. These firms assert that the Chinese government sets stringent trade restrictions for U.S. entertainment products, even as it turns a blind eye to Chinese companies that copy American movies and television shows and sell them on the black market.”

With Deep Knowledge, Thinking Can Penetrate Beyond Surface Structure
If knowledge of how to solve a problem never transferred to problems with new surface structures, schooling would be inefficient or even futile—but of course, such transfer does occur. When and why is complex,5 but two factors are especially relevant for educators: familiarity with a problem’s deep structure and the knowledge that one should look for a deep structure. I’ll address each in turn. When one is very familiar with a problem’s deep-structure, knowledge about how to solve it transfers well. That familiarity can come from long-term, repeated experience with one problem, or with various manifestations of one type of problem (i.e., many problems that have different surface structures, but the same deep structure). After repeated exposure to either or both, the subject simply perceives the deep structure as part of the problem description.

The Cambridge Handbook of Thinking and Reasoning --- Click Here

The Miniature Guide To Critical Thinking Concepts & Tools --- Click Here


 


Appendix 4

Update Message from Robert Bjork

 

I read your online article with great interest. You do an excellent job of making concrete some very important points in your article, including, of course, some of the points I tried to emphasize in my 1994 article.

The other important thing that comes through in your article is that computer-based individualized learning techologies, for all of their potential, are not something magical: They are tools that can be used in counterproductive and ill-advised ways as well as in innovative and productive ways. Starting last year, for example, there was an initiative at UCLA to have an internet site for every course at UCLA. That initiative—well meaning, in my own opinion—has raised cries of alarm from certain professors about infringement of intellectual property. Those concerns are legitimate in a few cases, I think, but I (and I alone, I sometimes think) have been concerned about the kind of issues you address in your article—namely, the potential for such "hi-tech" resources to be used in ways that impede, rather than promote, learning. Instructors now get day-to-day pressures from students to put most everything on the web—overheads, outlines, lecture notes, etc. The web site then becomes a kind of remedial device. Students decide that they can skip the lecture, or, when they do attend, that they don’t need to take good notes, understand the lecture, or ask questions when things are confusing, because they can (hopefully) get a repetition on the course site.

There are, of course, creative ways to use a course site—ways that make the learner an active participant in the learning process, but such exercises/materials to enrich a course demand the professor’s time and energies and will not necessarily be used or appreciated by students, who are prone to view such enriching exercises as an additional course burden.

When I teach the graduate course on learning and memory this fall term, I would like to consider using your article as one of the packet of course readings. I printed a copy from the web site, but that copy is not one that would lend itself to being reproduced for the course. Would you be willing to either send me a hard word-processing copy or attach one to an email message?

A couple related articles of my own that appeared after the Bjork "Memory and Metamemory Considerations" article are:

Bjork, R.A. Institutional impediments to effective training. (1994). In D. Druckman and R.A.Bjork (Eds.), Learning, remembering, believing:

Enhancing human performance (pp.295-306). Washington, DC: National Academy Press.

Bjork, R. A. (in press). Assessing our own competence: Heuristics and illusions. In D. Gopher and A. Koriat (Eds.), Attention and Peformance XVII. Cognitive Regulation of Performance: Interaction of Theory and Application. Cambridge, MA: MIT Press. (41 pages)

I’ll have my assistant send copies of those articles to you.

Best regards,

RAB

Robert A. Bjork, Editor

PSYCHOLOGICAL REVIEW
Department of Psychology
UCLA
Los Angeles, CA 90095-1563
(310-825-7028; fax 310-206-5895)


"Are we over estimating remembering and underestimating learning?"
by Joe Proctor, Williams College
Memory And Learning: Recent Research
Main Category: Psychology /
Psychiatry Also Included In: Neurology /
Neuroscience Article Date: 08 Mar 2010 - 2:00 PST
http://www.medicalnewstoday.com/articles/181496.php

Are we over estimating remembering and underestimating learning?

Current research by Nate Kornell, an assistant professor of psychology at Williams College, and Robert A. Bjork of the University of California, Los Angeles address this question and was recently published in the Journal of Experimental Psychology.

In their paper titled A Stability Bias in Human Memory: Overestimating Remembering and Underestimating Learning, Kornell and Bjork write: "To manage one's own conditions of learning effectively requires gaining an understanding of the activities and processes that do and do not support learning."

In psychology, experts use the term metacognition to talk about how people think about their own cognitive processes - in essence, thinking about thinking.

To probe the way people think about their capacity for remembering, Kornell and Bjork asked people to look at a list of words and predict how well they would be able to remember the words after subsequent periods of study and testing.

Their results led the researchers to the suggestion that people are under confident in their learning abilities and overconfident in their memories. That is, people failed to predict that they would be able to remember more words after studying more - although in reality, they learned far more -- instead basing their predictions on current memory. Kornell and Bjork call this a "stability bias" in memory.

Kornell's work also has been published in Scientific American, Psychological Science, Current Directions in Psychological Science, and Applied Cognitive Psychology, among other journals.

 

Appendix 5

The word "metacognition" arises once again.

"Assessing the Impact of Instructional Technology on Student Achievement," by Lorraine Sherry, Shelley Billig, Daniel Jesse, and Deborah Watson-Acosta, T.H.E. Journal, February 2001, pp. 40-43 --- http://www.thejournal.com/magazine/vault/A3297.cfm 

Four separate simplified path analysis models were tested. The first pair addressed process and product outcomes for class motivation, and the second pair addressed school motivation. The statistically significant (p < .05) results were as follows:

  • Motivation was related to metacognition. The relationship between class motivation and metacognition was slightly stronger (R = .307, p < the relationship between school motivation and metacognition (R = .282, p < .0001).
  • The relationship between metacognition and inquiry learning (Beta = .546, p < .0001) was stronger than the relationship between metacognition and application of skills (Beta = .282, p < .0001).
  • The relationship between inquiry learning and the student learning process outcome (Beta = .384, p = .001) was stronger than the relationship between application of skills and the student learning process outcome (Beta = -.055, not significant).
  • The relationship between application of skills and the student product outcome (Beta = .371, p = .004) was stronger than the relationship between inquiry learning and the student product outcome (Beta = .063, not significant).

Clearly, correlation does not imply causality. However, when each of these elements was considered as an independent variable, there was a corresponding change in associated dependent variables. For example, there was a significant correlation between motivation and metacognition, indicating that students' enthusiasm for learning with technology may stimulate students' metacognitive (strategic) thinking processes. The significant correlations between motivation, metacognition, inquiry learning, and the student learning process score indicate that motivation may drive increases in the four elements connected by the first path. Similarly, the significant correlations between motivation, metacognition, application of skills, and the student product score indicate that motivation may drive increases in the four elements connected by the second path.

Based on the significant correlations of the two teacher measurements of student achievement with the student survey data, these data validated the evaluation team's extension of the Developing Expertise model to explain increases in student performance as a result of engaging in technology-supported learning activities. Moreover, nearly all students across the project met the standards for both the teacher-created student product assessment and the learning process assessment. This indicates that, in general, the project had a positive impact on student achievement.

Conclusions

These preliminary findings suggest that teachers should emphasize the use of metacognitive skills, application of skills, and inquiry learning as they infuse technology into their respective academic content areas. Moreover, these activities are directly in line with the Vermont Reasoning and Problem Solving Standards, and with similar standards in other states. The ISTE/NETS standards for assessment and evaluation also suggest that teachers:

  • Apply technology in assessing student learning of subject matter using a variety of assessment techniques.
  • Use technology resources to collect and analyze data, interpret results, and communicate findings to improve instructional practice and maximize student learning.
  • Apply multiple evaluation methods to determine students' appropriate use of technology resources for learning, communication and productivity.

Rockman (1998) suggests that "A clear assessment strategy that goes beyond standardized tests enables school leaders, policymakers, and the community to understand the impact of technology on teaching and learning." RMC Research Corporation's extension of the Sternberg model can be used to organize and interpret a variety of student self-perceptions, teacher observations of student learning processes, and teacher-scored student products. It captures the overlapping kinds of expertise that students developed throughout their technology-related activities.

One of the greatest challenges facing the Technology Innovation Challenge Grants and the Preparing Tomorrow's Teachers To Use Technology (PT3) grants is to make a link between educational technology innovations, promising practices for teaching and learning with technology, and increases in student achievement. We believe that this model may be replicable in other educational institutions, including schools, districts, institutions of higher learning, and grant-funded initiatives. However, to use this model, participating teachers must be able to clearly identify the standards they are addressing in their instruction, articulate the specific knowledge and skills that are to be fostered by using technology, carefully observe student behavior in creating and refining their work, and create and benchmark rubrics that they intend to use to evaluate student work.


 

Appendix 6
Update in February 2003

Accounting Professors in Support of Online Testing That, Among Other Things, Reduces Cheating
These same professors became widely known for their advocacy of self-learning in place of lecturing

"In Support of the E-Test," by Elia Powers, Inside Higher Ed, August 29, 2007 --- http://www.insidehighered.com/news/2007/08/29/e_test

Critics of testing through the computer often argue that it’s difficult to tell if students are doing their own work. It’s also unclear to some professors whether using the technology is worth their while. A new study makes the argument that giving electronic tests can actually reduce cheating and save faculty time.

Anthony Catanach Jr. and Noah Barsky, both associate professors of accounting at the Villanova School of Business, came to that conclusion after speaking with faculty members and analyzing the responses of more than 100 students at Villanova and Philadelphia University. Both Catanach and Barsky teach a course called Principles of Managerial Accounting that utilizes the WebCT Vista e-learning platform. The professors also surveyed undergraduates at Philadelphia who took tests electronically.

The Villanova course follows a pattern of Monday lecture, Wednesday case assignment, Friday assessment. The first two days require in-person attendance, while students can check in Friday from wherever they are.

“It never used to make sense to me why at business schools you have Friday classes,” Catanach said. “As an instructor it’s frustrating because 30 percent of the class won’t show up, so you have to redo material. We said, how can we make that day not lose its effectiveness?”

The answer, he and Barsky determined, was to make all electronically submitted group work due on Fridays and have that be electronic quiz day. That’s where academic integrity came into play. Since the professors weren’t requiring students to be present to take the exams, they wanted to deter cheating. Catanach said programs like the one he uses mitigate the effectiveness of looking up answers or consulting friends.

In electronic form, questions are given to students in random order so that copying is difficult. Professors can change variables within a problem to make sure that each test is unique while also ensuring a uniform level of difficulty. The programs also measure how much time a student spends on each question, which could signal to an instructor that a student might have slowed to use outside resources. Backtracking on questions generally is not permitted. Catanach said he doesn’t pay much attention to time spent on individual questions. And since he gives his students a narrow time limit to finish their electronic quizzes, consulting outside sources would only lead students to be rushed by the end of the exam, he added.

Forty-five percent of students who took part in the study reported that the electronic testing system reduced the likelihood of their cheating during the course.

Stephen Satris, director of the Center for Academic Integrity at Clemson University, said he applauds the use of technology to deter academic dishonesty. Students who take these courses might think twice about copying or plagiarizing on other exams, he said.

“It’s good to see this program working,” Satris said. “It does an end run around cheating.”

The report also makes the case that both faculty and students save time with e-testing. Catanach is up front about the initial time investment: For instructors to make best use of the testing programs, they need to create a “bank” of exam questions and code them by topic, learning objectives and level of difficulty. That way, the program knows how to distribute questions. (He said instructors should budget roughly 10 extra hours per week during the course for this task.)

The payoff, he said, comes later in the term. In the study, professors reported recouping an average of 80 hours by using the e-exams. Faculty don’t have to hand-grade tests (that often being a deterrent for the Friday test, Catanach notes), and graduate students or administrative staff can help prepare the test banks, the report points out.

Since tests are taken from afar, class time can be used for other purposes. Students are less likely to ask about test results during sessions, the study says, because the computer program gives them immediate results and points to pages where they can find out why their answers were incorrect. Satris said this type of system likely dissuades students from grade groveling, because the explanations are all there on the computer. He said it also make sense in other ways.

“I like that professors can truly say, ‘I don’t know what’s going to be on the test. There’s a question bank; it’s out of my control,’ ” he said.

And then there’s the common argument about administrative efficiency: An institution can keep a permanent electronic record of its students.

Survey results showed that Villanova students, who Catanach said were more likely to have their own laptop computers and be familiar with e-technology, responded better to the electronic testing system than did students at Philadelphia, who weren’t as tech savvy. Both Catanach and Satris said the e-testing programs are not likely to excite English and philosophy professors, whose disciplines call for essay questions rather than computer-graded content.

From a testing perspective, Catanach said the programs can be most helpful for faculty with large classes who need to save time on grading. That’s why the programs have proven popular at community colleges in some of the larger states, he said.

“It works for almost anyone who wants to have periodic assessment,” he said. “How much does the midterm and final motivate students to keep up with material? It doesn’t. It motivates cramming. This is a tool to help students keep up with the material.”

August 29, 2007 reply from Stokes, Len [stokes@SIENA.EDU]

I am also a strong proponent of active learning strategies. I have the luxury of a small class size. Usually fewer than 30 so I can adapt my classes to student interaction and can have periodic assessment opportunities as it fits the flow of materials rather than the calendar. I still think a push toward smaller classes with more faculty face time is better than computer tests. One lecture and one case day does not mean active learning. It is better than no case days but it is still a lecture day. I don’t have real lecture days every day involves some interactive material from the students.

While I admit I can’t pick up all trends in grading the tests, but I do pick up a lot of things so I have tendency to have a high proportion of essays and small problems. I then try to address common errors in class and also can look at my approach to teaching the material.

Len

Bob Jensen attempts to make a case that self learning is more effective for metacognitive reasons --- http://www.trinity.edu/rjensen/265wp.htm
This document features the research of Tony Catanach, David Croll, Bob Grinaker, and  Noah Barsky.

Bob Jensen's threads on "Online Education Effectiveness and Testing" are at http://www.trinity.edu/rjensen/Assess.htm#OnlineOffCampus

Bob Jensen's threads on the myths of online education are at http://www.trinity.edu/rjensen/000aaa/thetools.htm#Myths


Team Learning versus Lecture Learning

May 22, 2010 message from J. S. Gangolly [gangolly@CSC.ALBANY.EDU]

Bob,
Team-based learning is an alternative to lecturing.
However, the students are tested individually as well as teams. 
As I understand, each session starts with an individual quizz and 
then the team discussion of problems (usually cases) and their solution.
Some references:
http://ctl.stanford.edu/Tomprof/postings/750.html   
http://papers.ssrn.com/sol3/papers.cfm?abstract_id=1546479  
http://www.cshe.unimelb.edu.au/pdfs/Team_problem.pdf  
http://www.ecampus.com/bk_searchresult.asp?qtype=ISBN&qsearch=9781579220860  
Jagdish

 


Accounting Professors in Support of Online Testing That, Among Other Things, Reduces Cheating
These same professors became widely known for their advocacy of self-learning in place of lecturing

"In Support of the E-Test," by Elia Powers, Inside Higher Ed, August 29, 2007 --- http://www.insidehighered.com/news/2007/08/29/e_test

Critics of testing through the computer often argue that it’s difficult to tell if students are doing their own work. It’s also unclear to some professors whether using the technology is worth their while. A new study makes the argument that giving electronic tests can actually reduce cheating and save faculty time.

Anthony Catanach Jr. and Noah Barsky, both associate professors of accounting at the Villanova School of Business, came to that conclusion after speaking with faculty members and analyzing the responses of more than 100 students at Villanova and Philadelphia University. Both Catanach and Barsky teach a course called Principles of Managerial Accounting that utilizes the WebCT Vista e-learning platform. The professors also surveyed undergraduates at Philadelphia who took tests electronically.

The Villanova course follows a pattern of Monday lecture, Wednesday case assignment, Friday assessment. The first two days require in-person attendance, while students can check in Friday from wherever they are.

“It never used to make sense to me why at business schools you have Friday classes,” Catanach said. “As an instructor it’s frustrating because 30 percent of the class won’t show up, so you have to redo material. We said, how can we make that day not lose its effectiveness?”

The answer, he and Barsky determined, was to make all electronically submitted group work due on Fridays and have that be electronic quiz day. That’s where academic integrity came into play. Since the professors weren’t requiring students to be present to take the exams, they wanted to deter cheating. Catanach said programs like the one he uses mitigate the effectiveness of looking up answers or consulting friends.

In electronic form, questions are given to students in random order so that copying is difficult. Professors can change variables within a problem to make sure that each test is unique while also ensuring a uniform level of difficulty. The programs also measure how much time a student spends on each question, which could signal to an instructor that a student might have slowed to use outside resources. Backtracking on questions generally is not permitted. Catanach said he doesn’t pay much attention to time spent on individual questions. And since he gives his students a narrow time limit to finish their electronic quizzes, consulting outside sources would only lead students to be rushed by the end of the exam, he added.

Forty-five percent of students who took part in the study reported that the electronic testing system reduced the likelihood of their cheating during the course.

Stephen Satris, director of the Center for Academic Integrity at Clemson University, said he applauds the use of technology to deter academic dishonesty. Students who take these courses might think twice about copying or plagiarizing on other exams, he said.

“It’s good to see this program working,” Satris said. “It does an end run around cheating.”

The report also makes the case that both faculty and students save time with e-testing. Catanach is up front about the initial time investment: For instructors to make best use of the testing programs, they need to create a “bank” of exam questions and code them by topic, learning objectives and level of difficulty. That way, the program knows how to distribute questions. (He said instructors should budget roughly 10 extra hours per week during the course for this task.)

The payoff, he said, comes later in the term. In the study, professors reported recouping an average of 80 hours by using the e-exams. Faculty don’t have to hand-grade tests (that often being a deterrent for the Friday test, Catanach notes), and graduate students or administrative staff can help prepare the test banks, the report points out.

Since tests are taken from afar, class time can be used for other purposes. Students are less likely to ask about test results during sessions, the study says, because the computer program gives them immediate results and points to pages where they can find out why their answers were incorrect. Satris said this type of system likely dissuades students from grade groveling, because the explanations are all there on the computer. He said it also make sense in other ways.

“I like that professors can truly say, ‘I don’t know what’s going to be on the test. There’s a question bank; it’s out of my control,’ ” he said.

And then there’s the common argument about administrative efficiency: An institution can keep a permanent electronic record of its students.

Survey results showed that Villanova students, who Catanach said were more likely to have their own laptop computers and be familiar with e-technology, responded better to the electronic testing system than did students at Philadelphia, who weren’t as tech savvy. Both Catanach and Satris said the e-testing programs are not likely to excite English and philosophy professors, whose disciplines call for essay questions rather than computer-graded content.

From a testing perspective, Catanach said the programs can be most helpful for faculty with large classes who need to save time on grading. That’s why the programs have proven popular at community colleges in some of the larger states, he said.

“It works for almost anyone who wants to have periodic assessment,” he said. “How much does the midterm and final motivate students to keep up with material? It doesn’t. It motivates cramming. This is a tool to help students keep up with the material.”

August 29, 2007 reply from Stokes, Len [stokes@SIENA.EDU]

I am also a strong proponent of active learning strategies. I have the luxury of a small class size. Usually fewer than 30 so I can adapt my classes to student interaction and can have periodic assessment opportunities as it fits the flow of materials rather than the calendar. I still think a push toward smaller classes with more faculty face time is better than computer tests. One lecture and one case day does not mean active learning. It is better than no case days but it is still a lecture day. I don’t have real lecture days every day involves some interactive material from the students.

While I admit I can’t pick up all trends in grading the tests, but I do pick up a lot of things so I have tendency to have a high proportion of essays and small problems. I then try to address common errors in class and also can look at my approach to teaching the material.

Len

Bob Jensen attempts to make a case that self learning is more effective for metacognitive reasons --- http://www.trinity.edu/rjensen/265wp.htm
This document features the research of Tony Catanach, David Croll, Bob Grinaker, and  Noah Barsky.

Bob Jensen's threads on "Online Education Effectiveness and Testing" are at http://www.trinity.edu/rjensen/Assess.htm#OnlineOffCampus

Bob Jensen's threads on the myths of online education are at http://www.trinity.edu/rjensen/000aaa/thetools.htm#Myths

 


Hi Yvonne,

For what it is worth, my advice to new faculty is at http://www.trinity.edu/rjensen/000aaa/newfaculty.htm 

One thing to remember is that the employers of our students (especially the public accounting firms) are very unhappy with our lecture/drill pedagogy at the introductory and intermediate levels. They believe that such pedagogy turns away top students, especially creative and conceptualizing students. Employers  believe that lecture/drill pedagogy attracts savant-like memorizers who can recite their lessons book and verse but have few creative talents and poor prospects for becoming leaders. The large accounting firms believed this so strongly that they donated several million dollars to the American Accounting Association for the purpose of motivating new pedagogy experimentation. This led to the Accounting Change Commission (AECC) and the mixed-outcome experiments that followed. See http://accounting.rutgers.edu/raw/aaa/facdev/aecc.htm 

The easiest pedagogy for faculty is lecturing, and it is appealing to busy faculty who do not have time for students outside the classroom. When lecturing to large classes it is even easier because you don't have to get to know the students and have a great excuse for using multiple choice examinations and graduate student teaching assistants. I always remember an economics professor at Michigan State University who said that when teaching basic economics it did not matter whether he had a live class of 300 students or a televised class of 3,000 students. His full-time teaching load was three hours per week in front of a TV camera. He was a very good lecturer and truly loved his three-hour per week job!

Lecturing appeals to faculty because it often leads to the highest teaching evaluations.  Students love faculty who spoon feed and make learning seem easy.  It's much easier when mom or dad spoon the pudding out of the jar than when you have to hold your own spoon and/or find your own jar.

An opposite but very effective pedagogy is the AECC (University of Virginia) BAM Pedagogy that entails live classrooms with no lectures. BAM instructors think it is more important for students to learn on their own instead of sitting through spoon-fed learning lectures. I think it takes a special kind of teacher to pull off the astoundingly successful BAM pedagogy. Interestingly, it is often some of our best lecturers who decided to stop lecturing because they experimented with the BAM and found it to be far more effective for long-term memory. The top BAM enthusiasts are Tony Catanach at Villanova University and David Croll at the University of Virginia. Note, however, that most BAM applications have been at the intermediate accounting level. I have my doubts (and I think BAM instructors will agree) that BAM will probably fail at the introductory level. You can read about the BAM pedagogy at http://www.trinity.edu/rjensen/265wp.htm 

At the introductory level we have what I like to call the Pincus (User Approach) Pedagogy. Karen Pincus is now at the University of Arkansas, but at the time that her first learning experiments were conducted, she taught basic accounting at the University of Southern California. The Pincus Pedagogy is a little like both the BAM and the case method pedagogies. However, instead of having prepared learning cases, the Pincus Pedagogy sends students to on-site field visitations where they observe on-site operations and are then assigned tasks to creatively suggest ways of improving existing accounting, internal control, and information systems. Like the BAM, the Pincus Pedagogy avoids lecturing and classroom drill. Therein lies the controversy. Students and faculty in subsequent courses often complain that the Pincus Pedagogy students do not know the fundamental prerequisites of basic accounting needed for intermediate and advanced-level accounting courses.  Two possible links of interest on the controversial Pincus Pedagogy are as follows:  

Where the Pincus Pedagogy and the BAM Pedagogy differ lies in subject matter itself and stress on creativity. The BAM focuses on traditional subject matter that is found in such textbooks as intermediate accounting textbooks. The BAM Pedagogy simply requires that students learn any way they want to learn on their own since students remember best what they learned by themselves. The Pincus Pedagogy does not focus on much of the debit and credit "rules" found in most traditional textbooks. Students are required to be more creative at the expense of memorizing the "rules."

The Pincus Pedagogy is motivated by the belief that traditional lecturing/drill pedagogy at the basic accounting and tax levels discourages the best and more-creative students to pursue careers in the accountancy profession. The BAM pedagogy is motivated more by the belief that lecturing is a poor pedagogy for long-term memory of technical details. What is interesting is that the leading proponents of getting away from the lecture/drill pedagogy (i.e., Karen Pincus and Anthony Catenach) were previously two of the very best lecturers in accountancy. If you have ever heard either of them lecture, I think you would agree that you wish all your lecturers had been only half as good. I am certain that both of these exceptional teachers would agree that lecturing is easier than any other alternatives. However, they do not feel that lecturing is the best alternative for top students.

Between lecturing and the BAM Pedagogy, we have case method teaching. Case method teaching is a little like lecturing and a little like the BAM with some instructors providing answers in case wrap ups versus some instructors forcing students to provide all the answers. Master case teachers at Harvard University seldom provide answers even in case wrap ups, and often the cases do not have any known answer-book-type solutions. The best Harvard cases have alternative solutions with success being based upon discovering and defending an alternative solution. Students sometimes interactively discover solutions that the case writers never envisioned. I generally find case teaching difficult at the undergraduate level if students do not yet have the tools and maturity to contribute to case discussions. Interestingly, it may be somewhat easier to use the BAM at the undergraduate level than Harvard-type cases. The reason is that BAM instructors are often dealing with more rule-based subject matter such as intermediate accounting or tax rather than conceptual subject matter such as strategic decision making, business valuation, and financial risk analysis.

The hardest pedagogy today is probably a Socratic pedagogy online with instant messaging communications where an instructor who's on call about 60 hours per week from his or her home. The online instructor monitors the chats and team communications between students in the course at most any time of day or night. Amy Dunbar can tell you about this tedious pedagogy since she's using it for tax courses and will be providing a workshop that tells about how to do it and how not to do it. The next scheduled workshop precedes the AAA Annual Meetings on August 1, 2003 in Hawaii. You can also hear Dr. Dunbar and view her PowerPoint show from a previous workshop at http://www.cs.trinity.edu/~rjensen/002cpe/02start.htm#2002 

In conclusion, always remember that there is no optimal pedagogy in all circumstances. All learning is circumstantial based upon such key ingredients as student maturity, student motivation, instructor talent, instructor dedication, instructor time, library resources, technology resources, and many other factors that come to bear at each moment in time. And do keep in mind that how you teach may determine what students you keep as majors and what you turn away. 

I tend to agree with the accountancy firms that contend that traditional lecturing probably turns away many of the top students who might otherwise major in accountancy. 

At the same time, I tend to agree with students who contend that they took accounting courses to learn accounting rather than economics, computer engineering, and behavioral science.

Bob Jensen

-----Original Message----- 
From: Lou&Bonnie [mailto:gyp1@EARTHLINK.NET]  
Sent: Thursday, January 16, 2003 5:03 PM

I am a beginning accounting instructor (part-time) at a local community college. I am applying for a full-time faculty position, but am having trouble with a question. Methodology in accounting--what works best for a diversified group of individuals. Some students work with accounting, but on a computer and have no understanding of what the information they are entering really means to some individuals who have no accounting experience whatsoever. What is the best methodology to use, lecture, overhead, classroom participation? I am not sure and I would like your feedback. Thank you in advance for your help. 

Yvonne


February 5,, 2003 message from James Borden [james.borden@villanova.edu]

Bob,

I thought you might be interested in another curriculum innovation that is taking place at Villanova, once again involving Tony Catanach, along with Noah Barsky. Noah is one of our young professors at Villanova who is an outstanding teacher, and has been committed to developing the Business Planning Model (BPM) approach to teaching Management Accounting for some time now. If you have any questions about the paper, feel free to contact Noah at noah.barsky@villanova.edu

Thank you for continuing to support the BAM approach to teaching Intermediate as well!

Jim

Note that Noah has a PDF file that he will probably send to you if you request it from him.


February 5, 2003 message from Milt Cohen

-----Original Message----- 
From: Milt Cohen, Accounting Instructor [mailto:uncmlt@JUNO.COM]  
Sent: Wednesday, February 05, 2003 9:54 PM 

A thought for the bright minded on this web site, after watching an H & R Block TV commercial where they boast about getting more refunds for taxpayers when they review their prior year's tax forms, do you think that if they find that the taxpayer OWES money for a prior year and......... if the taxpayer says, "hell no, I won't pay" ............are they obligated to report the short pay (tax owed) to the IRS?

Interesting letter in the current Journal of Accountancy , Feb. 2003 issue on pages 11 and 12, a professor of Accounting at Mesa State College in Grand Junction, Colorado, writes about how poorly the current text book used in his college classes do not provide explanation as to how transactions (of businesses) get recorded into the books and records of a business. And that the emphasis is on financial statement analysis. I can't help recalling how I (and my classmates) slaved over work problems in accounting classes in the 1950s. And how I emphasis (in my classes) the bookkeeping procedures. It's hard to realize that college Accounting classes ignore the bookkeeping phase of the profession. Without a bookkeeper, there is no basis for financial statement analysis.

Just a couple of thoughts.


Appendix 7

How the Brain Deals With Information Overload

"Attention Must Be Paid," by Scott McLemee, Inside Higher Ed, November 26, 2008 --- http://www.insidehighered.com/views/2008/11/26/mclemee 

Actually looking at the e-mail would no doubt have informed me that it was a matter of paying a modest fee to some enterprising soul, probably in the Cayman Islands. Instead, I deleted this message on the basis of the subject line alone, along with a dozen other such communications. Meanwhile, my eyeballs were unwittingly drawn to a video loop of a woman screaming in terror – horrified at high credit card interest rates, which she could reduce via a company that advertises with my e-mail provider.

Then my cell phone emitted a short burst of music, announcing that someone had just left a text message.

All par for the course, of course. (At least I wasn’t driving.) The demands on our attention have now become a matter for professional expertise: An organization for specialists, the Information Overload Research Group, was formally incorporated as a nonprofit this summer and held its first conference in August. A substantial technical literature on interruption now exists. And one recent consideration of the world economic crisis suggests it has been exacerbated by all the data now sloshing around the globe: “We have far too much information today and that impedes our decision-making abilities and throttles our ability to resolve crises.”

The weak link in the information age seems to be our human hard-wiring. So one gathers from The Overflowing Brain: Information Overload and the Limits of Working Memory (Oxford University Press) by Torkel Klingberg, who is a professor of developmental cognitive neuroscience at the Stockholm Brain Institute. A review of recent research on how attention and memory actually function within our gray matter, it is a work of scientific popularization rather than a handbook on how to minimize the cognitive drain of distraction.

But there may be some advantage to knowing how the systems in our heads actually operate – and it is Klingberg’s contention that, in spite of everything, those systems may actually benefit from the sometimes excessive demands our environments now place on our capacity to process the data flux. The human brain itself has not changed much in either anatomy or volume over the past 40,000 years. So at one level it seems natural that we should experience a cognitive bottleneck in handling the masses of information being hurled at us daily.

To simplify Klingberg’s already pared-down analysis, we can distinguish between two kinds of attention. One is controlled attention: the directed effort to apply one’s concentration to a particular task. The other is stimulus-driven attention, which is an involuntary response to something happening in the environment. (You can tune out the conversations going on around you in a restaurant. But if a waiter drops a tray full of dishes, it is going to impose itself on your awareness.)

But it’s not as if these forms of attention are – as it may seem – different manifestations of the same state of consciousness: researchers have found from tests that the controlled and stimulus-driven attention “seem fairly independent of one another,” says Klingberg, which may mean “that there are different parts of the brain, or different brain processes” involved in them.

Likewise, there is a distinction between the kind of memory that allows you to recall an event from five years ago and a set of information connected with a problem you are trying to solve. Your recollections of yesteryear are part of long-term memory, which can be mysteriously capacious. By contrast, there are definite limitations on how much task-oriented data can be held in your “working memory.” (Evidently there are grounds for debate among researchers over whether or not this is the same as “short-term memory,” but we’ll just stick to Klingberg’s preferred usage.)

As with the forms of attention, the distinction between long-term and working memory corresponds to different processes within the brain, occurring within different parts of its geography. But there is evidence that (as you might expect) working memory and controlled attention are closely related. People who score lower on tests for the ability to retain information in their working memory tend to have more difficulty in focusing attention on a complex task. “It might not come as too much of a surprise,” says Klingberg, “to find that working memory capacity correlates highly with reading comprehension.”

Klingberg reports that a two-year study in his lab showed that it was possible to increase working-memory capacity: “children who had done a certain type of computerized memory task, such as remembering positions in a four-by-four grid and clicking a mouse button, improved at other, noncomputerized types of working memory too.... We had shown that the systems are not static and that the limits of working memory capacity can be stretched.”

Further study suggested that this improvement also corresponded to increased problem-solving skills. Our brains may still have many of the same fundamental limitations as the Cro-Magnon model, but there is also some degree of plasticity in how we can use and develop it.

Which brings us to Klingberg’s most surprising and even counterintuitive suggestion. Multitasking often threatens to overload the working memory. But at the same time, it’s clear that we can actually manage it, at least to some degree – reading a newspaper while walking on a treadmill, for example, and occasionally glancing up at the TV screen to see what’s breaking on CNN.

“There is, fortunately, no research suggesting that exposure to mentally more demanding or challenging situations impairs our powers of concentration,” writes Klingberg. “Indeed, there is much that points to the contrary: it is in situations that push the boundaries of our abilities that we train our brains the most.”

But even if our basic ability to process information is increasing, a growing “discrepancy between demand and capacity” may account for the common sense of losing focus.

“You are very possibly 10 percent better at talking on the phone while erasing spam today than you were three years ago. On the other hand, the number of e-mails you receive per day has probably shot up about 200 percent. There is, therefore, no contradiction between the feeling that your abilities are inadequate and the improvement of those abilities.”

Well, that is some comfort – if not much. It’s been said that the scarcest resource in an information society is not information but attention. Klingberg’s book, interesting as it is, does not leave the reader with any way around that. In any case, a great deal of the “information” (such as my Ph.D. offer this morning) turns out to be noise, rather than anything meaningful. It’s necessary to pay just enough attention to decide not to pay any more attention – a kind of catch-22.

Which is why it sometimes feels like one’s brain is being nibbled by carnivorous gnats. It would be good if Dr. Klingberg and his colleagues would apply themselves to finding a salve. Or better yet, a repellent.


"Better Learning With Sites and Sound," by Andy Guess, Inside Higher Ed, December 3, 2008 --- http://www.insidehighered.com/news/2008/12/03/audio

Students in four graduate courses at West Virginia University worked on and submitted group projects in two different ways, alternating for each assignment: using Microsoft Word to save, track changes, add comments and send files back and forth as e-mail attachments; and sharing files and editing them online using Buzzword. According to the study, the students “were more likely to use graphics, charts, links, etc. in Buzzword because of the ease of inclusion” than in Word, possibly as a function of the interface’s comparative ease of use.

Perhaps more significantly, the study found that they were “more likely to explain more complex concepts using a combination of text and non-text based materials. The majority of participants ... expressed the view that it was easier to express themselves at a higher cognitive level when they could present material using multiple media sources.” They also had higher levels of satisfaction.

Although the study had a small sample size, Ice suggested in an interview that the “multiple forms of sensory input” such as charts, links and graphics not only make the information more understandable to the reader “but apparently ... students are learning more from that process as well"; a process that’s not too different from the wiki editing experience. He is preparing a larger follow-up study with at least six different institutions around the world.

In theory, then, collaborations using Web-based editing tools can potentially boost understanding, at least visually.

But learning doesn’t just occur in the visual realm. Ice co-authored a study, currently under review, that examines how listening to spoken words while also reading at the same time can improve students’ learning experiences. In particular, he and his colleagues attempted a method in which professors record comments on students’ written assignments, which students can then listen to as they read along at corresponding points in the text. They can also record their own responses and continue back and forth in a sort of audio conversation.

While the Web-based collaboration tools are free, Ice’s method makes use of embedded audio features in Adobe Acrobat Pro. If institutions own the software, however, students can listen to the audio (and record their own additions) on the free and commonly used Acrobat Reader. (Adobe provided 60 copies of Acrobat Pro for the study but no additional funding or support.)

The forthcoming paper found that students in the audio study were at least three times more likely to take professors’ comments into account in their final assignments if they were in audio form as opposed to written. What they found, Ice said, was that “students are actually listening to the instructor and reading what they wrote so they have two sensory modes working at the same time,” which could actually improve cognition.

Since the paper was produced, Ice added, additional research has confirmed that the findings are generalizable over many different contexts, such as types of learners and types of institutions.

But a central component of the effect is what the authors call the “asynchronous audio feedback” aspect of the comments: that students can listen to previously recorded audio while they’re reading what it is referring to.

“I’ve tried other methods, too, where you send the students a document and then also send them a [separate] sound file, and the effect is not nearly as strong; as a matter of fact, it’s barely significant when you do that,” Ice said.

Continued in article


"15 Years of Cutting-Edge Thinking on Understanding the Mind," Edited by Maria Popova, The Atlantic, September 14, 2011 ---
http://www.theatlantic.com/life/archive/2011/09/15-years-of-cutting-edge-thinking-on-understanding-the-mind/245006/
Bringing Jonathan Haidt, Martin Seligman, Alison Gopnik, Steven Pinker, Philip Zimbardo, and others together between two covers

For the past 15 years, literary-agent-turned-crusader-of-human-progress John Brockman has been a remarkable curator of curiosity, long before either "curator" or "curiosity" was a frivolously tossed around buzzword. His Edge.org has become an epicenter of bleeding-edge insight across science, technology and beyond, hosting conversations with some of our era's greatest thinkers (and, once a year, asking them some big questions). Last month marked the release of The Mind, the first volume in The Best of Edge Series, presenting eighteen provocative, landmark pieces -- essays, interviews, transcribed talks -- from the Edge archive. The anthology reads like a who's who of Brain Pickings favorites across psychology, evolutionary biology, social science, technology, and more. And, perhaps equally interestingly, the tome -- most of the materials in which are available for free online -- is an implicit manifesto for the enduring power of books as curatorial capsules of ideas. Brockman writes in the book's introduction:

While there's no doubt about the value of online presentations, the role of books, whether bound and printed or presented electronically, is still an invaluable way to present important ideas. Thus, we are pleased to be able to offer this series of books to the public.

Here's a small sampling of the treasure chest between The Mind's covers:

In "Eudaemonia: The Good Life" (2004), Martin Seligman, father of positive psychology whom you might recall as the author of Flourish and Learned Optimism, one of our 7 essential books on optimism, explores what he calls the "third form of happiness," which lies in:

...knowing what your highest straights are and deploying those in the service of something you believe in is larger than you are. There's no shortcut to that. That's what life is about. There will likely be a pharmacology of pleasure, and there may be a pharmacology of positive emotion generally, but it's unlikely there'll be an interesting pharmacology of flow. And it's impossible that there'll be a pharmacology of meaning.

In "Moral Psychology and the Misunderstanding of Religion" (2007), psychologist Jonathan Haidt (whose The Happiness Hypothesis you might recall as one of our 7 favorite books on happiness) notes:

[I]t might seem obvious to you that contractual societies are good, modern, creative, and free, whereas beehive societies reek of feudalism, fascism, and patriarchy. And, as a secular liberal I agree that contractual societies such as those of Western Europe offer the best hope for living peacefully together in our increasingly diverse modern nations (although it remains to be seen if Europe can solve its current diversity problems). I just want to make one point, however, that should give constructualists pause: surveys have long shown that religious believers in the United States are happier, healthier, longer-lived, and more generous to charity and to each other than are secular people.
In "Amazing Babies" (2009), psychologist and philosopher Alison Gopnik laid the foundations for her The Philosophical Baby, one of this year's must-read books by TED Global speakers:
We've known for a long time that human children are the best learning machines in the universe, but it has always been like the mystery of the hummingbirds. We know that they fly, but we don't know how they can possibly do it. We could say that babies learn, but we didn't know how.

Harvard's Steven Pinker, whose illuminating insights on violence and human nature you might recall and who penned one of our 5 favorite books on language, wrote in "Organs of Computation" (1997), long before the hype of contemporary quasi-sciences like neuromarketing:

Most of the assumptions about the mind that underlie current discussions are many decades out of date. [L]ook at the commentaries on human affairs by pundits and social critics. They say we're 'conditioned' to do this, or 'brainwashed' to do that, or 'socialized' to believe such and such. Where do these ideas come from? From the behaviorism of the 1920s, from bad Cold War movies from the 1950s, from folklore about the effects of family upbringing that behavior genetics has shown to be false. The basic understanding that the human mind is a remarkably complex processor of information, an 'organ of extreme perfection and complication,' to use Darwin's phrase, has not made it into the mainstream of intellectual life.

Continued in article

Bob Jensen's threads on metacognition are at
http://www.trinity.edu/rjensen/265wp.htm

 

 


Appendix 8

The Importance of Paying Attention for Longer Periods of Time

Once Mischel began analyzing the results, he noticed that low delayers, the children who rang the bell quickly, seemed more likely to have behavioral problems, both in school and at home. They got lower S.A.T. scores. They struggled in stressful situations, often had trouble paying attention, and found it difficult to maintain friendships. The child who could wait fifteen minutes had an S.A.T. score that was, on average, two hundred and ten points higher than that of the kid who could wait only thirty seconds.
"Don’t! The secret of self-control," by Jonah Lehrer, The New Yorker, May 18, 2009 ---
http://www.newyorker.com/reporting/2009/05/18/090518fa_fact_lehrer
Jim Mahar clued me into this link.

In the late nineteen-sixties, Carolyn Weisz, a four-year-old with long brown hair, was invited into a “game room” at the Bing Nursery School, on the campus of Stanford University. The room was little more than a large closet, containing a desk and a chair. Carolyn was asked to sit down in the chair and pick a treat from a tray of marshmallows, cookies, and pretzel sticks. Carolyn chose the marshmallow. Although she’s now forty-four, Carolyn still has a weakness for those air-puffed balls of corn syrup and gelatine. “I know I shouldn’t like them,” she says. “But they’re just so delicious!” A researcher then made Carolyn an offer: she could either eat one marshmallow right away or, if she was willing to wait while he stepped out for a few minutes, she could have two marshmallows when he returned. He said that if she rang a bell on the desk while he was away he would come running back, and she could eat one marshmallow but would forfeit the second. Then he left the room.

Although Carolyn has no direct memory of the experiment, and the scientists would not release any information about the subjects, she strongly suspects that she was able to delay gratification. “I’ve always been really good at waiting,” Carolyn told me. “If you give me a challenge or a task, then I’m going to find a way to do it, even if it means not eating my favorite food.” Her mother, Karen Sortino, is still more certain: “Even as a young kid, Carolyn was very patient. I’m sure she would have waited.” But her brother Craig, who also took part in the experiment, displayed less fortitude. Craig, a year older than Carolyn, still remembers the torment of trying to wait. “At a certain point, it must have occurred to me that I was all by myself,” he recalls. “And so I just started taking all the candy.” According to Craig, he was also tested with little plastic toys—he could have a second one if he held out—and he broke into the desk, where he figured there would be additional toys. “I took everything I could,” he says. “I cleaned them out. After that, I noticed the teachers encouraged me to not go into the experiment room anymore.”

Footage of these experiments, which were conducted over several years, is poignant, as the kids struggle to delay gratification for just a little bit longer. Some cover their eyes with their hands or turn around so that they can’t see the tray. Others start kicking the desk, or tug on their pigtails, or stroke the marshmallow as if it were a tiny stuffed animal. One child, a boy with neatly parted hair, looks carefully around the room to make sure that nobody can see him. Then he picks up an Oreo, delicately twists it apart, and licks off the white cream filling before returning the cookie to the tray, a satisfied look on his face.

Most of the children were like Craig. They struggled to resist the treat and held out for an average of less than three minutes. “A few kids ate the marshmallow right away,” Walter Mischel, the Stanford professor of psychology in charge of the experiment, remembers. “They didn’t even bother ringing the bell. Other kids would stare directly at the marshmallow and then ring the bell thirty seconds later.” About thirty per cent of the children, however, were like Carolyn. They successfully delayed gratification until the researcher returned, some fifteen minutes later. These kids wrestled with temptation but found a way to resist.

The initial goal of the experiment was to identify the mental processes that allowed some people to delay gratification while others simply surrendered. After publishing a few papers on the Bing studies in the early seventies, Mischel moved on to other areas of personality research. “There are only so many things you can do with kids trying not to eat marshmallows.”

But occasionally Mischel would ask his three daughters, all of whom attended the Bing, about their friends from nursery school. “It was really just idle dinnertime conversation,” he says. “I’d ask them, ‘How’s Jane? How’s Eric? How are they doing in school?’ ” Mischel began to notice a link between the children’s academic performance as teen-agers and their ability to wait for the second marshmallow. He asked his daughters to assess their friends academically on a scale of zero to five. Comparing these ratings with the original data set, he saw a correlation. “That’s when I realized I had to do this seriously,” he says. Starting in 1981, Mischel sent out a questionnaire to all the reachable parents, teachers, and academic advisers of the six hundred and fifty-three subjects who had participated in the marshmallow task, who were by then in high school. He asked about every trait he could think of, from their capacity to plan and think ahead to their ability to “cope well with problems” and get along with their peers. He also requested their S.A.T. scores.

Once Mischel began analyzing the results, he noticed that low delayers, the children who rang the bell quickly, seemed more likely to have behavioral problems, both in school and at home. They got lower S.A.T. scores. They struggled in stressful situations, often had trouble paying attention, and found it difficult to maintain friendships. The child who could wait fifteen minutes had an S.A.T. score that was, on average, two hundred and ten points higher than that of the kid who could wait only thirty seconds.

Carolyn Weisz is a textbook example of a high delayer. She attended Stanford as an undergraduate, and got her Ph.D. in social psychology at Princeton. She’s now an associate psychology professor at the University of Puget Sound. Craig, meanwhile, moved to Los Angeles and has spent his career doing “all kinds of things” in the entertainment industry, mostly in production. He’s currently helping to write and produce a film. “Sure, I wish I had been a more patient person,” Craig says. “Looking back, there are definitely moments when it would have helped me make better career choices and stuff.”

Mischel and his colleagues continued to track the subjects into their late thirties—Ozlem Ayduk, an assistant professor of psychology at the University of California at Berkeley, found that low-delaying adults have a significantly higher body-mass index and are more likely to have had problems with drugs—but it was frustrating to have to rely on self-reports. “There’s often a gap between what people are willing to tell you and how they behave in the real world,” he explains. And so, last year, Mischel, who is now a professor at Columbia, and a team of collaborators began asking the original Bing subjects to travel to Stanford for a few days of experiments in an fMRI machine. Carolyn says she will be participating in the scanning experiments later this summer; Craig completed a survey several years ago, but has yet to be invited to Palo Alto. The scientists are hoping to identify the particular brain regions that allow some people to delay gratification and control their temper. They’re also conducting a variety of genetic tests, as they search for the hereditary characteristics that influence the ability to wait for a second marshmallow.

If Mischel and his team succeed, they will have outlined the neural circuitry of self-control. For decades, psychologists have focussed on raw intelligence as the most important variable when it comes to predicting success in life. Mischel argues that intelligence is largely at the mercy of self-control: even the smartest kids still need to do their homework. “What we’re really measuring with the marshmallows isn’t will power or self-control,” Mischel says. “It’s much more important than that. This task forces kids to find a way to make the situation work for them. They want the second marshmallow, but how can they get it? We can’t control the world, but we can control how we think about it.”

Walter Mischel is a slight, elegant man with a shaved head and a face of deep creases. He talks with a Brooklyn bluster and he tends to act out his sentences, so that when he describes the marshmallow task he takes on the body language of an impatient four-year-old. “If you want to know why some kids can wait and others can’t, then you’ve got to think like they think,” Mischel says.

Mischel was born in Vienna, in 1930. His father was a modestly successful businessman with a fondness for café society and Esperanto, while his mother spent many of her days lying on the couch with an ice pack on her forehead, trying to soothe her frail nerves. The family considered itself fully assimilated, but after the Nazi annexation of Austria, in 1938, Mischel remembers being taunted in school by the Hitler Youth and watching as his father, hobbled by childhood polio, was forced to limp through the streets in his pajamas. A few weeks after the takeover, while the family was burning evidence of their Jewish ancestry in the fireplace, Walter found a long-forgotten certificate of U.S. citizenship issued to his maternal grandfather decades earlier, thus saving his family.

The family settled in Brooklyn, where Mischel’s parents opened up a five-and-dime. Mischel attended New York University, studying poetry under Delmore Schwartz and Allen Tate, and taking studio-art classes with Philip Guston. He also became fascinated by psychoanalysis and new measures of personality, such as the Rorschach test. “At the time, it seemed like a mental X-ray machine,” he says. “You could solve a person by showing them a picture.” Although he was pressured to join his uncle’s umbrella business, he ended up pursuing a Ph.D. in clinical psychology at Ohio State.

But Mischel noticed that academic theories had limited application, and he was struck by the futility of most personality science. He still flinches at the naïveté of graduate students who based their diagnoses on a battery of meaningless tests. In 1955, Mischel was offered an opportunity to study the “spirit possession” ceremonies of the Orisha faith in Trinidad, and he leapt at the chance. Although his research was supposed to involve the use of Rorschach tests to explore the connections between the unconscious and the behavior of people when possessed, Mischel soon grew interested in a different project. He lived in a part of the island that was evenly split between people of East Indian and of African descent; he noticed that each group defined the other in broad stereotypes. “The East Indians would describe the Africans as impulsive hedonists, who were always living for the moment and never thought about the future,” he says. “The Africans, meanwhile, would say that the East Indians didn’t know how to live and would stuff money in their mattress and never enjoy themselves.”

Mischel took young children from both ethnic groups and offered them a simple choice: they could have a miniature chocolate bar right away or, if they waited a few days, they could get a much bigger chocolate bar. Mischel’s results failed to justify the stereotypes—other variables, such as whether or not the children lived with their father, turned out to be much more important—but they did get him interested in the question of delayed gratification. Why did some children wait and not others? What made waiting possible? Unlike the broad traits supposedly assessed by personality tests, self-control struck Mischel as potentially measurable.

Continued in article

Jensen Comment
I think this entire article has a lot to do with metacognition and the importance of self-learning because it is often easier to pay attention when learning on your own.
Bob Jensen's threads on learning memory are at http://www.trinity.edu/rjensen/265wp.htm




How to Train the Aging Brain

"How to Train the Aging Brain," by Barbara Strauch, The New York Times, December 29, 2009 ---
http://www.nytimes.com/2010/01/03/education/edlife/03adult-t.html?hpw

I LOVE reading history, and the shelves in my living room are lined with fat, fact-filled books. There’s “The Hemingses of Monticello,” about the family of Thomas Jefferson’s slave mistress; there’s “House of Cards,” about the fall of Bear Stearns; there’s “Titan,” about John D. Rockefeller Sr.

The problem is, as much as I’ve enjoyed these books, I don’t really remember reading any of them. Certainly I know the main points. But didn’t I, after underlining all those interesting parts, retain anything else? It’s maddening and, sorry to say, not all that unusual for a brain at middle age: I don’t just forget whole books, but movies I just saw, breakfasts I just ate, and the names, oh, the names are awful. Who are you?

Brains in middle age, which, with increased life spans, now stretches from the 40s to late 60s, also get more easily distracted. Start boiling water for pasta, go answer the doorbell and — whoosh — all thoughts of boiling water disappear. Indeed, aging brains, even in the middle years, fall into what’s called the default mode, during which the mind wanders off and begin daydreaming.

Given all this, the question arises, can an old brain learn, and then remember what it learns? Put another way, is this a brain that should be in school?

As it happens, yes. While it’s tempting to focus on the flaws in older brains, that inducement overlooks how capable they’ve become. Over the past several years, scientists have looked deeper into how brains age and confirmed that they continue to develop through and beyond middle age.

Many longheld views, including the one that 40 percent of brain cells are lost, have been overturned. What is stuffed into your head may not have vanished but has simply been squirreled away in the folds of your neurons.

One explanation for how this occurs comes from Deborah M. Burke, a professor of psychology at Pomona College in California. Dr. Burke has done research on “tots,” those tip-of-the-tongue times when you know something but can’t quite call it to mind. Dr. Burke’s research shows that such incidents increase in part because neural connections, which receive, process and transmit information, can weaken with disuse or age.

But she also finds that if you are primed with sounds that are close to those you’re trying to remember — say someone talks about cherry pits as you try to recall Brad Pitt’s name — suddenly the lost name will pop into mind. The similarity in sounds can jump-start a limp brain connection. (It also sometimes works to silently run through the alphabet until landing on the first letter of the wayward word.)

This association often happens automatically, and goes unnoticed. Not long ago I started reading “The Prize,” a history of the oil business. When I got to the part about Rockefeller’s early days as an oil refinery owner, I realized, hey, I already know this from having read “Titan.” The material was still in my head; it just needed a little prodding to emerge.

Recently, researchers have found even more positive news. The brain, as it traverses middle age, gets better at recognizing the central idea, the big picture. If kept in good shape, the brain can continue to build pathways that help its owner recognize patterns and, as a consequence, see significance and even solutions much faster than a young person can.

The trick is finding ways to keep brain connections in good condition and to grow more of them.

“The brain is plastic and continues to change, not in getting bigger but allowing for greater complexity and deeper understanding,” says Kathleen Taylor, a professor at St. Mary’s College of California, who has studied ways to teach adults effectively. “As adults we may not always learn quite as fast, but we are set up for this next developmental step.”

Educators say that, for adults, one way to nudge neurons in the right direction is to challenge the very assumptions they have worked so hard to accumulate while young. With a brain already full of well-connected pathways, adult learners should “jiggle their synapses a bit” by confronting thoughts that are contrary to their own, says Dr. Taylor, who is 66.

Teaching new facts should not be the focus of adult education, she says. Instead, continued brain development and a richer form of learning may require that you “bump up against people and ideas” that are different. In a history class, that might mean reading multiple viewpoints, and then prying open brain networks by reflecting on how what was learned has changed your view of the world.

“There’s a place for information,” Dr. Taylor says. “We need to know stuff. But we need to move beyond that and challenge our perception of the world. If you always hang around with those you agree with and read things that agree with what you already know, you’re not going to wrestle with your established brain connections.”

Such stretching is exactly what scientists say best keeps a brain in tune: get out of the comfort zone to push and nourish your brain. Do anything from learning a foreign language to taking a different route to work.

“As adults we have these well-trodden paths in our synapses,” Dr. Taylor says. “We have to crack the cognitive egg and scramble it up. And if you learn something this way, when you think of it again you’ll have an overlay of complexity you didn’t have before — and help your brain keep developing as well.”

Jack Mezirow, a professor emeritus at Columbia Teachers College, has proposed that adults learn best if presented with what he calls a “disorienting dilemma,” or something that “helps you critically reflect on the assumptions you’ve acquired.”

Dr. Mezirow developed this concept 30 years ago after he studied women who had gone back to school. The women took this bold step only after having many conversations that helped them “challenge their own ingrained perceptions of that time when women could not do what men could do.”

Such new discovery, Dr. Mezirow says, is the “essential thing in adult learning.”

“As adults we have all those brain pathways built up, and we need to look at our insights critically,” he says. “This is the best way for adults to learn. And if we do it, we can remain sharp.”

And so I wonder, was my cognitive egg scrambled by reading that book on Thomas Jefferson? Did I, by exploring the flaws in a man I admire, create a suitably disorienting dilemma? Have I, as a result, shaken up and fed a brain cell or two?

And perhaps it doesn’t matter that I can’t, at times, recall the given name of the slave with whom Jefferson had all those children. After all, I can Google a simple name.

Sally.

Barbara Strauch is The Times’s health editor; her book “The Secret Life of the Grown-Up Brain” will be published in April.

Jensen Comment
At my age, this NYT article hits close to home. One way I exercise my aging brain is messaging to the AECM.

One thing I most certainly note in my old brain are “tots” --- those tip-of-the-tongue times. What helps my memory is a massive Web site that lets me look up tidbits that jog my memory --- I should call it my Website for Tots --- http://www.trinity.edu/rjensen/threads.htm


Appendix 10

That Placebo Effect in Research: Dan Ariely on Tennis Shoes and Toilet Paper


Dan Ariely is James B. Duke Professor of Behavioral Economics at Duke University and is head of the eRationality research group at the
MIT Media Lab.
Dan Ariely --- http://en.wikipedia.org/wiki/Dan_Ariely

Published works

 

"The Science Behind Exercise Footwear," by Dan Ariely, MIT's Technology Review, January 5, 2010 ---
http://www.technologyreview.com/blog/post.aspx?bid=355&bpid=24614&nlid=2647

A few weeks ago Reebok unveiled a walking shoe purported to tone muscles to a greater extent than your average sneaker. All you had to do was slip on a pair of EasyTone and the rest would take care of itself.

Exercise without exercise? Great!

Considering the abracadabra-like quality of the shoe, it’s no surprise that it’s been selling like hotcakes. The question of course is “ does it work”?

According to a recent New York Times article on the topic Reebok has accumulated “15,000 hours’ worth of wear-test data from shoe users who say they notice the difference.” (The company also quotes a study as support, but it’s one they commissioned themselves and only carries a sample size of five.) The two women quoted in the article further echo this sentiment.

Reebok’s head of advanced innovation (and EasyTone mastermind), Bill McInnis, says the shoe works because it offers the kind of imbalance that you get with stability balls at the gym. Unlike other sneakers, which are made flat with comfort in mind, the EasyTone is purposely outfitted with air-filled toe-and-heal “balance pods” in order to simulate the muscle engagement required to walk through sand. With every step, air shifts from one pod to the other, causing the person’s foot to sink and forcing their leg and backside muscles into a workout.

But as the Times article proposes at the end (without explicitly using the term), the shoe’s success could instead come from the placebo effect. Thanks to Reebok’s marketing efforts, buyers pick up the shoes already convinced of their success, a mind frame that may then cause them to walk faster or harder or longer, thereby producing the expected workout – just not for the expected reason.

And there are some reasons to suspect this kind of placebo effect: In a paper by Alia Crum and Ellen Langer. Titled “Mind-Set Matters: Exercise and the Placebo Effect.” In their research they told some maids working in hotels that the work they do (cleaning hotel rooms) is good exercise and satisfies the Surgeon General’s recommendations for an active lifestyle. Other maids were not given this information. 4 weeks later, the informed group perceived themselves to be getting significantly more exercise than before, their weight was lower and they even showed a decrease in blood pressure, body fat, waist-to-hip ratio, and body mass index.

So, maybe exercise affects health are part placebo?

Irrationally Yours

Dan

A One-Hour Video on What it Means to Be Predictably Irrational (July 25, 2008) --- http://financialrounds.blogspot.com/
The video is also at http://www.youtube.com/watch?v=VZv--sm9XXU
This is quite interesting!

From the Financial Rounds Blog on January 25, 2008 --- http://financialrounds.blogspot.com/

"Dan Ariely (Duke University) - Predictably Irrational

Here's a video of Dan Ariely (author of "Predictably Irrational") in his recent talk for the Google Authors program. Ariely has written a fascinating book about some of the cognitive and behavioral biases that most of us exhibit. If you listen carefully, you'll find that he even gives a hint about how to increase your student evaluations --- http://financialrounds.blogspot.com/

 

Summary of what it means to be "predictably irrational" --- http://en.wikipedia.org/wiki/Predictably_Irrational

New York Times Book Review
"Emonomics," by David Berreby, The New York Times, March 16, 2008 ---
http://www.nytimes.com/2008/03/16/books/review/Berreby-t.html?_r=1&oref=slogin

For years, the ideology of free markets bestrode the world, bending politics as well as economics to its core assumption: market forces produce the best solution to any problem. But these days, even Bill Gates says capitalism’s work is “unsatisfactory” for one-third of humanity, and not even Hillary Clinton supports Bill Clinton’s 1990s trade pacts.

Another sign that times are changing is “Predictably Irrational,” a book that both exemplifies and explains this shift in the cultural winds. Here, Dan Ariely, an economist at M.I.T., tells us that “life with fewer market norms and more social norms would be more satisfying, creative, fulfilling and fun.” By the way, the conference where he had this insight wasn’t sponsored by the Federal Reserve, where he is a researcher. It came to him at Burning Man, the annual anarchist conclave where clothes are optional and money is banned. Ariely calls it “the most accepting, social and caring place I had ever been.”

Obviously, this sly and lucid book is not about your grandfather’s dismal science. Ariely’s trade is behavioral economics, which is the study, by experiments, of what people actually do when they buy, sell, change jobs, marry and make other real-life decisions.

To see how arousal alters sexual attitudes, for example, Ariely and his colleagues asked young men to answer a questionnaire — then asked them to answer it again, only this time while indulging in Internet pornography on a laptop wrapped in Saran Wrap. (In that state, their answers to questions about sexual tastes,, violence and condom use were far less respectable.) To study the power of suggestion, Ariely’s team zapped volunteers with a little painful electricity, then offered fake pain pills costing either 10 cents or $2.50 (all reduced the pain, but the more expensive ones had a far greater effect). To see how social situations affect honesty, they created tests that made it easy to cheat, then looked at what happened if they reminded people right before the test of a moral rule. (It turned out that being reminded of any moral code — the Ten Commandments, the non-existent “M.I.T. honor system” — caused cheating to plummet.)

These sorts of rigorous but goofy-sounding experiments lend themselves to a genial, gee-whiz style, with which Ariely moves comfortably from the lab to broad social questions to his own life (why did he buy that Audi instead of a sensible minivan?). He is good-tempered company — if he mentions you in this book, you are going to be called “brilliant,” “fantastic” or “delightful” — and crystal clear about all he describes. But “Predictably Irrational” is a far more revolutionary book than its unthreatening manner lets on. It’s a concise summary of why today’s social science increasingly treats the markets-know-best model as a fairy tale.

At the heart of the market approach to understanding people is a set of assumptions. First, you are a coherent and unitary self. Second, you can be sure of what this self of yours wants and needs, and can predict what it will do. Third, you get some information about yourself from your body — objective facts about hunger, thirst, pain and pleasure that help guide your decisions. Standard economics, as Ariely writes, assumes that all of us, equipped with this sort of self, “know all the pertinent information about our decisions” and “we can calculate the value of the different options we face.” We are, for important decisions, rational, and that’s what makes markets so effective at finding value and allocating work. To borrow from H. L. Mencken, the market approach presumes that “the common people know what they want, and deserve to get it good and hard.”

What the past few decades of work in psychology, sociology and economics has shown, as Ariely describes, is that all three of these assumptions are false. Yes, you have a rational self, but it’s not your only one, nor is it often in charge. A more accurate picture is that there are a bunch of different versions of you, who come to the fore under different conditions. We aren’t cool calculators of self-interest who sometimes go crazy; we’re crazies who are, under special circumstances, sometimes rational.

Ariely is not out to overthrow rationality. Instead, he and his fellow social scientists want to replace the “rational economic man” model with one that more accurately describes the real laws that drive human choices. In a chapter on “relativity,” for example, Ariely writes that evaluating two houses side by side yields different results than evaluating three — A, B and a somewhat less appealing version of A. The subpar A makes it easier to decide that A is better — not only better than the similar one, but better than B. The lesser version of A should have no effect on your rating of the other two buildings, but it does. Similarly, he describes the “zero price effect,” which marketers exploit to convince us to buy something we don’t really want or need in order to collect a “free” gift. “FREE! gives us such an emotional charge that we perceive what is being offered as immensely more valuable than it really is,” Ariely writes. None of this is rational, but it is predictable.

What the reasoning self should do, he says, is set up guardrails to manage things during those many, many moments when reason is not in charge. (Though one might ask why the reasoning self should always be in charge, an assumption Ariely doesn’t examine too closely.)

For example, Ariely writes, we know our irrational self falls easily into wanting stuff we can’t afford and don’t need. So he proposes a credit card that encourages planning and self-control. After $50 is spent on chocolate this month — pfft, declined! He has in fact suggested this to a major bank. Of course, he knew that his idea would cut into the $17 billion a year that American banks make on consumer credit-card interest, but what the heck: money isn’t everything.

 

An Experiment With Toilet Paper and Other Messages --- http://www.predictablyirrational.com/

Other videos on being Predictably Irrational

 

Great Minds in Management:  The Process of Theory Development --- http://www.trinity.edu/rjensen//theory/00overview/GreatMinds.htm


CBS Sixty Minutes Video
Mozart of Chess:  Magness Carlsen
The Youngest Number One Chess Player in History

Magnus is number one in the chess world. That is sensational. In a very short segment, there are any number of possible "angles" to convey his remarkable talents and achievements to the public at large. While admittedly hackneyed in virtually all respects, I generally have no substantive quarrel with most of the editorial selections of 60 Minutes in this regard. But the part about the potential of following in the footsteps of Bobby Fischer's mental illness seems deceptively truncated. I find it very hard to believe that, in any extended discussion of the subject, Magnus would really leave one with the impression that he is truly concerned that he will end up in the same sorry state that Fischer did. The problematic -- or symptomatic -- details of Fischer's early life (including up until Magnus's current age, for example) are well known. In any discussion, surely Magnus would convey that, while the Fischer saga naturally may give rise to momentary self-reflective wonder, it is plain that Magnus's own family life, overall socialization, behavior, and outlook bear no resemblance whatsoever to that of Fischer. Anything Magnus conveyed along these lines obviously ended up on the cutting room floor.

I applaud all the attention that Magnus garners within the mainstream media. It's good for chess. His publicist would be wise, however, to develop a list of questions and "angles" to explore that are more interesting and penetrating (even for a general audience) than what we see again and again.

Jensen Comment
This was one of the most interesting television segments I ever watched.

The video demonstrates how master chess players are super confident with a compulsion not just to beat their opponents but to annihilate them.

The only opponent to ever intimidate Magness was the great Russian great Garry Kasparov. Kasparov was always one of my heroes until I learned how rude and unsportsmanlike he was in a match with Magness Carlsen (a kid). This is probably the only time in the world that Magness was intimidated to a point where he settled for a tie when he could've won the match.

 

"How experts recall chess positions," by Daniel Simons, The Invisible Gorilla, February 15, 2012 ---
http://theinvisiblegorilla.com/blog/2012/02/15/how-experts-recall-chess-positions/

. . .

This question, how do chess experts evaluate positions to find the best move, has been studied for decades, dating back to the groundbreaking work of Adriaan de Groot and later to work by William Chase and Herbert Simonde Groot interviewed several chess players as they evaluated positions, and he argued that experts and weaker players tended to “look” about the same number of moves ahead and to evaluate similar numbers of moves with roughly similar speed.  The relatively small differences between experts and novices suggested that their advantages came not from brute force calculation ability but from something else: knowledge.  According to De Groot, the core of chess expertise is the ability to recognize huge number of chess positions (or parts of positions) and to derive moves from them.  In short, their greater efficiency came not from evaluating more outcomes, but from considering only the better options. [Note: Some of the details of de Groot’s claims, which he made before the appropriate statistical tests were in widespread use, did not hold up to later scrutiny—experts do consider somewhat more options, look a bit deeper, and process positions faster than less expert players (Holding, 1992). But de Groot was right about the limited nature of expert search and the importance of knowledge and pattern recognition in expert performance.]

Continued in the article (with an interesting concluding video)

 


Appendix 11

Computer Trained Yet Deeply Intuitive

"The Future of Decision Making: Less Intuition, More Evidence," Simoleon Sense, January 11, 2010 ---
http://www.simoleonsense.com/the-future-of-decision-making-less-intuition-more-evidence/

Awesome article (covering decision making, Kahneman, etc) via Harvard

Big thanks & h/t to Michael & Stuart

Click Here Fore: The Future of Decision Making: Less Intuition, More Evidence

Introduction (Via Harvard Blogs)

Human intuition can be astonishingly good, especially after it’s improved by experience. Savvy poker players are so good at reading their opponents’ cards and bluffs that they seem to have x-ray vision. Firefighters can, under extreme duress, anticipate how flames will spread through a building. And nurses in neonatal ICUs can tell if a baby has a dangerous infection even before blood test results come back from the lab.

The lexicon to describe this phenomenon is mostly mystical in nature. Poker players have a sixth sense; firefighters feel the blaze’s intentions; Nurses just know what seems like an infection. They can’t even tell us what data and cues they use to make their excellent judgments; their intuition springs from a deep place that can’t be easily examined. . Examples like these give many people the impression that human intuition is generally reliable, and that we should rely more on the decisions and predictions that come to us in the blink of an eye.

Findings (Via Harvard Blogs)


* It takes a long time to build good intuition. Chess players, for example, need 10 years of dedicated study and competition to assemble a sufficient mental repertoire of board patterns.

* Intuition only works well in specific environments, ones that provide a person with good cues and rapid feedback . Cues are accurate indications about what’s going to happen next. They exist in poker and firefighting, but not in, say, stock markets. Despite what chartists think, it’s impossible to build good intuition about future market moves because no publicly available information provides good cues about later stock movements. Feedback from the environment is information about what worked and what didn’t. It exists in neonatal ICUs because babies stay there for a while. It’s hard, though, to build medical intuition about conditions that change after the patient has left the care environment, since there’s no feedback loop.

* We apply intuition inconsistently. Even experts are inconsistent. One study determined what criteria clinical psychologists used to diagnose their patients, and then created simple models based on these criteria. Then, the researchers presented the doctors with new patients to diagnose and also diagnosed those new patients with their models. The models did a better job diagnosing the new cases than did the humans whose knowledge was used to build them. The best explanation for this is that people applied what they knew inconsistently — their intuition varied. Models, though, don’t have intuition.

* It’s easy to make bad judgments quickly. We have a many biases that lead us astray when making assessments. Here’s just one example. If I ask a group of people “Is the average price of German cars more or less than $100,000?” and then ask them to estimate the average price of German cars, they’ll “anchor” around BMWs and other high-end makes when estimating. If I ask a parallel group the same two questions but say “more or less than $30,000″ instead, they’ll anchor around VWs and give a much lower estimate. How much lower? About $35,000 on average, or half the difference in the two anchor prices. How information is presented affects what we think.

* We can’t know tell where our ideas come from. There’s no way for even an experienced person to know if a spontaneous idea is the result of legitimate expert intuition or of a pernicious bias. In other words, we have lousy intuition about our intuition.

Click Here Fore: The Future of Decision Making: Less Intuition, More Evidence

"Video: Daniel Kahneman - The Psychology of Large Mistakes and Important Decisions" Simoleon Sense, July 27, 2009 ---
http://www.simoleonsense.com/daniel-kahneman-psychology-of-large-mistakes-and-decisions/

"I was going to say I saw a duckie and a horsie, but I changed my mind."
Computer Trained Yet Deeply Intuitive

Not that Carlsen lacks computational prowess, though. He often calculates 20 moves ahead and can comfortably play several games simultaneously while blindfolded simply by hearing each move in notation. The fear surrounding any such beautiful mind is that a life spent probing the edges of the infinite — the possible permutations of a chess game outnumber the estimated number of atoms in the universe — will eventually lead to madness. Grand masters say Carlsen's precociousness is reminiscent of Bobby Fischer's. The great American player spent his later years in isolation, reappearing only to spout anti-Semitic conspiracy theories. "It's easy to get obsessed with chess," Carlsen says. "That's what happened with Fischer and Paul Morphy," another prodigy lost to madness. "I don't have that same obsession."

Remember this Charles Shultz Cartoon

Lucy Van Pelt: Aren't the clouds beautiful? They look like big balls of cotton. I could just lie here all day and watch them drift by. If you use your imagination, you can see lots of things in the cloud's formations. What do you think you see, Linus?
 
Linus Van Pelt: Well, those clouds up there look to me look like the map of the British Honduras on the Caribbean. [points up] That cloud up there looks a little like the profile of Thomas Eakins, the famous painter and sculptor. And that group of clouds over there... [points] ...gives me the impression of the Stoning of Stephen. I can see the Apostle Paul standing there to one side.
 
Lucy Van Pelt: Uh huh. That's very good. What do you see in the clouds, Charlie Brown?
 
Charlie Brown: Well... I was going to say I saw a duckie and a horsie, but I changed my mind.

"A Bold Opening for Chess Player," by Magnus Carlsen, Time Magazine, January 11, 2010, Page 43 ---
http://www.time.com/time/magazine/article/0,9171,1950683,00.html 

Vladimir Kramnik, former world chess champion and current No. 4, is playing in the first round of the London Chess Classic, the most competitive chess tournament to be played in the U.K. capital in 25 years. Tall, handsome and expressionless, he looks exactly as a man who has mastered a game of nearly infinite variation should: like a high-end assassin. Today, however, he is getting methodically and mercilessly crushed.

His opponent is a teenager who seems to be having difficulty staying awake. Magnus Carlsen yawns, fidgets, slumps in his chair. He gets up and wanders over to the other games, staring at the boards like a curious toddler. Every now and then, he returns to his own game and moves one of his pieces, inexorably building an attack so fierce that by the 43rd move Kramnik sees the hopelessness of his position and resigns.

Genius can appear anywhere, but the origins of Carlsen's talent are particularly mysterious. In November, Carlsen, then 18, became the youngest world No. 1 in the game's history. He hails from Norway — a "small, poxy chess nation with almost no history of success," as the English grand master Nigel Short sniffily describes it — and unlike many chess prodigies who are full-time players by age 12, Carlsen stayed in school until last year. His father Henrik, a soft-spoken engineer, says he has spent more time urging his young son to complete his schoolwork than to play chess. Even now, Henrik will interrupt Carlsen's chess studies to drag him out for a family hike or museum trip. "I still have to pinch my arm," Henrik says. "This certainly is not what we had in mind for Magnus."

Even pro chess players — a population inured to demonstrations of extraordinary intellect — have been electrified by Carlsen's rise. A grand master at 13 (the third youngest in history) and a conqueror of top players at 15, he is often referred to as the Mozart of chess for the seeming ease of his mastery. In September, he announced a coaching contract with Garry Kasparov, arguably the greatest player of all time, who quit chess in 2005 to pursue a political career in Russia. "Before he is done," Kasparov says, "Carlsen will have changed our ancient game considerably."

In conversation, Carlsen offers only subtle clues to his intelligence. His speech, like his chess, is technical, grammatically flawless and logically irresistible. He dresses neatly but shows a teenager's discomfort with formality. (He rarely makes it through a game without his shirt coming untucked.) He would seem older than 19 but for his habit of giggling and his coltlike aversion to eye contact.

Carlsen joins chess's élite at a time of unprecedented change. He is one of a generation of players who learned the game from computers. To this day, he's not certain if he has an actual board at home. "I might have one somewhere. I'm not sure," he says. Powerful chess programs, which now routinely beat the best human competitors, have allowed grand masters to study positions at a deeper level than was possible before. Short says top players can now spend almost an entire game trading moves that have been scripted by the same program and that such play by rote has removed some of the mystique of chess. He likens chess computers to "chainsaws chopping down the Amazon." (Read a Q&A with Carlsen.)

But Kasparov says Carlsen's mastery is rooted in a "deep intuitive sense no computer can teach" and that his pupil "has a natural feel for where to place the pieces." According to Kasparov, Carlsen has a knack for sensing the potential energy in each move, even if its ultimate effect is too far away for anyone — even a computer — to calculate. In the grand-master commentary room, where chess's clerisy gather to analyze play, the experts did not even consider several of Carlsen's moves during his game with Kramnik until they saw them and realized they were perfect. "It's hard to explain," Carlsen says. "Sometimes a move just feels right."

Not that Carlsen lacks computational prowess, though. He often calculates 20 moves ahead and can comfortably play several games simultaneously while blindfolded simply by hearing each move in notation. The fear surrounding any such beautiful mind is that a life spent probing the edges of the infinite — the possible permutations of a chess game outnumber the estimated number of atoms in the universe — will eventually lead to madness. Grand masters say Carlsen's precociousness is reminiscent of Bobby Fischer's. The great American player spent his later years in isolation, reappearing only to spout anti-Semitic conspiracy theories. "It's easy to get obsessed with chess," Carlsen says. "That's what happened with Fischer and Paul Morphy," another prodigy lost to madness. "I don't have that same obsession." (Read: "Fischer vs. Spassky: Battle of the Brains.")

Although firmly atop the chess rankings, thanks in part to his victory in London, Carlsen must now fight his way through a series of qualifying competitions in order to earn a chance to play for the world-championship title — the game's highest prize, which is contested every two or three years. His father says he is more concerned about "whether chess will make him a happy person." It seems to be doing just that. "I love the game. I love to compete," Carlsen says. Asked how long he will continue to enjoy chess and where the game will take him, Carlsen pauses to ponder the variables. "It's too difficult to predict," he concludes. So far, at least, he's been making all the right moves.

Fascinating History
"The Chess Master and the Computer,"  By Garry Kasparov, New York Books, February 11, 2010 ---
http://www.nybooks.com/articles/23592

In 1985, in Hamburg, I played against thirty-two different chess computers at the same time in what is known as a simultaneous exhibition. I walked from one machine to the next, making my moves over a period of more than five hours. The four leading chess computer manufacturers had sent their top models, including eight named after me from the electronics firm Saitek.

It illustrates the state of computer chess at the time that it didn't come as much of a surprise when I achieved a perfect 32–0 score, winning every game, although there was an uncomfortable moment. At one point I realized that I was drifting into trouble in a game against one of the "Kasparov" brand models. If this machine scored a win or even a draw, people would be quick to say that I had thrown the game to get PR for the company, so I had to intensify my efforts. Eventually I found a way to trick the machine with a sacrifice it should have refused. From the human perspective, or at least from my perspective, those were the good old days of man vs. machine chess.

Eleven years later I narrowly defeated the supercomputer Deep Blue in a match. Then, in 1997, IBM redoubled its efforts—and doubled Deep Blue's processing power—and I lost the rematch in an event that made headlines around the world. The result was met with astonishment and grief by those who took it as a symbol of mankind's submission before the almighty computer. ("The Brain's Last Stand" read the Newsweek headline.) Others shrugged their shoulders, surprised that humans could still compete at all against the enormous calculating power that, by 1997, sat on just about every desk in the first world.

It was the specialists—the chess players and the programmers and the artificial intelligence enthusiasts—who had a more nuanced appreciation of the result. Grandmasters had already begun to see the implications of the existence of machines that could play—if only, at this point, in a select few types of board configurations—with godlike perfection. The computer chess people were delighted with the conquest of one of the earliest and holiest grails of computer science, in many cases matching the mainstream media's hyperbole. The 2003 book Deep Blue by Monty Newborn was blurbed as follows: "a rare, pivotal watershed beyond all other triumphs: Orville Wright's first flight, NASA's landing on the moon...."

The AI crowd, too, was pleased with the result and the attention, but dismayed by the fact that Deep Blue was hardly what their predecessors had imagined decades earlier when they dreamed of creating a machine to defeat the world chess champion. Instead of a computer that thought and played chess like a human, with human creativity and intuition, they got one that played like a machine, systematically evaluating 200 million possible moves on the chess board per second and winning with brute number-crunching force. As Igor Aleksander, a British AI and neural networks pioneer, explained in his 2000 book, How to Build a Mind:

By the mid-1990s the number of people with some experience of using computers was many orders of magnitude greater than in the 1960s. In the Kasparov defeat they recognized that here was a great triumph for programmers, but not one that may compete with the human intelligence that helps us to lead our lives.

It was an impressive achievement, of course, and a human achievement by the members of the IBM team, but Deep Blue was only intelligent the way your programmable alarm clock is intelligent. Not that losing to a $10 million alarm clock made me feel any better.

My hopes for a return match with Deep Blue were dashed, unfortunately. IBM had the publicity it wanted and quickly shut down the project. Other chess computing projects around the world also lost their sponsorship. Though I would have liked my chances in a rematch in 1998 if I were better prepared, it was clear then that computer superiority over humans in chess had always been just a matter of time. Today, for $50 you can buy a home PC program that will crush most grandmasters. In 2003, I played serious matches against two of these programs running on commercially available multiprocessor servers—and, of course, I was playing just one game at a time—and in both cases the score ended in a tie with a win apiece and several draws.

Inevitable or not, no one understood all the ramifications of having a super-grandmaster on your laptop, especially what this would mean for professional chess. There were many doomsday scenarios about people losing interest in chess with the rise of the machines, especially after my loss to Deep Blue. Some replied to this with variations on the theme of how we still hold footraces despite cars and bicycles going much faster, a spurious analogy since cars do not help humans run faster while chess computers undoubtedly have an effect on the quality of human chess.

Another group postulated that the game would be solved, i.e., a mathematically conclusive way for a computer to win from the start would be found. (Or perhaps it would prove that a game of chess played in the best possible way always ends in a draw.) Perhaps a real version of HAL 9000 would simply announce move 1.e4, with checkmate in, say, 38,484 moves. These gloomy predictions have not come true, nor will they ever come to pass. Chess is far too complex to be definitively solved with any technology we can conceive of today. However, our looked-down-upon cousin, checkers, or draughts, suffered this fate quite recently thanks to the work of Jonathan Schaeffer at the University of Alberta and his unbeatable program Chinook.

The number of legal chess positions is 1040, the number of different possible games, 10120. Authors have attempted various ways to convey this immensity, usually based on one of the few fields to regularly employ such exponents, astronomy. In his book Chess Metaphors, Diego Rasskin-Gutman points out that a player looking eight moves ahead is already presented with as many possible games as there are stars in the galaxy. Another staple, a variation of which is also used by Rasskin-Gutman, is to say there are more possible chess games than the number of atoms in the universe. All of these comparisons impress upon the casual observer why brute-force computer calculation can't solve this ancient board game. They are also handy, and I am not above doing this myself, for impressing people with how complicated chess is, if only in a largely irrelevant mathematical way.

This astronomical scale is not at all irrelevant to chess programmers. They've known from the beginning that solving the game—creating a provably unbeatable program—was not possible with the computer power available, and that effective shortcuts would have to be found. In fact, the first chess program put into practice was designed by legendary British mathematician Alan Turing in 1952, and he didn't even have a computer! He processed the algorithm on pieces of paper and this "paper machine" played a competent game.

Rasskin-Gutman covers this well-traveled territory in a book that achieves its goal of being an overview of overviews, if little else. The history of the study of brain function is covered in the first chapter, tempting the reader to skip ahead. You might recall axons and dendrites from high school biology class. We also learn about cholinergic and aminergic systems and many other things that are not found by my computer's artificially intelligent English spell-checking system—or referenced again by the author. Then it's on to similarly concise, if inconclusive, surveys of artificial intelligence, chess computers, and how humans play chess.

There have been many unintended consequences, both positive and negative, of the rapid proliferation of powerful chess software. Kids love computers and take to them naturally, so it's no surprise that the same is true of the combination of chess and computers. With the introduction of super-powerful software it became possible for a youngster to have a top- level opponent at home instead of need ing a professional trainer from an early age. Countries with little by way of chess tradition and few available coaches can now produce prodigies. I am in fact coaching one of them this year, nineteen-year-old Magnus Carlsen, from Norway, where relatively little chess is played.

The heavy use of computer analysis has pushed the game itself in new directions. The machine doesn't care about style or patterns or hundreds of years of established theory. It counts up the values of the chess pieces, analyzes a few billion moves, and counts them up again. (A computer translates each piece and each positional factor into a value in order to reduce the game to numbers it can crunch.) It is entirely free of prejudice and doctrine and this has contributed to the development of players who are almost as free of dogma as the machines with which they train. Increasingly, a move isn't good or bad because it looks that way or because it hasn't been done that way before. It's simply good if it works and bad if it doesn't. Although we still require a strong measure of intuition and logic to play well, humans today are starting to play more like computers.

The availability of millions of games at one's fingertips in a database is also making the game's best players younger and younger. Absorbing the thousands of essential patterns and opening moves used to take many years, a process indicative of Malcolm Gladwell's "10,000 hours to become an expert" theory as expounded in his recent book Outliers. (Gladwell's earlier book, Blink, rehashed, if more creatively, much of the cognitive psychology material that is re-rehashed in Chess Metaphors.) Today's teens, and increasingly pre-teens, can accelerate this process by plugging into a digitized archive of chess information and making full use of the superiority of the young mind to retain it all. In the pre-computer era, teenage grandmasters were rarities and almost always destined to play for the world championship. Bobby Fischer's 1958 record of attaining the grandmaster title at fifteen was broken only in 1991. It has been broken twenty times since then, with the current record holder, Ukrainian Sergey Karjakin, having claimed the highest title at the nearly absurd age of twelve in 2002. Now twenty, Karjakin is among the world's best, but like most of his modern wunderkind peers he's no Fischer, who stood out head and shoulders above his peers—and soon enough above the rest of the chess world as well.

Excelling at chess has long been considered a symbol of more general intelligence. That is an incorrect assumption in my view, as pleasant as it might be. But for the purposes of argument and investigation, chess is, in Russkin-Gutman's words, "an unparalleled laboratory, since both the learning process and the degree of ability obtained can be objectified and quantified, providing an excellent comparative framework on which to use rigorous analytical techniques."

Here I agree wholeheartedly, if for different reasons. I am much more interested in using the chess laboratory to illuminate the workings of the human mind, not the artificial mind. As I put it in my 2007 book, How Life Imitates Chess, "Chess is a unique cognitive nexus, a place where art and science come together in the human mind and are then refined and improved by experience." Coincidentally the section in which that phrase appears is titled "More than a metaphor." It makes the case for using the decision-making process of chess as a model for understanding and improving our decision-making everywhere else.

This is not to say that I am not interested in the quest for intelligent machines. My many exhibitions with chess computers stemmed from a desire to participate in this grand experiment. It was my luck (perhaps my bad luck) to be the world chess champion during the critical years in which computers challenged, then surpassed, human chess players. Before 1994 and after 2004 these duels held little interest. The computers quickly went from too weak to too strong. But for a span of ten years these contests were fascinating clashes between the computational power of the machines (and, lest we forget, the human wisdom of their programmers) and the intuition and knowledge of the grandmaster.

In what Rasskin-Gutman explains as Moravec's Paradox, in chess, as in so many things, what computers are good at is where humans are weak, and vice versa. This gave me an idea for an experiment. What if instead of human versus machine we played as partners? My brainchild saw the light of day in a match in 1998 in León, Spain, and we called it "Advanced Chess." Each player had a PC at hand running the chess software of his choice during the game. The idea was to create the highest level of chess ever played, a synthesis of the best of man and machine.

Although I had prepared for the unusual format, my match against the Bulgarian Veselin Topalov, until recently the world's number one ranked player, was full of strange sensations. Having a computer program available during play was as disturbing as it was exciting. And being able to access a database of a few million games meant that we didn't have to strain our memories nearly as much in the opening, whose possibilities have been thoroughly catalogued over the years. But since we both had equal access to the same database, the advantage still came down to creating a new idea at some point.

Having a computer partner also meant never having to worry about making a tactical blunder. The computer could project the consequences of each move we considered, pointing out possible outcomes and countermoves we might otherwise have missed. With that taken care of for us, we could concentrate on strategic planning instead of spending so much time on calculations. Human creativity was even more paramount under these conditions. Despite access to the "best of both worlds," my games with Topalov were far from perfect. We were playing on the clock and had little time to consult with our silicon assistants. Still, the results were notable. A month earlier I had defeated the Bulgarian in a match of "regular" rapid chess 4–0. Our advanced chess match ended in a 3–3 draw. My advantage in calculating tactics had been nullified by the machine.

This experiment goes unmentioned by Russkin-Gutman, a major omission since it relates so closely to his subject. Even more notable was how the advanced chess experiment continued. In 2005, the online chess-playing site Playchess.com hosted what it called a "freestyle" chess tournament in which anyone could compete in teams with other players or computers. Normally, "anti-cheating" algorithms are employed by online sites to prevent, or at least discourage, players from cheating with computer assistance. (I wonder if these detection algorithms, which employ diagnostic analysis of moves and calculate probabilities, are any less "intelligent" than the playing programs they detect.)

Lured by the substantial prize money, several groups of strong grandmasters working with several computers at the same time entered the competition. At first, the results seemed predictable. The teams of human plus machine dominated even the strongest computers. The chess machine Hydra, which is a chess-specific supercomputer like Deep Blue, was no match for a strong human player using a relatively weak laptop. Human strategic guidance combined with the tactical acuity of a computer was overwhelming.

The surprise came at the conclusion of the event. The winner was revealed to be not a grandmaster with a state-of-the-art PC but a pair of amateur American chess players using three computers at the same time. Their skill at manipulating and "coaching" their computers to look very deeply into positions effectively counteracted the superior chess understanding of their grandmaster opponents and the greater computational power of other participants. Weak human + machine + better process was superior to a strong computer alone and, more remarkably, superior to a strong human + machine + inferior process.

Continued in article

 


"Critical Thinking:  Why It's So Hard to Teach," by Daniel T. Willingham ---
http://www.aft.org/pubs-reports/american_educator/issues/summer07/Crit_Thinking.pdf

Also see Simorleon Sense --- http://www.simoleonsense.com/critical-thinking-why-is-it-so-hard-to-teach/

“Critical thinking is not a set of skills that can be deployed at any time, in any context. It is a type of thought that even 3-year-olds can engage in—and even trained scientists can fail in.”

“Knowing that one should think critically is not the same as being able to do so. That requires domain knowledge and practice.”

So,  Why Is Thinking Critically So Hard?
Educators have long noted that school attendance and even academic success are no guarantee that a student will graduate an effective thinker in all situations. There is an odd tendency for rigorous thinking to cling to particular examples or types of problems. Thus, a student may have learned to estimate the answer to a math problem before beginning calculations as a way of checking the accuracy of his answer, but in the chemistry lab, the same student calculates the components of a compound without noticing that his estimates sum to more than 100 percent. And a student who has learned to thoughtfully discuss the causes of the American Revolution from both the British and American perspectives doesn’t even think to question how the Germans viewed World War II. Why are students able to think critically in one situation, but not in another? The brief answer is: Thought processes are intertwined with what is being thought about. Let’s explore this in depth by looking at a particular kind of critical thinking that has been studied extensively: problem solving.

Imagine a seventh-grade math class immersed in word problems. How is it that students will be able to answer one problem, but not the next, even though mathematically both word problems are the same, that is, they rely on the same mathematical knowledge? Typically, the students are focusing on the scenario that the word problem describes (its surface structure) instead of on the mathematics required to solve it (its deep structure). So even though students have been taught how to solve a particular type of word problem, when the teacher or textbook changes the scenario, students still struggle to apply the solution because they don’t recognize that the problems are mathematically the same.

Thinking Tends to Focus on a Problem’s “Surface Structure”
To understand why the surface structure of a problem is so distracting and, as a result, why it’s so hard to apply familiar solutions to problems that appear new, let’s first consider how you understand what’s being asked when you are given a problem. Anything you hear or read is automatically interpreted in light of what you already know about similar subjects. For example, suppose you read these two sentences: “After years of pressure from the film and television industry, the President has filed a formal complaint with China over what U.S. firms say is copyright infringement. These firms assert that the Chinese government sets stringent trade restrictions for U.S. entertainment products, even as it turns a blind eye to Chinese companies that copy American movies and television shows and sell them on the black market.”

With Deep Knowledge, Thinking Can Penetrate Beyond Surface Structure
If knowledge of how to solve a problem never transferred to problems with new surface structures, schooling would be inefficient or even futile—but of course, such transfer does occur. When and why is complex,5 but two factors are especially relevant for educators: familiarity with a problem’s deep structure and the knowledge that one should look for a deep structure. I’ll address each in turn. When one is very familiar with a problem’s deep-structure, knowledge about how to solve it transfers well. That familiarity can come from long-term, repeated experience with one problem, or with various manifestations of one type of problem (i.e., many problems that have different surface structures, but the same deep structure). After repeated exposure to either or both, the subject simply perceives the deep structure as part of the problem description.


"Beyond Critical Thinking," by Michael S. Roth, Chronicle of Higher Education's Chronicle Review, January 3, 2010 ---
http://chronicle.com/article/Beyond-Critical-Thinking/63288/

The antivocational dimension of the humanities has been a source of pride and embarrassment for generations. The persistence of this reputed uselessness is puzzling given the fact that an education in the humanities allows one to develop skills in reading, writing, reflection, and interpretation that are highly prized in our economy and culture. Sure, specific training in a discrete set of skills might prepare you for Day 1 of the worst job you'll ever have (your first), but the humanities teach elements of mind and heart that you will draw upon for decades of innovative and focused work. But we do teach a set of skills, or an attitude, in the humanities that may have more to do with our antipractical reputation than the antivocational notion of freedom embedded in the liberal arts. This is the set of skills that usually goes under the rubric of critical thinking.

Although critical thinking first gained its current significance as a mode of interpretation and evaluation to guide beliefs and actions in the 1940s, the term took off in education circles after Robert H. Ennis published "A Concept of Critical Thinking" in the Harvard Educational Review in 1962. Ennis was interested in how we teach the "correct assessment of statements," and he offered an analysis of 12 aspects of this process. Ennis and countless educational theorists who have come after him have sung the praises of critical thinking. There is now a Foundation for Critical Thinking and an industry of consultants to help you enhance this capacity in your teachers, students, or yourself.

A common way to show that one has sharpened one's critical thinking is to display an ability to see through or undermine statements made by (or beliefs held by) others. Thus, our best students are really good at one aspect of critical thinking­—being critical. For many students today, being smart means being critical. To be able to show that Hegel's concept of narrative foreclosed the non-European, or that Butler's stance on vulnerability contradicts her conception of performativity, or that a tenured professor has failed to account for his own "privilege"—these are marks of sophistication, signs of one's ability to participate fully in the academic tribe. But this participation, being entirely negative, is not only seriously unsatisfying; it is ultimately counterproductive.

The skill at unmasking error, or simple intellectual one-upmanship, is not completely without value, but we should be wary of creating a class of self-satisfied debunkers or, to use a currently fashionable word on campuses, people who like to "trouble" ideas. In overdeveloping the capacity to show how texts, institutions, or people fail to accomplish what they set out to do, we may be depriving students of the capacity to learn as much as possible from what they study. In a humanities culture in which being smart often means being a critical unmasker, our students may become too good at showing how things don't make sense. That very skill may diminish their capacity to find or create meaning and direction in the books they read and the world in which they live. Once outside the university, our students continue to score points by displaying the critical prowess for which they were rewarded in school. They wind up contributing to a cultural climate that has little tolerance for finding or making meaning, whose intellectuals and cultural commentators delight in being able to show that somebody else is not to be believed.

I doubt that this is a particularly contemporary development. In the 18th century there were complaints about an Enlightenment culture that prized only skepticism and that was satisfied only with disbelief. Our contemporary version of this trend, though, has become skeptical even about skepticism. We no longer have the courage of our lack of conviction. Perhaps that's why we teach our students that it's cool to say that they are engaged in "troubling" an assumption or a belief. To declare that one wanted to disprove a view would show too much faith in the ability to tell truth from falsehood. And to declare that one was receptive to learning from someone else's view would show too much openness to being persuaded by an idea that might soon be deconstructed (or simply mocked).

In training our students in the techniques of critical thinking, we may be giving them reasons to remain guarded—which can translate into reasons not to learn. The confident refusal to be affected by those with whom we disagree seems to have infected much of our cultural life: from politics to the press, from siloed academic programs (no matter how multidisciplinary) to warring public intellectuals. As humanities teachers, however, we must find ways for our students to open themselves to the emotional and cognitive power of history and literature that might initially rub them the wrong way, or just seem foreign. Critical thinking is sterile without the capacity for empathy and comprehension that stretches the self.

One of the crucial tasks of the humanities should be to help students cultivate the willingness and ability to learn from material they might otherwise reject or ignore. This material will often surprise students and sometimes upset them. Students seem to have learned that teaching-evaluation committees take seriously the criticism that "the professor, or the material, made me uncomfortable." This complaint is so toxic because being made uncomfortable may be a necessary component of an education in the humanities. Creating a humanistic culture that values the desire to learn from unexpected and uncomfortable sources as much as it values the critical faculties would be an important contribution to our academic and civic life.

But the contemporary humanities should do more than supplement critical thinking with empathy and a desire to understand others from their own point of view. We should also supplement our strong critical engagement with cultural and social norms by developing modes of teaching that allow our students to enter in the value-laden practices of a particular culture to understand better how these values are legitimated: how the values are lived as legitimate. Current thinking in the humanities is often strong at showing that values that are said to be shared are really imposed on more-vulnerable members of a particular group. Current thinking in the humanities is also good at showing the contextualization of norms, whether the context is generated by an anthropological, historical, or other disciplinary matrix. But in both of these cases we ask our students to develop a critical distance from the context or culture they are studying.

Many humanities professors have become disinclined to investigate with our students how we generate the values we believe in, or the norms according to which we go about our lives. In other words, we have been less interested in showing how we make a norm legitimate than in sharpening our tools for delegitimization. The philosopher Robert Pippin has recently made a similar point, and has described how evolutionary biology and psychology have moved into this terrain, explaining moral values as the product of the same dynamic that gives rise to the taste for sweets. Pippin argues, on the contrary, that "the practical autonomy of the normative is the proper terrain of the humanities," and he has an easy task of showing how the pseudoscientific evolutionary "explanation" of our moral choices is a pretty flimsy "just-so" story.

If we humanities professors saw ourselves more often as explorers of the normative than as critics of normativity, we would have a better chance to reconnect our intellectual work to broader currents in public culture. This does not have to mean an acceptance of the status quo, but it does mean an effort to understand the practices of cultures (including our own) from the point of view of those participating in them. This would include an understanding of how cultures change. For many of us, this would mean complementing our literary or textual work with participation in community, with what are often called service-learning courses. For others, it would mean approaching our object of study not with the anticipated goal of exposing weakness or mystification but with the goal of turning ourselves in such a way as to see how what we study might inform our thinking and our lives.

I realize that I am arguing for a mode of humanistic education that many practice already. It is a mode that can take language very seriously, but rather than seeing it as the master mediator between us and the world, a matrix of representations always doomed to fail, it sees language as itself a cultural practice to be understood from the point of view of those using it.

The fact that language fails according to some impossible criterion, or that we fail in our use of it, is no news, really. It is part of our finitude, but it should not be taken as the key marker of our humanity. The news that is brought by the humanities is a way of turning the heart and the spirit so as to hear possibilities of various forms of life in which we might participate. When we learn to read or look or listen intensively, we are not just becoming adept at exposing falsehood or at uncovering yet more examples of the duplicities of culture and society. We are partially overcoming our own blindness by trying to understand something from another's artistic, philosophical, or historical point of view. William James put it perfectly in a talk to teachers and students entitled "On a Certain Blindness in Human Beings": "The meanings are there for others, but they are not there for us." James saw the recognition of this blindness as key to education as well as to the development of democracy and civil society. Of course hard-nosed critical thinking may help in this endeavor, but it also may be a way we learn to protect ourselves from the acknowledgment and insight that humanistic study has to offer. As students and as teachers we sometimes crave that protection because without it we risk being open to changing who we are. In order to overcome this blindness, we risk being very uncomfortable indeed.

It is my hope that humanists will continue offering criticism, making connections, and finding ways to acknowledge practices that seem at first opaque or even invisible. In supporting a transition from critical thinking to practical exploration, I am echoing a comment made by my undergraduate philosophy teacher Louis Mink, and echoed by my graduate mentor, Richard Rorty. Years before Dick Rorty deconstructed the idea of the "philosopher as referee," Louis Mink suggested that critics "exchange the judge's wig for the guide's cap." I think we may say the same for humanists, who can, in his words, "show us details and patterns and relations which we would not have seen or heard for ourselves."

My humanities teachers enriched my life by showing me details and pattern and relations. In so doing they also helped me to acquire tools that have energetically shaped my scholarship and my interactions with colleagues and students. It is my hope that as guides, not judges, we can show our students how to engage in the practice of exploring objects, norms, and values that inform diverse cultures. In doing so, students will develop the ability to converse with others about shaping the objects, norms, and values that will inform their own lives. They will develop the ability to add value to (and not merely criticize values in) whatever organizations in which they participate. They will often reject roads that others have taken, and they will sometimes chart new paths. But guided by the humanities, they will increase their ability to find together ways of living that have meaning and direction, illuminating paths immensely practical and sustaining.

Michael S. Roth is an intellectual historian and president of Wesleyan University. This essay was part of a lecture commemorating the 50th anniversary of the founding of Wesleyan's Center for the Humanities.

January 5, 2009 reply from Barbara Scofield [barbarawscofield@GMAIL.COM]

At the University of Kentucky in the 1990s I took a faculty development course in "Integrative Studies," which was required of the medical students at that time, and then offered one summer to all faculty. In the discussion segments the faculty participants were asked to always provide comments that were an addition to the comments of the other participants. In other words, we couldn't begin with "Yes, but ..." We were supposed to find common ground and build from there. Some faculty found this impossible to do, even when the facilitator emphasized it over and over again. My remembrance is that the business and agriculture faculty had an easier time with the cooperative nature of the course than the liberal arts folks.

Barbara W. Scofield, PhD, CPA Chair of Graduate Business Studies Professor of Accounting The University of Texas of the Permian Basin 4901 E. University Dr. Odessa, TX 79762
432-552-2183 (Office) 817-988-5998 (Cell)

BarbaraWScofield@gmail.com

The Cambridge Handbook of Thinking and Reasoning --- Click Here

The Miniature Guide To Critical Thinking Concepts & Tools --- Click Here

Bob Jensen's threads on critical thinking are at
http://www.trinity.edu/rjensen/HigherEdControversies.htm#CriticalThinking


College Degrees Without Instructors

Competency-Based Assessment --- http://www.trinity.edu/rjensen/competency.htm

There are a few really noteworthy competency-based distance education programs including Western Governors University (WGU) and the Chartered Accountancy School of Business (CASB)  in Canada. But these competency-based programs typically have assigned instructors and bear the costs of those instructors. The instructors, however, do not assign grades to students.

It appears that the Southern New Hampshire University (a private institution) is taking competency-based distance education to a new level by eliminating the instructors. It should be noted that SNHU has both an onsite campus and online degree programs.

"Online Education Is Everywhere. What’s the Next Big Thing?" by Marc Parry, Chronicle of Higher Education, August 31, 2011 ---
http://chronicle.com/blogs/wiredcampus/online-education-is-everywhere-whats-the-next-big-thing/32898?sid=wc&utm_source=wc&utm_medium=en

. . .

The vision is that students could sign up for self-paced online programs with no conventional instructors. They could work at their own speeds through engaging online content that offers built-in assessments, allowing them to determine when they are ready to move on. They could get help through networks of peers who are working on the same courses; online discussions could be monitored by subject experts. When they’re ready, students could complete a proctored assessment, perhaps at a local high school, or perhaps online. The university’s staff could then grade the assessment and assign credit.

And the education could be far cheaper, because there would be no expensive instructor and students could rely on free, open educational resources rather than expensive textbooks. Costs to the student might include the assessment and the credits.

“The whole model hinges on excellent assessment, a rock-solid confidence that the student has mastered the student-learning outcomes,” the memo says. “If we know with certainty that they have, we should no longer care if they raced through the course or took 18 months, or if they worked on their courses with the support of a local church organization or community center or on their own. The game-changing idea here is that when we have assessment right, we should not care how a student achieves learning. We can blow up the delivery models and be free to try anything that shows itself to work.”

Continued in article

"A Russian University Gets Creative Against Corruption:  With surveillance equipment and video campaigns, rector aims to eliminate bribery at Kazan State," by Anna Nemtsova, Chronicle of Higher Education, January 17, 2010 ---
http://chronicle.com/article/A-Russian-University-Gets/63522/

Jensen Comment
In its early history, the University of Chicago had competency-based programs where grades were assigned solely on the basis of scores on final examinations. Students did not have to attend class.

Bob Jensen's threads on competency-based assessment ---
http://www.trinity.edu/rjensen/competency.htm 

Bob Jensen's threads on distance education alternatives are at
http://www.trinity.edu/rjensen/Crossborder.htm

Bob Jensen's threads on higher education controversies are at
http://www.trinity.edu/rjensen/HigherEdControversies.htm

I should point out that this is very similar to the AAA's Innovation in Accounting Education Award Winning BAM Pedagogy commenced at the University of Virginia (but there were instructors who did not teach) ---
http://www.trinity.edu/rjensen/265wp.htm


 

Appendix 13

Updates

Metacognition in Learning

To my knowledge, Bob Jensen is the first author to discuss the importance of metacognition in learning.
That paper focuses on the metacognitive advantages of self-learning (with blood, sweat, and tears) over memorizing answers given out by teachers.
"Metacognitive Concerns in Designs and Evaluations of Computer Aided Education and Training: Are We Misleading Ourselves About Measures of Success?"
http://www.trinity.edu/rjensen/265wp.htm

 

Now we have a second paper on he importance of metacognition in learning
The paper below focuses on the metacognitive mindset
"Accounting Students' Metacognition: The Association of Performance, Calibration Error, and Mindset," by Susan P. Ravenscroft, Tammy R. Waymire, and Timothy D. West, Issues in Accounting Education, Vol. 27, No. 3, August 2012, pp. 707-732 (not free) ---
http://aaajournals.org/doi/full/10.2308/iace-50148

In recognition of the evolving body of knowledge in the accounting profession, the American Institute of Certified Public Accountants (AICPA 2010) highlights the importance of viewing learning as a lifelong process that requires self-awareness and extends beyond the academic setting. Metacognition, the assessment and regulation of one's own learning, is a crucial element in lifelong learning. We draw upon judgment of learning research and introduce mindset theory to explore the relationship among (1) exam performance, (2) calibration error, measured as expected minus actual exam scores, and (3) mindset, a person's basic beliefs about learning and ability (Dweck 2000, 2006) in the accounting classroom. We find strong evidence that exam performance is inversely related to calibration error (Kruger and Dunning 1999). We also find modest evidence that a growth mindset is associated with improved performance and decreased calibration error. While the mindset results were not entirely consistent with prior research in educational psychology, we explore possible reasons and future directions for accounting education research.

. . .

DISCUSSION Limitations

Our sample consisted of students taught by a single instructor at a single institution who took an elective governmental and nonprofit accounting course during one of three semesters. This course is typically viewed as difficult and as needed for the CPA examination. While this could restrict the generalizability of the results, we do not believe that it does so seriously. We are aware of no research findings indicating that judgments of learning or mindsets differ across social demographics. Instead, the findings on which we relied are found across broad categories of groups. However, to establish generalizability, we hope to use multiple institutions, instructors, and courses in future research.

Another limitation is the restriction of range that we found in the independent variable of mindset. Dweck and Molden (2005) note that when they assess children or adults, they find that about 40 percent endorse the fixed view of mindset, another 40 percent endorse the growth view, and about 20 percent are undecided. Given that a majority of the subjects were categorized as having a growth mindset, the likelihood of seeing a significant relationship was decreased. Because we did not manipulate this variable, we could not create a full range of mindsets for our analysis. Moreover, we have a restricted range of performance. Students taking the governmental and nonprofit accounting course have all succeeded in a competitive accounting program, with average GPAs above that required for remaining the program. Both of these restrictions bias against finding statistically significant relationships, and we believe that the results can, therefore, still be of benefit to a broad range of accounting educators. Discussion of Results

The initial goal of this study was to better understand why accounting students sometimes lack self-awareness about their own abilities and skills, and to explore factors that may assist accounting educators. The study's results point to three implications for accounting educators. First, consistent with Kruger and Dunning (1999), we found that students who overestimate their abilities likely do so because they lack the technical knowledge to evaluate their own performance, as evidenced by lower performance. We also found, in the first two exams, evidence of a magnitude effect that suggests that high-performing students calibrate more accurately than low-performing students do, expressed in absolute terms. This may affirm observations by accounting faculty and help them in assisting students with their self-regulated learning and self-insight.

Second, in exploring the average calibration errors of high- and low-performing students, we found that low-performing students tend to improve their calibration accuracy, while high-performing students tend to become increasingly underconfident relative to their performance. These results demonstrate the concerns that accounting educators may have for both low performers and high performers. Low performers' lack the self-awareness of their technical skills to accurately calibrate their own performance, and this may cause them to continue to underperform. High performers fail to recognize their strong technical skills and may become overly critical of their own performance.

Third, in exploring the role of mindset regarding an individual's approach to learning and response to failure, we predicted that students with a growth mindset (i.e., those who were motivated by learning, resilient, and focused on learning from feedback) would demonstrate higher exam performance, improvement in performance, lower calibration error, and improvement in their calibration. We find modest evidence supporting these predictions. Mindset was significantly associated with performance on only one of three exams, and improvement from Exam 1 to Exam 2. Mindset was not associated with level of calibration error, but was associated with improvement in calibration from Exam 1 to Exam 2 and from Exam 1 to Exam 3. We expected growth mindset to be more consistently associated with the level of, and improvements in, calibration error; however, we believe that the short, one-semester timeline may make it more difficult to capture the impact of mindset. Furthermore, we present evidence that the final exam (Exam 3) may reflect unique resource allocation decisions on the part of students that may affect both the performance and calibration error results. Although inconsistent, the results provide some modest evidence that encouraging a growth mindset may offer benefits to students in improving their performance and calibration accuracy.

Mindset theory originated as a way to explain why students have differing goals and reactions to failure (Dweck and Leggett 1988), but as the research in this area has continued, the significance and implications of mindset have grown. For instance, more recent work implies that mindsets—although malleable experimentally—represent a fundamental view of the world, quoting Piaget to the effect that worldviews of children “can be as important to their functioning as the logical reasoning he studied for much of his career,” (Molden and Dweck 2006, 200). Molden and Dweck (2006) survey research showing that mindset plays a role in many behaviors, including goal setting, attributions, strategies, grades, perceptions of others, responses to stereotypes, self-esteem, and self-regulatory strategies.

In our setting, senior-level and graduate accounting students who have met stringent admissions criteria and who are very grade-conscious may hold strong achievement goals. The connection between mindset and performance may be altered in the presence of strongly held achievement goals (grade-based as opposed to learning-based). Dweck and other researchers (e.g., Dweck and Leggett 1988; Shunk 1995) observe that the positive effect of mindset on achievement can be overridden by the effect of goals. Shunk (1995, 317) discusses the interaction of goals and mindset, and notes that sometimes “success-oriented persons who perform poorly on one occasion will work harder and improve their performance on another.” The integration of the goals literature may, therefore, be helpful in future exploration of the role of mindset in the accounting education setting and extending the results presented in this study. Furthermore, because research suggests that business students generally approach studying in a more superficial way than non-business students (Arum and Roska 2011), future research studies could be conducted across academic disciplines, preferably including students in and outside the college of business to make comparisons among groups.

In sum, our study presents evidence of an inverse relationship between performance and calibration error in an accounting education setting, and offers an initial step in understanding the role mindset plays in metacognitive self-awareness of accounting students. Although this research represents an early effort to introduce mindset concepts within the accounting education literature, our results and the underlying research suggest that faculty could introduce the concept of mindset to students, which could be particularly useful for those students with fixed mindsets. Introducing the concept of a growth mindset leads naturally into a discussion of the effort that is necessary for deep learning, and could motivate a discussion with student involvement about the students' study approaches and preparation for tests. Finally, recent research (Anseel et al. 2009) suggests that the beneficial effects of faculty feedback to students can be amplified if students are appropriately guided to reflect on their performance. Mindset, in conjunction with feedback, offers promise as a way to encourage learning and self-awareness.


"Designing for Emergence: The Role of the Instructor in Student-Centered Learning," by Mary Stewart, Hybrid Pedagogy, August 21, 2014 ---
http://www.hybridpedagogy.com/journal/designing-emergence-role-instructor-student-centered-learning/


Jensen Comment
I seldom say that any of my tidbits are "must reads." This is an exception. I think it is a must read for all students and faculty!

"Book Review: 'The Organized Mind' by Daniel J. Levitin --- Our minds were designed to succeed in an environment utterly unlike the information overload we now face," by Christopher Chabris, The Wall Street Journal, August 15, 2014 ---
http://online.wsj.com/articles/book-review-the-organized-mind-by-daniel-j-levitin-1408137852?tesla=y&mod=djemMER_h&mg=reno64-wsj

Dr. Chabris is a psychology professor at Union College and co-author, with Daniel Simons, of "The Invisible Gorilla: How Our Intuitions Deceive Us."

More than a century ago, Sigmund Freud wrote the "Psychopathology of Everyday Life." Over two decades ago, Donald Norman published the "Psychology of Everyday Things." Three years ago, David Myers called a new edition of his textbook "Psychology in Everyday Life." The word "everyday" has a special appeal in such titles, since so many psychology books, especially of the self-help variety, are written for the self with major problems to contend with—love, illness, grief, identity, conflict—leaving the small tasks of mundane functioning to common sense, or perhaps to business writers who purvey "habits" and "disciplines."

In "The Organized Mind," Daniel J. Levitin, a cognitive neuroscientist at McGill University, makes an ambitious attempt to bring research in neuroscience and cognitive psychology to bear on the more ordinary parts of our lives. He focuses on the daily challenges of professionals, managers and knowledge workers. But we are all knowledge workers now, since everyone uses Facebook, FB -0.90% communicates by email, and must process, store and retrieve an ever-growing volume of information. In this impressively wide-ranging and thoughtful work, Mr. Levitin stresses the many ways in which evolution designed our minds to succeed in an environment that was utterly unlike the world of information overload we now face. And he aims to help us cope by providing concrete suggestions for solving the daily problems of modern existence.

Mr. Levitin begins by explaining why we are in the mess we are in. The capacities of our brains grew out of solutions to the problems that our ancestor species confronted when living in the natural world. We have very good memories for routes we walk and for places where things are located because those are the most important things for primates and mammals to be keep track of. And our tendency to be attracted by anything new had great value when new things were likely to be important threats or opportunities. But these capacities may be maladapted to the challenges of current life, especially the man-made parts of it.

Memories tuned for routes and places are simply not designed to store the near-infinity of unique passwords (random strings of letters, numbers and punctuation) that Internet security demands. Decision-making systems that put a premium on novelty betray us when millions upon millions of new data packets are mere finger-taps away. "Every status update you read on Facebook, every tweet or text message you get from a friend," Mr. Levitin writes, "is competing for resources in your brain with important things like whether to put your savings in stocks or bonds, where you left your passport, or how to reconcile with a close friend you just had an argument with." All this piling up taxes our abilities to process information, remember it and make decisions.

What to do? Mr. Levitin devotes several meaty chapters to specific domains—including domestic matters, social connections and time management—in which we tend to fall short of what is needed for peace of mind and productivity. He also considers how to teach younger people to cope with the information-rich environment they will grow up in. Throughout, he mixes anecdote and science, first-person narrative and tips for successful living. On one page you may read a detailed description of a brain-imaging study, on the next a quotation from the man who was once in charge of managing President Obama's mail, and after that a formula for generating strong but memorable passwords.

A good way to deal with overload, Mr. Levitin suggests, is to offload the responsibilities of "personal management," tasks like being on time and staying in touch with friends or associates. This strategy is routinely adopted by members of a category that Mr. Levitin calls HSPs—Highly Successful Persons. A few months ago, I was excited to learn that my newest Twitter TWTR -1.26% followers were Phil Ivey, one of the best poker players in the world, and the popular actor Taye Diggs —both undoubted HSPs. Then I found out that they employ other people to manage their social media. Mr. Levitin tells us that he met Jimmy Carter back in the mid-1970s, when he was first running for president, and Mr. Carter spoke "as though we had all the time in the world." He could focus on the task at hand, Mr. Levitin notes, because his aides were worrying about where he needed to be and when, freeing him to "let go of those inner nagging voices and be there."

Mr. Levitin isn't recommending that we all hire personal assistants, an unrealistic approach unless you happen to be an HSP or a Real Housewife. We don't need human helpers because computational ones become better all the time. A calendar app that buzzes quietly 15 minutes before each appointment is better and cheaper than a human who has to knock on your door and interrupt your conversation or train of thought. Sites like Orbitz and Kayak are faster and more flexible for booking almost any trip than human travel agents ever were. A well-curated Twitter feed will keep you up on news about your work and hobbies in a way that no personal assistant ever could. Indeed, we have outsourced to Google GOOGL -0.16% a massive volume of "research" chores that used to take anywhere from minutes to months of trawling through reference sources, making phone calls and visiting archives. These conveniences come with frustrations (also known as "first-world problems"), but to focus on such trees is to miss the forest of improvements that we enjoy today.

To a surprising extent, Mr. Levitin's advice for organizing our minds consists not of learning mental tricks or doing brain exercises but of organizing our surroundings—literally, the physical world we inhabit every day. This effort can reduce needless demands on our cognitive abilities, especially on our capacity for paying attention, which he rightly calls "the most essential mental resource for any organism." The suggestions range from the simple (keep in plain sight the things you need to access most often) to the detailed, such as setting up various filing systems, including a junk drawer and "miscellaneous" file.

Explaining filing systems, Mr. Levitin writes, "the key to creating useful categories in our homes is to limit the number of types of things they contain to one or at most four (respecting the capacity limitations of working memory)." By keeping items that share a common use in a single place (e.g., supplies for a party) you reduce the burden on memory, since you must only associate one location ("top middle kitchen island drawer") with one category ("birthday party") rather than several locations (different drawers and cabinets) with several items (colored napkins, paper plates, plastic cups). The same idea applies to files of important documents and media, whether physical or digital.

In the case of the junk drawer, Mr. Levitin explains that, while we have a powerful instinct to categorize things, we don't benefit from creating categories that have only a single member. Uncategorizable stuff should go together in its own space because this is the easiest way for the human mind to keep track of it. If you have a lot of folders with just one document in each, you might soon forget that those folders even exist. It would be better if their contents remained together in a bigger "misc" folder.

When it comes to teaching younger generations how to organize their minds, Mr. Levitin makes some surprising points. Today's college students are thought of as "digital natives" who are inherently skilled with computers and the Internet because they grew up with them from their earliest days. But they have trouble distinguishing media outlets and websites that at least try to report news and facts objectively from those that are deliberately partisan or ideological. Even medical students aren't good at telling high-quality journals (whose research reports should be given more trust) from low-quality ones (whose reports should be regarded with skepticism). Given that our minds mostly evolved long before the invention of reading and writing, let alone mass media, it stands to reason that a fine eye for evaluating the quality of sources must be learned, and even taught, rather than assumed to be part of our standard equipment.

What to do about this? It turns out that librarians have already responded by writing guides to evaluating sources. These include questions like "is the page current?" and "what is the domain?" (A page from nih.gov will have more authoritative medical advice than one from autismspeaks.org.) These suggestions point to considerations that many of us take for granted but that are increasingly crucial for everyone to grasp, now that Google's cornucopia is rarely farther away than our hip pockets.

Some of Mr. Levitin's recommendations may seem like little more than common sense or reiterations of techniques developed from generations of experience. Has anyone not heard of a junk drawer? But if there is one lesson to be drawn from the past century of research on human behavior it is that common sense is a much poorer guide to life than, well, common sense would have us believe. Common sense is often contradicted by empirical evidence—we assume, for instance, that we remember important events in precise detail, when research shows that such memories become distorted and decay over time. And elements of common sense frequently contradict one another. How can opposites attract but birds of a feather flock together? When common sense can be shown to be consistent with solid scientific principles, we should prize it all the more.

"The Organized Mind" is an organized book, but it also rewards dipping in at any point, for there are fascinating facts and examples throughout. Mr. Levitin concisely explains Kolmogorov complexity (a way to measure how much information a message or algorithm contains, which can help with communicating and storing data optimally). He provides a handy inventory of things to keep extras of in your luggage, so you don't have to remember them and then scurry around to gather them up before you leave for a trip. (Most important, keep a phone charger in your bag at all times.) He even lists the rules that govern how interstate highways are numbered—with examples and a map. The point is that if you learn the system, you don't need to memorize the specific directions and junctions of dozens of individual highways. (Nowadays, though, we outsource this knowledge to GPS-equipped cars and phones.) An appendix explains how to construct simple 2x2 tables to properly interpret important percentages and probabilities like the likelihood of having a serious disease given a positive diagnostic test.

Like any neuroscience-based book, "The Organized Mind" has to confront the problem of the still-tentative nature of many of the most fascinating findings and resist the ever-present temptation to pick a few new ideas to weave a just-so story. Books with titles like "the new science of X" or "the neuroscience of Y" can almost be counted on to be wrong—often breathlessly so. Mr. Levitin mostly eludes this trap by sticking to established principles, such as the limited capacities of memory and attention or the biases that plague our thinking (e.g., the difficulty of reasoning rationally about risks). None of these principles will rise or fall at the next conference or in the next edition of a scholarly journal.

Clearly, most of us could use more organization in our lives and in our minds. But does a push for organization and focus, for the optimal use of our scarce attention, come at any cost? Perhaps. Students who use "study drugs" like Ritalin or Adderall to stay awake and concentrate on work and deadlines report that they find it easier to complete projects but feel that the results are less creative than they otherwise would be. Such a tradeoff—if it is even more than anecdotal—could be the natural result of "powering through" a task in a fixed time frame, regardless of whether drugs or other forces were involved.

But such trade-offs do not mean that organization and creativity are enemies. Without enough organization to complete a project, no amount of creativity will have an effect. And some of the most creative people, Mr. Levitin notes, were also some of the most fanatical organizers of their output. Michael Jackson employed a full-time archivist, and John Lennon "kept boxes and boxes of work tapes of songs in progress, carefully labeled." Perhaps this habit explains why the ex-Beatle seemed to go on releasing album after album of new material after his death. In any case, it's a mistake to think creativity suffers from organization, that a messy desk or office is a sign of genius. What does benefit creativity is exposing one's mind to a variety of influences, sources and types of information. A mind that can stay focused despite diverse stimulation, and that can produce enough ideas so that some of them might be truly great, must be an organized one.

Continued in article

 


Bob Jensen's threads on asynchronous versus synchronous learning ---
http://www.trinity.edu/rjensen/255wp.htm

 

 




 

Click here to return to the top of this document.