Bob Jensen's Threads on
Assessment
Bob
Jensen at Trinity
University
George Carlin - Who Really Controls America ---
Click Here
"More kids pass tests if we simplify the tests --- Why education will never be
fixed."
The Downfall of Lecturing
Coaches Graham and Gazowski
Teaching Evaluations and RateMyProfessor
Concept Knowledge and Assessment of Deep
Understanding
Onsite Versus Online Differences for Faculty
Online Versus Onsite for
Students.
Onsite Versus Online Education
(including controls for online examinations and assignments)
Student Engagement
Students Reviewing Each Others' Projects
Online Education Effectiveness and Testing
What Works in Education?
Predictors of Success
Minimum Grades as a School Policy
Team Grading
Too Good to Grade:
How can these students get into doctoral programs and law school if their
prestigious universities will not disclose grades and class rankings? Why
grade at all in this case?
Software for faculty and departmental
performance evaluation and management
K-12 School and College Assessment and College Admission Testing
Civil Rights Groups That Favor
Standardized Testing
Computer-Based Assessment
Computer Grading of Essays
Assessment in General (including the
debate over whether academic research itself should be assessed)
AICPA Educational Competency Assessment for Accounting
Students
Assessment
Issues: Measurement and No-Significant-Differences
Dangers of Self Assessment
The Criterion Problem
Success Stories in Education Technology
Research Versus Teaching
"Favorite Teacher" Versus "Learned the Most"
Grade Inflation Versus Teaching Evaluations
Student Evaluations and Learning Styles
Assessment Takes Center Stage in Online Learning: The
Saga of Western Governors University
Measures of Quality in Internet-Based Distance
Learning
Number Watch: How to Lie With Statistics
Drop Out Problems
On
the Dark Side
Accreditation Issues
Software
for Online Examinations and Quizzes
Onsite Versus Online Education
(including controls for online examinations and assignments)
The term "electroThenic
portfolio," or "ePortfolio," is on everyone's lips. What
does this mean?
Research Versus Teaching
"Favorite Teacher" Versus "Learned the Most"
Grade Inflation Versus Course Evaluations
Work Experience Substitutes for College Credits
Certification Examinations
Should attendance guarantee passing?
Peer Review Controversies in Academic Journals
Research Questions About the Corporate Ratings
Game
Differences between "popular teacher"
versus "master teacher"
versus "mastery learning"
versus "master educator."
Are student usages of FaceBook correlated with lower grades?
Concerns About Social Networking, Blogging, and Twittering in
Education ---
http://www.trinity.edu/rjensen/ListservRoles.htm
Bob Jensen's threads on Cognitive Processes and Artificial Intelligence
are at
http://www.trinity.edu/rjensen/000aaa/thetools.htm#CognitiveProcesses
Degrees Versus Piecemeal Distance (Online)
Education
Bob Jensen's threads on memory and metacognition are at
http://www.trinity.edu/rjensen/265wp.htm
Full Disclosure to Consumers of Higher Education (including assessment
of colleges and the Spellings Commission Report) ---
http://www.trinity.edu/rjensen/HigherEdControversies.htm#FullDisclosure
Also see
http://www.trinity.edu/rjensen/HigherEdControversies.htm#Bok
Publish Exams Online ---
http://www.examprofessor.com/main/index.cfm
Controversies in Higher Education ---
http://www.trinity.edu/rjensen/HigherEdControversies.htm
Bob Jensen's threads on cheating and plagiarism ---
http://www.trinity.edu/rjensen/plagiarism.htm
Effort Reporting Technology for Higher Education ---
http://www.huronconsultinggroup.com/uploadedFiles/ECRT_email.pdf
Some Thoughts on Competency-Based Training
and Education ---
http://www.trinity.edu/rjensen/competency.htm
You can download (for free) hours of
MP3 audio and the PowerPoint presentation slides from several of the best
education technology workshops that I ever organized. --- http://www.cs.trinity.edu/~rjensen/002cpe/02start.htm
Asynchronous Learning Advantages and
Disadvantages ---
http://www.trinity.edu/rjensen/255wp.htm
Dark Sides of Education Technologies ---
http://www.trinity.edu/rjensen/000aaa/0000start.htm
For threaded audio and email
messages from early pioneers in distance education, go http://www.trinity.edu/rjensen/ideasmes.htm
Full Disclosure to Consumers of Higher Education at
http://www.trinity.edu/rjensen/HigherEdControversies.htm#FullDisclosure
From PhD Comics: Helpers for Filling Out Teaching Evaluations ---
http://www.phdcomics.com/comics.php?f=847
As David Bartholomae observes, “We make a huge
mistake if we don’t try to articulate more publicly what it is we value in
intellectual work. We do this routinely for our students — so it should not be
difficult to find the language we need to speak to parents and legislators.” If
we do not try to find that public language but argue instead that we are not
accountable to those parents and legislators, we will only confirm what our
cynical detractors say about us, that our real aim is to keep the secrets of our
intellectual club to ourselves. By asking us to spell out those secrets and
measuring our success in opening them to all, outcomes assessment helps make
democratic education a reality.
Gerald Graff, "Assessment Changes
Everything," Inside Higher Ed, February 21, 2008 ---
http://www.insidehighered.com/views/2008/02/21/graff
Gerald Graff is professor of English at the University of Illinois at Chicago
and president of the Modern Language Association. This essay is adapted from a
paper he delivered in December at the MLA annual meeting, a version of which
appears on the MLA’s Web site and is reproduced here with the association’s
permission. Among Graff’s books are Professing Literature, Beyond the
Culture Wars and Clueless in Academe: How School Obscures the Life of the Mind.
Would-be lawyers in Wisconsin who have challenged
the state’s policy of allowing graduates of state law schools to practice law
without passing the state’s bar exam will have their day in court after all, the
Associated Press reported. A federal appeals court has reinstated a lawsuit
challenging the practice, which apparently is unique in the United States.
Katherine Mangan, "Appeals Court Reinstates Lawsuit Over Wisconsin's Bar-Exam
Exemption," Chronicle of Higher Education, January 29, 2008 ---
Click Here
"How Do People Learn," Sloan-C Review, February 2004 ---
http://www.aln.org/publications/view/v3n2/coverv3n2.htm
Like some of the
other well known cognitive and affective taxonomies, the Kolb figure
illustrates a range of interrelated learning activities and styles beneficial
to novices and experts. Designed to emphasize reflection on learners’
experiences, and progressive conceptualization and active experimentation,
this kind of environment is congruent with the aim of lifelong learning. Randy
Garrison points out that:
From a content
perspective, the key is not to inundate students with information. The first
responsibility of the teacher or content expert is to identify the central
idea and have students reflect upon and share their conceptions. Students
need to be hooked on a big idea if learners are to be motivated to be
reflective and self-directed in constructing meaning. Inundating learners
with information is discouraging and is not consistent with higher order
learning . . . Inappropriate assessment and excessive information will
seriously undermine reflection and the effectiveness of asynchronous
learning.
Reflection on a big
question is amplified when it enters collaborative inquiry, as multiple styles
and approaches interact to respond to the challenge and create solutions. In
How People Learn: Brain, Mind, Experience, and School, John Bransford and
colleagues describe a legacy cycle for collaborative inquiry, depicted in a
figure by Vanderbilt University researchers (see image, lower left).
Continued in the article
December 12, 2003 message from Tracey Sutherland [return@aaahq.org]
THE EDUCATIONAL COMPETENCY ASSESSMENT (ECA) WEB
SITE IS LIVE! http://www.aicpa-eca.org
The AICPA provides this resource to help educators
integrate the skills-based competencies needed by entry-level accounting
professionals. These competencies, defined within the AICPA Core Competency
Framework Project, have been derived from academic and professional competency
models and have been widely endorsed within the academic community. Created by
educators for educators, the evaluation and educational strategies resources
on this site are offered for your use and adaptation.
The ECA site contains a LIBRARY that, in addition to
the Core Competency Database and Education Strategies, provides information
and guidance on Evaluating Competency Coverage and Assessing Student
Performance.
To assist you as you assess student performance and
evaluate competency coverage in your courses and programs, the ECA ORGANIZERS
guide you through the process of gathering, compiling and analyzing evidence
and data so that you may document your activities and progress in addressing
the AICPA Core Competencies.
The ECA site can be accessed through the Educator's
page of aicpa.org, or at the URL listed above.
The Downfall of Lecturing
My Hero at the American Accounting Association
Meetings in San Antonio on August 13, 2002 --- Amy Dunbar
How to students evaluate Amy Dunbar's online tax courses?
This link is a pdf doc that I will be presenting at a
CPE session with Bob Jensen, Nancy Keeshan, and Dennis Beresford at the AAA on
Tuesday. I updated the paper I wrote that summarized the summer 2001 online
course. You might be interested in the exhibits, particularly Exhibit II,
which summarizes student responses to the learning tools over the two summers.
This summer I used two new learning tools: synchronous classes (I used
Placeware) and RealPresenter videos. My read of the synchronous class comments
is that most students liked having synchronous classes, but not often and not
long ones! 8 of the 57 responding students thought the classes were a waste of
time. 19 of my students, however, didn't like the RealPresenter videos, partly
due to technology problems. Those who did like them, however, really liked
them and many wanted more of them. I think that as students get faster access
to the Internet, the videos will be more useful.
http://www.sba.uconn.edu/users/adunbar/genesis_of_an_online_course_2002.pdf
Amy Dunbar
UConn
Education is an admirable thing, but it is well to
remember from time to time that nothing that is worth learning can be taught.
Oscar Wilde
"The Objective of Education is Learning, Not Teaching (audio version
available)," University of Pennsylvania's Knowledge@Wharton, August 20, 2008
---
http://knowledge.wharton.upenn.edu/article.cfm;jsessionid=9a30b5674a8d333e4d18?articleid=2032
In their book, Turning Learning Right Side
Up: Putting Education Back on Track, authors Russell L. Ackoff and
Daniel Greenberg point out that today's education system is seriously flawed
-- it focuses on teaching rather than learning. "Why should children -- or
adults -- be asked to do something computers and related equipment can do
much better than they can?" the authors ask in the following excerpt from
the book. "Why doesn't education focus on what humans can do better than the
machines and instruments they create?"
"Education is an admirable thing, but it is well to
remember from time to time that nothing that is worth learning can be
taught."
-- Oscar Wilde
Traditional education focuses on teaching, not
learning. It incorrectly assumes that for every ounce of teaching there is
an ounce of learning by those who are taught. However, most of what we learn
before, during, and after attending schools is learned without its being
taught to us. A child learns such fundamental things as how to walk, talk,
eat, dress, and so on without being taught these things. Adults learn most
of what they use at work or at leisure while at work or leisure. Most of
what is taught in classroom settings is forgotten, and much or what is
remembered is irrelevant.
In most schools, memorization is mistaken for
learning. Most of what is remembered is remembered only for a short time,
but then is quickly forgotten. (How many remember how to take a square root
or ever have a need to?) Furthermore, even young children are aware of the
fact that most of what is expected of them in school can better be done by
computers, recording machines, cameras, and so on. They are treated as poor
surrogates for such machines and instruments. Why should children -- or
adults, for that matter -- be asked to do something computers and related
equipment can do much better than they can? Why doesn't education focus on
what humans can do better than the machines and instruments they create?
When those who have taught others are asked who in
the classes learned most, virtually all of them say, "The teacher." It is
apparent to those who have taught that teaching is a better way to learn
than being taught. Teaching enables the teacher to discover what one thinks
about the subject being taught. Schools are upside down: Students should be
teaching and faculty learning.
After lecturing to undergraduates at a major
university, I was accosted by a student who had attended the lecture. After
some complimentary remarks, he asked, "How long ago did you teach your first
class?"
I responded, "In September of 1941."
"Wow!" The student said. "You mean to say you have
been teaching for more than 60 years?"
"Yes."
"When did you last teach a course in a subject that
existed when you were a student?"
This difficult question required some thought.
After a pause, I said, "September of 1951."
"Wow! You mean to say that everything you have
taught in more than 50 years was not taught to you; you had to
learn on your own?"
"Right."
"You must be a pretty good learner."
I modestly agreed.
The student then said, "What a shame you're not
that good a teacher."
The student had it right; what most faculty members
are good at, if anything, is learning rather than teaching. Recall that in
the one-room schoolhouse, students taught students. The teacher served as a
guide and a resource but not as one who force-fed content into students'
minds.
Ways of Learning
There are many different ways of learning; teaching
is only one of them. We learn a great deal on our own, in independent study
or play. We learn a great deal interacting with others informally -- sharing
what we are learning with others and vice versa. We learn a great deal by
doing, through trial and error. Long before there were schools as we know
them, there was apprenticeship -- learning how to do something by trying it
under the guidance of one who knows how. For example, one can learn more
architecture by having to design and build one's own house than by taking
any number of courses on the subject. When physicians are asked whether they
leaned more in classes or during their internship, without exception they
answer, "Internship."
In the educational process, students should be
offered a wide variety of ways to learn, among which they could choose or
with which they could experiment. They do not have to learn different things
the same way. They should learn at a very early stage of "schooling" that
learning how to learn is largely their responsibility -- with the help they
seek but that is not imposed on them.
The objective of education is learning, not
teaching.
There are two ways that teaching is a powerful tool
of learning. Let's abandon for the moment the loaded word teaching, which is
unfortunately all too closely linked to the notion of "talking at" or
"lecturing," and use instead the rather awkward phrase explaining something
to someone else who wants to find out about it. One aspect of explaining
something is getting yourself up to snuff on whatever it is that you are
trying to explain. I can't very well explain to you how Newton accounted for
planetary motion if I haven't boned up on my Newtonian mechanics first. This
is a problem we all face all the time, when we are expected to explain
something. (Wife asks, "How do we get to Valley Forge from home?" And
husband, who does not want to admit he has no idea at all, excuses himself
to go to the bathroom; he quickly Googles Mapquest to find out.) This is one
sense in which the one who explains learns the most, because the person to
whom the explanation is made can afford to forget the explanation promptly
in most cases; but the explainers will find it sticking in their minds a lot
longer, because they struggled to gain an understanding in the first place
in a form clear enough to explain.
The second aspect of explaining something that
leaves the explainer more enriched, and with a much deeper understanding of
the subject, is this: To satisfy the person being addressed, to the point
where that person can nod his head and say, "Ah, yes, now I understand!"
explainers must not only get the matter to fit comfortably into their own
worldview, into their own personal frame of reference for understanding the
world around them, they also have to figure out how to link their frame of
reference to the worldview of the person receiving the explanation, so that
the explanation can make sense to that person, too. This involves an intense
effort on the part of the explainer to get into the other person's mind, so
to speak, and that exercise is at the heart of learning in general. For, by
practicing repeatedly how to create links between my mind and another's, I
am reaching the very core of the art of learning from the ambient culture.
Without that skill, I can only learn from direct experience; with that
skill, I can learn from the experience of the whole world. Thus, whenever I
struggle to explain something to someone else, and succeed in doing so, I am
advancing my ability to learn from others, too.
Learning through Explanation
This aspect of learning through explanation has
been overlooked by most commentators. And that is a shame, because both
aspects of learning are what makes the age mixing that takes place in the
world at large such a valuable educational tool. Younger kids are always
seeking answers from older kids -- sometimes just slightly older kids (the
seven-year old tapping the presumed life wisdom of the
so-much-more-experienced nine year old), often much older kids. The older
kids love it, and their abilities are exercised mightily in these
interactions. They have to figure out what it is that they understand about
the question being raised, and they have to figure out how to make their
understanding comprehensible to the younger kids. The same process occurs
over and over again in the world at large; this is why it is so important to
keep communities multi-aged, and why it is so destructive to learning, and
to the development of culture in general, to segregate certain ages
(children, old people) from others.
Continued in article
Bob Jensen's threads on assessment, learning, and technology in education
are at
http://www.trinity.edu/rjensen/000aaa/0000start.htm
"Web Surfing in the Classroom: Sound Familiar?" by Catherine Rampell,
Chronicle of Higher Education, May 15, 2008 ---
http://chronicle.com/wiredcampus/index.php?id=3004&utm_source=wc&utm_medium=en
Over at the New York Times’s
Freakonomics blog, Yale Law School professor Ian
Ayres praises the University of Chicago Law School’s decision to
eliminate Internet access in some classrooms. But
more importantly, he recounts an amusing sketch from the Yale’s “Law Revue”
skit night, which is worth sharing in full:
One of the skits had a group of students sitting at
desks, facing the audience, listening to a professor drone on.
All of the students were looking at laptops except
for one, who had a deck of cards and was playing solitaire. The professor
was outraged and demanded that the student explain why she was playing
cards. When she answered “My laptop is broken,” I remember there was
simultaneously a roar of laughter from the student body and a gasp from the
professors around me. In this one moment, we learned that something new was
happening in class.
Bob Jensen's threads on higher education controversies are at
http://www.trinity.edu/rjensen/HigherEdControversies.htm
Random Thoughts (about learning from a retired professor of
engineering) ---
http://www4.ncsu.edu/unity/lockers/users/f/felder/public/Columns.html
Dr. Felder's column in Chemical Engineering Education
Focus is heavily upon active learning and group learning.
Bob Jensen's threads on learning are in the following links:
http://www.trinity.edu/rjensen/assess.htm
http://www.trinity.edu/rjensen/255wp.htm
http://www.trinity.edu/rjensen/265wp.htm
March 3, 2005 message from Carolyn Kotlas [kotlas@email.unc.edu]
WHAT LEADS TO ACHIEVING SUCCESS IN DISTANCE
EDUCATION?
"Achieving Success in Internet-Supported
Learning in Higher Education," released February 1, 2005, reports on the
study of distance education conducted by the Alliance for Higher Education
Competitiveness (A-HEC). A-HEC surveyed 21 colleges and universities to
"uncover best practices in achieving success with the use of the Internet
in higher education." Some of the questions asked by the study included:
"Why do institutions move online? Are there
particular conditions under which e-Learning will be successful?"
"What is the role of leadership and by whom?
What level of investment or commitment is necessary for success?"
"How do institutions evaluate and measure
success?"
"What are the most important and successful
factors for student support and faculty support?"
"Where do institutions get stuck? What are the
key challenges?"
The complete report is available online, at no cost,
at http://www.a-hec.org/e-learning_study.html.
The "core focus" of the nonprofit Alliance
for Higher Education Competitiveness (A-HEC) "is on communicating how
higher education leaders are creating positive change by crystallizing their
mission, offering more effective academic programs, defining their role in
society, and putting in place balanced accountability measures." For more
information, go to http://www.a-hec.org/ .
Individual membership in A-HEC is free.
Hi Yvonne,
For what it is worth, my advice to new
faculty is at http://www.trinity.edu/rjensen/000aaa/newfaculty.htm
One thing to remember is that the
employers of our students (especially the public accounting firms) are very
unhappy with our lecture/drill pedagogy at the introductory and intermediate
levels. They believe that such pedagogy turns away top students, especially
creative and conceptualizing students. Employers believe that
lecture/drill pedagogy attracts savant-like memorizers who can recite their
lessons book and verse but have few creative talents and poor prospects for
becoming leaders. The large accounting firms believed this so strongly that they
donated several million dollars to the American Accounting Association for the
purpose of motivating new pedagogy experimentation. This led to the Accounting
Change Commission (AECC) and the mixed-outcome experiments that followed. See http://accounting.rutgers.edu/raw/aaa/facdev/aecc.htm
The easiest pedagogy for faculty is
lecturing, and it is appealing to busy faculty who do not have time for students
outside the classroom. When lecturing to large classes it is even easier because
you don't have to get to know the students and have a great excuse for using
multiple choice examinations and graduate student teaching assistants. I always
remember an economics professor at Michigan State University who said that when
teaching basic economics it did not matter whether he had a live class of 300
students or a televised class of 3,000 students. His full-time teaching load was
three hours per week in front of a TV camera. He was a very good lecturer and
truly loved his three-hour per week job!
Lecturing appeals to faculty because it
often leads to the highest teaching evaluations. Students love faculty who
spoon feed and make learning seem easy. It's much easier when mom or dad
spoon the pudding out of the jar than when you have to hold your own spoon
and/or find your own jar.
An opposite but very effective pedagogy
is the AECC (University of Virginia) BAM Pedagogy that entails live classrooms
with no lectures. BAM instructors think it is more important for students to
learn on their own instead of sitting through spoon-fed learning lectures. I
think it takes a special kind of teacher to pull off the astoundingly successful
BAM pedagogy. Interestingly, it is often some of our best lecturers who decided
to stop lecturing because they experimented with the BAM and found it to be far
more effective for long-term memory. The top BAM enthusiasts are Tony Catanach
at Villanova University and David Croll at the University of Virginia. Note,
however, that most BAM applications have been at the intermediate accounting
level. I have my doubts (and I think BAM instructors will agree) that BAM will
probably fail at the introductory level. You can read about the BAM pedagogy at http://www.trinity.edu/rjensen/265wp.htm
At the introductory level we have what
I like to call the Pincus (User Approach) Pedagogy. Karen Pincus is now at the
University of Arkansas, but at the time that her first learning experiments were
conducted, she taught basic accounting at the University of Southern California.
The Pincus Pedagogy is a little like both the BAM and the case method
pedagogies. However, instead of having prepared learning cases, the Pincus
Pedagogy sends students to on-site field visitations where they observe on-site
operations and are then assigned tasks to creatively suggest ways of improving
existing accounting, internal control, and information systems. Like the BAM,
the Pincus Pedagogy avoids lecturing and classroom drill. Therein lies the
controversy. Students and faculty in subsequent courses often complain that the
Pincus Pedagogy students do not know the fundamental prerequisites of basic
accounting needed for intermediate and advanced-level accounting courses.
Two possible links of interest on the controversial Pincus Pedagogy are as
follows:
Where the Pincus Pedagogy and the BAM
Pedagogy differ lies in subject matter itself and stress on creativity. The BAM
focuses on traditional subject matter that is found in such textbooks as
intermediate accounting textbooks. The BAM Pedagogy simply requires that
students learn any way they want to learn on their own since students remember
best what they learned by themselves. The Pincus Pedagogy does not focus on much
of the debit and credit "rules" found in most traditional textbooks.
Students are required to be more creative at the expense of memorizing the
"rules."
The Pincus Pedagogy is motivated by the
belief that traditional lecturing/drill pedagogy at the basic accounting and tax
levels discourages the best and more-creative students to pursue careers in the
accountancy profession. The BAM pedagogy is motivated more by the belief that
lecturing is a poor pedagogy for long-term memory of technical details. What is
interesting is that the leading proponents of getting away from the
lecture/drill pedagogy (i.e., Karen Pincus and Anthony Catenach) were previously
two of the very best lecturers in accountancy. If you have ever heard either of
them lecture, I think you would agree that you wish all your lecturers had been
only half as good. I am certain that both of these exceptional teachers would
agree that lecturing is easier than any other alternatives. However, they do not
feel that lecturing is the best alternative for top students.
Between lecturing and the BAM Pedagogy,
we have case method teaching. Case method teaching is a little like lecturing
and a little like the BAM with some instructors providing answers in case wrap
ups versus some instructors forcing students to provide all the answers. Master
case teachers at Harvard University seldom provide answers even in case wrap
ups, and often the cases do not have any known answer-book-type solutions. The
best Harvard cases have alternative solutions with success being based upon
discovering and defending an alternative solution. Students sometimes
interactively discover solutions that the case writers never envisioned. I
generally find case teaching difficult at the undergraduate level if students do
not yet have the tools and maturity to contribute to case discussions.
Interestingly, it may be somewhat easier to use the BAM at the undergraduate
level than Harvard-type cases. The reason is that BAM instructors are often
dealing with more rule-based subject matter such as intermediate accounting or
tax rather than conceptual subject matter such as strategic decision making,
business valuation, and financial risk analysis.
The hardest pedagogy today is probably
a Socratic pedagogy online with instant messaging communications where an
instructor who's on call about 60 hours per week from his or her home. The
online instructor monitors the chats and team communications between students in
the course at most any time of day or night. Amy Dunbar can tell you about this
tedious pedagogy since she's using it for tax courses and will be providing a
workshop that tells about how to do it and how not to do it. The next scheduled
workshop precedes the AAA Annual Meetings on August 1, 2003 in Hawaii. You can
also hear Dr. Dunbar and view her PowerPoint show from a previous workshop at http://www.cs.trinity.edu/~rjensen/002cpe/02start.htm#2002
In conclusion, always remember that
there is no optimal pedagogy in all circumstances. All learning is
circumstantial based upon such key ingredients as student maturity, student
motivation, instructor talent, instructor dedication, instructor time, library
resources, technology resources, and many other factors that come to bear at
each moment in time. And do keep in mind that how you teach may determine what
students you keep as majors and what you turn away.
I tend to agree with the accountancy
firms that contend that traditional lecturing probably turns away many of the
top students who might otherwise major in accountancy.
At the same time, I tend to agree with
students who contend that they took accounting courses to learn accounting
rather than economics, computer engineering, and behavioral science.
Bob Jensen
-----Original
Message-----
From: Lou&Bonnie [mailto:gyp1@EARTHLINK.NET]
Sent: Thursday, January 16, 2003 5:03 PM
I am a beginning
accounting instructor (part-time) at a local community college. I am applying
for a full-time faculty position, but am having trouble with a question.
Methodology in accounting--what works best for a diversified group of
individuals. Some students work with accounting, but on a computer and have no
understanding of what the information they are entering really means to some
individuals who have no accounting experience whatsoever. What is the best
methodology to use, lecture, overhead, classroom participation? I am not sure
and I would like your feedback. Thank you in advance for your help.
Yvonne
January 20, 2003 reply from Thomas C. Omer
[omer@UIC.EDU]
Don’t forget about
Project Discovery going on at the University of Illinois Champaign-Urbana
Thomas C. Omer
Associate Professor
Department of Accounting University of Illinois At Chicago
The Art of Discovery: Finding the forest in spite of the trees.
Thanks for reminding me Tom. A good
link for Project Discovery is at http://accounting.rutgers.edu/raw/aaa/facdev/aeccuind.htm
January 17, 2003 reply from David R. Fordham
[fordhadr@JMU.EDU]
I'll add an
endorsement to Bob's advice to new teachers. His page should be required
reading for Ph.D.s.
And I'll add one more
tidbit.
Most educators
overlook the distinction between "lectures" and
"demonstrations".
There is probably no
need for any true "lecture" in the field of accounting at the
college level, even though it is still the dominant paradigm at most
institutions.
However, there is
still a great need for "live demonstrations", **especially** at the
introductory level.
Accounting is a
complex process. Introductory students in ANY field learn more about complex
processes from demonstrations than probably any other method.
Then, they move on
and learn more from "practicing" the process, once they've learned
the steps and concepts of the process. And for intermediate and advanced
students, practice is the best place to "discover" the nuances and
details.
While
"Discovery" is probably the best learning method of all, it is
frequently very difficult to "discover" a complex process correctly
from its beginning, on your own. Thus, a quick demonstration can often be of
immense value at the introductory level. It's an efficient way of
communicating sequences, relationships, and dynamics, all of which are present
in accounting processes.
Bottom line: You can
(and should) probably eliminate "lectures" from your classes. You
should not entirely eliminate "demonstrations" from your classes.
Unfortunately, most
education-improvement reform literature does not draw the distinction: anytime
the teacher is doing the talking in front of a class, using blackboard and
chalk or PowerPoint, they label it "lecture" and suggest you don't
do it! This is, in my view, oversimplification, and very bad advice.
Your teaching will
change a whole lot (for the better!) once you realize that students only need
demonstrations of processes. You will eliminate a lot of material you used to
"lecture" on. This will make room for all kinds of other things that
will improve your teaching over the old "lecture" method:
discussions, Socratic dialogs, cases and dilemmas, even some entertainment
here and there.
Plus, the
"lectures" you retain will change character. Take your cue from Mr.
Wizard or Bill Nye the Science Guy, who appear to "lecture" (it's
about the only thing you can do in front of a camera!), but whose entire
program is pretty much devoted to demonstration. Good demonstrations do more
than just demonstrate, they also motivate! Most lectures don't!
Another two pennies
from the verbose one...
David R.
Fordham
PBGH Faculty Fellow
James Madison University
January 16, 2003 message from Peter French [pjfrench@CELESTIAL.COM.AU]
I found this source http://www.thomson.com/swcp/gita.html
and also Duncan Williamson has some very good basic material on his sites http://duncanwil.co.uk/index.htm
; http://www.duncanwil.co.uk/objacc.html
;
Don't forget the world lecture hall at http://www.utexas.edu/world/lecture/
;
This reminds me of how I learned ... the 'real
learning' in the workplace...
I remember my first true life consolidation - 130
companies in 1967. We filled a wall with butchers paper and had 'callers',
'writers' and 'adders' who called out the information to others who wrote out
the entries and others who did the adding. I was 25 and quite scared. The
Finance Director knew this and told me [1] to stick with 'T' accounts to be
sure I was making the right entry - just stick the ones you are sure in and
don't even think about the other entry - it must 'balance' it out; [2] just
because we are dealing with 130 companies and several hundreds of millions of
dollars don't lose sight of the fact that really it is no different from the
corner store. I have never forgotten the simplistic approach. He said - if the
numbers scare you, decimalise them to 100,000's in your mind - it helps ...
and it did. He often used to say the Dr/Cr entries out aloud
I entered teaching aged 48 after having been in
industry and practice for nearly 30 years. Whether i am teaching introductory
accounting, partnership formation/dissolution, consolidations, asset
revaluation, tax affect accounting, I simply write up the same basic entries
on the white board each session - I never use an overhead for this, I always
write it up and say it out aloud, and most copy/follow me - and then recap and
get on with the lesson. I always take time out to 'flow chart' what we are
doing so that they never loose sight of the real picture ... this simple
system works, and have never let my students down.
There have been several movements away form rote
learning in all levels of education - often with disastrous consequences. It
has its place and I am very proud to rely on it. This works and when it isn't
broken, I am not about to try to fix it.
Good luck - it is the greatest responsibility in the
world, and gives the greatest job satisfaction. It is worth every hour and
every grey hair. To realise that you have enabled someone to change their
lives, made a dream come true, eclipses every successful takeover battle or
tax fight that I won i have ever had.
Good luck - may it be to you what is has been to me.
Peter French
January 17, 2003 reply from Michael O'Neil, CPA Adjunct Prof. Weber [Marine8105@AOL.COM]
I am currently teaching high school students, some of
whom will hopefully go on to college. Parents expect you to teach the
children, which really amounts to lecturing, or going over the text material.
When you do this they do not read the textbook, nor do they know how to use
the textbook to answer homework questions. If you don't lecture then the
parents will blame you for "not" teaching their children the
material.
I agree that discovery is the best type of learning,
and the most fun. I teach geometry and accounting/consumer finance. Geometry
leans itself to discovery, but to do so you need certain materials. At our
level (high school) we are also dealing several other issues you don't have at
the college level. In my accounting classes I teach the debit/credit, etc. and
then have them do a lot of work using two different accounting programs. When
they make errors I have them discover the error and correct it. They probably
know very little about posting, and the formatting of financial statements
although we covered it. Before we used the programs we did a lot of pencil
work.
Even when I taught accounting at the college and
junior college level I found students were reluctant to, and not well prepared
to, use their textbooks. Nor were they inclined to DO their homework.
I am sure that many of you have noticed a drop off in
quality of students in the last years. I wish I could tell you that I see that
it will change, but I do not see any effort in that direction. Education
reminds me of a hot air balloon being piloted by people who lease the balloon
and have no idea how to land it. They are just flying around enjoying the
view. If we think in terms of bankruptcy education is ready for Chapter 11.
Mike ONeil
January 17, 2003 reply from Chuck Pier
[texcap@HOTMAIL.COM]
While not in
accounting, I would like to share some information on my wife's experience
with online education. She has a background (10 years) as a public school
teacher and decided to get her graduate degree in library science. Since I was
about to finish my doctoral studies and we knew we would be moving she wanted
to find a program that would allow her to move away and not lose too many
hours in the transfer process. What she found was the online program at the
University of North Texas (UNT) in Denton. Through this program she will be
able to complete a 36 hour American Library Association accredited Master's
degree in Library Science and only spend a total of 9 days on campus. The 9
days are split into a one day session and 2 four day sessions, which can be
combined into 1 five and 1 four day session. Other than these 9 days the
entire course is conducted over the internet. The vast majority is
asynchronous, but there are some parts conducted in a synchronous manner.
She has completed
about 3/4 of the program and is currently in Denton for her last on campus
session. While I often worry about the quality of online programs, after
seeing how much work and time she is required to put in, I don't think I
should worry as much. I can honestly say that I feel she is getting a better,
more thorough education than most traditional programs. I know at a minimum
she has covered a lot more material.
All in all her
experience has been positive and this program fit her needs. I think the MLS
program at UNT has been very successful to date and appears to be growing
quite rapidly. It may serve as a role model for programs in other areas.
Chuck Pier
Charles A.
Pier
Assistant Professor Department of Accounting
Walker College of Business
Appalachian State University
Boone, NC 28608 email: pierca@appstate.edu
828-262-6189
Concept Knowledge and Assessment of
Deep UnderstandingWhat questions might classroom teachers ask of their students,
the answers to which would allow a strong inference that the students
"understood"?
"The Assessment of “Understanding,” by Lloyd Bond, Carnegie Foundation for
Advancement in Teaching ---
Click Here
Study to remember and you will forget.
Study to understand and you will remember.
—Anonymous
I once sat on the dissertation
committee of a graduate student in mathematics education who had examined
whether advanced graduate students in math and science education could
explain the logic underlying a popular procedure for extracting square roots
by hand. Few could explain why the procedure worked. Intrigued by the
results, she decided to investigate whether they could explain the logic
underlying long division. To her surprise, most in her sample could not. All
of the students were adept at division, but few understood why the procedure
worked.
In a series of studies at Johns Hopkins University,
researchers found that first year physics students could unerringly solve
fairly sophisticated problems in classical physics involving moving bodies,
but many did not understand the implications of their answers for the
behavior of objects in the real world. For example, many could not draw the
proper trajectories of objects cut from a swinging pendulum that their
equations implied.
What then does it mean to “understand” something—a
concept, a scientific principle, an extended rhetorical argument, a
procedure or algorithm? What questions might classroom teachers ask of their
students, the answers to which would allow a strong inference that the
students “understood”? Every educator from kindergarten through graduate and
professional school must grapple almost daily with this fundamental
question. Do my students really “get it”? Do they genuinely understand the
principle I was trying to get across at a level deeper than mere
regurgitation? Rather than confront the problem head on, some teachers,
perhaps in frustration, sidestep it. Rather then assign projects or
construct examinations that probe students’ deep understanding, they require
only that students apply the learned procedures to problems highly similar
to those discussed in class. Other teachers with the inclination, time and
wherewithal often resort to essay tests that invite their students to probe
more deeply, but as often as not their students decline the invitation and
stay on the surface.
I have thought about issues surrounding the
measurement of understanding on and off for years, but have not
systematically followed the literature on the topic. On a lark, I conducted
three separate Google searches and obtained the following results:
- “nature of understanding” 41,600 hits
- “measurement of understanding” 66,000 hits
- “assessment of understanding” 34,000 hits
Even with the addition of “classroom” to the
search, the number of hits exceeded 9,000 for each search. The listings
covered the spectrum—from suggestions to elementary school teachers on how
to detect “bugs” in children’s understanding of addition and subtraction, to
discussions of laboratory studies of brain activity during problem solving,
to abstruse philosophical discussions in hermeneutics and epistemology.
Clearly, this approach was taking me everywhere, which is to say, nowhere.
Fully aware that I am ignoring much that has been
learned, I decided instead to draw upon personal experience—some 30 years in
the classroom—to come up with a list of criteria that classroom teachers
might use to assess understanding. The list is undoubtedly incomplete, but
it is my hope that it will encourage teachers to not only think more
carefully about how understanding might be assessed, but also—and perhaps
more importantly—encourage them to think more creatively about the kinds of
activities they assign their classes. These activities should stimulate
students to study for understanding, rather than for mere regurgitation at
test time.
The student who understands a principle, rule,
procedure or concept should be able to do the following tasks (these are
presented in no particular order and their actual difficulties are an
empirical question):
Construct problems that illustrate the
concept, principle, rule or procedure in question.
As the two anecdotes above illustrate, students may know how to use a
procedure or solve specific textbook problems in a domain, but may still not
fully understand the principle involved. A more stringent test of
understanding would be that they can construct problems themselves that
illustrate the principle. In addition to revealing much to instructors about
the nature of students’ understanding, problem construction by students can
be a powerful learning experience in its own right, for it requires the
student to think carefully about such things as problem constraints and data
sufficiency.
Identify and, if possible, correct a
flawed application of a principle or procedure.
This is basically a check on conceptual and procedural knowledge. If a
student truly understands a concept, principle or procedure, she should be
able to recognize when it is faithfully and properly applied and when it is
not. In the latter case, she should be able to explain and correct the
misapplication.
Distinguish between instances and
non-instances of a principle; or stated somewhat differently, recognize and
explain “problem isomorphs,” that is, problems that differ in their context
or surface features, but are illustrations of the same underlying principle.
In a famous and highly cited study by Michelene Chi and her colleagues at
the Learning Research and Development Center, novice physics students and
professors of physics were each presented with problems typically found in
college physics texts and asked to sort or categorized them into groups that
“go together” in some sense. They were then asked to explain the basis for
their categorization. The basic finding (since replicated in many different
disciplines) was that the novice physics students tended to sort problems on
the basis of their surface features (e.g., pulley problems, work problems),
whereas the experts tended to sort problems on the basis of their “deep
structure,” the underlying physical laws that they illustrated (e.g.,
Newton’s third law of motion, the second law of thermodynamics). This
profoundly revealing finding is usually discussed in the context of
expert-novice comparisons and in studies of how proficiency develops, but it
is also a powerful illustration of deep understanding.
Explain a principle or concept to a
naïve audience.
One of the most difficult questions on an examination I took in graduate
school was the following: “How would you explain factor analysis to your
mother?” That I remember this question over 30 years later is strong
testimony to the effect it had on me. I struggled mightily with it. But the
question forced me to think about the underlying meaning of factor analysis
in ways that had not occurred to me before.
Mathematics educator and researcher, Liping Ma, in
her classic exposition Knowing and Teaching Elementary Mathematics
(Lawrence Erlbaum, 1999), describes the difficulty some fifth and sixth
grade teachers in the United States encounter in explaining fundamental
mathematical concepts to their charges. Many of the teachers in her sample,
for example, confused division by 1/2 with division by two. The teachers
could see on a verbal level that the two were different but they could
neither explain the difference nor the numerical implications of that
difference. It follows that they could not devise simple story problems and
other exercises for fifth and sixth graders that would demonstrate the
difference.
To be sure, students may well understand a
principle, procedure or concept without being able to do all of the above.
But a student who can do none of the above almost certainly does not
understand, and students who can perform all of the above tasks flawlessly
almost certainly do understand.
Continued in article
Jensen Comment
This is a huge problem in accounting education, because so many of us teach "how
to" procedures, often very complex procedures, without really knowing whether
our students truly understand the implications of what they are doing for
decision makers who use accounting information, for fraud detection, for fraud
prevention, etc. For example, when teaching rules for asset capitalization
versus expensing, it might help students better understand if they
simultaneously learned about how and why Worldcom understated earnings by over a
billion dollars by capitalizing expenditures that should have been expensed ---
http://www.trinity.edu/rjensen/FraudEnron.htm#WorldCom
Also see
http://www.trinity.edu/rjensen/265wp.htm
Education is an admirable thing, but it is well to
remember from time to time that nothing that is worth learning can be taught.
Oscar Wilde
"The Objective of Education is Learning, Not Teaching (audio version
available)," University of Pennsylvania's Knowledge@Wharton, August 20, 2008
---
http://knowledge.wharton.upenn.edu/article.cfm;jsessionid=9a30b5674a8d333e4d18?articleid=2032
In their book, Turning Learning Right Side
Up: Putting Education Back on Track, authors Russell L. Ackoff and
Daniel Greenberg point out that today's education system is seriously flawed
-- it focuses on teaching rather than learning. "Why should children -- or
adults -- be asked to do something computers and related equipment can do
much better than they can?" the authors ask in the following excerpt from
the book. "Why doesn't education focus on what humans can do better than the
machines and instruments they create?"
"Education is an admirable thing, but it is well to
remember from time to time that nothing that is worth learning can be
taught."
-- Oscar Wilde
Traditional education focuses on teaching, not
learning. It incorrectly assumes that for every ounce of teaching there is
an ounce of learning by those who are taught. However, most of what we learn
before, during, and after attending schools is learned without its being
taught to us. A child learns such fundamental things as how to walk, talk,
eat, dress, and so on without being taught these things. Adults learn most
of what they use at work or at leisure while at work or leisure. Most of
what is taught in classroom settings is forgotten, and much or what is
remembered is irrelevant.
In most schools, memorization is mistaken for
learning. Most of what is remembered is remembered only for a short time,
but then is quickly forgotten. (How many remember how to take a square root
or ever have a need to?) Furthermore, even young children are aware of the
fact that most of what is expected of them in school can better be done by
computers, recording machines, cameras, and so on. They are treated as poor
surrogates for such machines and instruments. Why should children -- or
adults, for that matter -- be asked to do something computers and related
equipment can do much better than they can? Why doesn't education focus on
what humans can do better than the machines and instruments they create?
When those who have taught others are asked who in
the classes learned most, virtually all of them say, "The teacher." It is
apparent to those who have taught that teaching is a better way to learn
than being taught. Teaching enables the teacher to discover what one thinks
about the subject being taught. Schools are upside down: Students should be
teaching and faculty learning.
After lecturing to undergraduates at a major
university, I was accosted by a student who had attended the lecture. After
some complimentary remarks, he asked, "How long ago did you teach your first
class?"
I responded, "In September of 1941."
"Wow!" The student said. "You mean to say you have
been teaching for more than 60 years?"
"Yes."
"When did you last teach a course in a subject that
existed when you were a student?"
This difficult question required some thought.
After a pause, I said, "September of 1951."
"Wow! You mean to say that everything you have
taught in more than 50 years was not taught to you; you had to
learn on your own?"
"Right."
"You must be a pretty good learner."
I modestly agreed.
The student then said, "What a shame you're not
that good a teacher."
The student had it right; what most faculty members
are good at, if anything, is learning rather than teaching. Recall that in
the one-room schoolhouse, students taught students. The teacher served as a
guide and a resource but not as one who force-fed content into students'
minds.
Ways of Learning
There are many different ways of learning; teaching
is only one of them. We learn a great deal on our own, in independent study
or play. We learn a great deal interacting with others informally -- sharing
what we are learning with others and vice versa. We learn a great deal by
doing, through trial and error. Long before there were schools as we know
them, there was apprenticeship -- learning how to do something by trying it
under the guidance of one who knows how. For example, one can learn more
architecture by having to design and build one's own house than by taking
any number of courses on the subject. When physicians are asked whether they
leaned more in classes or during their internship, without exception they
answer, "Internship."
In the educational process, students should be
offered a wide variety of ways to learn, among which they could choose or
with which they could experiment. They do not have to learn different things
the same way. They should learn at a very early stage of "schooling" that
learning how to learn is largely their responsibility -- with the help they
seek but that is not imposed on them.
The objective of education is learning, not
teaching.
There are two ways that teaching is a powerful tool
of learning. Let's abandon for the moment the loaded word teaching, which is
unfortunately all too closely linked to the notion of "talking at" or
"lecturing," and use instead the rather awkward phrase explaining something
to someone else who wants to find out about it. One aspect of explaining
something is getting yourself up to snuff on whatever it is that you are
trying to explain. I can't very well explain to you how Newton accounted for
planetary motion if I haven't boned up on my Newtonian mechanics first. This
is a problem we all face all the time, when we are expected to explain
something. (Wife asks, "How do we get to Valley Forge from home?" And
husband, who does not want to admit he has no idea at all, excuses himself
to go to the bathroom; he quickly Googles Mapquest to find out.) This is one
sense in which the one who explains learns the most, because the person to
whom the explanation is made can afford to forget the explanation promptly
in most cases; but the explainers will find it sticking in their minds a lot
longer, because they struggled to gain an understanding in the first place
in a form clear enough to explain.
The second aspect of explaining something that
leaves the explainer more enriched, and with a much deeper understanding of
the subject, is this: To satisfy the person being addressed, to the point
where that person can nod his head and say, "Ah, yes, now I understand!"
explainers must not only get the matter to fit comfortably into their own
worldview, into their own personal frame of reference for understanding the
world around them, they also have to figure out how to link their frame of
reference to the worldview of the person receiving the explanation, so that
the explanation can make sense to that person, too. This involves an intense
effort on the part of the explainer to get into the other person's mind, so
to speak, and that exercise is at the heart of learning in general. For, by
practicing repeatedly how to create links between my mind and another's, I
am reaching the very core of the art of learning from the ambient culture.
Without that skill, I can only learn from direct experience; with that
skill, I can learn from the experience of the whole world. Thus, whenever I
struggle to explain something to someone else, and succeed in doing so, I am
advancing my ability to learn from others, too.
Learning through Explanation
This aspect of learning through explanation has
been overlooked by most commentators. And that is a shame, because both
aspects of learning are what makes the age mixing that takes place in the
world at large such a valuable educational tool. Younger kids are always
seeking answers from older kids -- sometimes just slightly older kids (the
seven-year old tapping the presumed life wisdom of the
so-much-more-experienced nine year old), often much older kids. The older
kids love it, and their abilities are exercised mightily in these
interactions. They have to figure out what it is that they understand about
the question being raised, and they have to figure out how to make their
understanding comprehensible to the younger kids. The same process occurs
over and over again in the world at large; this is why it is so important to
keep communities multi-aged, and why it is so destructive to learning, and
to the development of culture in general, to segregate certain ages
(children, old people) from others.
Continued in article
Bob Jensen's threads on assessment, learning, and technology in education
are at
http://www.trinity.edu/rjensen/000aaa/0000start.htm
In particular note the document on assessment ---
http://www.trinity.edu/rjensen/assess.htm
June 18, 2006
message from Bob Kennelly
[bob_kennelly@YAHOO.COM]
I am a data analyst with the Federal Government,
recently assigned a project to integrate our accounting codes with XBRL
accounting codes, primarily for the quarterly reporting of banking
financial information.
For the past few weeks, i've been searching the
WEB looking for educational materials that will help us map, rollup and
orr olldown the data that we recieve from the banks that we regulate, to
the more generic XBRL accounting codes.
Basically, i'm hoping to provide my team members
with the tools to help them make more informed decisions on how to
classify accounting codes and capture their findings for further review
and discussion.
To my suprise there isn't the wealth of accounting
information that i thought there would be on the WEB, but i am very
relieved to have found Bob Jensen's site and in particular an article
which refers to the kind of information gathering
approaches that i'm hoping to discover!
Here is the brief on that article:
"Using Hypertext in Instructional Material: Helping Students Link
Accounting Concept Knowledge to Case Applications," by Dickie Crandall
and Fred Phillips, Issues in Accounting Education, May 2002, pp. 163-184
---
We studied whether instructional material that
connects accounting concept discussions with sample case applications
through hypertext links would enable students to better understand how
concepts are to be applied to practical case situations.
Results from a laboratory experiment indicated
that students who learned from such hypertext-enriched instructional
material were better able to apply concepts to new accounting cases than
those who learned from instructional material that contained identical
content but lacked the concept-case application hyperlinks.
Results also indicated that the learning benefits
of concept-case application hyperlinks in instructional material were
greater when the hyperlinks were self-generated by the students rather
than inherited from instructors, but only when students had generated
appropriate links.
Could anyone be so kind as to please suggest other
references, articles or tools that will help us better understand and
classify the broad range of accounting terminologies and methodologies
please?
Thanks very much!
Bob Kennelly
OFHEO
June 19, 2006 reply from Bob Jensen
Hi Bob,
You may find the following documents of related interest:
"Internet Financial Reporting: The Effects of Hyperlinks and Irrelevant
Information on Investor Judgments," by Andrea S. Kelton (Ph.D. Dissertation
at the University of Tennessee) ---
http://www.mgt.ncsu.edu/pdfs/accounting/kelton_dissertation_1-19-06.pdf
Extendible Adaptive Hypermedia Courseware: Integrating Different Courses
and Web Material
Lecture Notes in Computer Science, Publisher: Springer Berlin /
Heidelberg ISSN: 0302-9743 Subject: Computer Science Volume 1892 / 2000
Title: Adaptive Hypermedia and Adaptive Web-Based Systems: International
Conference, AH 2000, Trento, Italy, August 2000. Proceedings Editors: P.
Brusilovsky, O. Stock, C. Strapparava (Eds.) ---
Click Here
"Concept, Knowledge, and Thought," G. C. Oden, Annual Review of
Psychology Vol. 38: 203-227 (Volume publication date January 1987) ---
Click Here
"A Framework for Organization and Representation of Concept Knowledge in
Autonomous Agents," by Paul Davidsson, Department of Computer Science,
University of Lund, Box 118, S–221 00 Lund, Sweden email:
Paul.Davidsson@dna.lth.se
"Active concept learning for image retrieval in dynamic databases," by
Dong, A. Bhanu, B. Center for Res. in Intelligent Syst., California Univ.,
Riverside, CA, USA; This paper appears in: Computer Vision, 2003.
Proceedings. Ninth IEEE International Conference on Publication Date: 13-16
Oct. 2003 On page(s): 90- 95 vol.1 ISSN: ISBN: 0-7695-1950-4 ---
Click Here
"Types and qualities of knowledge," by Ton de Jong, Monica G.M.
Ferguson-Hessler, Educational Psychologist 1996, Vol. 31, No. 2,
Pages 105-113 ---
Click Here
Also note
http://www.trinity.edu/rjensen/assess.htm#DownfallOfLecturing
Hope this helps
Bob Jensen
Assessing-to-Learn Physics: Project Website ---
http://a2l.physics.umass.edu/
Bob Jensen's threads on science and medicine tutorials are at
http://www.trinity.edu/rjensen/Bookbob2.htm#Science
Onsite Versus Online Differences for Faculty
Soaring Popularity of E-Learning Among Students But Not Faculty
How many U.S. students took at least on online course from a legitimate college
in Fall 2005?
More students are taking online college courses than
ever before, yet the majority of faculty still aren’t warming up to the concept
of e-learning, according to a national survey from the country’s largest
association of organizations and institutions focused on online education . . .
‘We didn’t become faculty to sit in front of a computer screen,’
Elia Powers, "Growing Popularity of E-Learning, Inside Higher Ed,
November 10, 2006 ---
http://www.insidehighered.com/news/2006/11/10/online
More students are taking online college courses
than ever before, yet the majority of faculty still aren’t warming up to the
concept of e-learning, according to a national survey from the country’s
largest association of organizations and institutions focused on online
education.
Roughly 3.2 million students took at least one
online course from a degree-granting institution during the fall 2005 term,
the Sloan Consortium said. That’s double the number who reported doing so in
2002, the first year the group collected data, and more than 800,000 above
the 2004 total. While the number of online course participants has increased
each year, the rate of growth slowed from 2003 to 2004.
The report, a joint partnership between the group
and the College Board, defines online courses as those in which 80 percent
of the content is delivered via the Internet.
The Sloan Survey of Online Learning,
“Making the Grade: Online Education in the United States, 2006,”
shows that 62 percent of chief academic officers say
that the learning outcomes in online education are now “as good as or
superior to face-to-face instruction,” and nearly 6 in 10 agree that
e-learning is “critical to the long-term strategy of their institution.”
Both numbers are up from a year ago.
Researchers at the Sloan Consortium, which is
administered through Babson College and Franklin W. Olin College of
Engineering, received responses from officials at more than 2,200 colleges
and universities across the country. (The report makes few references to
for-profit colleges, a force in the online market, in part because of a lack
of survey responses from those institutions.)
Much of the report is hardly surprising. The bulk
of online students are adult or “nontraditional” learners, and more than 70
percent of those surveyed said online education reaches students not served
by face-to-face programs.
What stands out is the number of faculty who still
don’t see e-learning as a valuable tool. Only about one in four academic
leaders said that their faculty members “accept the value and legitimacy of
online education,” the survey shows. That number has remained steady
throughout the four surveys. Private nonprofit colleges were the least
accepting — about one in five faculty members reported seeing value in the
programs.
Elaine Allen, co-author of the report and a Babson
associate professor of statistics and entrepreneurship, said those numbers
are striking.
“As a faculty member, I read that response as, ‘We
didn’t become faculty to sit in front of a computer screen,’ ” Allen said.
“It’s a very hard adjustment. We sat in lectures for an hour when we were
students, but there’s a paradigm shift in how people learn.”
Barbara Macaulay, chief academic officer at UMass
Online, which offers programs through the University of Massachusetts, said
nearly all faculty members teaching the online classes there also teach
face-to-face courses, enabling them to see where an online class could fill
in the gap (for instance, serving a student who is hesitant to speak up in
class).
She said she isn’t surprised to see data
illustrating the growing popularity of online courses with students, because
her program has seen rapid growth in the last year. Roughly 24,000 students
are enrolled in online degree and certificate courses through the university
this fall — a 23 percent increase from a year ago, she said.
“Undergraduates see it as a way to complete their
degrees — it gives them more flexibility,” Macaulay said.
The Sloan report shows that about 80 percent of
students taking online courses are at the undergraduate level. About half
are taking online courses through community colleges and 13 percent through
doctoral and research universities, according to the survey.
Nearly all institutions with total enrollments
exceeding 15,000 students have some online offerings, and about two-thirds
of them have fully online programs, compared with about one in six at the
smallest institutions (those with 1,500 students or fewer), the report
notes. Allen said private nonprofit colleges are often set in enrollment
totals and not looking to expand into the online market.
The report indicates that two-year colleges are particularly willing to be
involved in online learning.
“Our institutions tend to embrace changes a little
more readily and try different pedagogical styles,” said Kent Phillippe, a
senior research associate at the American Association of Community Colleges.
The report cites a few barriers to what it calls the “widespread adoption of
online learning,” chief among them the concern among college officials that
some of their students lack the discipline to succeed in an online setting.
Nearly two-thirds of survey respondents defined that as a barrier.
Allen, the report’s co-author, said she thinks that
issue arises mostly in classes in which work can be turned in at any time
and lectures can be accessed at all hours. “If you are holding class in real
time, there tends to be less attrition,” she said. The report doesn’t
differentiate between the live and non-live online courses, but Allen said
she plans to include that in next year’s edition.
Few survey respondents said acceptance of online
degrees by potential employers was a critical barrier — although liberal
arts college officials were more apt to see it as an issue.
November 10, 2006 reply from John Brozovsky
[jbrozovs@vt.edu]
Hi Bob:
One reason why might be what I have seen. The
in residence accounting students that I talk with take online classes
here because they are EASY and do not take much work. This would be very
popular with students but not generally so with faculty.
John
November 10, 2006 reply from Bob Jensen
Hi John,
Then there is a quality control problem whereever this is a fact. It
would be a travesty if any respected college had two or more categories of
academic standards or faculty assignments.
Variations in academic standards have long been a problem between
part-time versus full-time faculty, although grade inflation can be higher
or lower among part-time faculty. In one instance, it’s the tenure-track
faculty who give higher grades because they're often more worried about
student evaluations. At the opposite extreme it is part-time faculty who
give higher grades for many reasons that we can think of if we think about
it.
One thing that I'm dead certain about is that highly motivated students
tend to do better in online courses ceteris paribus. Reasons are mainly that
time is used more efficiently in getting to class (no wasted time driving or
walking to class), less wasted time getting teammates together on team
projects, and fewer reasons for missing class.
Also online alternatives offer some key advantages for certain types of
handicapped students ---
http://www.trinity.edu/rjensen/000aaa/thetools.htm
My opinions on learning advantages of E-Learning were heavily influenced
by the most extensive and respected study of online versus onsite learning
experiments in the SCALE experiments
using full-time resident students at the University of Illinois ---
http://www.trinity.edu/rjensen/255wp.htm#Illinois
In the SCALE experiments cutting across 30 disciplines, it was generally
found that motivated students learned better online then their onsite
counterparts having the same instructors. However, there was no significant
impact on students who got low grades in online versus onsite treatment
groups.
I think the main problem with faculty is that online teaching tends to
burn out instructors more frequently than onsite instructors. This was also
evident in the SCALE experiments. When done correctly, online courses are
more communication intent between instructors and faculty. Also, online
learning takes more preparation time if it is done correctly.
My hero for online learning is still Amy Dunbar who
maintains high standards for everything:
http://www.cs.trinity.edu/~rjensen/002cpe/02start.htm
http://www.trinity.edu/rjensen/book01q4.htm#Dunbar
Bob Jensen
November 10, 2006 reply from John Brozovsky
[jbrozovs@vt.edu]
Hi Bob:
Also why many times it is not done 'right'. Not
done right they do not get the same education. Students generally do not
complain about getting 'less for their money'. Since we do not do online
classes in department the ones the students are taking are the university
required general education and our students in particular are not unhappy
with being shortchanged in that area as they frequently would have preferred
none anyway.
John
Bob Jensen's threads on open sharing and education technology are at
http://www.trinity.edu/rjensen/000aaa/0000start.htm
Bob Jensen's threads on online training and education alternatives are at
http://www.trinity.edu/rjensen/crossborder.htm
Motivations for Distance Learning ---
http://www.trinity.edu/rjensen/000aaa/updateee.htm#Motivations
Bob Jensen's threads on the dark side of online learning and teaching are
at
http://www.trinity.edu/rjensen/000aaa/theworry.htm
Question
Why should teaching a course online take twice as much time as teaching it
onsite?
Answer
Introduction to Economics: Experiences of teaching this course online
versus onsite
With a growing number of courses offered online and
degrees offered through the Internet, there is a considerable interest in online
education, particularly as it relates to the quality of online instruction. The
major concerns are centering on the following questions: What will be the new
role for instructors in online education? How will students' learning outcomes
be assured and improved in online learning environment? How will effective
communication and interaction be established with students in the absence of
face-to-face instruction? How will instructors motivate students to learn in the
online learning environment? This paper will examine new challenges and barriers
for online instructors, highlight major themes prevalent in the literature
related to “quality control or assurance” in online education, and provide
practical strategies for instructors to design and deliver effective online
instruction. Recommendations will be made on how to prepare instructors for
quality online instruction.
Yi Yang and Linda F. Cornelious, "Preparing Instructors for Quality
Online Instruction, Working Paper ---
http://www.westga.edu/%7Edistance/ojdla/spring81/yang81.htm
Jensen Comment: The bottom line is that teaching the course online took
twice as much time because "largely from increased student contact and
individualized instruction and not from the use of technology per se."
Online teaching is more likely to result in instructor burnout. These
and other issues are discussed in my "dark side" paper at
http://www.trinity.edu/rjensen/000aaa/theworry.htm
April 1, 2005 message from Carolyn Kotlas
[kotlas@email.unc.edu]
COMPUTERS IN THE CLASSROOM AND OPEN BOOK EXAMS
In "PCs in the Classroom & Open Book Exams" (UBIQUITY, vol. 6, issue 9,
March 15-22, 2005), Evan Golub asks and supplies some answers to questions
regarding open-book/open-note exams. When classroom computer use is allowed
and encouraged, how can instructors secure the open-book exam environment?
How can cheating be minimized when students are allowed Internet access
during open-book exams? Golub's suggested solutions are available online at
http://www.acm.org/ubiquity/views/v6i9_golub.html
Ubiquity is a free, Web-based publication of the
Association for Computing Machinery (ACM), "dedicated to fostering critical
analysis and in-depth commentary on issues relating to the nature,
constitution, structure, science, engineering, technology, practices, and
paradigms of the IT profession." For more information, contact: Ubiquity,
email: ubiquity@acm.org ; Web:
http://www.acm.org/ubiquity/
For more information on the ACM, contact: ACM, One Astor Plaza, 1515
Broadway, New York, NY 10036, USA; tel: 800-342-6626 or 212-626-0500; Web:
http://www.acm.org/
NEW EDUCAUSE E-BOOK ON THE NET GENERATION
EDUCATING THE NET GENERATION, a new EDUCAUSE
e-book of essays edited by Diana G. Oblinger and James L. Oblinger,
"explores the Net Gen and the implications for institutions in areas such as
teaching, service, learning space design, faculty development, and
curriculum." Essays include: "Technology and Learning Expectations of the
Net Generation;" "Using Technology as a Learning Tool, Not Just the Cool New
Thing;" "Curricula Designed to Meet 21st-Century Expectations;" "Faculty
Development for the Net Generation;" and "Net Generation Students and
Libraries." The entire book is available online at no cost at
http://www.educause.edu/educatingthenetgen/
.
EDUCAUSE is a nonprofit association whose mission
is to advance higher education by promoting the intelligent use of
information technology. For more information, contact: Educause, 4772 Walnut
Street, Suite 206, Boulder, CO 80301-2538 USA; tel: 303-449-4430; fax:
303-440-0461; email:
info@educause.edu; Web:
http://www.educause.edu/
See also:
GROWING UP DIGITAL: THE RISE OF THE NET GENERATION
by Don Tapscott McGraw-Hill, 1999; ISBN: 0-07-063361-4
http://www.growingupdigital.com/
EFFECTIVE E-LEARNING DESIGN
"The unpredictability of the student context and
the mediated relationship with the student require careful attention by the
educational designer to details which might otherwise be managed by the
teacher at the time of instruction." In "Elements of Effective e-Learning
Design" (INTERNATIONAL REVIEW OF RESEARCH IN OPEN AND DISTANCE LEARNING,
March 2005) Andrew R. Brown and Bradley D. Voltz cover six elements of
effective design that can help create effective e-learning delivery. Drawing
upon examples from The Le@rning Federation, an initiative of state and
federal governments of Australia and New Zealand, they discuss lesson
planning, instructional design, creative writing, and software
specification. The paper is available online at
http://www.irrodl.org/content/v6.1/brown_voltz.html
International Review of Research in Open and
Distance Learning (IRRODL) [ISSN 1492-3831] is a free, refereed ejournal
published by Athabasca University - Canada's Open University. For more
information, contact Paula Smith, IRRODL Managing Editor; tel: 780-675-6810;
fax: 780-675-672; email:
irrodl@athabascau.ca
; Web:
http://www.irrodl.org/
The Le@rning Federation (TLF) is an "initiative
designed to create online curriculum materials and the necessary
infrastructure to ensure that teachers and students in Australia and New
Zealand can use these materials to widen and enhance their learning
experiences in the classroom." For more information, see
http://www.thelearningfederation.edu.au/
RECOMMENDED READING
"Recommended Reading" lists items that have been
recommended to me or that Infobits readers have found particularly
interesting and/or useful, including books, articles, and websites published
by Infobits subscribers. Send your recommendations to
carolyn_kotlas@unc.ed u for possible
inclusion in this column.
Author Clark Aldrich recommends his new book:
LEARNING BY DOING: A COMPREHENSIVE GUIDE TO
SIMULATIONS, COMPUTER GAMES, AND PEDAGOGY IN E-LEARNING AND OTHER
EDUCATIONAL EXPERIENCES Wiley, April 2005 ISBN: 0-7879-7735-7 hardcover
$60.00 (US)
Description from Wiley website:
"Designed for learning professionals and drawing on
both game creators and instructional designers, Learning by Doing explains
how to select, research, build, sell, deploy, and measure the right type of
educational simulation for the right situation. It covers simple approaches
that use basic or no technology through projects on the scale of computer
games and flight simulators. The book role models content as well, written
accessibly with humor, precision, interactivity, and lots of pictures. Many
will also find it a useful tool to improve communication between themselves
and their customers, employees, sponsors, and colleagues."
The table of contents and some excerpts are
available at
http://www.wiley.com/WileyCDA/WileyTitle/productCd-0787977357.html
Aldrich is also author of SIMULATIONS AND THE FUTURE OF LEARNING: AN
INNOVATIVE (AND PERHAPS REVOLUTIONARY) APPROACH TO E-LEARNING. See
http://www.wiley.com/WileyCDA/WileyTitle/productCd-0787969621.html
for more information or to request an evaluation copy of this title.
Also see
Looking at Learning….Again, Part 2 ---
http://www.learner.org/resources/series114.html
Bob Jensen's documents on education technology are at
http://www.trinity.edu/rjensen/000aaa/0000start.htm
More on this topic appears in the module below.
"Nationally Recognized Assessment and Higher Education Study Center
Findings as Resources for Assessment Projects," by Tracey Sutherland,
Accounting Education News, 2007 Winter Issue, pp. 5-7
While nearly all accounting programs are wrestling
with various kinds of assessment initiatives to meet local assessment plans
and/or accreditation needs, most colleges and universities participate in
larger assessment projects whose results may not be shared at the
College/School level. There may be information available on your campus
through campus-level assessment and institutional research that generate
data that could be useful for your accounting program/school assessment
initiatives. Below are examples of three such research projects, and some of
their recent findings about college students.
- The Cooperative Institutional Research Program
(CIRP) The American Freshman: National Norms for 2006
- The 2006 Report of the National Survey of
Student Engagement
- From the National Freshman Attitudes Report
2007
Some things in the The 2006 Report of the National Survey of Student
Engagement especially caught my eye:
Promising Findings from the National Surveyof Student
Engagement
• Student engagement is positively
related to first-year and senior student grades and to persistence
between the first and second year of college.
• Student engagement has
compensatory effects on grades andpersistence of students from
historically underserved backgrounds.
• Compared with campus-basedstudents,
distance education learners reported higher levels ofacademic challenge,
engaged more often in deep learning activities, and reported greater
developmental gains from college.
• Part-time working students
reported grades comparable to other students and also perceived the
campus to be as supportive of their academic and social needs as
theirnon-working peers.
• Four out of five beginning
college students expected that reflective learning activities would be
an important part of their first-year experience.
Disappointing Findings from the
National
Survey of Student Engagement
• Students spend on average only about
13–14 hours a week preparingfor class, far below what faculty members say is
necessary to do well in their classes.
• Students study less during the first
year of college than they expected to at the start of the academic year.
• Women are less likely than men to
interact with faculty members outside of class including doing research with
a faculty member.
• Distance education students are less
involved in active and collaborative learning.
• Adult learners were much lesslikely
to have participated in such enriching educational activities as community
service, foreign language study, a culminating senior experience, research
with faculty,and co-curricular activities.
• Compared with other students,
part-time students who are working had less contact with facultyand
participated less in active and collaborative learning activities and
enriching educational experiences.
Some additional 2006 NSSE findings
• Distance education studentsreported higher levels of
academic challenge, and reported engaging more often in deep learning
activities such as the reflective learning activities. They also reported
participating less in collaborative learning experiences and worked more
hours off campus.
• Women students are more likely to be engaged in foreign
language coursework.
• Male students spent more time engaged in working with
classmates on projects outside of class.
• Almost half (46%) of adult students were working more than
30 hours per week and about three-fourths were caring for dependents. In
contrast, only 3% of traditional age students worked more than 30 hours per
week, and about four fifths spend no time caring for dependents.
Bob Jensen's threads on higher education controversies are at
http://www.trinity.edu/rjensen/HigherEdControversies.htm
Students Reviewing Each Others' Projects
January 30, 2009 message from David Fordham, James Madison University
[fordhadr@JMU.EDU]
I teach an MBA section of "Introduction to
Information Security". One of the course requirements is an Information
Security Policy Manual for a hypothetical company. Students submit their
manuals electronically, with the only identifying information being their
name as the title of the file. I strip off all other identifying information
(Tools-Options, File-Properties, etc.) from the document and change the name
of the file to "Student 1" "Student 2" etc.
Then, I distribute the file to two other students
for blind review.
In reality, each author receives THREE (3) reviews,
because I myself provide a review, in addition to the two students. I do NOT
identify the reviewers, either, so the author gets three reviews, but does
not know which one is mine and which are the other two student reviews. Two
are blind, and one is mine, but all the student gets is "review 1", "review
2", and "review 3". I am NOT always "review 3".
This has proven to be very effective. Each student
gets to actually SEE two other students' work up close and personal and has
to put thought into evaluating it, and in so doing, can compare their peers'
work to their own. Plus, each student then gets three reviews from three
other individuals, making a total of FIVE (5) different perspectives which
to compare with their own.
This "reviewed" submission is the "mid-term"
submission. The students then have the option (all of them take it!) to
revise their manual if they wish for the final submission. The quality of
the final product is day-and-night difference from what I used to get: truly
professional level work. Hence, I'm a believer in the system.
(Plus, I can rage all I want in my review of the
first submission if its really bad, and the student doesn't know it's me!)
Incidentally, part of the course grade is how well
they review their two assigned manuals... I expect good comments,
constructive criticism, useful suggestions, etc. Because the students are
all in the executive MBA program, and because this approach is novel, I
usually get some really good participation and high-quality reviews.
No, it doesn't save me a lot of time, since I still
personally "grade" (e.g., do a review of) each submission. But I'm doing it
to save time, I'm doing it because it gives high value to the student. I
can, however, easily see where peer review would be a fantastic time-saver
when a professor gives lengthy assignments to large numbers of students.
David Fordham
JMU
Online Versus Onsite for Students
"Students prefer online courses: Classes popular with on-campus
students," CNN, January 13, 2006 ---
http://www.cnn.com/2006/EDUCATION/01/13/oncampus.online.ap/index.html
At least 2.3 million people took some kind of
online course in 2004, according to a recent survey by The Sloan Consortium,
an online education group, and two-thirds of colleges offering
"face-to-face" courses also offer online ones. But what were once two
distinct types of classes are looking more and more alike -- and often
dipping into the same pool of students.
At some schools, online courses -- originally
intended for nontraditional students living far from campus -- have proved
surprisingly popular with on-campus students. A recent study by South
Dakota's Board of Regents found 42 percent of the students enrolled in its
distance-education courses weren't so distant: they were located on campus
at the university that was hosting the online course.
Numbers vary depending on the policies of
particular colleges, but other schools also have students mixing and
matching online and "face-to-face" credits. Motives range from lifestyle to
accommodating a job schedule to getting into high-demand courses.
Classes pose challenges Washington State University
had about 325 on-campus undergraduates taking one or more distance courses
last year. As many as 9,000 students took both distance and in-person
classes at Arizona State Univesity last year.
"Business is really about providing options to
their customers, and that's really what we want to do," said Sheila Aaker,
extended services coordinator at Black Hills State.
Still, the trend poses something of a dilemma for
universities.
They are reluctant to fill slots intended for
distance students with on-campus ones who are just too lazy to get up for
class. On the other hand, if they insist the online courses are just as
good, it's hard to tell students they can't take them. And with the student
population rising and pressing many colleges for space, they may have little
choice.
In practice, the policy is often shaded. Florida
State University tightened on-campus access to online courses several years
ago when it discovered some on-campus students hacking into the system to
register for them. Now it requires students to get an adviser's permission
to take an online class.
Online, in-person classes blending Many schools,
like Washington State and Arizona State, let individual departments and
academic units decide who can take an online course. They say students with
legitimate academic needs -- a conflict with another class, a course they
need to graduate that is full -- often get permission, though they still
must take some key classes in person.
In fact, the distinction between online and
face-to-face courses is blurring rapidly. Many if not most traditional
classes now use online components -- message boards, chat rooms, electronic
filing of papers. Students can increasingly "attend" lectures by downloading
a video or a podcast.
At Arizona State, 11,000 students take fully online
courses and 40,000 use the online course management system, which is used by
many "traditional" classes. Administrators say the distinction between
online and traditional is now so meaningless it may not even be reflected in
next fall's course catalogue.
Arizone State's director of distance learning, Marc
Van Horne, says students are increasingly demanding both high-tech delivery
of education, and more control over their schedules. The university should
do what it can to help them graduate on time, he says.
"Is that a worthwhile goal for us to pursue? I'd
say 'absolutely,"' Van Horne said. "Is it strictly speaking the mission of a
distance learning unit? Not really."
Then there's the question of whether students are
well served by taking a course online instead of in-person. Some teachers
are wary, saying showing up to class teaches discipline, and that lectures
and class discussions are an important part of learning.
But online classes aren't necessarily easier.
Two-thirds of schools responding to a recent survey by The Sloan Consortium
agreed that it takes more discipline for students to succeed in an online
course than in a face-to-face one.
"It's a little harder to get motivated," said
Washington State senior Joel Gragg, who took two classes online last year
(including "the psychology of motivation"). But, he said, lectures can be
overrated -- he was still able to meet with the professor in person when he
had questions -- and class discussions are actually better online than in a
college classroom, with a diverse group exchanging thoughtful postings.
"There's young people, there's old people, there's
moms, professional people," he said. "You really learn a lot more."
Bob Jensen's threads on distance education and training alternatives are
at
http://www.trinity.edu/rjensen/crossborder.htm
The 2006 National Survey of Student Engagement, released November 13,
2006, for the first time offers a close look at distance education, offering
provocative new data suggesting that e-learners report higher levels of
engagement, satisfaction and academic challenge than their on-campus peers ---
http://nsse.iub.edu/NSSE_2006_Annual_Report/index.cfm
"The Engaged E-Learner," by Elizabeth Redden, Inside Higher Ed,
November 13, 2006 ---
http://www.insidehighered.com/news/2006/11/13/nsse
The 2006
National Survey of Student Engagement, released
today, for the first time offers a close look at distance education,
offering provocative new data suggesting that e-learners report higher
levels of engagement, satisfaction and academic challenge than their
on-campus peers.
Beyond the numbers, however, what institutions
choose to do with the data promises to attract extra attention to this
year’s report.
NSSE is one of the few standardized measures of
academic outcomes that most officials across a wide range of higher
education institutions agree offers something of value.Yet NSSE does not
release institution-specific data, leaving it to colleges to choose whether
to publicize their numbers.
Colleges are under mounting pressure, however, to
show in concrete, measurable ways that they are successfully educating
students, fueled in part by the recent release of the
report from the
Secretary of Education’s Commission on the Future of Higher Education,
which emphasizes the need for the development of
comparable measures of student learning. In the commission’s report and in
college-led efforts to heed the commission’s call,
NSSE has been embraced as one way to do that. In this climate, will a
greater number of colleges embrace transparency and release their results?
Anywhere between one-quarter and one-third of the
institutions participating in NSSE choose to release some data, said George
Kuh, NSSE’s director and a professor of higher education at Indiana
University at Bloomington. But that number includes not only those
institutions that release all of the data, but also those that pick and
choose the statistics they’d like to share.
In the “Looking Ahead” section that concluded the
2006 report, the authors note that NSSE can “contribute to the higher
education improvement and accountability agenda,” teaming with institutions
to experiment with appropriate ways to publicize their NSSE data and
developing common templates for colleges to use. The report cautions that
the data released for accountability purposes should be accompanied by other
indicators of student success, including persistence and graduation rates,
degree/certificate completion rates and measurements of post-college
endeavors.
“Has this become a kind of a watershed moment when
everybody’s reporting? No. But I think what will happen as a result of the
Commission on the Future of Higher Ed, Secretary (Margaret) Spelling’s
workgroup, is that there is now more interest in figuring out how to do
this,” Kuh said.
Charles Miller, chairman of the Spellings
commission, said he understands that NSSE’s pledge not to release
institutional data has encouraged colleges to participate — helping the
survey, first introduced in 1999, get off the ground and gain wide
acceptance. But Miller said he thinks that at this point, any college that
chooses to participate in NSSE should make its data public.
“Ultimately, the duty of the colleges that take
public funds is to make that kind of data public. It’s not a secret that the
people in the academy ought to have. What’s the purpose of it if it’s just
for the academy? What about the people who want to get the most for their
money?”
Participating public colleges are already obliged
to provide the data upon request, but Miller said private institutions,
which also rely heavily on public financial aid funds, should share that
obligation.
Kuh said that some colleges’ reluctance to
publicize the data stems from a number of factors, the primary reason being
that they are not satisfied with the results and feel they might reflect
poorly on the institution.
In addition, some college officials fear that the
information, if publicized, may be misused, even conflated to create a
rankings system. Furthermore, sharing the data would represent a shift in
the cultural paradigm at some institutions used to keeping sensitive data to
themselves, Kuh said.
“The great thing about NSSE and other measures like
it is that it comes so close to the core of what colleges and universities
are about — teaching and learning. This is some of the most sensitive
information that we have about colleges and universities,” Kuh said.
But Miller said the fact that the data get right to
the heart of the matter is precisely why it should be publicized. “It
measures what students get while they’re at school, right? If it does that,
what’s the fear of publishing it?” Miller asked. “If someone would say,
‘It’s too hard to interpret,’ then that’s an insult to the public.” And if
colleges are afraid of what their numbers would suggest, they shouldn’t
participate in NSSE at all, Miller said.
However, Douglas Bennett, president of Earlham
College in Indiana and chair of NSSE’s National Advisory Board, affirmed
NSSE’s commitment to opening survey participation to all institutions
without imposing any pressure that they should make their institutional
results public. “As chair of the NSSE board, we believe strongly that
institutions own their own data and what they do with it is up to them.
There are a variety of considerations institutions are going to take into
account as to whether or not they share their NSSE data,” Bennett said.
However, as president of Earlham, which releases
all of its NSSE data and even releases its accreditation reports, Bennett
said he thinks colleges, even private institutions, have a professional and
moral obligation to demonstrate their effectiveness in response to
accountability demands — through NSSE or another means a college might deem
appropriate.
This Year’s Survey
The 2006 NSSE survey, which is based on data from
260,000 randomly-selected first-year and senior students at 523 four-year
institutions(NSSE’s companion survey, the
Community College Survey of
Student Engagement, focuses on two-year colleges)
looks much more deeply than previous iterations of the survey did into the
performance of online students.
Distance learning students outperform or perform on
par with on-campus students on measures including level of academic
challenge; student-faculty interaction; enriching educational experiences;
and higher-order, integrative and reflective learning; and gains in
practical competence, personal and social development, and general
education. They demonstrate lower levels of engagement when it comes to
active and collaborative learning.
Karen Miller, a professor of education at the
University of Louisville who studies online learning, said the results
showing higher or equal levels of engagement among distance learning
students make sense: “If you imagine yourself as an undergraduate in a
fairly large class, you can sit in that class and feign engagement. You can
nod and make eye contact; your mind can be a million miles away. But when
you’re online, you’ve got to respond, you’ve got to key in your comments on
the discussion board, you’ve got to take part in the group activities.
Plus, Miller added, typing is a more complex
psycho-motor skill than speaking, requiring extra reflection. “You see what
you have said, right in front of your eyes, and if you realize it’s kind of
half-baked you can go back and correct it before you post it.”
Also, said Kuh, most of the distance learners
surveyed were over the age of 25. “Seventy percent of them are adult
learners. These folks are more focused; they’re better able to manage their
time and so forth,” said Kuh, who added that many of the concerns
surrounding distance education focus on traditional-aged students who may
not have mastered their time management skills.
Among other results from the 2006 NSSE survey:
- Those students who come to college less
well-prepared academically or from historically underrepresented groups
tend to benefit from
engagement in educationally purposeful
activities even more than their peers do.
- First-year and senior students spend an
average of about 13 to 14 hours per week preparing for classes, much
less than what faculty members say is needed.
- Student engagement is positively correlated to
grades and persistence between the first and second year of college.
- New students study fewer hours during their
first year than they expected to when starting college.
- First-year students at research universities
are more likely than students at other types of institutions to
participate in a learning community.
- First-year students at liberal arts colleges
participate in class discussions more often and view their faculty more
positively than do students at other institutions.
- Seniors at master’s level colleges and
universities give class presentations and work with their peers on
problems in class more than students at other types of institutions do.
Bob Jensen's threads on distance education and training alternatives
around the world are at
http://www.trinity.edu/rjensen/Crossborder.htm
Soaring Popularity of E-Learning Among Students But Not Faculty
How many U.S. students took at least on online course from a legitimate college
in Fall 2005?
More students are taking online college courses than
ever before, yet the majority of faculty still aren’t warming up to the concept
of e-learning, according to a national survey from the country’s largest
association of organizations and institutions focused on online education . . .
‘We didn’t become faculty to sit in front of a computer screen,’
Elia Powers, "Growing Popularity of E-Learning, Inside Higher Ed,
November 10, 2006 ---
http://www.insidehighered.com/news/2006/11/10/online
More students are taking online college courses
than ever before, yet the majority of faculty still aren’t warming up to the
concept of e-learning, according to a national survey from the country’s
largest association of organizations and institutions focused on online
education.
Roughly 3.2 million students took at least one
online course from a degree-granting institution during the fall 2005 term,
the Sloan Consortium said. That’s double the number who reported doing so in
2002, the first year the group collected data, and more than 800,000 above
the 2004 total. While the number of online course participants has increased
each year, the rate of growth slowed from 2003 to 2004.
The report, a joint partnership between the group
and the College Board, defines online courses as those in which 80 percent
of the content is delivered via the Internet.
The Sloan Survey of Online Learning,
“Making the Grade: Online Education in the United States, 2006,”
shows that 62 percent of chief academic officers say
that the learning outcomes in online education are now “as good as or
superior to face-to-face instruction,” and nearly 6 in 10 agree that
e-learning is “critical to the long-term strategy of their institution.”
Both numbers are up from a year ago.
Researchers at the Sloan Consortium, which is
administered through Babson College and Franklin W. Olin College of
Engineering, received responses from officials at more than 2,200 colleges
and universities across the country. (The report makes few references to
for-profit colleges, a force in the online market, in part because of a lack
of survey responses from those institutions.)
Much of the report is hardly surprising. The bulk
of online students are adult or “nontraditional” learners, and more than 70
percent of those surveyed said online education reaches students not served
by face-to-face programs.
What stands out is the number of faculty who still
don’t see e-learning as a valuable tool. Only about one in four academic
leaders said that their faculty members “accept the value and legitimacy of
online education,” the survey shows. That number has remained steady
throughout the four surveys. Private nonprofit colleges were the least
accepting — about one in five faculty members reported seeing value in the
programs.
Elaine Allen, co-author of the report and a Babson
associate professor of statistics and entrepreneurship, said those numbers
are striking.
“As a faculty member, I read that response as, ‘We
didn’t become faculty to sit in front of a computer screen,’ ” Allen said.
“It’s a very hard adjustment. We sat in lectures for an hour when we were
students, but there’s a paradigm shift in how people learn.”
Barbara Macaulay, chief academic officer at UMass
Online, which offers programs through the University of Massachusetts, said
nearly all faculty members teaching the online classes there also teach
face-to-face courses, enabling them to see where an online class could fill
in the gap (for instance, serving a student who is hesitant to speak up in
class).
She said she isn’t surprised to see data
illustrating the growing popularity of online courses with students, because
her program has seen rapid growth in the last year. Roughly 24,000 students
are enrolled in online degree and certificate courses through the university
this fall — a 23 percent increase from a year ago, she said.
“Undergraduates see it as a way to complete their
degrees — it gives them more flexibility,” Macaulay said.
The Sloan report shows that about 80 percent of
students taking online courses are at the undergraduate level. About half
are taking online courses through community colleges and 13 percent through
doctoral and research universities, according to the survey.
Nearly all institutions with total enrollments
exceeding 15,000 students have some online offerings, and about two-thirds
of them have fully online programs, compared with about one in six at the
smallest institutions (those with 1,500 students or fewer), the report
notes. Allen said private nonprofit colleges are often set in enrollment
totals and not looking to expand into the online market.
The report indicates that two-year colleges are particularly willing to be
involved in online learning.
“Our institutions tend to embrace changes a little
more readily and try different pedagogical styles,” said Kent Phillippe, a
senior research associate at the American Association of Community Colleges.
The report cites a few barriers to what it calls the “widespread adoption of
online learning,” chief among them the concern among college officials that
some of their students lack the discipline to succeed in an online setting.
Nearly two-thirds of survey respondents defined that as a barrier.
Allen, the report’s co-author, said she thinks that
issue arises mostly in classes in which work can be turned in at any time
and lectures can be accessed at all hours. “If you are holding class in real
time, there tends to be less attrition,” she said. The report doesn’t
differentiate between the live and non-live online courses, but Allen said
she plans to include that in next year’s edition.
Few survey respondents said acceptance of online
degrees by potential employers was a critical barrier — although liberal
arts college officials were more apt to see it as an issue.
November 10, 2006 reply from John Brozovsky
[jbrozovs@vt.edu]
Hi Bob:
One reason why might be what I have seen. The
in residence accounting students that I talk with take online classes
here because they are EASY and do not take much work. This would be very
popular with students but not generally so with faculty.
John
November 10, 2006 reply from Bob Jensen
Hi John,
Then there is a quality control problem whereever this is a fact. It
would be a travesty if any respected college had two or more categories of
academic standards or faculty assignments.
Variations in academic standards have long been a problem between
part-time versus full-time faculty, although grade inflation can be higher
or lower among part-time faculty. In one instance, it’s the tenure-track
faculty who give higher grades because they're often more worried about
student evaluations. At the opposite extreme it is part-time faculty who
give higher grades for many reasons that we can think of if we think about
it.
One thing that I'm dead certain about is that highly motivated students
tend to do better in online courses ceteris paribus. Reasons are mainly that
time is used more efficiently in getting to class (no wasted time driving or
walking to class), less wasted time getting teammates together on team
projects, and fewer reasons for missing class.
Also online alternatives offer some key advantages for certain types of
handicapped students ---
http://www.trinity.edu/rjensen/000aaa/thetools.htm
My opinions on learning advantages of E-Learning were heavily influenced
by the most extensive and respected study of online versus onsite learning
experiments in the SCALE experiments
using full-time resident students at the University of Illinois ---
http://www.trinity.edu/rjensen/255wp.htm#Illinois
In the SCALE experiments cutting across 30 disciplines, it was generally
found that motivated students learned better online then their onsite
counterparts having the same instructors. However, there was no significant
impact on students who got low grades in online versus onsite treatment
groups.
I think the main problem with faculty is that online teaching tends to
burn out instructors more frequently than onsite instructors. This was also
evident in the SCALE experiments. When done correctly, online courses are
more communication intent between instructors and faculty. Also, online
learning takes more preparation time if it is done correctly.
My hero for online learning is still Amy Dunbar who
maintains high standards for everything:
http://www.cs.trinity.edu/~rjensen/002cpe/02start.htm
http://www.trinity.edu/rjensen/book01q4.htm#Dunbar
Bob Jensen
November 10, 2006 reply from John Brozovsky
[jbrozovs@vt.edu]
Hi Bob:
Also why many times it is not done 'right'. Not
done right they do not get the same education. Students generally do not
complain about getting 'less for their money'. Since we do not do online
classes in department the ones the students are taking are the university
required general education and our students in particular are not unhappy
with being shortchanged in that area as they frequently would have preferred
none anyway.
John
Bob Jensen's threads on open sharing and education technology are at
http://www.trinity.edu/rjensen/000aaa/0000start.htm
Bob Jensen's threads on online training and education alternatives are at
http://www.trinity.edu/rjensen/crossborder.htm
Motivations for Distance Learning ---
http://www.trinity.edu/rjensen/000aaa/updateee.htm#Motivations
Bob Jensen's threads on the dark side of online learning and teaching are
at
http://www.trinity.edu/rjensen/000aaa/theworry.htm
October 5, 2006 message from Carolyn Kotlas
[kotlas@email.unc.edu]
STUDENTS' PERCEPTIONS OF ONLINE LEARNING
"The ultimate question for educational research is
how to optimize instructional designs and technology to maximize learning
opportunities and achievements in both online and face-to-face
environments." Karl L.Smart and James J. Cappel studied two undergraduate
courses -- an elective course and a required course -- that incorporated
online modules into traditional classes. Their research of students'
impressions and satisfaction with the online portions of the classes
revealed mixed results:
-- "participants in the elective course rated
use of the learning modules slightly positive while students in the
required course rated them slightly negative"
-- "while students identified the use of
simulation as the leading strength of the online units, it was also the
second most commonly mentioned problem of these units"
-- "students simply did not feel that the
amount of time it took to complete the modules was worth what was
gained"
The complete paper, "Students' Perceptions of Online Learning: A
Comparative Study" (JOURNAL OF INFORMATION TECHNOLOGY EDUCATION, vol. 5,
2006, pp. 201-19), is available online at
http://jite.org/documents/Vol5/v5p201-219Smart54.pdf.
Current and back issues of the Journal of Information Technology
Education (JITE) [ISSN 1539-3585 (online) 1547-9714 (print)] are available
free of charge at
http://jite.org/.
The peer-reviewed journal is published annually by the Informing Science
Institute. For more information contact: Informing Science Institute, 131
Brookhill Court, Santa Rosa, California 95409 USA; tel: 707-531-4925; fax:
480-247-5724;
Web:
http://informingscience.org/.
I have heard some faculty argue that
asynchronous Internet courses just do not mesh with Trinity's on-campus mission.
The Scale Experiments at the University of Illinois indicate that many students
learn better and prefer online courses even if they are full-time, resident
students. The University of North Texas is finding out the same thing. There may
be some interest in what our competition may be in the future even for
full-time, on-campus students at private as well as public colleges and
universities.
On January 17, 2003, Ed Scribner forwarded this article from The Dallas
Morning News
Students Who Live
on Campus Choosing Internet Courses Syndicated From: The Dallas Morning
News
DALLAS - Jennifer
Pressly could have walked to a nearby lecture hall for her U.S. history class
and sat among 125 students a few mornings a week.
But the 19-year-old
freshman at the University of North Texas preferred rolling out of bed and
attending class in pajamas at her dorm-room desk. Sometimes she would wait
until Saturday afternoon.
The teen from
Rockwall, Texas, took her first college history class online this fall
semester. She never met her professor and knew only one of her 125 classmates:
her roommate.
"I take
convenience over lectures," she said. "I think I would be bored to
death if I took it in lecture."
She's part of a
controversial trend that has surprised many university officials across the
country. Given a choice, many traditional college students living on campus
pick an online course. Most universities began offering courses via the
Internet in the late 1990s to reach a different audience - older students who
commute to campus and are juggling a job and family duties.
During the last year,
UNT began offering an online option for six of its highest-enrollment courses
that are typically taught in a lecture hall with 100 to 500 students. The
online classes, partly offered as a way to free up classroom space in the
growing school, filled up before pre-registration ended, UNT officials said.
At UNT, 2,877 of the about 23,000 undergraduates are taking at least one
course online.
Nationwide, colleges
are reporting similar experiences, said Sally Johnstone, director of WCET, a
Boulder, Colo., cooperative of state higher education boards and universities
that researches distance education. Kansas State University, in a student
survey last spring, discovered that 80 percent of its online students were
full-time and 20 percent were part-time, the opposite of the college's
expectations, Johnstone said.
"Why pretend
these kids want to be in a class all the time? They don't, but kids don't come
to campus to sit in their dorm rooms and do things online exclusively,"
she said. "We're in a transition, and it's a complex one."
The UT Telecampus, a
part of the University of Texas System that serves 15 universities and
research facilities, began offering online undergraduate classes in
state-required courses two years ago. Its studies show that 80 percent of the
2,260 online students live on campus, and the rest commute.
Because they are
restricted to 30 students each, the UT System's online classes are touted as a
more intimate alternative to lecture classes, said Darcy Hardy, director of
the UT Telecampus.
"The
freshman-sophomore students are extremely Internet-savvy and understand more
about online options and availability than we could have ever imagined,"
Hardy said.
Online education
advocates say professors can reach students better online than in lecture
classes because of the frequent use of e-mail and online discussion groups.
Those who oppose the idea say they worry that undergraduates will miss out on
the debate, depth and interaction of traditional classroom instruction.
UNT, like most
colleges, is still trying to figure out the effect on its budget. The
professorial salary costs are the same, but an online course takes more money
to develop. The online students, however, free up classroom space and
eliminate the need for so many new buildings in growing universities. The
price to enroll is typically the same for students, whether they go to a
classroom or sit at their computer.
Mike Campbell, a
history professor at UNT for 36 years, does not want to teach an online class,
nor does he approve of offering undergraduate history via the Internet.
"People
shouldn't be sitting in the dorms doing this rather than walking over
here," he said. "That is based on a misunderstanding of what matters
in history."
In his class of 125,
he asks students rhetorical questions they answer en masse to be sure they're
paying attention, he said. He goes beyond the textbook, discussing such topics
as the moral and legal issues surrounding slavery.
He said he compares
the online classes to the correspondence courses he hated but had to teach
when he came to UNT in 1966. Both methods are too impersonal, he said,
recalling how he mailed assignments and tests to correspondence students.
UNT professors who
teach online say the courses are interactive, unlike correspondence courses.
Matt Pearcy has
lectured 125 students for three hours at a time.
"You'd try to be
entertaining," he said. "You have students who get bored after 45
minutes, no matter what you're doing. They're filling out notes, doing their
to-do list, reading their newspaper in front of you."
In his online U.S.
history class at UNT, students get two weeks to finish each lesson. They read
text, complete click-and-drag exercises, like one that matches terms with
historical figures, and take quizzes. They participate in online discussions
and group projects, using e-mail to communicate.
"Hands-down, I
believe this is a more effective way to teach," said Pearcy, who is based
in St. Paul, Minn. "In this setting, they go to the class when they're
ready to learn. They're interacting, so they're paying attention."
Pressly said she
liked the hands-on work in the online class. She could do crossword puzzles to
reinforce her history lessons. Or she could click an icon and see what Galileo
saw through his telescope in the 17th century.
"I took more
interest in this class than the other ones," she said.
The class, though,
required her to be more disciplined, she said, and that added stress. Two
weeks in a row, she waited till 11:57 p.m. Sunday - three minutes before the
deadline - to turn in her assignment.
Online courses aren't
for everybody.
"The thing about
sitting in my dorm, there's so much to distract me," said Trevor Shive, a
20-year-old freshman at UNT. "There's the Internet. There's TV. There's
radio."
He said students on
campus should take classes in the real, not virtual, world.
"They've got
legs; they can walk to class," he said.
Continued in the article at http://www.dallasnews.com/
January 17, 2003 response from John L. Rodi
[jrodi@IX.NETCOM.COM]
I would have added
one additional element. Today I think too many of us tend to teach accounting
the way you teach drivers education. Get in the car turn on the key and off
you go. If something goes wrong with the car you a sunk since you nothing
conceptually. Furthermore, it makes you a victim of those who do. Conceptual
accounting education teaches you to respond to choices, that is not only how
to drive but what to drive. Thanks for the wonderful analogy.
John Rodi
El Camino College
January 21 reply
from
On the subject of
technology and teaching accounting, I wonder how many of you are in the SAP
University Alliance and using it for accounting classes. I just teach advanced
financial accounting, and have not found a use for it there. However, I have
often felt that there is a place for it in intro financial, in managerial and
in AIS. On the latter, there is at least one good text book containing SAP
exercises and problems.
Although there are
over 400 universities in the world in the program, one of the areas where use
is lowest is accounting courses. The limitation appears to be related to a
combination of the learning curve for professors, together with an uncertainty
as to how it can be used to effectively teach conceptual material or otherwise
fit into curricula.
Gerald Trites,
FCA
Professor of Accounting and Information Systems
St Francis Xavier University
Antigonish, Nova Scotia
Website - http://www.stfx.ca/people/gtrites
The SAP University Alliance homepage is
at http://www.sap.com/usa/company/ua/
In today's
fast-paced, technically advanced society, universities must master the latest
technologies, not only to achieve their own business objectives
cost-effectively but also to prepare the next generation of business leaders.
To meet the demands for quality teaching, advanced curriculum, and more
technically sophisticated graduates, your university is constantly searching
for innovative ways of acquiring the latest information technology while
adhering to tight budgetary controls.
SAP™ can
help. A world leader in the development of business software, SAP is making
its market-leading, client/server-based enterprise software, the R/3®
System, available to the higher education community. Through our SAP
University Alliance Program, we are proud to offer you the world's most
popular software of its kind for today's businesses. SAP also provides setup,
follow-up consulting, and R/3 training for faculty - all at our expense. The
SAP R/3 System gives you the most advanced software capabilities used by
businesses of all sizes and in all industries around the world.
There are many ways a
university can benefit from an educational alliance with SAP. By partnering
with SAP and implementing the R/3 System, your university can:
- Take advantage
of a powerful cross-functional teaching tool
Because R/3 is a comprehensive, integrated business system with a proven
track record in the real world, it is an excellent tool for teaching
students how a business really works.
- Access advanced
software technology
Sophisticated in both architecture and functionality, R/3 is the world's
most advanced business enterprise software available today. Faculty and
students have the opportunity to stay in the forefront of business
software innovation.
- Enhance
marketability
Experience with R/3 is prized by corporate recruiters. Students
well-versed in the principles of management and the uses of R/3 are highly
marketable to SAP, our customers, and partners.
- Attract leading
educators
Prominent educators in business and information technology may find the
university's alliance with SAP attractive in terms of access to research
opportunities, advanced software, and users of R/3.
- Pursue research
opportunities
Faculty members can pursue research in many areas of business and
information technology.
- Broaden
outreach
SAP maintains an extensive network of contacts with leading consulting
firms that work as our partners in implementing R/3. What's more, our
customers are some of the largest and most prestigious corporations around
the world. As an Alliance member, your university can tap into this
network of contacts to broaden your reach into the business community.
- Stay in touch
with industry and product trends
SAP strategic business units work closely with customers, user groups,
industry associations, and leading consulting firms to ensure that we
continue to deliver leading-edge capability. As an Alliance member, your
university can keep abreast of new enterprise computing ideas and trends
through the SAP strategic business units.
January 6, 2006 message from Carolyn Kotlas
[kotlas@email.unc.edu]
No Significant Difference Phenomenon website
http://www.nosignificantdifference.org/
The website is a companion piece to Thomas L.
Russell's book THE NO SIGNIFICANT DIFFERENCE PHENOMENON, a bibliography of
355 research reports, summaries, and papers that document no significant
differences in student outcomes between alternate modes of education
delivery.
DISTANCE LEARNING AND FACULTY CONCERNS
Despite the growing number of distance learning
programs, faculty are often reluctant to move their courses into the online
medium. In "Addressing Faculty Concerns About Distance Learning" (ONLINE
JOURNAL OF DISTANCE LEARNING ADMINISTRATION, vol. VIII, no. IV, Winter 2005)
Jennifer McLean discusses several areas that influence faculty resistance,
including: the perception that technical support and training is lacking,
the fear of being replaced by technology, and the absence of a
clearly-understood institutional vision for distance learning. The paper is
available online at
http://www.westga.edu/%7Edistance/ojdla/winter84/mclean84.htm
The Online Journal of Distance Learning
Administration is a free, peer-reviewed quarterly published by the Distance
and Distributed Education Center, The State University of West Georgia, 1600
Maple Street, Carrollton, GA 30118 USA; Web:
http://www.westga.edu/~distance/jmain11.html .
December 10, 2004 message from Carolyn Kotlas [kotlas@email.unc.edu]
E-LEARNING ONLINE PRESENTATIONS
The University of Calgary Continuing Education
sponsors Best Practices in E-Learning, a website that provides a forum for
anyone working in the field to share their best practices. This month's
presentations include:
-- "To Share or Not To Share: There is No
Question" by Rosina Smith Details a new model for permitting "the
reuse, multipurposing, and repurposing of existing content"
-- "Effective Management of Distributed Online
Educational Content" by Gary Woodill "[R]eviews the history of
online educational content, and argues that the future is in distributed
content learning management systems that can handle a wide diversity of
content types . . . identifies 40 different genres of online educational
content (with links to examples)"
Presentations are in various formats, including
Flash, PDF, HTML, and PowerPoint slides. Registered users can interact with
the presenters and post to various discussion forums on the website. There is
no charge to register and view presentations. You can also subscribe to their
newsletter which announces new presentations each month. (Note: No archive of
past months' presentations appears to be on the website.)
For more information, contact: Rod Corbett, University of Calgary
Continuing Education; tel:403-220-6199 or 866-220-4992 (toll-free); email: rod.corbett@ucalgary.ca
; Web: http://elearn.ucalgary.ca/showcase/.
NEW APPROACHES TO
EVALUATING ONLINE LEARNING
"The clear
implication is that online learning is not good enough and needs to prove its
worth before gaining full acceptance in the pantheon of educational practices.
This comparative frame of reference is specious and irrelevant on several
counts . . ." In "Escaping the Comparison Trap: Evaluating Online
Learning on Its Own Terms (INNOVATE, vol. 1, issue 2, December 2004/January
2005), John Sener writes that, rather than being inferior to classroom
instruction, "[m]any online learning practices have demonstrated superior
results or provided access to learning experiences not previously
possible." He describes new evaluation models that are being used to
judge online learning on its own merits. The paper is available online at http://www.innovateonline.info/index.php?view=article&id=11&action=article.
You will need to
register on the Innovate website to access the paper; there is no charge for
registration and access.
Innovate [ISSN
1552-3233] is a bimonthly, peer-reviewed online periodical published by the
Fischler School of Education and Human Services at Nova Southeastern
University. The journal focuses on the creative use of information technology
(IT) to enhance educational processes in academic, commercial, and government
settings. Readers can comment on articles, share material with colleagues and
friends, and participate in open forums. For more information, contact James
L. Morrison, Editor-in-Chief, Innovate; email: innovate@nova.edu
; Web: http://www.innovateonline.info/.
I read the following for a scheduled program of the 29th Annual Accounting
Education Conference, October 17-18, 2003 Sponsored by the Texas CPA
Society, San Antonio Airport Hilton.
WEB-BASED AND
FACE-TO-FACE INSTRUCTION:
A COMPARISON OF LEARNING OUTCOMES IN A FINANCIAL ACCOUNTING
COURSE
Explore the results
of a study conducted over a four-semester period that focused on the same
graduate level financial accounting course that was taught using web-based
instruction and face-to-face instruction. Discuss the comparison of
student demographics and characteristics, course satisfaction, and comparative
statistics related to learning outcomes.
Doug Rusth/associate
professor/University of Houston at Clear Lake/Clear Lake
Bob Jensen's threads on asynchronous
versus synchronous learning are at http://www.trinity.edu/rjensen/255wp.htm
Note in particular the research outcomes of The Scale Experiment at the
University of Illinois --- http://www.trinity.edu/rjensen/255wp.htm#Illinois
Once again, my advice to new faculty
is at http://www.trinity.edu/rjensen/000aaa/newfaculty.htm
Minimum Grades as a School Policy
Question
Should a student who gets a zero (for not doing anything) or 23% (for doing
something badly) on an assignment, exam, or term paper be automatically (as a
matter of school policy) upgraded to a 60% no matter what proportion the grade
is toward a course's final grade?
Should a student get 60% even if he or she fails to show up for an examination?
Jensen Comment
This could lead to some strategies like "don't spend any time on the term paper
and concentrate on passing the final examination or vice versa."
Such strategies are probably not in the spirit of the course design, especially
when the instructor intended for students to have to write a paper.
"Time to Add Basket Weaving as a Course," by Ben Baker, The Irascible
Professor, June 22, 2008 ---
http://irascibleprofessor.com/comments-06-22-08.htm
Bob Jensen's threads on higher education controversies are at
http://www.trinity.edu/rjensen/HigherEdControversies.htm
Bob Jensen's threads on assessment are at
http://www.trinity.edu/rjensen/assess.htm
Issues in Group
Grading
December 6, 2004 message from Glen Gray
[glen.gray@CSUN.EDU]
When I have students
do group projects, I require each team member complete a peer review form
where the team member evaluates the other team members on 8 attributes using a
scale from 0 to 4. On this form they also give their team members an overall
grade. In a footnote it is explained that an “A” means the team member
receives the full team grade; a “B” means a 10% reduction from the team
grade; a “C” means 20% discount; a “D” means 30% discount; “E”
means 40%, and an “F” means a 100% discount (in other words, the team
member should get a zero).
I assumed that the
form added a little peer pressure to the team work process. In the past,
students were usually pretty kind to each other. But now I have a situation
where the team members on one team have all given either E’s of F’s to one
of their team members. Their written comments about this guy are all pretty
consistent.
Now, I worried if I
actually enforce the discount scale, things are going to get messy and the
s*** is going to hit the fan. I’m going to have one very upset student. He
is going to be mad at his fellow teammates.
Has anyone had
similar experience? What has the outcome been? Is there a confidentially issue
here? In other words, are the other teammates also going to be upset that I
revealed their evaluations? Is there going to be a lawsuit coming over the
horizon?
Glen L. Gray, PhD,
CPA
Dept. of Accounting & Information Systems
College of Business & Economics
California State University, Northridge
Northridge, CA 91330-8372
http://www.csun.edu/~vcact00f
Most of the replies to the message
above encouraged being clear at the beginning that team evaluations would affect
the final grade and then sticking to that policy.
December 5, 2004 reply from David Fordham, James Madison University
[fordhadr@JMU.EDU]
Glen, the fact that
you are in California, by itself, makes it much more difficult to predict the
lawsuit question. I've seen some lawsuits (and even worse, legal outcomes)
from California that are completely unbelievable... Massachussetts too.
But that said, I can
share my experience that I have indeed given zero points on a group grade to
students where the peer evaluations indicated unsatisfactory performance. My
justification to the students in these "zero" cases has always been,
"it was clear from your peers that you were not part of the group effort,
and thus have not earned the points for the group assignment".
I never divulge any
specific comments, but I do tell the student that I am willing to share the
comments with an impartial arbiter if they wish to have a third party confirm
my evidence. To date, no student has ever contested the decision.
Every other semester
or so, I have to deduct points to some degree for unsatisfactory work as
judged by peers. So far, I've had no problems making it stick, and in most
cases, the affected student willingly admits their deficiency, although
usually with excuses and rationales.
But I'm not in
California, and the legal precedents here are unlike those in your neck of the
woods.
If I were on the west
coast, however, I'd probably be likely to at least try to stick to my
principles as far as my university legal counsel would allow. Then, if my
counsel didn't support me, I'd look for employment in a part of the country
with a more reasonable legal environment (although that is getting harder to
find every day).
Good luck,
David Fordham
December 5, 2004 reply from Amy Dunbar
Sometimes groups do
blow up. Last summer I had one group ask me to remove a member. Another group
had a nonfunctioning member, based on the participation scores. I formed an
additional group comprised of just those two. They finally learned how to
work. Needless to say they weren’t happy with me, but the good thing about
teaching is that every semester we get a fresh start!
Another issue came up
for the first time, at least that I noticed. I learned that one group made a
pact to rate each other high all semester long regardless of work level, and I
still am not sure how I am going to avoid that problem next time around. The
agreement came to light when one of the students was upset that he did so
poorly on my exams. He told his senior that he had no incentive to do the
homework because he could just get the answers from the other group members,
and he didn’t have to worry about being graded down because of the
agreement. The student was complaining that the incentive structure I set up
hurt him because he needed more push do the homework. The senior told me after
the class ended. Any suggestions?
TEXAS IS GOING TO THE
ROSE BOWL!!!!!!!!! Go Horns! Oops, that just slipped out.
Amy Dunbar
A Texas alum married to a Texas fanatic
December 6, 2004 reply from Tracey Sutherland
[tracey@AAAHQ.ORG]
Glen, My first
thought on reading your post was that if things get complicated it could be
useful to have a context for your grading policy that clearly establishes that
it falls within common practice (in accounting and in cooperative college
classrooms in general). Now you've already built some context from within
accounting by gathering some responses here from a number of colleagues for
whom this is a regular practice. Neal's approach can be a useful counterpart
to peer evaluation for triangulation purposes -- sometimes students will
report that they weren't really on-point for one reason or another (I've done
this with good result but only with upper-level grad students). If the issue
becomes more complicated because the student challenges your approach up the
administrative ladder, you could provide additional context for the
consistency of your approach in general by referencing the considerable body
of literature on these issues in the higher education research literature --
you are using a well-established approach that's been frequently tested. A
great resource if you need it is Barbara Millis and Phil Cottell's book
"Cooperative Learning for Higher Education Faculty" published by
Oryx Press (American Council on Education Series on Higher Education). They do
a great job of annotating the major work in the area in a short, accessible,
and concise book that also includes established criteria used for evaluating
group work and some sample forms for peer assessment and self-assessment for
group members (also just a great general resource for well-tested
cooperative/group activities -- and tips for how to manage implementing them).
Phil Cottell is an accounting professor (Miami U.) and would be a great source
of information should you need it.
Your established
grading policy indicates that there would be a reduction of grade when team
members give poor peer evaluations -- which wouldn't necessarily mean that you
would reveal individual's evaluations but that a negative aggregate evaluation
would have an effect -- and that would protect confidentiality consistently
with your policy. It seems an even clearer case because all group members have
given consistently negative evaluations -- as long as it's not some weird
interpersonal thing -- something that sounds like that would be a red flag for
the legal department. I hate it that we so often worry about legal
ramifications . . . but then again it pays to be prepared!
Peace of the
season,
Tracey
December 6, 2004 reply from Bob Jensen
I once listened to an award winning
AIS professor from a very major university (that after last night won't be
going to the Orange Bowl this year) say that the best policy is to promise
everybody an A in the course. My question then is what the point of the
confidential evaluations would be other than to make the professor feel bad at
the end of the course?
Bob Jensen
Too Good to Grade: How can these
students get into doctoral programs and law school if their prestigious
universities will not disclose grades and class rankings? Why grade at all
in this case?
Students at some top-ranked B-schools have a secret. It's something they
can't share even if it means losing a job offer. It's one some have worked hard
for and should be proud of, but instead they keep it to themselves. The secret
is their grades.
At four of the nation's 10 most elite B-schools --
including Harvard, Stanford, and Chicago -- students have adopted policies that
prohibit them or their schools from disclosing grades to recruiters. The idea is
to reduce competitiveness and eliminate the risk associated with taking
difficult courses. But critics say the only thing nondisclosure reduces is one
of the most important lessons B-schools should teach: accountability (see
BusinessWeek, 9/12/05,
"Join the Real World, MBAs").
It's a debate that's flaring up on B-school campuses
across the country. (For more on this topic, log on to our
B-Schools Forum.) And nowhere is it more
intense than at University of Pennsylvania's Wharton School, where students,
faculty, and administrators have locked horns over a school-initiated proposal
that would effectively end a decade of grade secrecy at BusinessWeek's No.
3-ranked B-school. It wouldn't undo disclosure rules but would recognize the top
25% of each class -- in effect outing everyone else. It was motivated, says
Vice-Dean Anjani Jain in a recent Wharton Journal article, by the "disincentivizing
effects" of grade nondisclosure, which he says faculty blame for lackluster
academic performance and student disengagement.
"Campus Confidential:
Four top-tier B-schools don't disclose grades. Now that policy is under attack,"
Business Week, September 12, 2005 ---
http://snipurl.com/BWSept122
Too Good to Grade: How can these
students get into doctoral programs and law schools if their prestigious
universities will not disclose grades and class rankings? Why grade at all
in this case?
Students at some top-ranked B-schools have a secret. It's something they
can't share even if it means losing a job offer. It's one some have worked hard
for and should be proud of, but instead they keep it to themselves. The secret
is their grades.
At four of the nation's 10 most elite B-schools --
including Harvard, Stanford, and Chicago -- students have adopted policies that
prohibit them or their schools from disclosing grades to recruiters. The idea is
to reduce competitiveness and eliminate the risk associated with taking
difficult courses. But critics say the only thing nondisclosure reduces is one
of the most important lessons B-schools should teach: accountability (see
BusinessWeek, 9/12/05,
"Join the Real World, MBAs").
It's a debate that's flaring up on B-school campuses
across the country. (For more on this topic, log on to our
B-Schools Forum.) And nowhere is it more
intense than at University of Pennsylvania's Wharton School, where students,
faculty, and administrators have locked horns over a school-initiated proposal
that would effectively end a decade of grade secrecy at BusinessWeek's No.
3-ranked B-school. It wouldn't undo disclosure rules but would recognize the top
25% of each class -- in effect outing everyone else. It was motivated, says
Vice-Dean Anjani Jain in a recent Wharton Journal article, by the "disincentivizing
effects" of grade nondisclosure, which he says faculty blame for lackluster
academic performance and student disengagement.
"Campus Confidential:
Four top-tier B-schools don't disclose grades. Now that policy is under attack,"
Business Week, September 12, 2005 ---
http://snipurl.com/BWSept122
Jensen Comment: Talk about moral hazard. What if 90% of the
applicants claim to be straight A graduates at the very top of the class,
and nobody can prove otherwise?
September 2, 2005 message from Denny Beresford
[DBeresford@TERRY.UGA.EDU]
Bob,
The impression I have (perhaps I'm misinformed) is that most MBA classes
result in nearly all A's and B's to students. If that's the case, I wonder
how much a grade point average really matters.
Denny Beresford
September 2, 2005 reply from Bob Jensen
One of the schools, Stanford,
in the 1970s lived with the Van Horn rule that dictated no more than 15% A
grades in any MBA class. I guess grade inflation has hit the top
business schools. Then again, maybe the students are just better than
we were.
I added the following to my
Tidbit on this:
Talk about moral hazard. What
if 90% of the applicants claim to be straight A graduates at the very top
of the class, and nobody can prove otherwise?
After your message Denny, I
see that perhaps it's not moral hazard. Maybe 90% of the students actually
get A grades in these business schools, in which nearly 90% would graduate
summa cum laude.
What a joke! It must be
nice teaching students who never hammer you on teaching evaluations because
you gave them a C or below.
The crucial quotation is
"faculty blame for lackluster academic performance and student
disengagement." Isn't this a laugh if they all get A and B grades for
"lackluster academic performance and student disengagement."
I think these top schools are
simply catering to their customers!
Bob Jensen
Harvard Business School Eliminates Ban on a Graduate's
Discretionary Disclosure of Grades
The era of the second-year slump at
Harvard Business School is over. Or maybe the days of
student cooperation are over. Despite strong student
opposition, the business school announced Wednesday that it
was ending its ban on sharing grades with potential
employers. Starting with new students who enroll in the
fall, M.B.A. candidates can decide for themselves whether to
share their transcripts. The ban on grade-sharing has been
enormously popular with students since it was adopted in
1998. Supporters say that it discouraged (or at least kept
to a reasonable level) the kind of cut-throat competition
for which business schools are known. With the ban, students
said they were more comfortable helping one another or
taking difficult courses. But a memo sent to students by Jay
O. Light, the acting dean, said that the policy was wrong.
“Fundamentally, I believe it is inappropriate for HBS to
dictate to students what they can and cannot say about their
grades during the recruiting process. I believe you and your
classmates earn your grades and should be accountable for
them, as you will be accountable for your performance in the
organizations you will lead in the future,” he wrote.
Scott Jaschik, "Survival of the Fittest MBA," Inside
Higher Ed, December 16, 2005 ---
http://www.insidehighered.com/news/2005/12/16/grades
Bob Jensen's threads on Controversies in Higher Education are at
http://www.trinity.edu/rjensen/HigherEdControversies.htm
Software for faculty and departmental performance evaluation and
management
May 30, 2006 message from Ed Scribner
[escribne@NMSU.EDU]
A couple of months ago I asked for any experiences
with systems that collect faculty activity and productivity data for
multiple reporting needs (AACSB, local performance evaluation, etc.). I said
I'd get back to the list with a summary of private responses.
No one reported any significant direct experience,
but many AECMers provided names and e-mail addresses of [primarily]
associate deans who had researched products from Sedona and Digital
Measures. Since my associate dean was leading the charge, I just passed
those addresses on to her.
We ended up selecting Digital Measures mainly
because of our local faculty input, the gist of which was that it had a more
professional "feel." My recollection is that the risk of data loss with
either system is acceptable and that the university "owns" the data. I
understand that a grad student is entering our data from the past five years
to get us started.
Ed Scribner
New Mexico State University
Las Cruces, NM, USA
Jensen Comment
The Digital Measures homepage is at
http://www.digitalmeasures.com/
Over 100 universities use Digital Measures'
customized solutions to connect administrators, faculty, staff, students,
and alumni. Take a look at a few of the schools and learn more about Digital
Measures.
Free from the Huron Consulting Group (Registration Required) ---
http://www.huronconsultinggroup.com/
Effort Reporting Technology for Higher Education ---
http://www.huronconsultinggroup.com/uploadedFiles/ECRT_email.pdf
-
Question Mark (Software for Test and
Tutorial Generation and Networking)
- Barron's Home Page
- Metasys Japan Software
- Question Mark America home page
- Using ExamProc for
OMR Exam Marking
- Vizija d.o.o. -
Educational Programs - Wisdom Tools
Yahoo Links
TechKnowLogia --- http://www.techknowlogia.org/
TechKnowLogia
is an international online journal that provides policy makers,
strategists, practitioners and technologists at the local, national and
global levels with a strategic forum to:
Explore the vital
role of different information technologies (print, audio, visual
and digital) in the development of human and knowledge capital;
Share policies,
strategies, experiences and tools in harnessing technologies for
knowledge dissemination, effective learning, and efficient
education services;
Review the latest
systems and products of technologies of today, and peek into the
world of tomorrow; and
Exchange information
about resources, knowledge networks and centers of expertise.
- Do
Technologies Enhance Learning?
- Brain
Research, Learning and Technology
- Technologies
at Work for: Critical Thinking, Science Instruction,
Teaching Practices, etc...
- Interactive
TV as an Educational Tool
- Complexity
of Integrating ICTs into Curriculum & Exams
- Use of
Digital Cameras to Enhance Learning
- Creating
Affordable Universal Internet Access
Bob Jensen's threads on education technologies are at http://www.trinity.edu/rjensen/000aaa/0000start.htm
"What's the Best Q&A Site?" by Wade Roush, MIT's
Technology Review, December 22, 2006 ---
http://www.technologyreview.com/InfoTech/17932/
Magellan Metasearch ---
http://sourceforge.net/projects/magellan2/
Many educators would like to put more materials on
the web, but they are concerned about protecting access to all or parts of
documents. For example, a professor may want to share a case with the world but
limit the accompanying case solution to selected users. Or a professor may want to
make certain lecture notes available but limit the access of certain copyrighted portions
to students in a particular course. If protecting parts of your documents is of
great interest, you may want to consider NetCloak from Maxum at http://www.maxum.com/ . You can download a free
trial version.
NetCloak Professional Edition
combines the power of Maxum's classic combo, NetCloak and NetForms, into a single CGI
application or WebSTAR API plug-in. With NetCloak Pro, you can use HTML forms on your web
site to create or update your web pages on the fly. Or you can store form data in text
files for importing into spreadsheets or databases off-line. Using NetCloak Pro, you can
easily create online discussion forums, classified ads, chat systems, self-maintaining
home pages, frequently-asked-question lists, or online order forms!
NetCloak Pro also gives your web
site access to e-mail. Users can send e-mail messages via HTML forms, and NetCloak Pro can
create or update web pages whenever an e-mail message is received by any e-mail address.
Imagine providing HTML archives of your favorite mailing lists in minutes!
NetCloak Pro allows users to
"cloak" pages individually or "cloak" individual paragraphs or text
strings. The level of security seems to be much higher than scripted passwords such
as scripted passwords in JavaScript or VBScript.
Eric Press led me to http://www.maxum.com/NetCloak/FAQ/FAQList.html
(Thank you Eric, and thanks for the "two lunches")
Richard Campbell responded as follows:
Alternatives to using Netcloak: 1.
Symantec http://www.symantec.com has a free
utility called Secret which will password-protect any type of file.
2. Winzip http://www.winzip.com has a another shareware
utility called Winzip - Self-Extractor, which has a password protect capability. The
advantage to this approach is that you can bundle different file types (.doc, xls) , zip
them and you can have them automatically install to a folder that you have named. If you
have a shareware install utility that creates a setup.exe routine, you also can have it
install automatically on the student's machine. The price of this product is about $30.
Full Disclosure to Consumers of Higher Education (including assessment
of colleges and the Spellings Commission Report) ---
http://www.trinity.edu/rjensen/HigherEdControversies.htm#FullDisclosure
Dropping a Bomb on Accreditation
The most provocative vision for changing accreditation
put forward at Tuesday’s meeting came from Robert C. Dickeson, president
emeritus of the University of Northern Colorado. Dickeson’s presentation was
loaded with irony, in some ways; a position paper he wrote in 2006 as a
consultant to Margaret Spellings’s Commission on the Future of Higher Education
was harshly critical of the current system of accreditation (calling it rife
with conflicts of interest and decidedly lacking in transparency) and suggested
replacing the regional accrediting agencies with a “national accreditation
foundation” that would establish national standards for colleges to meet.
Dickeson’s presentation Tuesday acknowledged that there remained legitimate
criticisms of accreditation’s rigor and agility, noting that many colleges and
accrediting agencies still lacked good information about student learning
outcomes “40 years after the assessment movement began in higher education.”
Doug Lederman, "Whither Accreditation," Inside Higher Ed, January 28,
2009 ---
http://www.insidehighered.com/news/2009/01/28/accredit
Dickerson's 2006 Position Paper "Dropping a Bomb on Accreditation" ---
http://insidehighered.com/news/2006/03/31/accredit
Here’s something
that may be useful when assessing a doctoral program. Note to key items listed
near the end of the document.
From the
Chronicle of Higher Education, November 7, 2008 ---
http://chronicle.com/weekly/v55/i11/11a00104.htm?utm_source=wb&utm_medium=en
Provosts around the country are anticipating — and
some are surely dreading — the long afternoons when they will go over
national rankings data for their graduate departments. No later than this
winter, after many delays, the National Research Council plans to release
its assessments of American doctoral programs.
Student-faculty ratios, time to degree, stipends,
faculty research productivity, and citation counts: Those numbers and many
others will be under national scrutiny.
But one university couldn't wait. Last year,
prodded by anxious faculty members worried about low Ph.D. production, Ohio
State University conducted a thorough review of its doctoral programs,
drawing heavily on data that its departments had compiled for the council's
questionnaire. The Ohio State experience provides a window on what may be
coming nationally.
The evaluations had teeth. Of the 90 doctoral
programs at Ohio State, five small ones were tagged as "candidates for
disinvestment or elimination": comprehensive vocational education (a
specialty track in the college of agriculture), soil science, welding
engineering, rehabilitation services, and technology education. Another 29
programs were instructed to reassess or restructure themselves.
Some programs got good news, however. Twenty-nine
that were identified as "high quality" or "strong" will share hundreds of
thousands of dollars in new student-fellowship subsidies.
Many faculty members say the assessments provided a
long-overdue chance for Ohio State to think strategically, identifying some
fields to focus on and others that are marginal. But the process has also
had its share of bumps. The central administration concluded that certain
colleges, notably the College of Biological Sciences, were too gentle in
their self-reports. And some people have complained that the assessments
relied too heavily on "input" variables, such as students' GRE scores.
Despite those concerns, the dean of Ohio State's
Graduate School, Patrick S. Osmer, says the assessment project has exceeded
his expectations. He hopes it can serve as a model for what other
institutions can do with their doctoral data. "The joy of working here," he
says, "is that we're trying to take a coordinated, logical approach to all
of these questions, to strengthen the university."
A Faculty Mandate
The seeds of the assessment project were planted in
2005, when a high-profile faculty committee issued a report warning that
Ohio State was generating proportionally fewer Ph.D.'s than were the other
Big Ten universities. "The stark fact is that 482 Ph.D. degrees ... granted
in 2003-4 is far below the number expected from an institution the size and
(self-declared) quality of OSU," the report read. (The 482 figure excluded
doctorates awarded by Ohio State's college of education.) At the University
of Wisconsin at Madison, for example, each tenure-track faculty member
generated an average of 0.4 Ph.D.'s each year. At Ohio State, the figure was
only 0.267.
The committee recommended several steps: Give the
central administration more power in graduate-level admissions. Organize
stipends, fellowships, and course work in ways that encourage students to
complete their doctorates in a timely manner. Stop giving doctoral-student
subsidies to students who are likely to earn only master's degrees. And
distribute subsidies from the central administration on a strategic basis,
rewarding the strongest programs and those with the most potential for
improvement.
"One thing that motivated all of this," says Paul
Allen Beck, a professor of political science and a former dean of social and
behavioral sciences at Ohio State, "was a feeling that the university had
not invested enough in Ph.D. education. Our universitywide fellowships were
not at a competitive level. We really felt that we should try to do a better
job of concentrating our university investments on the very best programs."
Ohio State officials had hoped to use the National
Research Council's final report itself for their evaluations. But after its
release was postponed for what seemed like the sixth or seventh time, they
moved forward without it.
In September 2007, Mr. Osmer asked the deans of
Ohio State's 18 colleges to report data about their doctoral students'
median time to degree, GRE scores, stipends, fellowships, job-placement
outcomes, and racial and ethnic diversity.
Many of those numbers were easy to put together,
because departments had compiled them during the previous year in response
to the council's questionnaire. But job placements — a topic that will not
be covered in the NRC report — were something that certain Ohio State
programs had not previously tracked.
"This was a huge new project for us and for some of
our departments as well," says Julie Carpenter-Hubin, director of
institutional research and planning. "But simply going around and talking to
faculty took care of most of it. It's really remarkable the degree to which
faculty members stay in touch with their former doctoral students and know
where they are. I think we wound up with placement data for close to 80
percent of our Ph.D. graduates, going 10 years back."
Defending Their Numbers
The reports that Ohio State's colleges generated
last fall contained a mixture of quantitative data — most prominently GRE
scores and time-to-degree numbers — and narrative arguments about their
departments' strengths. The College of Social and Behavioral Sciences, for
example, noted that several recent Ph.D.s in economics, political science,
and psychology had won tenure-track positions at Ivy League institutions.
When they had to report poor-looking numbers,
departments were quick to cite reasons and contexts. The anthropology
program said its median time to degree of 7.3 years might seem high when
compared with those of other degree courses, but is actually lower than the
national average for anthropology students, who typically spend years doing
fieldwork. Economics said its retention-and-completion rate, which is less
than 50 percent, might look low but is comparable to those in other highly
ranked economics departments, where students are often weeded out by
comprehensive exams at the end of the first year.
In April 2008, a committee appointed and led by Mr.
Osmer, the graduate-school dean, digested the colleges' reports and issued a
report card, ranking the 90 doctoral programs in six categories. (See table
on following page.)
The panel did not meekly accept the colleges'
self-evaluations. The College of Biological Sciences, for example, had
reported that it lacked enough data to draw distinctions among its programs.
But the committee's report argued, among other things, that the small
program in entomology appeared to draw relatively little outside research
support, and that its students had lower GRE scores than those in other
biology programs. (Entomology and all other doctoral programs in biology
were among the 29 programs that Mr. Osmer's committee deemed in need of
reassessment or restructuring.)
The report's points about entomology — and about
the general organization of the college — were controversial among the
faculty members, says Matthew S. Platz, a professor of chemistry who became
interim dean of biological sciences in July. But faculty members have taken
the lead in developing new designs for the college, he says, to answer many
of the central administration's concerns.
"I'm delighted by the fact that at the grass-roots
level, faculty members have been talking about several types of
reorganization," Mr. Platz says. "And I'm hopeful that two or three of them
will be approved by the end of the year."
'Unacceptably Low Quality'
The five doctoral degrees named as candidates for
the ax have also stirred controversy.
Jerry M. Bigham, a professor of soil science and
director of Ohio State's School of Environment and Natural Resources, says
he was disappointed but not entirely surprised by the committee's suggestion
that his program could be terminated. The soil-science program has existed
on its own only since 1996; before that it was one of several
specializations offered by the doctoral program in agronomy.
"In essence, we've had students and faculty members
spread across three programs," he says. So he understands why the university
might want to place soil sciences under a larger umbrella, in order to
reduce overhead and streamline the administration.
At the same time, he says, several people were
offended by the Osmer committee's blunt statement that soil-science students
are of "unacceptably low quality."
The panel's analysis of the students' GRE scores
was "just a snapshot, and I think it really has to be viewed with caution,"
Mr. Bigham says. "Even though we're a small program, our students have won
university fellowships and have been recognized for their research. So I
would really object to any characterization of our students as being weak."
The final verdict on the five programs is
uncertain. The colleges that house them might propose folding them into
larger degree courses. Or they might propose killing them outright. All such
proposals, which are due this fall, are subject to approval by the central
administration.
Jason W. Marion, president of the university's
Council of Graduate Students, says its members have generally supported the
doctoral-assessment project, especially its emphasis on improving stipends
and fellowships. But some students, he adds, have expressed concern about an
overreliance on GRE scores at the expense of harder-to-quantify "output"
variables like job-placement outcomes.
Mr. Osmer replies that job placement actually has
been given a great deal of weight. "Placing that alongside the other
variables really helped our understanding of these programs come together,"
he says.
At this summer's national workshop sessions of the
Council of Graduate Schools, Mr. Osmer was invited to lecture about Ohio
State's assessment project and to discuss how other institutions might make
use of their own National Research Council data. William R. Wiener, a vice
provost at Marquette University who also spoke on Mr. Osmer's panel, calls
the Ohio State project one example of how universities are becoming smarter
about assessments.
"Assessments need to have reasonable consequences,"
Mr. Wiener says. "I think more universities realize that they need to create
a culture of assessment, and that improving student learning needs to
permeate everything that we do."
Mr. Beck, the former social-sciences dean at Ohio
State, says that even for relatively strong departments — his own
political-science department was rated "high quality" by Mr. Osmer's
committee — a well-designed assessment process can be eye-opening.
"These programs just kind of float along, guided by
their own internal pressures," says Mr. Beck. But "the departments here were
forced to take a hard look at themselves, and they sometimes saw things that
they didn't like."
|
HOW
OHIO
STATE U. RATES DOCTORAL PROGRAMS Until recently, Ohio
State University used a simple, quantity-based formula to distribute
student-support money to its doctoral programs. In essence, the more
credit hours taken by students in a program each quarter, the more
money the program collected. But last year the university introduced
quality-control measures. It used them to make choices about which
programs to invest in — and, more controversially, which ones to
eliminate.
Measures used:
- Students' time to
degree Students' GRE scores
- Graduates' job
placements, 1996-2005 Student diversity
- The program's share
of Ph.D. production (both nationally and among Ohio State's
peers)
- "Overall program
quality and centrality to the university's mission"
Resulting ratings:
- High quality: 12 programs
- Strong: 17 programs
- Good: 16 programs
- New and/or in transition; cannot be fully
assessed: 11 programs
- Must reassess and/or restructure: 29
programs
- Candidates for disinvestment or
elimination: 5 programs
What the ratings mean:
- Programs rated "high quality" and "strong"
will share new funds from the central administration for
graduate-student stipends.
- "Good" programs have been asked to make
improvements in specific areas. Their support will not
significantly change.
- Colleges with doctoral programs that were
deemed in need of reassessment or restructuring were asked to
submit new strategic plans this fall. Those plans are subject to
approval by Ohio State's provost.
- The new strategic plans will also deal
with programs deemed candidates for disinvestment or
elimination. Those programs might be folded into larger degree
courses, or killed outright.
|
Bob
Jensen's threads on assessment are at
http://www.trinity.edu/rjensen/assess.htm
"Minnesota Colleges Seek Accountability by
the Dashboard Light," by Paul Basken, Chronicle of Higher Education,
June 18, 2008 ---
http://chronicle.com/daily/2008/06/3423n.htm
When your car starts sputtering, it's easy to look
at the dashboard and see if you're running out of gas. What if you could do
the same with your local college?
Minnesota's system of state colleges and
universities believes it can show the way.
After two years of preparation, the 32-college
system unveiled on Tuesday its new Accountability Dashboard. The service is
based on a Web site that displays a series of measures—tuition rates,
graduates' employment rates, condition of facilities—that use
speedometer-type gauges to show exactly how the Minnesota system and each of
its individual colleges is performing.
The idea is in response to the growing demand,
among both policy makers and the public, for colleges to provide more useful
and accessible data about how well they are doing their jobs.
"There's a great call across the country for
accountability and transparency, and I don't think it's going to go away,"
said James H. McCormick, chancellor of the 374,000-student system. "It's
just a new way of doing business."
Shining a Light
The information in the new format was already
publicly available. But its presentation in the dashboard format, along with
comparisons with statewide and national figures as well as the system's own
goals, will put pressure on administrators and faculty members for
improvement, Mr. McCormick and other state education officials told
reporters.
"The dashboard shines a light on where we need to
improve," said Ruth Grendahl, vice chairman of the Board of Trustees of the
Minnesota State Colleges and Universities.
Among the areas the dashboard already indicates as
needing improvement is the cost of attending Minnesota's state colleges. The
gauges for tuition and fees at all 30 of the system's two-year institutions
show needles pointing to "needs attention," a reflection of the fact that
their costs are higher than those of 80 percent of their peers nationwide.
The dashboard shows the system faring better in
other areas, such as licensure-examination pass rates and degree-completion
rates, in which the average figures are in the "meets expectations" range.
Other measures, like "innovation" and "student engagement," don't yet show
results, as the necessary data are still being collected or the criteria
have not yet been defined.
Tool of Accountability
Many private companies already use dashboard-type
displays in their computer systems to help monitor business performance, but
the data typically serve an internal function rather than being a tool for
public accountability.
The Minnesota dashboard stems in part from the
system's work through the National Association of System Heads, or NASH, on
a project to improve the education of minority and low-income students. The
project is known as Access to Success.
Continued in article
Jensen Comment
Those in my generation might appreciate the fact that this car has a "NASH"
dashboard. The problem is that when a car's dashboard signals troubles such as
oil leaks and overheating, owner's can easily trade in or junk a clunker
automobile. This is not so simple in the politics of state universities.
May 2, 2008 message from Carolyn Kotlas
[kotlas@email.unc.edu]
REPORT ON E-LEARNING RETURNS ON INVESTMENT
"Within the academic community there remains a
sizable proportion of sceptics who question the value of some of the tools
and approaches and perhaps an even greater proportion who are unaware of the
full range of technological enhancements in current use. Amongst senior
managers there is a concern that it is often difficult to quantify the
returns achieved on the investment in such technologies. . . . JISC infoNet,
the Association for Learning Technology (ALT) and The Higher Education
Academy were presented with the challenge of trying to make some kind of
sense of the diversity of current e-learning practice across the sector and
to seek out evidence that technology-enhanced learning is delivering
tangible benefits for learners, teachers and institutions."
The summary of the project is presented in the
recently-published report, "Exploring Tangible Benefits of e-Learning: Does
Investment Yield Interest?" Some benefits were hard to measure and quantify,
and the case studies were limited to only sixteen institutions. However,
according to the study, there appears to be "clear evidence" of many good
returns on investment in e-learning. These include improved student pass
rates, improved student retention, and benefits for learners with special
needs.
A copy of the report is available at
http://www.jiscinfonet.ac.uk/publications/camel-tangible-benefits.pdf
A two-page briefing paper is available at
http://www.jisc.ac.uk/media/documents/publications/bptangiblebenefitsv1.pdf
JISC infoNet, a service of the Joint Information
Systems Committee, "aims to be the UK's leading advisory service for
managers in the post-compulsory education sector promoting the effective
strategic planning, implementation and management of information and
learning technology." For more information, go to
http://www.jiscinfonet.ac.uk/
Association for Learning Technology (ALT), formed
in 1993, is "the leading UK body bringing together practitioners,
researchers, and policy makers in learning technology." For more
information, go to
http://www.alt.ac.uk/
The mission of The Higher Education Academy, owned
by two UK higher education organizations (Universities UK and GuildHE), is
to "help institutions, discipline groups, and all staff to provide the best
possible learning experience for their students." For more information, go
to
http://www.heacademy.ac.uk/
Bob Jensen's threads on asynchronous learning are at
http://www.trinity.edu/rjensen/255wp.htm
Also see
http://www.trinity.edu/rjensen/265wp.htm
Assessment Issues ---
http://www.trinity.edu/rjensen/assess.htm
Threads on Costs and Instructor Compensation (somewhat outdated) ---
http://www.trinity.edu/rjensen/distcost.htm
Bob Jensen's education technology threads are linked at
http://www.trinity.edu/rjensen/000aaa/0000start.htm
Question
Guess which parents most strongly object to grade inflation?
Hint: Parents Say Schools Game System, Let Kids Graduate Without Skills
The Bredemeyers represent a new voice in special
education: parents disappointed not because their children are failing, but
because they're passing without learning. These families complain that schools
give their children an easy academic ride through regular-education classes,
undermining a new era of higher expectations for the 14% of U.S. students who
are in special education. Years ago, schools assumed that students with
disabilities would lag behind their non-disabled peers. They often were taught
in separate buildings and left out of standardized testing. But a combination of
two federal laws, adopted a quarter-century apart, have made it national policy
to hold almost all children with disabilities to the same academic standards as
other students.
John Hechinger and Daniel Golden, "Extra Help: When Special Education Goes
Too Easy on Students," The Wall Street Journal, August 21, 2007, Page A1
---
http://online.wsj.com/article/SB118763976794303235.html?mod=todays_us_page_one
Bob Jensen's threads on grade inflation are at
http://www.trinity.edu/rjensen/Assess.htm#GradeInflation
Bob Jensen's fraud updates are at
http://www.trinity.edu/rjensen/FraudUpdates.htm
Question
What Internet sites help you compare neighboring K-12 schools?
"Grading Neighborhood Schools: Web Sites Compare A Variety of Data, Looking
Beyond Scores," by Katherine Boehret, The Wall Street Journal, February
20, 2008; Page D6 ---
I performed various school queries
using
Education.com
Inc., GreatSchools Inc.'s
GreatSchools.net and
SchoolMatters.com by typing in a ZIP Code, city,
district or school name. Overall, GreatSchools and Education.com offered the
most content-packed environments, loading their sites with related articles
and offering community feedback on education-related issues by way of blog
posts or surveys. And though GreatSchools is 10 years older than
Education.com, which made its debut in June, the latter has a broader
variety of content and considers its SchoolFinder feature -- newly available
as of today -- just a small part of the site.
Both Education.com and
GreatSchools.net base a good portion of their data on information gathered
by the Department of Education and the National Center for Education
Statistics, the government entity that collects and analyzes data related to
education.
SchoolMatters.com, a service of
Standard & Poor's, is more bare-bones, containing quick statistical
comparisons of schools. (S&P is a unit of McGraw-Hill Cos.) This site gets
its content from various sources, including state departments of education,
private research firms, the Census and National Public Education Finance
Survey. This is evidenced by lists, charts and pie graphs that would make
Ross Perot proud. I learned about where my alma mater high school got its
district revenue in 2005: 83% was local, 15% was state and 2% was federal.
But I couldn't find district financial information for more recent years on
the site.
All three sites base at least some
school-evaluation results on test scores, a point that some of their users
critique. Parents and teachers, alike, point out that testing doesn't always
paint an accurate picture of a school and can be skewed by various
unacknowledged factors, such as the number of students with disabilities.
Education.com's SchoolFinder feature is starting
with roughly 47,000 schools in 10 states: California, Texas, New York,
Florida, Illinois, Pennsylvania, Ohio, Michigan, New Jersey and Georgia. In
about two months, the site hopes to have data for all states, totaling about
60,000 public and charter schools. I was granted early access to
SchoolFinder, but only Michigan was totally finished during my testing.
SchoolFinder lets you narrow your results by type
(public or charter), student-to-teacher ratio, school size or Adequate
Yearly Progress (AYP), a measurement used to determine each school's annual
progress. Search results showed specific details on teachers that I didn't
see on the other sites, such as how many teachers were fully credentialed in
a particular school and the average years of experience held by a school's
teachers.
The rest of the Education.com site contains over
4,000 articles written by well-known education sources like the New York
University Child Study Center, Reading is Fundamental and the Autism Society
of America. It also contains a Web magazine and a rather involved
discussion-board community where members can ask questions of like-minded
parents and the site's experts, who respond with advice and suggestions of
articles that might be helpful.
Private schools aren't required to release test
scores, student or teacher statistics, so none of the sites had as much data
on private schools. However, GreatSchools.net at least offered basic results
for most private-school queries that I performed, such as a search for
Salesianum School in Delaware (where a friend of mine attended) that
returned the school's address, a list of the Advanced Placement exams it
offered from 2006 to 2007 and six rave reviews from parents and former
students.
GreatSchools.net makes it easy to compare schools,
even without knowing specific names. After finding a school, I was able to
easily compare that school with others in the geographic area or school
district -- using a chart with numerous results on one screen. After
entering my email address, I saved schools to My School List for later
reference.
I couldn't find each school's AYP listed on
GreatSchools.net, though these data were on Education.com and
SchoolMatters.com.
SchoolMatters.com doesn't provide articles, online
magazines or community forums. Instead, it spits out data -- and lots of it.
A search for "Philadelphia" returned 324 schools in a neat comparison chart
that could, with one click, be sorted by grade level, reading test scores,
math test scores or students per teacher. (The Julia R. Masterman Secondary
School had the best reading and math test scores in Philadelphia, according
to the site.)
SchoolMatters.com didn't have nearly as much user
feedback as Education.com or GreatSchools.net. But stats like a school's
student demographics, household income distribution and the district's
population age distribution were accessible thanks to colorful pie charts.
These three sites provide a good overall idea of
what certain schools can offer, though GreatSchools.net seems to have the
richest content in its school comparison section. Education.com excels as a
general education site and will be a comfort to parents in search of
reliable advice. Its newly added SchoolFinder, while it's in early stages
now, will only improve this resource for parents and students.
May 2, 2007 message from Carnegie President
[carnegiepresident@carnegiefoundation.org]
A different way to think about ... accountability
Alex McCormick's timely essay brings to our attention one of the most
intriguing paradoxes associated with high-stakes measurement of educational
outcomes. The more importance we place on going public with the results of
an assessment, the higher the likelihood that the assessment itself will
become corrupted, undermined and ultimately of limited value. Some policy
scholars refer to the phenomenon as a variant of "Campbell's Law," named for
the late Donald Campbell, an esteemed social psychologist and methodologist.
Campbell stated his principle in 1976: "The more any quantitative social
indicator is used for social decisionmaking, the more subject it will be to
corruption pressures and the more apt it will be to distort and corrupt the
social processes it is intended to monitor."
In the specific case of the Spellings Commission
report, Alex points out that the Secretary's insistence that information be
made public on the qualities of higher education institutions will place
ever higher stakes on the underlying measurements, and that very visibility
will attenuate their effectiveness as accountability indices. How are we to
balance the public's right to know with an institution's need for the most
reliable and valid information? Alex McCormick's analysis offers us another
way to think about the issue.
Carnegie has created a forum—Carnegie
Conversations—where you can engage publicly with the author and read and
respond to what others have to say about this article at
http://www.carnegiefoundation.org/perspectives/april2007 .
Or you may respond to Alex privately through
carnegiepresident@carnegiefoundation.org .
If you would like to unsubscribe to Carnegie
Perspectives, use the same address and merely type "unsubscribe" in the
subject line of your email to us.
We look forward to hearing from you.
Sincerely,
Lee S. Shulman
President The Carnegie Foundation for the Advancement of Teaching
Jensen Comment
The fact that an assessment provides incentives to cheat is not a reason to not
assess. The fact that we assign grades to students gives them incentives to
cheat. That does not justify ceasing to assess, because the assessment process
is in many instances the major incentive for a student to work harder and learn
more. The fact that business firms have to be audited and produce financial
statements provides incentives to cheat. That does not justify not holding
business firms accountable. Alex McCormick's analysis and Shulman's concurrence
is a bit one-sided in opposing the Spellings Commission recommendations.
Also see Full Disclosure to Consumers of Higher Education at
http://www.trinity.edu/rjensen/HigherEdControversies.htm#FullDisclosure
School Assessment and College Admission Testing
July 25, 2006 query from Carol Flowers
[cflowers@OCC.CCCD.EDU]
I am looking for a study that I saw. I was unsure
if someone in this group had supplied the link, originally. It was a very
honest and extremely comprehensive evaluation of higher education. In it,
the
Higher Education Evaluation and Research Group was
constantly quoted. But, what organizations it is affiliated with, I am
unsure.
They commented on the lack of student academic
preparedness in our educational system today along with other challenging
areas that need to be addressed inorder to serve the population with which
we now deal.
If anyone remembers such a report, please forward
to me the url.
Thank You!
July 25, 2006 reply from Bob Jensen
Hi Carol,
I think the HEERG is
affiliated with the Chancellor's Office of the California Community
Colleges. It is primarily focused upon accountability and assessment of
these colleges.
HEERG ---
http://snipurl.com/HEERG
Articles related to your query include the
following:
Leopards in the Temple ---
http://www.insidehighered.com/views/2006/06/12/caesar
Accountability, Improvement and Money ---
http://www.insidehighered.com/views/2005/05/03/lombardi
Grade Inflation and Abdication ---
http://www.insidehighered.com/views/2005/06/03/lombardi
Students Read Less. Should We Care? ---
http://www.insidehighered.com/views/2005/08/23/lombardi
Missing the Mark: Graduation Rates and University
Performance ---
http://www.insidehighered.com/views/2005/02/14/lombardi2
Assessment of Learning Achievements of College Graduates
"Getting the Faculty On Board," by Freeman A. Hrabowski III, Inside Higher
Ed, June 23, 2006 ---
http://www.insidehighered.com/views/2006/06/23/hrabowski
But as assessment becomes a national imperative,
college and university leaders face a major challenge: Many of our faculty
colleagues are skeptical about the value of external mandates to measure
teaching and learning, especially when those outside the academy propose to
define the measures. Many faculty members do not accept the need for
accountability, but the assessment movement’s success will depend upon
faculty because they are responsible for curriculum, instruction and
research. All of us — policy makers, administrators and faculty — must work
together to develop language, strategies and practices that help us
appreciate one another and understand the compelling need for assessment —
and why it is in the best interest of faculty and students.
Why is assessment important? We know from the work
of researchers like Richard Hersh, Roger Benjamin, Mark Chun and George Kuh
that college enrollment will be increasing by more than 15 percent
nationally over the next 15 years (and in some states by as much as 50
percent). We also know that student retention rates are low, especially
among students of color and low-income students. Moreover, of every 10
children who start 9th grade, only seven finish high school, five start
college, and fewer than three complete postsecondary degrees. And there is a
20 percent gap in graduation rates between African Americans (42 percent)
and whites (62 percent). These numbers are of particular concern given the
rising higher education costs, the nation’s shifting demographics, and the
need to educate more citizens from all groups.
At present, we do not collect data on student
learning in a systematic fashion and rankings on colleges and universities
focus on input measures, rather than on student learning in the college
setting. Many people who have thought about this issue agree: We need to
focus on “value added” assessment as an approach to determine the extent to
which a university education helps students develop knowledge and skills.
This approach entails comparing what students know at the beginning of their
education and what they know upon graduating. Such assessment is especially
useful when large numbers of students are not doing well — it can and should
send a signal to faculty about the need to look carefully at the “big
picture” involving coursework, teaching, and the level of support provided
to students and faculty.
Many in the academy, however, continue to resist
systematic and mandated assessment in large part because of problems they
see with K-12 initiatives like No Child Left Behind — e.g., testing that
focuses only on what can be conveniently measured, unacceptable coaching by
teachers, and limiting what is taught to what is tested. Many academics
believe that what is most valuable in the college experience cannot be
measured during the college years because some of the most important effects
of a college education only become clearer some time after graduation.
Nevertheless, more institutions are beginning to understand that value-added
assessment can be useful in strengthening teaching and learning, and even
student retention and graduation rates.
It is encouraging that a number of institutions are
interested in implementing value-added assessment as an approach to evaluate
student progress over time and to see how they compare with other
institutions. Such strategies are more effective when faculty and staff
across the institution are involved. Examples of some best practices include
the following:
- Constantly talking with colleagues about both
the challenges and successful initiatives involving undergraduate
education.
- Replicating successful initiatives (best
practices from within and beyond the campus), in order to benefit as
many students as possible.
- Working continuously to improve learning based
on what is measured — from advising practices and curricular issues to
teaching strategies — and making changes based on what we learn from
those assessments.
- Creating accountability by ensuring that
individuals and groups take responsibility for different aspects of
student success.
- Recruiting and rewarding faculty who are
committed to successful student learning (including examining the
institutional reward structure).
- Taking the long view by focusing on
initiatives over extended periods of time — in order to integrate best
practices into the campus culture.
We in the academy need to think broadly about
assessment. Most important, are we preparing our students to succeed in a
world that will be dramatically different from the one we live in today?
Will they be able to think critically about the issues they will face,
working with people from all over the globe? It is understandable that
others, particularly outside the university, are asking how we demonstrate
that our students are prepared to handle these issues.
Assessment is becoming a national imperative, and
it requires us to listen to external groups and address the issues they are
raising. At the same time, we need to encourage and facilitate discussions
among our faculty — those most responsible for curriculum, instruction, and
research — to grapple with the questions of assessment and accountability.
We must work together to minimize the growing tension among groups — both
outside and inside the university — so that we appreciate and understand
different points of view and the compelling need for assessment.
Bob Jensen's threads on controversies in higher education are at
http://www.trinity.edu/rjensen/HigherEdControversies.htm
NCLB = No Child Left Behind Law
A September 2007 Thomas B. Fordham Institute report
found NCLB's assessment system "slipshod" and characterized by "standards that
are discrepant state to state, subject to subject, and grade to grade." For
example, third graders scoring at the sixth percentile on Colorado's state
reading test are rated proficient. In South Carolina the third grade proficiency
cut-off is the sixtieth percentile.
Peter Berger, "Some Will Be Left
Behind," The Irascible Professor, November 10, 2007 ---
http://irascibleprofessor.com/comments-11-10-07.htm
"This is Only a Test," by Peter Berger, The Irascible
Professor, December 5, 2005 ---
http://irascibleprofessor.com/comments-12-05-05.htm
Back in 2002 President Bush predicted "great
progress" once schools began administering the annual testing regime
mandated by No Child Left Behind. Secretary of Education Rod Paige echoed
the President's sentiments. According to Mr. Paige, anyone who opposed NCLB
testing was guilty of "dismissing certain children" as "unteachable."
Unfortunately for Mr. Paige, that same week The New
York Times documented "recent" scoring errors that had "affected millions of
students" in "at least twenty states." The Times report offered a pretty
good alternate reason for opposing NCLB testing. Actually, it offered
several million pretty good alternate reasons.
Here are a few more.
There's nothing wrong with assessing what students
have learned. It lets parents, colleges, and employers know how our kids are
doing, and it lets teachers know which areas need more teaching. That's why
I give quizzes and tests and one of the reasons my students write essays.
Of course, everybody who's been to school knows
that some teachers are tougher graders than others. Traditional standardized
testing, from the Iowa achievement battery to the SATs, was supposed to help
us gauge the value of one teacher's A compared to another's. It provided a
tool with which we could compare students from different schools.
This works fine as long as we recognize that all
tests have limitations. For example, for years my students took a nationwide
standardized social studies test that required them to identify the
President who gave us the New Deal. The problem was the seventh graders who
took the test hadn't studied U.S. history since the fifth grade, and FDR
usually isn't the focus of American history classes for ten-year-olds. He
also doesn't get mentioned in my eighth grade U.S. history class until May,
about a month after eighth graders took the test.
In other words, wrong answers about the New Deal
only meant we hadn't gotten there yet. That's not how it showed up in our
testing profile, though. When there aren't a lot of questions, getting one
wrong can make a surprisingly big difference in the statistical soup.
Multiply our FDR glitch by the thousands of
curricula assessed by nationwide testing. Then try pinpointing which schools
are succeeding and failing based on the scores those tests produce. That's
what No Child Left Behind pretends to do.
Testing fans will tell you that cutting edge
assessments have eliminated inconsistencies like my New Deal hiccup by
"aligning" the tests with new state of the art learning objectives and grade
level expectations. The trouble is these newly minted goals are often
hopelessly vague, arbitrarily narrow, or so unrealistic that they're pretty
meaningless. That's when they're not obvious and the same as they always
were.
New objectives also don't solve the timing problem.
For example, I don't teach poetry to my seventh grade English students.
That's because I know that their eighth grade English teacher does an
especially good job with it the following year, which means that by the time
they leave our school, they've learned about poetry. After all, does it
matter whether they learn to interpret metaphors when they're thirteen or
they're fourteen as long as they learn it?
Should we change our program, which matches our
staff's expertise, just to suit the test's arbitrary timing? If we don't,
our seventh graders might not make NCLB "adequate yearly progress." If we
do, our students likely won't learn as much.
Which should matter more?
Even if we could perfectly match curricula and test
questions, modern assessments would still have problems. That's because most
are scored according to guidelines called rubrics. Rubric scoring requires
hastily trained scorers, who typically aren't teachers or even college
graduates, to determine whether a student's essay "rambles" or "meanders."
Believe it or not, that choice represents a twenty-five percent variation in
the score. Or how about distinguishing between "appropriate sentence
patterns" and "effective sentence structure," or language that's "precise
and engaging" versus "fluent and original."
These are the flip-a-coin judgments at the heart of
most modern assessments. Remember that the next time you read about which
schools passed and which ones failed.
Unreliable scoring is one reason the General
Accountability Office condemned data "comparisons between states" as
"meaningless." It's why CTB/McGraw-Hill had to recall and rescore 120,000
Connecticut writing tests after the scores were released. It's why New York
officials discarded the scores from its 2003 Regents math exam. A 2001
Brookings Institution study found that "fifty to eighty percent of the
improvement in a school's average test scores from one year to the next was
temporary" and "had nothing to do with long-term changes in learning or
productivity." A senior RAND analyst warned that today's tests aren't
identifying "good schools" and "bad schools." Instead, "we're picking out
lucky and unlucky schools."
Students aren't the only victims of faulty scoring.
Last year the Educational Testing Service conceded that more than ten
percent of the candidates taking its 2003-2004 nationwide Praxis teacher
licensing exam incorrectly received failing scores, which resulted in many
of them not getting jobs. ETS attributed the errors to the "variability of
human grading."
The New England Common Assessment Program,
administered for NCLB purposes to all students in Vermont, Rhode Island, and
New Hampshire, offers a representative glimpse of the cutting edge. NECAP is
heir to all the standard problems with standardized test design, rubrics,
and dubiously qualified scorers.
NECAP security is tight. Tests are locked up, all
scrap paper is returned to headquarters for shredding, and testing scripts
and procedures are painstakingly uniform. Except on the mathematics exam,
each school gets to choose if its students can use calculators.
Whether or not you approve of calculators on math
tests, how can you talk with a straight face about a "standardized" math
assessment if some students get to use them and others don't? Still more
ridiculous, there's no box to check to show whether you used one or not, so
the scoring results don't even differentiate between students and schools
that did and didn't.
Finally, guess how NECAP officials are figuring out
students' scores. They're asking classroom teachers. Five weeks into the
year, before we've even handed out a report card to kids we've just met,
we're supposed to determine each student's "level of proficiency" on a
twelve point scale. Our ratings, which rest on distinguishing with allegedly
statistical accuracy between "extensive gaps," "gaps," and "minor gaps," are
a "critical piece" and "key part of the NECAP standard setting process."
Let's review. Because classroom teachers' grading
standards aren't consistent enough from one school to the next, we need a
standardized testing program. To score the standardized testing program,
every teacher has to estimate within eight percentage points how much their
students know so test officials can figure out what their scores are worth
and who passed and who failed.
If that makes sense to you, you've got a promising
future in education assessment. Unfortunately, our schools and students
don't.
"College Board Asks Group Not to Post Test Analysis," by Diana Jean
Schemol, The New York Times, December 4, 2004 --- http://www.nytimes.com/2004/12/04/education/04college.html?oref=login
The College Board, which owns the SAT college
entrance exam, is demanding that a nonprofit group critical of standardized
tests remove from its Web site data that breaks down scores by race, income
and sex.
The demand, in a letter to The National Center for
Fair and Open Testing, also known as FairTest, accuses the group of infringing
on the College Board's copyright.
"Unfortunately, your misuse overtly bypasses our
ownership and significantly impacts the perceptions of students, parents and
educators regarding the services we provide," the letter said.
The move by the College Board comes amid growing
criticism of the exams, with more and more colleges and universities raising
questions about their usefulness as a gauge of future performance and
discarding them as requirements for admission. The College Board is
overhauling parts of the exam and will be using a new version beginning in
March
FairTest has led opposition to the exams, and
releases the results to support its accusation of bias in the tests, a claim
rejected by test makers, who contend the scores reflect true disparities in
student achievement. FairTest posts the information in easily accessible
charts, and Robert A. Schaeffer, its spokesman, said they were the Web site's
most popular features.
In its response to the College Board letter, which
FairTest posted on its Web site on Tuesday, the group said it would neither
take down the data nor seek formal permission to use it. FairTest has been
publicly showing the data for nearly 20 years, Mr. Schaeffer said, until now
without objection from the testing company, which itself releases the data in
annual reports it posts on its Web site.
"You can't copyright numbers like that,"
Mr. Schaeffer said. "It's all about public education and making the
public aware of score gaps and the potential for bias in the exams."
Devereux Chatillon, a specialist on copyright law at
Sonnenschein, Nath & Rosenthal in New York, said case law supported
FairTest's position. "Facts are not copyrightable," Ms. Chatillon
said. In addition, she said, while the College Board may own the exam, the
real authors of the test results are those taking the exams.
Continued in article
2004 Senior Test Scores: ACT --- http://www.fairtest.org/nattest/ACT%20Scores%202004%20Chart.pdf
2004 Senior Test Scores: SAT --- http://www.fairtest.org/nattest/SAT%20Scoresn%202004%20Chart.pdf
Fair Test Reacts to the SAT Outcomes --- http://www.fairtest.org/univ/2004%20SAT%20Score%20Release.html
Fair Test Home --- http://www.fairtest.org/
Jensen Comment:
If there is to be a test that sets apart students that demonstrate higher
ability, motivation, and aptitude for college studies, how would it differ from
the present Princeton tests that have been designed and re-designed over and
over again? I cannot find any Fair Test models of what such a test would
look like. One would assume that by its very name Fair Test still agrees
that some test is necessary. However, the group's position seems to
be that no national test is feasible that will give the same means and standard
deviations for all groups (males, females, and race categories). Fair Test
advocates "assessments based on students' actual performances, not
one-shot, high-stakes exams."
Texas has such a Fair Test system in place for admission to any state
university. The President of the University of Texas, however, wants the
system to be modified since his top-rated institution is losing all of its
admission discretion and may soon be overwhelmed with more admissions than can
be seated in classrooms. My module on this issue, which was a special
feature on 60 Minutes from CBS, is at http://www.trinity.edu/rjensen/book04q4.htm#60Minutes
The problem with performance-based systems (such as the requirement that any
state university in Texas must accept any graduate in the top 10% of the
graduating class from any Texas high school) is that high schools in the U.S.
generally follow the same grading scale as Harvard University. Most
classes give over half the students A grades. Some teachers give A grades
just for attendance or effort apart from performance. This means that when
it comes to isolating the top 10% of each graduating class, we're talking in
terms of Epsilon differences. I hardly think Epsilon is a fair criterion
for admission to college. Also, as was pointed out on 60 Minutes,
students with 3.9 grade averages from some high schools tend to score much lower
than students with 3.0 grade averages from other high schools. This might
achieve better racial mix but hardly seems fair to the 3.0 student who was
unfortunate enough to live near a high school having a higher proportion of top
students. That was the theme
of the 60 Minutes CBS special contrasting a 3.9 low SAT student who got
into UT versus a 3.0 student who had a high SAT but was denied admission to UT.
What we really need is to put more resources into fair chances for those who
test poorly or happen to fall Epsilon below that hallowed 10% cut off. in a
performance-based system. This may entail more time and remedial effort on
the part of students before or after entering college.
Mount Holyoke Dumps the SAT
Mount Holyoke College, which decided in 2001 to make
the SAT optional, is finding very little difference in academic performance
between students who provided their test scores and those who didn't. The
women's liberal arts college is in the midst of one of the most extensive
studies to date about the impact of dropping the SAT -- a research project
financed with $290,000 from the Mellon Foundation. While the study isn't
complete, the college is releasing some preliminary results. So far, Mount
Holyoke has found that there is a difference of 0.1 point in the grade-point
average of those who do and do not submit SAT scores. That is equivalent to
approximately one letter grade in one course over a year of study. Those
results are encouraging to Mount Holyoke officials about their decision in 2001.
Scott Jaschik, "Not Missing the SAT," Inside Higher Ed March 9, 2005
--- http://www.insidehighered.com/insider/not_missing_the_sat
Jensen Comment:
These results differ from the experiences of the University of Texas system
where grades and test scores differ greatly between secondary
schools. Perhaps Mount Holyoke is not getting applications from
students in the poorer school districts. See http://www.trinity.edu/rjensen/book04q4.htm#60Minutes
Dangers of Self Assessment
My undergraduate students can’t accurately predict
their academic performance or skill levels. Earlier in the semester, a writing
assignment on study styles revealed that 14 percent of my undergraduate English
composition students considered themselves “overachievers.” Not one of those
students was receiving an A in my course by midterm. Fifty percent were
receiving a C, another third was receiving B’s and the remainder had earned
failing grades by midterm. One student wrote, “overachievers like myself began a
long time ago.” She received a 70 percent on her first paper and a low C at
midterm.
Shari Wilson, "Ignorant of Their
Ignorance," Inside Higher Ed, November 16, 2006 ---
http://www.insidehighered.com/views/2006/11/16/wilson
Jensen comment
This does not bode well for self assessment.
Do middle-school students understand how well they actually learn?
Given national mandates to ‘leave no child behind,’
grade-school students are expected to learn an enormous amount of course
material in a limited amount of time. “Students have too much to learn, so it’s
important they learn efficiently,” says Dr. John Dunlosky, Kent State professor
of psychology and associate editor of Journal of Experimental Psychology:
Learning, Memory and Cognition. Today, students are expected to understand and
remember difficult concepts relevant to state achievement tests. However, a
major challenge is the student’s ability to judge his own learning. “Students
are extremely over confident about what they’re learning,” says Dunlosky.
Dunlosky and his colleague, Dr. Katherine Rawson, Kent State assistant professor
of psychology, study metacomprehension, or the ability to judge your own
comprehension and learning of text materials. Funded by the U.S. Department of
Education, their research primarily focuses on fifth, seventh and eighth graders
as well as college-aged students, and how improving metacomprehension can, in
turn, improve students’ self-regulated learning.
PhysOrg, November 26, 2007 ---
http://physorg.com/news115318315.html
AICPA Educational Competency Assessment for
Accounting Students
Educational
Competency Assessment (ECA) Web Site --- http://www.aicpa-eca.org/
The AICPA recently won a National Association of Colleges and Employers (NACE)
Excellence Award for Educational Programming for developing this ECA site to
help accounting educators integrate the skill-based competencies needed by
entry-level accounting professionals.
The AICPA provides this resource to help educators
integrate the skills-based competencies needed by entry-level accounting
professionals. These competencies, defined within the AICPA Core Competency
Framework Project, have been derived from academic and professional competency
models and have been widely endorsed within the academic community. Created by
educators for educators, the evaluation and educational strategies resources
on this site are offered for your use and adaptation.
The ECA site contains a LIBRARY that, in addition to
the Core Competency Database and Education Strategies, provides information
and guidance on Evaluating Competency Coverage and Assessing Student
Performance.
To assist you as you assess student performance and
evaluate competency coverage in your courses and programs, the ECA ORGANIZERS
guide you through the process of gathering, compiling and analyzing evidence
and data so that you may document your activities and progress in addressing
the AICPA Core Competencies.
Online Education Effectiveness and Testing
Learning Effectiveness in Corporate Universities
A
group of colleges that serve adult students on Monday
formally announced their effort
to measure and report their effectiveness, focusing on outcomes in specific
programs. The initiative known as “Transparency by Design,
on which Inside Higher Ed reported earlier,
has grown to include a mix of 10 nonprofit and for-profit institutions: Capella
University, Charter Oak State College, Excelsior College, Fielding Graduate
University, Franklin University, Kaplan University, Regis University, Rio Salado
College, Western Governors University, and Union Institute & University.
Inside Higher Ed, October 23, 2007 ---
http://www.insidehighered.com/news/2007/10/23/qt
"Keeping an Eye on Online Students," by Andrea L. Foster, Chronicle of
Higher Education, July 21, 2008 ---
http://chronicle.com/wiredcampus/index.php?id=3181&utm_source=wc&utm_medium=en
Technology vendors are eager to sell college
officials hardware and software designed to verify the identify of online
students—and thereby prevent cheating. A free article in The Chronicle
describes some of the technologies that colleges are trying out to make
certain that the person taking an online exam is, in fact, the student
enrolled in the course. The technologies include Web cameras that watch
students taking tests and scanners that capture students’ fingerprints.
A provision in a bill reauthorizing the Higher
Education Act is fueling much of the interest in this issue. A paper
released in February by the Western Interstate Commission for Higher
Education says the provision—while not onerous to most distance-learning
providers—could “drive up the cost of these important education programs.”
And some online institutions fear that the
provision would require them to have their students travel to distant
locations to take proctored exams on paper. The result? Some states would
conclude that the institutions have a “physical presence” in their states,
and would subject the institutions to “a whole new set of state
regulations,” says John F. Ebersole, president of Excelsior College.
"Keeping an Eye on Online Students," by Andrea L. Foster, Chronicle of
Higher Education, July 21, 2008 ---
http://chronicle.com/wiredcampus/index.php?id=3181&utm_source=wc&utm_medium=en
Technology vendors are eager to sell college
officials hardware and software designed to verify the identify of online
students—and thereby prevent cheating. A free article in The Chronicle
describes some of the technologies that colleges are trying out to make
certain that the person taking an online exam is, in fact, the student
enrolled in the course. The technologies include Web cameras that watch
students taking tests and scanners that capture students’ fingerprints.
A provision in a bill reauthorizing the Higher
Education Act is fueling much of the interest in this issue. A paper
released in February by the Western Interstate Commission for Higher
Education says the provision—while not onerous to most distance-learning
providers—could “drive up the cost of these important education programs.”
And some online institutions fear that the
provision would require them to have their students travel to distant
locations to take proctored exams on paper. The result? Some states would
conclude that the institutions have a “physical presence” in their states,
and would subject the institutions to “a whole new set of state
regulations,” says John F. Ebersole, president of Excelsior College.
Question
What are some of the features of UserView from TechSmith for evaluating student
learning
Some of the reviews of the revised “free” Sound Recorder in Windows Vista are
negative. It’s good to learn that Richard Campbell is having a good experience
with it when recording audio and when translating the audio into text files ---
http://microsoft.blognewschannel.com/archives/2006/05/24/windows-vista-sound-recorder
For those of you on older systems as well as Vista there is a free recorder
called Audacity that I like ---
http://audacity.sourceforge.net/
I really like Audacity. There are some Wiki tutorials at
http://audacity.sourceforge.net/help/tutorials
Some video tutorials are linked at
http://youtube.com/results?search_query=audacity+tutorial&search=Search
I have some dated threads on speech recognition at
http://www.trinity.edu/rjensen/speech.htm Mac users can find options
at http://www.macspeech.com/
In addition, I like Camtasia (recording screen shots and camera video) and
Dubit (for recording audio and editing audio) from TechSmith ---
http://www.techsmith.com/
TechSmith products are very good, but they are not free downloads.
UserView ---
http://www.techsmith.com/uservue/features.asp
TechSmith has a newer product called UserView that really sounds exciting,
although I’ve not yet tried it. It allows you to view and record what is
happening on someone else’s computer like a student’s computer. Multiple
computers can be viewed at the same time. Images and text can be recorded.
Pop-up comments can be inserted by the instructor to text written by students.
UserView can be used for remote testing!
Userview offers great hope for teaching disabled students such as sight
and/or hearing impaired students ---
http://www.trinity.edu/rjensen/000aaa/thetools.htm#Handicapped
"Ways to prevent cheating on online exams," by Gail E. Krovitz,
eCollege Newsletter, Vol 8, Issue 6 November 15, 2007 ---
http://www.ecollege.com/Educators_Voice.learn
- Write
every exam as if it is open book. As much as we try to
convince ourselves otherwise, we need to assume that students
use resources on their exams (the book, Internet search engines
and so on) and write our exams accordingly. Are all of our
questions asking for information that can be gathered quickly
from the textbook or from a simple Internet search? Then we
should re-think our questions (see following guideline).
Open-book exams have the potential to test higher level thinking
skills, instead of just memorizing facts. Unfortunately, scores
on open-book exams are often lower, as students don’t take exam
preparation as seriously when they know they can use their book,
so training in open-book exam-taking skills would be helpful
(Rakes).
- Write
effective multiple-choice exam questions. Because it is
so easy to use prohibited materials during online exams, it is
foolish to design tests that simply test factual information
that is easily looked up. Although it is difficult to do, online
exams are most effective when they test higher order thinking
skills (application, synthesis and evaluation) and ask questions
that cannot be answered by glancing at the book or a quick
internet search. See Christe, Dewey and Rohrer for more
information about developing quality multiple-choice questions.
- Set
tight time limits per question. Even with open book
exams (and especially for ones that are not open book), it is
important to give a tight time frame for the test, so students
will not have time to look up each question in the book. The
time limit chosen will obviously vary depending on subject
matter, type of questions asked, etc. For strict fact recall,
instructors might start by giving a total time based on allowing
60- 90 seconds per question and then adjusting as necessary
based on their student body. More time would need to be given
for higher-level thinking questions or for those involving
calculations.
- Use
large question pools to offer different, randomly-selected
questions to each student. See “Tip: getting the most
out of exam question pools” for a good description of using
question pools in the eCollege system. The question pools must
be large enough to minimize overlap of questions between tests.
Rowe provides a chart comparing the average number of questions
in common for two students with different question pool sizes
and different numbers of questions drawn from the pool. For
example, 5 questions drawn from a pool of 10 questions results
in 2.5 questions in common between two students, while 5
questions drawn from a pool of 25 questions results in only 1
question in common between two students. You can consult the
mathematical formula or go with common sense: a larger question
pool is better for reducing the likelihood that students will
get the same questions.
-
Manually create different versions of the exam with the same
general question pools, but with scrambled answers for each
question. For example, in one version of the exam, the
correct answer could be B, while the answer choices are
scrambled in the other version so the correct answer is D. You
could use the Group function to assign half of the class to one
exam, and the other half the class to the other one. Cizek cites
research showing that scrambling questions and answer choices
does reduce cheating, while simply changing the order of the
same questions does not reduce cheating. In fact, in a study of
student’s perceived effectiveness of cheating prevention
strategies, having scrambled test forms was the number one
factor perceived by students to prevent cheating (Cizek).
- Assign
a greater number of smaller tests instead of one or two large
ones. This reduces the incentive to cheat, as each test
isn’t as likely to make or break a student’s grade; the pressure
of the midterm and final-only structure in some classes is a
strong incentive to cheat on those exams. Also, this increases
the logistical difficulties of cheating if a student is relying
on someone else to help them or to take the test for them.
- Provide
a clear policy for what happens if students cheat… and enforce
it! There are many important things instructors can do
from this perspective, such as discussing what constitutes
cheating, the importance of academic honesty, any honor codes in
place, what measures will be in place to prevent and detect
cheating and the punishments for cheating. If students perceive
that the instructor does not care about cheating, then incidents
of both spontaneous and planned cheating increase (Cizek).
Students know that most cheaters don’t get caught and that
punishments aren’t harsh for those who do get caught (Kleiner
and Lord). Research has found that punishment for cheating is
one of the main deterrents to cheating (Kleiner and Lord).
- Set the
exam Gradebook Review Date for after the exam has closed.
The Gradebook Review Date is when the students can access their
graded exam in the Gradebook. If this date is set before the end
of the exam, students who take the exam early could access their
exam in the Gradebook (and usually the correct answers as well)
and distribute the questions to students who would take the exam
later.
- Revise
tests every term. Sooner or later exam questions are
likely to get out into the student world and get distributed
between students. This is especially possible when students view
their graded exams in the Gradebook, as they have all the time
in the world to copy or print their questions (usually with the
correct answers provided). Periodic changes to the test bank can
help minimize the impact of this. Minor changes such as
rewording the questions and changing the order of answers
(especially if different versions with scrambled answers are not
used) can help extend the useful life of a test bank.
- Use
ExamGuardTM if the feature is available at
your school. ExamGuard prohibits the following actions while
students are taking online exams: printing, copying and pasting
anything into or from the assessment, surfing the Web, opening
or using other applications, using Windows system keys functions
or clicking on any other area within the course. Also note that
ExamGuard prohibits students from printing or copying exam
materials while viewing the exam in the Gradebook. If you are
interested in learning more about ExamGuard, please contact your
Account Executive or Client Services Consultant.
- Give
proctored exams in a traditional classroom. While this
is not an option for many online courses, it is a route that
some schools take, especially if they largely serve a local
population. With proctored exams, instructors feel more in
control of the testing environment and more able to combat
cheating in a familiar classroom setting (or at least to have
cheating levels on par with those seen in a traditional exam
setting). In a study on cheating in math or fact-based courses,
Trenholm concludes that proctoring is “the single greatest tool
we presently have to uphold the integrity of the educational
process in instruction in online MFB (math or fact based)
courses” (p. 297). Also, Cizek showed that attentive proctoring
reduced cheating directly and by giving the impression that
academic integrity is valued.
December 1, 2007 reply from Charles Wankel
[wankelc@VERIZON.NET]
Thanks Bob for sharing.
Some of the points seem to fall back to
face-to-face course ideas but others were very helpful. I found the emphasis
on higher order thinking skills (application, synthesis and evaluation) to
be a great one. I am going to try to work on putting synthesis into my
students’ assignments and projects.
Charlie Wankel
St. John’s University,
New York
December 1, 2007 reply from David Raggay
[draggay@TSTT.NET.TT]
Please be so kind as to refer me to the specific
article or articles wherein I can find a discussion on “higher order
thinking skills (application, synthesis and evaluation)”
Thanks,
David Raggay,
IFRS Consultants,
Trinidad and Tobago
December 1, 2007 reply from Bob Jensen
Hi David,
There are several tacks to take on this question. Charlie provides some
key words (see above).
I prefer to think of higher order metacognition ---
http://en.wikipedia.org/wiki/Metacognition
For specific examples in accounting education see
http://www.trinity.edu/rjensen/265wp.htm
One of the main ideas is to make students do their own discovery learning.
Blood, sweat, and tears are the best teachers.
Much of the focus in metacognitive learning is how to examine/discover
what students have learned on their own and how to control cheating when
assessing discovery and concept learning ---
http://www.trinity.edu/rjensen/assess.htm
Higher order learning attempts to make students think more conceptually.
In particular, note the following quotation from Bob Kennelly at
http://www.trinity.edu/rjensen/assess.htm#ConceptKnowledge
We studied whether instructional material that connects accounting
concept discussions with sample case applications through hypertext links
would enable students to better understand how concepts are to be applied to
practical case situations.
Results from a laboratory experiment indicated that students who learned
from such hypertext-enriched instructional material were better able to
apply concepts to new accounting cases than those who learned from
instructional material that contained identical content but lacked the
concept-case application hyperlinks.
Results also indicated that the learning benefits of concept-case
application hyperlinks in instructional material were greater when the
hyperlinks were self-generated by the students rather than inherited from
instructors, but only when students had generated appropriate links.
Along broader lines we might think of it in terms of self-organizing of
atomic-level knowledge ---
http://en.wikipedia.org/wiki/Self-organization
Issues are still in great dispute on the issues of over 80 suggested
“learning styles” ---
http://en.wikipedia.org/wiki/Learning_styles
Assessment and control of
cheating are still huge problems.
Bob Jensen
December 2, 2007 reply from Henry Collier
[henrycollier@aapt.net.au]
G’day Bob …
I’m not sure whether David is asking for the Bloom citation or not. I do not
disagree with your post in any way, but wonder if David is looking for the
‘start’ of the art/science. I have also suggested that he may want to look
at Bob Gagne’s approach to the same issues. Perhaps William Graves Perry’s
1970 book could / would also be useful.
Best
regards from spring time in New South Wales where the roses in my garden are
blooming and very pretty.
Henry
New Technology for Proctoring Distance Education Examinations
"Proctor 2.0," by Elia Powers, Inside Higher Ed, June 2, 2006 ---
http://www.insidehighered.com/news/2006/06/02/proctor
Bob Jensen's threads on online versus onsite assessment are at
http://www.trinity.edu/rjensen/assess.htm#OnsiteVersusOnline
Bob Jensen's threads on cheating are at
http://www.trinity.edu/rjensen/Plagiarism.htm
Accounting Professors in Support of Online Testing That, Among Other
Things, Reduces Cheating
These same professors became widely known for their advocacy of self-learning in
place of lecturing
"In Support of the E-Test," by Elia Powers, Inside Higher Ed, August
29, 2007 ---
http://www.insidehighered.com/news/2007/08/29/e_test
Critics
of testing through the computer often argue that it’s
difficult to tell if students are doing their own work. It’s
also unclear to some professors whether using the technology
is worth their while.
A new study makes the argument that giving
electronic tests can actually reduce cheating and save
faculty time.
Anthony
Catanach Jr. and Noah Barsky, both associate professors of
accounting at the Villanova School of Business, came to that
conclusion after speaking with faculty members and analyzing
the responses of more than 100 students at Villanova and
Philadelphia University. Both Catanach and Barsky teach a
course called Principles of Managerial Accounting that
utilizes the WebCT Vista e-learning platform. The professors
also surveyed undergraduates at Philadelphia who took tests
electronically.
The
Villanova course follows a pattern of Monday lecture,
Wednesday case assignment, Friday assessment. The first two
days require in-person attendance, while students can check
in Friday from wherever they are.
“It never
used to make sense to me why at business schools you have
Friday classes,” Catanach said. “As an instructor it’s
frustrating because 30 percent of the class won’t show up,
so you have to redo material. We said, how can we make that
day not lose its effectiveness?”
The answer,
he and Barsky determined, was to make all electronically
submitted group work due on Fridays and have that be
electronic quiz day. That’s where academic integrity came
into play. Since the professors weren’t requiring students
to be present to take the exams, they wanted to deter
cheating. Catanach said programs like the one he uses
mitigate the effectiveness of looking up answers or
consulting friends.
In
electronic form, questions are given to students in random
order so that copying is difficult. Professors can change
variables within a problem to make sure that each test is
unique while also ensuring a uniform level of difficulty.
The programs also measure how much time a student spends on
each question, which could signal to an instructor that a
student might have slowed to use outside resources.
Backtracking on questions generally is not permitted.
Catanach said he doesn’t pay much attention to time spent on
individual questions. And since he gives his students a
narrow time limit to finish their electronic quizzes,
consulting outside sources would only lead students to be
rushed by the end of the exam, he added.
Forty-five
percent of students who took part in the study reported that
the electronic testing system reduced the likelihood of
their cheating during the course.
Stephen
Satris, director of the Center for Academic Integrity at
Clemson University, said he applauds the use of technology
to deter academic dishonesty. Students who take these
courses might think twice about copying or plagiarizing on
other exams, he said.
“It’s good
to see this program working,” Satris said. “It does an end
run around cheating.”
The report
also makes the case that both faculty and students save time
with e-testing. Catanach is up front about the initial time
investment: For instructors to make best use of the testing
programs, they need to create a “bank” of exam questions and
code them by topic, learning objectives and level of
difficulty. That way, the program knows how to distribute
questions. (He said instructors should budget roughly 10
extra hours per week during the course for this task.)
The payoff,
he said, comes later in the term. In the study, professors
reported recouping an average of 80 hours by using the
e-exams. Faculty don’t have to hand-grade tests (that often
being a deterrent for the Friday test, Catanach notes), and
graduate students or administrative staff can help prepare
the test banks, the report points out.
Since tests
are taken from afar, class time can be used for other
purposes. Students are less likely to ask about test results
during sessions, the study says, because the computer
program gives them immediate results and points to pages
where they can find out why their answers were incorrect.
Satris said this type of system likely dissuades students
from grade groveling, because the explanations are all there
on the computer. He said it also make sense in other ways.
“I like that
professors can truly say, ‘I don’t know what’s going to be
on the test. There’s a question bank; it’s out of my
control,’ ” he said.
And then
there’s the common argument about administrative efficiency:
An institution can keep a permanent electronic record of its
students.
Survey
results showed that Villanova students, who Catanach said
were more likely to have their own laptop computers and be
familiar with e-technology, responded better to the
electronic testing system than did students at Philadelphia,
who weren’t as tech savvy. Both Catanach and Satris said the
e-testing programs are not likely to excite English and
philosophy professors, whose disciplines call for essay
questions rather than computer-graded content.
From a
testing perspective, Catanach said the programs can be most
helpful for faculty with large classes who need to save time
on grading. That’s why the programs have proven popular at
community colleges in some of the larger states, he said.
“It works
for almost anyone who wants to have periodic assessment,” he
said. “How much does the midterm and final motivate students
to keep up with material? It doesn’t. It motivates cramming.
This is a tool to help students keep up with the material.”
August 29, 2007 reply from Stokes, Len
[stokes@SIENA.EDU]
I am also a strong proponent of active learning
strategies. I have the luxury of a small class size. Usually fewer than 30
so I can adapt my classes to student interaction and can have periodic
assessment opportunities as it fits the flow of materials rather than the
calendar. I still think a push toward smaller classes with more faculty face
time is better than computer tests. One lecture and one case day does not
mean active learning. It is better than no case days but it is still a
lecture day. I don’t have real lecture days every day involves some
interactive material from the students.
While I admit I can’t pick up all trends in grading
the tests, but I do pick up a lot of things so I have tendency to have a
high proportion of essays and small problems. I then try to address common
errors in class and also can look at my approach to teaching the material.
Len
Bob Jensen attempts to make a case that self learning is more effective
for metacognitive reasons ---
http://www.trinity.edu/rjensen/265wp.htm
This document features the research of Tony Catanach, David Croll, Bob
Grinaker, and Noah
Barsky.
Bob Jensen's threads on the myths of online education are at
http://www.trinity.edu/rjensen/000aaa/thetools.htm#Myths
Barbara gave me permission to post the following message on March 15, 2006
My reply follows her message.
Professor Jensen:
I need your help in working with regulators who are
uncomfortable with online education.
I am currently on the faculty at the University of
Dallas in Irving, Texas and I abruptly learned yesterday that the Texas
State Board of Public Accountancy distinguishes online and on campus
offering of ethics courses that it approves as counting for students to meet
CPA candidacy requirements. Since my school offers its ethics course in both
modes, I am suddenly faced with making a case to the TSBPA in one week's
time to avoid rejection of the online version of the University of Dallas
course.
I have included in this email the "story" as I
understand it that explains my situation. It isn't a story about accounting
or ethics, it is a story about online education.
I would like to talk to you tomorrow because of
your expertise in distance education and involvement in the profession. In
addition, I am building a portfolio of materials this week for the Board
meeting in Austin March 22-23 to make a case for their approval (or at least
not rejection) of the online version of the ethics course that the Board
already accepts in its on campus version. I want to include compelling
research-based material demonstrating the value of online learning, and I
don't have time to begin that literature survey myself. In addition, I want
to be able to present preliminary results from reviewers of the University
of Dallas course about the course's merit in presentation of the content in
an online delivery.
Thank you for any assistance that you can give me.
Barbara W. Scofield
Associate Professor of Accounting
University of Dallas
1845 E Northgate Irving, TX 75062
972-721-5034
scofield@gsm.udallas.edu
A statement of the University of Dallas and Texas
State Board of Public Accountancy and Online Learning
The TSBPA approved the University of Dallas ethics
program in 2004. The course that was approved was a long-standing course,
required in several different graduate programs, called Business Ethics. The
course was regularly taught on campus (since 1995) and online (since 2001).
The application for approval of the ethics course
did not ask for information about whether the class was on campus or online
and the syllabus that was submitted happened to be the syllabus of an on
campus section. The TSBPA's position (via Donna Hiller) is that the Board
intended to approve only the on campus version of the course, and that the
Board inferred it was an on campus course because the sample syllabus that
was submitted was an on campus course.
Therefore the TSBPA (via Donna Hiller) is requiring
that University of Dallas students who took the online version of the ethics
course retake the exact same course in its on campus format. While the TSBPA
(via Donna Hiller) has indicated that the online course cannot at this time
be approved and its scheduled offering in the summer will not provide
students with an approved course, Donna Hiller, at my request, has indicated
that she will take this issue to the Board for their decision next week at
the Executive Board Meeting on March 22 and the Board Meeting on March 23.
There are two issues:
1. Treatment of students who were relying on
communication from the Board at the time they took the class that could
reasonably have been interpreted to confer approval of both the online and
on campus sections of the ethics course.
2. Status of the upcoming summer online ethics
class.
My priority is establishing the status of the
upcoming summer online ethics class. The Board has indicated through its
pilot program with the University of Texas at Dallas that there is a place
for online ethics classes in the preparation of CPA candidates. The
University of Dallas is interested in providing the TSBPA with any
information or assessment necessary to meet the needs of the Board to
understand the online ethics class at the University of Dallas. Although not
currently privy to the Board specific concerns about online courses, the
University of Dallas believes that it can demonstrate sufficient credibility
for the course because of the following factors:
A. The content of the online course is the same as
the on campus course. Content comparison can be provided. B. The
instructional methods of the online course involve intense
student-to-student, instructor-to-student, and student-to-content
interaction at a level equivalent to an on campus course. Empirical
information about interaction in the course can be provided.
C. The instructor for the course is superbly
qualified and a long-standing ethics instructor and distance learning
instructor. The vita of the instructor can be provided.
D. There are processes for course assessment in
place that regularly prompt the review of this course and these assessments
can be provided to the board along with comparisons with the on campus
assessments.
E. The University of Dallas will seek to coordinate
with the work done by the University of Texas at Dallas to provide
information at least equivalent to that provided by the University of Texas
at Dallas and to meet at a minimum the tentative criteria for online
learning that UT Dallas has been empowered to recommend to the TSBPA.
Contact with the University of Texas at Dallas has been initiated.
When the online ethics course is granted a path to
approval by the Board, I am also interested in addressing the issue of TSBPA
approval of students who took the class between the original ethics course
approval date and March 13, 2006, the date that the University of Dallas
became aware of the TSBPA intent (through Donna Hiller) that the TSBPA
distinguished online and on campus ethics classes.
The University of Dallas believes that the online
class in fact provided these students with a course that completely
fulfilled the general intent of the Board for education in ethics, since it
is the same course as the approved on campus course (see above). The
decision on the extent of commitment of the Board to students who relied on
the Board's approval letter may be a legal issue of some sort that is
outside of the current decision-making of the Board, but I want the Board
take the opportunity to consider that the reasonableness of the students'
position and the students' actual preparation in ethics suggest that there
should also be a path created to approval of online ethics courses taken at
the University of Dallas during this prior time period. The currently
proposed remedy of a requirement for students to retake the very same course
on campus that students have already taken online appears excessively costly
to Texans and the profession of accounting by delaying the entry of
otherwise qualified individuals into public accountancy. High cost is
justified when the concomitant benefits are also high. However, the benefit
to Texans and the accounting profession from students who retake the ethics
course seems to exist only in meeting the requirements of regulations that
all parties diligently sought to meet in the first place and not in
producing any actual additional learning experiences.
A reply to her from Bob Jensen
Hi
Barbara,
May I
share your questions and my responses in the next edition of New
Bookmarks? This might be helpful to your efforts when others become
informed. I will be in my office every day except for March 17. My phone
number is 210-999-7347. However, I can probably be more helpful via
email.
As
discouraging as it may seem, if students know what is expected of them
and must demonstrate what they have learned, pedagogy does not seem to
matter. It can be online or onsite. It can be lecture or cases. It can
be no teaching at all if there are talented and motivated students who
are given great learning materials. This is called the well-known “No
Significant Difference” phenomenon ---
http://www.nosignificantdifference.org/
I think
you should stress that insisting upon onsite courses is discriminatory
against potential students whose life circumstances make it difficult or
impossible to attend regular classes on campus.
I think
you should make the case that online education is just like onsite
education in the sense that learning depends on the quality and
motivations of the students, faculty, and university that sets the
employment and curriculum standards for quality. The issue is not onsite
versus online. The issue is quality of effort.
The most
prestigious schools like Harvard and Stanford and Notre Dame have a
large number of credit and non-credit courses online. Entire accounting
undergraduate and graduate degree programs are available online from
such quality schools as the University of Wisconsin and the University
of Maryland. See my guide to online training and education programs is
at
http://www.trinity.edu/rjensen/crossborder.htm
My main
introductory document on the future of distance education is at
http://www.trinity.edu/rjensen/000aaa/updateee.htm
Anticipate and deal with the main arguments against online education.
The typical argument is that onsite students have more learning
interactions with themselves and with the instructor. This is absolutely
false if the distance education course is designed to promote online
interactions that do a better job of getting into each others’ heads.
Online courses become superior to onsite courses.
Amy
Dunbar teaches intensely interactive online courses with Instant
Messaging. See Dunbar, A. 2004. “Genesis of an Online Course.” Issues in
Accounting Education (2004),19 (3):321-343.
ABSTRACT:
This paper presents a descriptive and evaluative analysis of the
transformation of a face-to-face graduate tax accounting course to an
online course. One hundred fifteen students completed the compressed
six-week class in 2001 and 2002 using WebCT, classroom environment
software that facilitates the creation of web-based educational
environments. The paper provides a description of the required
technology tools and the class conduct. The students used a combination
of asynchronous and synchronous learning methods that allowed them to
complete the coursework on a self-determined schedule, subject to
semi-weekly quiz constraints. The course material was presented in
content pages with links to Excel® problems, Flash examples, audio and
video files, and self-tests. Students worked the quizzes and then met in
their groups in a chat room to resolve differences in answers. Student
surveys indicated satisfaction with the learning methods.
I might
add that Amy is a veteran world class instructor both onsite and online.
She’s achieved all-university awards for onsite teaching in at least
three major universities. This gives her the credentials to judge how
well her online courses compare with her outstanding onsite courses.
A free
audio download of a presentation by Amy Dunbar is available at
http://www.cs.trinity.edu/~rjensen/002cpe/02start.htm#2002
The
argument that students cannot be properly assessed for learning online
is more problematic. Clearly it is easier to prevent cheating with
onsite examinations. But there are ways of dealing with this problem.
My best example of an online graduate program that is extremely
difficult is the Chartered Accountant School of Business (CASB) masters
program for all of Western Canada. Students are required to take some
onsite testing even though this is an online degree program. And CASB
does a great job with ethics online. I was engaged to formally assess
this program and came away extremely impressed. My main contact there is
Don Carter
carter@casb.com . If you are really serious about this, I would
invite Don to come down and make a presentation to the Board. Don will
convince them of the superiority of online education.
You can
read some about the CASB degree program at
http://www.casb.com/
You can
read more about assessment issues at
http://www.trinity.edu/rjensen/assess.htm
I think a
lot of the argument against distance education comes from faculty
fearful of one day having to teach online. First there is the fear of
change. Second there is the genuine fear that is entirely justified ---
if online teaching is done well it is more work and strain than onsite
teaching. The strain comes from increased hours of communication with
each and every student.
Probably
the most general argument in favor of onsite education is that students
living on campus have the social interactions and maturity development
outside of class. This is most certainly a valid argument. However, when
it comes to issues of learning of course content, online education can
be as good as or generally better than onsite classes. Students in
online programs are often older and more mature such that the on-campus
advantages decline in their situations. Online students generally have
more life, love, and work experiences already under their belts. And
besides, you’re only talking about ethics courses rather than an entire
undergraduate or graduate education.
I think
if you deal with the learning interaction and assessment issues that you
can make a strong case for distance education. There are some “dark
side” arguments that you should probably avoid. But if you care to read
about them, go to
http://www.trinity.edu/rjensen/000aaa/theworry.htm
Bob Jensen
March 15,
2006 reply from Bruce Lubich
[BLubich@UMUC.EDU]
Bob, as a director and teacher in a graduate
accounting program that is exclusively online, I want to thank you for your
support and eloquent defense of online education. Unfortunately, Texas's
predisposition against online teaching also shows up in its education
requirements for sitting for the CPA exam. Of the 30 required upper division
accounting credits, at least 15 must "result from physical attendance at
classes meeting regularly on the campus" (quote from the Texas State Board
of Public Accountancy website at www.tsbpa.state.tx.us/eq1.htm)
Cynically speaking, it seems the state of Texas
wants to be sure its classrooms are occupied.
Barbara, best of luck with your testimony.
Bruce Lubich
Program Director,
Accounting Graduate School of Management and Technology
University of Maryland University College
March 15, 2006 reply from David Albrecht
[albrecht@PROFALBRECHT.COM]
At my school, Bowling Green, student credits for
on-line accounting majors classes are never approved by the department
chair. He says that you can't trust the schools that are offering these.
When told that some very reputable schools are offering the courses, he
still says no because when the testing process is done on-line or not in the
physical presence of the professor the grades simply can't be trusted.
David Albrecht
March 16, 2006 reply from Bob Jensen
Hi David,
One tack against a luddites like that is to propose a compromise that
virtually accepts all transfer credits from AACSB-accredited universities.
It's difficult to argue that standards vary between online and onsite
courses in a given program accredited by the AACSB. I seriously doubt that
the faculty in that program would allow a double academic standard.
In fact, on transcripts it is often impossible to distinguish online from
onsite credits from a respected universities, especially when the same
course is offered online and onsite (i.e., merely in different sections).
You might explain to your department chair that he's probably been
accepting online transfer credits for some time. The University of North
Texas and other major universities now offer online courses to full-time
resident students who live on campus. Some students and instructors find
this to be a better approach to learning.
And you ask him why Bowling Green's assessment rigor is not widely known
to be vastly superior to online courses from nearly all major universities
that now offer distance education courses and even total degree programs,
including schools like the Fuqua Graduate School at Duke, Stanford
University (especially computer science and engineering online courses that
bring in over $100 million per year), the University of Maryland, the
University of Wisconsin, the University of Texas, Texas Tech, and even,
gasp, The Ohio State University.
You might tell your department chair that by not offering some online
alternatives, Bowling Green is not getting the most out of its students. The
University of Illinois conducted a major study that found that students
performed better in online versus onsite courses when matched pair sections
took the same examinations.
And then you might top it off by asking your department chair how he
justifies denying credit for Bowling Green's own distance education courses
---
http://adultlearnerservices.bgsu.edu/index.php?x=opportunities
The following is a quotation from the above Bowling Green site:
*****************************
The advancement of computer technology has
provided a wealth of new opportunities for learning. Distance education
is one example of technology’s ability to expand our horizons and gain
from new experiences. BGSU offers many distance education courses and
two baccalaureate degree completion programs online.
The Advanced Technological Education Degree
Program is designed for individuals who have completed a two-year
applied associate’s degree. The Bachelor of Liberal Studies Degree
Program is ideal for students with previous college credit who would
like flexibility in course selection while completing a liberal
education program.
Distance Education Courses and Programs ---
http://ideal.bgsu.edu/ONLINE/
***************************
Bob Jensen
March 16, 2006 reply from Amy Dunbar
[Amy.Dunbar@BUSINESS.UCONN.EDU]
Count me in the camp that just isn't that concerned
about online cheating. Perhaps that is because my students are graduate
students and my online exams are open-book, timed exams, and a different
version is presented to each student (much like a driver's license exam). In
my end-of-semester survey, I ask whether students are concerned about
cheating, and on occasion, I get one who is. But generally the response is
no.
The UConn accounting department was just reviewed
by the AACSB, and they were impressed by our MSA online program. They
commented that they now believed that an online MSA program was possible. I
am convinced that the people who are opposed to online education are
unwilling to invest the time to see how online education is implemented.
Sure there will be bad examples, but there are bad examples of face to face
(FTF) teaching. How many profs do you know who simply read powerpoint slides
to a sleeping class?! Last semester, I received the School of Business
graduate teaching award even though I teach only online classes. I believe
that the factor that really matters is that the students know you care about
whether they are learning. A prof who cares interacts with students. You can
do that online as well as FTF.
Do I miss FTF teaching -- you bet I do. But once I
focused on what the student really needs to learn, I realized, much to my
dismay, interacting FTF with Dunbar was not a necessary condition.
Amy Dunbar
March 16, 2006 message from Carol Flowers
[cflowers@OCC.CCCD.EDU]
To resolve this issue and make me
more comfortable with the grade a student earns, I have all my online exams
proctored. I schedule weekends (placing them in the schedule of classes) and
it is mandatory that they take the exams during this weekend period
(Fir/Sat) at our computing center. It is my policy that if they can't take
the paced exams during those periods, then the class is not one that they
can participate in. This is no different from having different times that
courses are offered. They have to make a choice in that situation, also, as
to which time will best serve their needs.
March 16, 2006 reply from David Fordham, James Madison
University
[fordhadr@JMU.EDU]
Our model is similar to Carol Flowers. Our on-line
MBA program requires an in-person meeting for four hours at the beginning of
every semester, to let the students and professor get to know each other
personally, followed by the distance-ed portion, concluding with another
four-hour in- person session for the final examination or other assessment.
The students all congregate at the Sheraton at Dulles airport, have dinner
together Friday night, spend Saturday morning taking the final for their
previous class, and spend Saturday afternoon being introduced to their next
class. They do this between every semester. So far, the on- line group has
outperformed (very slightly, and not statistically significant due to small
sample sizes) the face-to-face counterparts being used as our control
groups. We believe the outperformance might have an inherent self- selection
bias since the distance-learners are usually professionals, whereas many of
our face-to-face students are full-time students and generally a bit younger
and more immature.
My personal on-line course consists of exactly the
same readings as my F2F class, and exactly the same lectures (recorded using
Tegrity) provided on CD and watched asynchronously, followed by on-line
synchronous discussion sessions (2-3 hours per week) where I call on random
students asking questions about the readings, lectures, etc., and engaging
in lively discussion. I prepare some interesting cases and application
dilemmas (mostly adapted from real world scenarios) and introduce dilemmas,
gray areas, controversy (you expected maybe peace and quiet from David
Fordham?!), and other thought-provoking issues for discussion. I have almost
perfect attendance in the on-line synchronous because the students really
find the discussions engaging. Surprisingly, I have no problem with
freeloaders who don't read or watch the recorded lectures. My major student
assessment vehicle is an individual policy manual, supplemented by the
in-person exam. Since each student's manual organization, layout, approach,
and perspective is so very different from the others, cheating is almost out
of the question. And the in-person exam is conducted almost like the CISP or
old CPA exams... total quiet, no talking, no leaving the room, nothing but a
pencil, etc.
And finally, no, you can't tell the difference on
our student's transcript as to whether they took the on-line or in-person
MBA. They look identical on the transcript.
We've not yet had any problem with anyone
"rejecting" our credential that I'm aware of.
Regarding our own acceptance of transfer credit, we
make the student provide evidence of the quality of each course (not the
degree) before we exempt or accept credit. We do not distinguish between
on-line or F2F -- nor do we automatically accept a course based on
institution reputation. We have on many occasions rejected AACSB- accredited
institution courses (on a course-by-course basis) because our investigation
showed that the course coverage or rigor was not up to the standard we
required. (The only "blanket" exception that we make is for certain familiar
Virginia community college courses in the liberal studies where history has
shown that the college and coursework reliably meets the standards -- every
other course has to be accepted on a course-by-course basis.)
Just our $0.02 worth.
David Fordham
James Madison University
DOES DISTANCE LEARNING WORK?
A LARGE SAMPLE, CONTROL GROUP STUDY OF STUDENT SUCCESS IN DISTANCE LEARNING
by
James Koch ---
http://www.usq.edu.au/electpub/e-jist/docs/vol8_no1/fullpapers/distancelearning.htm
The relevant public policy question is this---Does
distance learning "work" in the sense that students experience as least as
much success when they utilize distance learning modes as compared to when
they pursue conventional bricks and mortar education? The answer to this
question is a critical in determining whether burgeoning distance learning
programs are cost-effective investments, either for students, or for
governments.
Of course, it is difficult to measure the
"learning" in distance learning, not the least because distance learning
courses now span nearly every academic discipline. Hence, most large sample
evaluative studies utilize students’ grades as an imperfect proxy for
learning. That approach is followed in the study reported here, as well.
A recent review of research in distance education
reported that 1,419 articles and abstracts appeared in major distance
education journals and as dissertations during the 1990-1999 period (Berge
and Mrozowski, 2001). More than one hundred of these studies focused upon
various measures of student success (such as grades, subsequent academic
success, and persistence) in distance learning courses. Several asked the
specific question addressed in this paper: Why do some students do better
than others, at least as measured by the grade they receive in their
distance learning course? A profusion of contradictory answers has emanated
from these studies (Berge and Mrozowski, 2001; Machtmes and Asher, 2000). It
is not yet clear how important to individual student success are factors
such as the student’s characteristics (age, ethnic background, gender,
academic background, etc.). However, other than knowing that experienced
faculty are more effective than less experienced faculty (Machtmes and
Asher, 2000), we know even less about how important the characteristics of
distance learning faculty are to student success, particularly where
televised, interactive distance learning is concerned.
Perhaps the only truly strong conclusion emerging
from previous empirical studies of distance learning is the oft cited "no
significant difference" finding (Saba, 2000). Indeed, an entire web site,
http://teleeducation.nb.ca/nosignificantdifference, exists that reports 355
such "no significant difference" studies. Yet, without quarreling with such
studies, they do not tell us why some students achieve better grades than
others when they utilize distance learning.
Several studies have suggested that student
learning styles and receptivity to distance learning influence student
success (see Taplin and Jegede, 2001, for a short survey). Unfortunately, as
Maushak et. al. (2001) point out, these intuitively sensible findings are
not yet highly useful, because they are not based upon large sample, control
group evidence that relates recognizable student learning styles to student
performance. Studies that rely upon "conversation and discourse analysis"
(Chen and Willits, 1999, provide a representative example) and interviews
with students are helpful, yet are sufficiently anecdotal that they are
unlikely to lead us to scientifically based conclusions about what works and
what does not.
This paper moves us several steps forward in terms
of our knowledge by means of a very large distance education sample (76,866
individual student observations) and an invaluable control group of students
who took the identical course at the same time from the same instructor, but
did so "in person" in a conventional "bricks and mortar" location. The
results indicate that gender, age, ethnic background, distance learning
experience, experience with the institution providing the instruction, and
measures of academic aptitude and previous academic success are
statistically significant determinants of student success. Similarly,
faculty characteristics such as gender, age, ethnic background, and
educational background are statistically significant predictors of student
success, though not necessarily in the manner one might hypothesize.
Continued in this working paper
January 6, 2006 message from Carolyn Kotlas
[kotlas@email.unc.edu]
No Significant Difference Phenomenon website
http://www.nosignificantdifference.org/
The website is a companion piece to Thomas L.
Russell's book THE NO SIGNIFICANT DIFFERENCE PHENOMENON, a bibliography of
355 research reports, summaries, and papers that document no significant
differences in student outcomes between alternate modes of education
delivery.
DISTANCE LEARNING AND FACULTY CONCERNS
Despite the growing number of distance learning
programs, faculty are often reluctant to move their courses into the online
medium. In "Addressing Faculty Concerns About Distance Learning" (ONLINE
JOURNAL OF DISTANCE LEARNING ADMINISTRATION, vol. VIII, no. IV, Winter 2005)
Jennifer McLean discusses several areas that influence faculty resistance,
including: the perception that technical support and training is lacking,
the fear of being replaced by technology, and the absence of a
clearly-understood institutional vision for distance learning. The paper is
available online at
http://www.westga.edu/%7Edistance/ojdla/winter84/mclean84.htm
The Online Journal of Distance Learning
Administration is a free, peer-reviewed quarterly published by the Distance
and Distributed Education Center, The State University of West Georgia, 1600
Maple Street, Carrollton, GA 30118 USA; Web:
http://www.westga.edu/~distance/jmain11.html
Bob Jensen's threads on faculty concerns are at
http://www.trinity.edu/rjensen/assess.htm
Also see Bob Jensen's threads on the dark side at
http://www.trinity.edu/rjensen/000aaa/theworry.htm
.QUESTIONING THE VALUE OF LEARNING
TECHNOLOGY
"The notion that the future of education lies
firmly in learning technology, seen as a tool of undoubted magnitude and a
powerful remedy for many educational ills, has penetrated deeply into the
psyche not only of those involved in delivery but also of observers,
including those in power within national governments." In a paper published
in 1992, Gabriel Jacobs expressed his belief that hyperlink technology would
be a "teaching resource that would transform passive learners into active
thinkers." In "Hypermedia and Discovery Based Learning: What Value?"
(AUSTRALASIAN JOURNAL OF EDUCATIONAL TECHNOLOGY, vol. 21, no. 3, 2005, pp.
355-66), he reconsiders his opinions, "the result being that the guarded
optimism of 1992 has turned to a deep pessimism." Jacob's paper is available
online at
http://www.ascilite.org.au/ajet/ajet21/jacobs.html .
The Australasian Journal of Educational Technology
(AJET) [ISSN 1449-3098 (print), ISSN 1449-5554 (online)], published three
times a year, is a refereed journal publishing research and review articles
in educational technology, instructional design, educational applications of
computer technologies, educational telecommunications, and related areas.
Back issues are available on the Web at no cost. For more information and
back issues go to
http://www.ascilite.org.au/ajet/ajet.html .
See Bob Jensen's threads on the dark side at
http://www.trinity.edu/rjensen/000aaa/theworry.htm
June 1, 2007 message from Carolyn Kotlas
[kotlas@email.unc.edu]
TEACHING THE "NET GENERATION"
The April/May 2007 issue of INNOVATE explores and
explains the learning styles and preferences of Net Generation learners.
"Net Generation learners are information seekers, comfortable using
technology to seek out information, frequently multitasking and using
multiple forms of media simultaneously. As a result, they desire
independence and autonomy in their learning processes."
Articles include:
"Identifying the Generation Gap in Higher
Education: Where Do theDifferences Really Lie?"
by Paula Garcia and Jingjing Qin, Northern Arizona University
"MyLiteracies: Understanding the Net Generation
through LiveJournals and Literacy Practices"
by Dana J. Wilber, Montclair State University
"Is Education 1.0 Ready for Web 2.0 Students?"
by John Thompson,Buffalo State College
The issue is available at
http://innovateonline.info/index.php.
Registration is required to access articles;
registration is free.
Innovate: Journal of Online Education [ISSN
1552-3233], an open-access, peer-reviewed online journal, is published
bimonthly by the Fischler School of Education and Human Services at Nova
Southeastern University.
The journal focuses on the creative use of
information technology (IT) to enhance educational processes in academic,
commercial, and governmental settings. For more information, contact James
L. Morrison, Editor-in-Chief; email:
innovate@nova.edu ;
Web: http://innovateonline.info/.
The journal also sponsors Innovate-Live webcasts
and discussion forums that add an interactive component to the journal
articles. To register for these free events, go to
http://www.uliveandlearn.com/PortalInnovate/.
See also:
"Motivating Today's College Students"
By Ian Crone
PEER REVIEW, vol. 9, no. 1, Winter 2007
http://www.aacu.org/peerreview/pr-wi07/pr-wi07_practice.cfm
Peer Review, published quarterly by the Association
of American Colleges and Universities (AACU), provides briefings on
"emerging trends and key debates in undergraduate liberal education. Each
issue is focused on a specific topic, provides comprehensive analysis, and
highlights changing practice on diverse campuses." For more information,
contact: AACU, 1818 R Street NW, Washington, DC 20009 USA;
tel: 202-387-3760; fax: 202-265-9532;
Web:
http://www.aacu.org/peerreview/.
For a perspective on educating learners on the
other end of the generational continuum see:
"Boomer Reality"
By Holly Dolezalek
TRAINING, vol. 44, no. 5, May 2007
http://www.trainingmag.com/msg/content_display/publications/e3if330208bec8f4014fac339db9fd0678e
Training [ISSN 0095-5892] is published monthly by
Nielsen Business Media, Inc., 770 Broadway, New York, NY 10003-9595 USA;
tel: 646-654-4500; email:
bmcomm@nielsen.com ;
Web: http://www.trainingmag.com.
Bob Jensen's threads on learning can be found at the following Web sites:
http://www.trinity.edu/rjensen/assess.htm
http://www.trinity.edu/rjensen/255wp.htm
http://www.trinity.edu/rjensen/265wp.htm
http://www.trinity.edu/rjensen/000aaa/0000start.htm
June 1, 2007 message from Carolyn Kotlas
[kotlas@email.unc.edu]
TECHNOLOGY AND CHANGE IN EDUCATIONAL PRACTICE
"Even if research shows that a particular
technology supports a certain kind of learning, this research may not reveal
the implications of implementing it. Without appropriate infrastructure or
adequate provisions of services (policy); without the facility or ability of
teachers to integrate it into their teaching practice (academics); without
sufficient support from technologists and/or educational technologists
(support staff), the likelihood of the particular technology or software
being educationally effective is questionable."
The current issue (vol. 19, no. 1, 2007) of the
JOURNAL OF EDUCATIONAL TECHNOLOGY & SOCIETY presents a selection of papers
from the Conference Technology and Change in Educational Practice which was
held at the London Knowledge Lab, Institute of Education, London in October
2005.
The papers cover three areas: "methodological
frameworks, proposing new ways of structuring effective research; empirical
studies, illustrating the ways in which technology impacts the working roles
and practices in Higher Education; and new ways of conceptualising
technologies for education."
Papers include:
"A Framework for Conceptualising the Impact of
Technology on Teaching and Learning"
by Sara Price and Martin Oliver, London Knowledge Lab, Institute of
Education
"New and Changing Teacher Roles in Higher Education
in a Digital Age"
by Jo Dugstad Wake, Olga Dysthe, and Stig Mjelstad, University of Bergen
"Academic Use of Digital Resources: Disciplinary
Differences and the Issue of Progression Revisited"
by Bob Kemp, Lancaster University, and Chris Jones, Open University
"The Role of Blogs In Studying the Discourse and
Social Practices of Mathematics Teachers"
by Katerina Makri and Chronis Kynigos, University of Athens
The issue is available at
http://www.ifets.info/issues.php?show=current.
The Journal of Educational Technology and Society
[ISSN 1436-4522]is a peer-reviewed, quarterly publication that "seeks
academic articles on the issues affecting the developers of educational
systems and educators who implement and manage such systems." Current and
back issues are available at
http://www.ifets.info/. The
journal is published by the International Forum of Educational Technology &
Society. For more information, see
http://ifets.ieee.org/.
Bob Jensen's threads on blogs and listservs are at
http://www.trinity.edu/rjensen/ListservRoles.htm
Bob Jensen's threads on education technologies are at
http://www.trinity.edu/rjensen/000aaa/0000start.htm
Bob Jensen's threads on distance education and training alternatives are at
http://www.trinity.edu/rjensen/Crossborder.htm
Civil Rights Groups That Favor Standardized Testing
"Teachers and Rights Groups Oppose Education Measure ," by Diana Jean Schemo,
The New York Times, September 11, 2007 ---
http://www.nytimes.com/2007/09/11/education/11child.html?_r=1&oref=slogin
The draft House bill to renew the federal No Child
Left Behind law came under sharp attack on Monday from civil rights groups
and the nation’s largest teachers unions, the latest sign of how difficult
it may be for Congress to pass the law this fall.
At a marathon hearing of the House Education
Committee, legislators heard from an array of civil rights groups, including
the Citizens’ Commission on Civil Rights, the National Urban League, the
Center for American Progress and Achieve Inc., a group that works with
states to raise academic standards.
All protested that a proposal in the bill for a
pilot program that would allow districts to devise their own measures of
student progress, rather than using statewide tests, would gut the law’s
intent of demanding that schools teach all children, regardless of poverty,
race or other factors, to the same standard.
Dianne M. Piché, executive director of the
Citizens’ Commission on Civil Rights, said the bill had “the potential to
set back accountability by years, if not decades,” and would lead to lower
standards for children in urban and high poverty schools.
“It strikes me as not unlike allowing my teenage
son and his friends to score their own driver’s license tests,” Ms. Piché
said, adding, “We’ll have one set of standards for the Bronx and one for
Westchester County, one for Baltimore and one for Bethesda.”
Continued in article
What works in education?
As I said previously, great teachers come in about as many varieties as
flowers. Click on the link below to read about some of the varieties
recalled by students from their high school days. I t should be noted that
"favorite teacher" is not synonymous with "learned the
most." Favorite teachers are often great at entertaining and/or
motivating. Favorite teachers often make learning fun in a variety of
ways.
However, students may actually learn the most from pretty dull teachers with
high standards and demanding assignments and exams. Also dull teachers may
also be the dedicated souls who are willing to spend extra time in one-on-one
sessions or extra-hour tutorials that ultimately have an enormous impact on
mastery of the course. And then there are teachers who are not so
entertaining and do not spend much time face-to-face that are winners because
they have developed learning materials that far exceed other teachers in terms
of student learning because of those materials.
The recollections below tend to lean toward entertainment and "fun"
teachers, but you must keep in mind that these were written after-the-fact by
former high school teachers. In high school, dull teachers tend not to be
popular before or after the fact. This is not
always the case when former students recall their college professors.
Handicapped Learning Aids Work Wonders ---
http://www.trinity.edu/rjensen/000aaa/thetools.htm#Handicapped
Asynchronous Learning ---
http://www.trinity.edu/rjensen/255wp.htm
Especially note the SCALE Experiments conducted at the University of
Illinois ---
http://www.trinity.edu/rjensen/255wp.htm#Illinois
"'A dozen roses to my favorite teacher," The Philadelphia Enquirer,
November 30, 2004 --- http://www.philly.com/mld/inquirer/news/special_packages/phillycom_teases/10304831.htm?1c
January 6, 2006 message from Carolyn Kotlas
[kotlas@email.unc.edu]
No Significant Difference Phenomenon website
http://www.nosignificantdifference.org/
The website is a companion piece to Thomas L.
Russell's book THE NO SIGNIFICANT DIFFERENCE PHENOMENON, a bibliography of
355 research reports, summaries, and papers that document no significant
differences in student outcomes between alternate modes of education
delivery.
Classroom Tips
Yes! 50 Scientifically Proven Ways to Be Persuasive
From the Financial Rounds Blog on May 4, 2009 ---
http://financialrounds.blogspot.com/
Using "Yes! 50 Scientifically Proven Ways to Be
Persuasive" In The Classroom I recently started reading Goldstein, Martin,
and Cialdini's "Yes!: 50 Scientifically Proven Ways to Be Persuasive." It
could easily be described as "Freakonomics for Social Psychology". It's a
fun, easy, and very informative read, with each chapter only about 1500-2000
words long, and highlighting one persuasion technique. So, you can knock out
a chapter in 10 minutes or so.
It's a very interesting introduction to the social
psychology literature on persuasion - it lists all the underlying research
in the appendix.
In addition to learning some interesting things,
I've also gotten some great ideas to use in my classes. I'll be discussing
these over the next few weeks, starting with
Chapters 1 & 2:
"The Bandwagon effect" One way to increase compliance with a request is to
mention that a lot of other people have done the same thing. In these
chapters, the authors mention a study where they tried to see if they could
increase the percentage of people staying in a hotel who reused towels at
least once during their stay. Their solution was simple. The hotels who do
this typically put a little card in the hotel room touting the benefits of
reusing towels. All they did was add a line to the extent that the majority
of people who stay in hotels do in fact reuse their towels at least once
during their stay. This dramatically increased the percentage of people who
chose to reuse.
In a related study, they added another line stating
that XX% of the people who stayed in this room reused towels. This increased
compliance even more.
Chapter 3:
"What common mistake causes messages to self-destruct?" The bandwagon effect
can also cause messages to backfire. In one study, they seeded the Petrified
Forest with fake pieces of petrified wood, and then posted signs stating
that "many past visitors have removed the petrified wood from the park,
changing the natural state of the petrified forest", accompanied by a
picture of several visitors to taking pieces of wood. These signs actually
increased the incidences of the behavior they were intended to stop. Here
are the applications to my classes: First off, to use the bandwagon effect
in my case course, I'm going to state figures (made up, of course) at the
beginning of class as to the average amount of time past students in that
class have spent preparing each week. I'm also going to tell my classes that
the average evaluation for the professors in the college ranges from 4.2 to
4.8 on a 5 point scale (I know, it's inflated, but it might be interesting
to see what happens if I state that several times during the semester). If I
really want to use the bandwagon effect, I'll mention that evaluations in
THAT particular class have been a bit higher.
As for avoiding the "self-destruct" part of the
bandwagon effect, I plan on spending less time talking about how many
students are absent. If I need to mention it, I'll focus on the flip side
that 94% of the students in this class make the vast majority of classes,
and commend them on that fact.
More to come later. It's a great book, and
inexpensive, too (the paperback is less than $20).
In Defense of "Traditional" Learning and Assessment
April 27, 2009 message from David Albrecht
[albrecht@PROFALBRECHT.COM]
Bob,
Here's another article from the CHE newsletter.
The conclusion from these latest two articles rings true.
Collegiate business courses in general, and collegiate accounting
courses, in particular, have taken their fair share of hits in recent
years, because of the lack of experiential learning built into the
curriculum and so many courses. The traditional approach to collegiate
instruction--lecture and (MC) testing--is too frequently assailed
because students don't become active participants in the learning
process. Never-the-less, accounting students across the country do pick
up on the rules of financial and tax accounting, and the logic of cost
accounting and auditing. I've frequently wondered where the missing
piece is, how a discredited approach to conducting college courses can
produce any learning results at all.
My own thinking had begun to focus on the recitation/homework aspect
built into so many of our courses, and the results of these two studies
seems to it up.
I have made extensive use of homework assignments over the years, to
the extent that I write my own problems. A HW set for a particular topic
moves from very short "drills" to comprehensive problems that set the
topic into a very realistic setting. What I do isn't unique. However, I
have my own idea about what is realistic.
Anyway, I find this latest news to be a validation for a part of what
we do, and welcome news indeed.
Access to the article below requires a subscription. The part of the
article not quoted IS important, as it pertains to real world
applications.
Dave Albrecht
******quotation begins******
http://chronicle.com/weekly/v55/i34/34a00101.htm
From the issue dated May 1, 2009 Close the
Book. Recall. Write It Down. That old study method still works,
researchers say. So why don't professors preach it?
By DAVID GLENN
The scene: A rigorous intro-level survey
course in biology, history, or economics. You're the instructor, and
students are crowding the lectern, pleading for study advice for the
midterm.
If you're like many professors, you'll tell
them something like this: Read carefully. Write down unfamiliar
terms and look up their meanings. Make an outline. Reread each
chapter.
That's not terrible advice. But some
scientists would say that you've left out the most important step:
Put the book aside and hide your notes. Then recall everything you
can. Write it down, or, if you're uninhibited, say it out loud.
Two psychology journals have recently
published papers showing that this strategy works, the latest
findings from a decades-old body of research. When students study on
their own, "active recall" ¬ recitation, for instance, or flashcards
and other self-quizzing ¬ is the most effective way to inscribe
something in long-term memory.
Yet many college instructors are only dimly
familiar with that research. And in March, when Mark A. McDaniel, a
professor of psychology at Washington University in St. Louis and
one author of the new studies, gave a talk at a conference of the
National Center for Academic Transformation, people fretted that the
approach was oriented toward robotic memorization, not true
learning.
Don't Reread
A central idea of Mr. McDaniel's work,
which appears in the April issue of Psychological Science and the
January issue of Contemporary Educational Psychology, is that it is
generally a mistake to read and reread a textbook passage. That
strategy feels intuitively right to many students ¬ but it's much
less effective than active recall, and it can give rise to a false
sense of confidence.
"When you've got your chemis-try book in
front of you, everything's right there on the page, it's all very
familiar and fluent," says Jeffrey D. Karpicke, an assistant
professor of psychology at Purdue University and lead author of a
paper in the May issue of Memory about students' faulty intuitions
about effective study habits.
"So you could say to yourself, 'Yeah, I
know this. Sure, this is all very familiar,'" Mr. Karpicke
continues. "But of course, when you go in to take a classroom test,
or in real life when you need to reconstruct your knowledge, the
book's not there. In our experiments, when students repeatedly read
something, it falsely inflates their sense of their own learning."
These findings about active recall are not
new or faddish or parochial. The research has been deepened and
systematized recently by scholars at the University of California at
Los Angeles and Washington University in St. Louis (where Mr.
Karpicke earned his doctorate in 2007). But the basic insight goes
back decades. One of the new papers tips its hat to a
recitation-based method known as "SQ3R," which was popularized in
Effective Study, a 1946 book by Francis P. Robinson.
So if this wisdom is so well-established ¬
at least among psychologists ¬ should colleges explicitly try to
coax students to use these study techniques? And if so, how? That is
the question that the authors of these papers are now pondering.
"I think it's a mistake for us to think
that just publishing this work in a few journals is going to have a
huge impact in the classroom," says Mr. McDaniel.
After a decade of working in this area, Mr.
McDaniel feels enough confidence in his findings that he is willing
to proselytize about them. He and his colleagues have also been
promoting the idea of frequent low-stakes classroom quizzes (The
Chronicle, June 8, 2007).
Among other things, Mr. McDaniel has
recently collaborated with a network of biology instructors who
would like to improve the pass rates in their introductory courses.
One of those scholars is Kirk Bartholomew,
an assistant professor of biology at Sacred Heart University. He
first crossed paths with Mr. McDaniel at a conference sponsored by a
textbook publisher.
"He basically confirmed my ideas ¬ that
after you've read something once, you've gotten what you're going to
get out of it, and then you need to go out and start applying the
information," Mr. Bartholomew says.
The two scholars collaborated on a Web
interface that encouraged students to try different study
techniques. The first round of research did not turn up any dramatic
patterns, Mr. Bartholomew says ¬ other than the unsurprising fact
that his students did better if they spent more time studying. But
he says that he looks forward to refining the system.
Rote learning?
In March, however, when Mr. McDaniel took
his message to the National Center for Academic Transformation
meeting, his talk was not entirely well received.
Several days after his appearance, he got a
note from Carol A. Twigg, the center's chief executive. "She said,
'We really loved having you, but you created some controversy
here,'" Mr. McDaniel says. According to Ms. Twigg's note, some
people worried that Mr. McDaniel's techniques might generate rote
memorization at the expense of deeper kinds of learning.
Michael R. Reder, director of Connecticut
College's Center for Teaching and Learning, had a similar reaction
to one of Mr. McDaniel's new papers on studying.
The paper seems perfectly valid on its own
terms and might offer a "useful tool," Mr. Reder says. But in his
view, the paper also "suggests an old model of learning. You know,
I'm going to give information to the students, and the students then
memorize that information and then spit it back."
Mr. McDaniel finds such reactions
frustrating. One experiment in his new paper suggests that a week
after reading a complex passage, people who recited the material
after reading it did much better at solving problems that involved
analyzing and drawing inferences from the material than did people
who simply read the passage twice.
"I don't think these techniques will
necessarily result in rote memorization," Mr. McDaniel says. "If you
ask people to free-recall, you can generate a better mental model of
a subject area, and in turn that can lead to better
problem-solving."
And in some college courses, he continues,
a certain amount of memorization is impossible to escape ¬ so it
might as well be done effectively.
In Biology 101, for example, "you've got a
heavily fact-laden course. When I talk to biology instructors at Big
Ten universities, they're working really hard to create interesting,
interactive courses where they've got 500 or 600 kids in a lecture
class. But no matter how engaging you make the course, the students
need to have the knowledge base to do the inquiry-based
problem-solving activities that you've designed."
continued in article
******quotation ends*******
"Imagining College Without Grades," by Scott Jaschik, Inside Higher
Ed, January 22, 2009 ---
http://www.insidehighered.com/news/2009/01/22/grades
Kathleen
O’Brien, senior vice president for academic affairs at
Alverno College, said she realized that it might seem like
the panelists were “tilting at windmills” with their vision
for moving past grades. But she said there may be an
alignment of ideas taking place that could move people away
from a sense that grades are inevitable. First, she noted
that several of the nation’s most
prestigious law schools have moved away from traditional
letter grades, citing a sense that
grades were squelching intellectual curiosity. This trend
adds clout to the discussion and makes it more difficult for
people to say that grades need to be maintained because
professional schools value them. Second, she noted that the
growing use of e-portfolios has dramatized the potential for
tools other than grades to convey what students learn.
Third, she noted that just about everyone views grade
inflation as having destroyed the reliability of grades.
Fourth, she said that with more students taking courses at
multiple colleges — including colleges overseas — the idea
of consistent and clear grading just doesn’t reflect the
mobility of students. And fifth, she noted the reactions in
the room, which are typical of academic groups in that most
professors and students are much more likely to complain
about grading than to praise its accuracy or value. This is
a case of an academic practice, she noted, that is
widespread even as many people doubt its utility.
At the same
time, O’Brien said that one thing holding back colleges from
moving was the sense of many people that doing away with
grades meant going easy on students. In fact, she said,
ending grades can mean much more work for both students and
faculty members. Done right, she said, eliminating grades
promotes rigor.
Continued in article
"Favorite Education Blogs of 2008," by Jay Mathews, The Washington
Post, April 7, 2008 ---
Click Here
Early last year, as
an experiment, I published a
list of what I and
commentator Walt Gardner considered our favorite education blogs. Neither
Gardner nor I had much experience with this most modern form of expression.
We are WAY older than the Web surfing generation. But the list proved
popular with readers, and I promised in that column to make this an annual
event.
Bernstein: The name is obviously a takeoff on the
foregoing. The author of this one occasionally posts elsewhere as well. This
site often provides some incisive and clear explanations of the key aspects
of educational policy.
Mathews: I agree, but have a bias here, too. This
is an Education Week blog, and I am on the board of trustees of the
nonprofit that publishes Ed Week.
My promise was actually more specific: "Next year,
through bribery or trickery, I hope to persuade Ken Bernstein, teacher and
blogger par excellence, to select his favorite blogs and then let me dump on
his choices, or something like that." As I learned long ago, begging works
even better than bribery or trickery, and Bernstein succumbed. Below are his
choices, with some comments from me, and a few of my favorites.
They are in no particular order of quality or
interest. Choosing blogs is a personal matter. Tastes differ widely and
often are not in sync with personal views on how schools should be improved.
I agree with all of Bernstein's choices, even though we disagree on many of
the big issues.
Bernstein is a splendid classroom teacher and a
fine writer, with a gift for making astute connections between
ill-considered policies and what actually happens to kids in school. He is a
social studies teacher at Eleanor Roosevelt High School in Prince George's
County and has been certified by the prestigious National Board for
Professional Teaching Standards. He is also a book reviewer and peer
reviewer for professional publications and ran panels on education at
YearlyKos conventions. He blogs on education, among other topics, at too
many sites to list. He describes his choices here as a few blogs he thinks
"are worthwhile to visit."
· Bridging Differences.
blogs.edweek.org/edweek/Bridging-Differences/
Bernstein: Deborah Meier and Diane Ravitch in the
past have had their differences on educational issues. They both serve at
the Steinhardt School of Education at New York University, and this shared
blog is as valuable as anything on the Web for the insights the two offer,
and for the quality of their dialog.
Mathews: I have a personal bias about this blog. I
know Meier and Ravitch well, consider them the best writers among education
pundits today and frequently bounce ideas off them.
· Eduwonk.
www.eduwonk.com/
Bernstein: I often disagree with Andrew J.
Rotherham, but his has been an influential voice on education policy for
some years, and even now, along with all else he does, he serves on the
Virginia Board of Education.
Mathews: I often agree with Rotherham, and my
editors sometimes complain that I quote him too much. But the guy is only 37
and is going to be an important influence on public school policy for the
rest of my life and long after.
· Edwize.
www.edwize.org/
Bernstein: The site is maintained by the United
Federation of Teachers, the New York affiliate of American Federation of
Teachers. They have a number of authors, many active in New York schools,
but they occasionally have posts from others. Full disclosure: I have been
invited to cross-post things I have written elsewhere.
Mathews: A nice mix of both comment on policy and
inside-the-classroom stuff from teachers.
· Education Policy Blog.
educationpolicyblog.blogspot.com/
Bernstein: The site describes itself as "a
multiblog about the ways that educational foundations can inform educational
policy and practice! The blog will be written by a group of people who are
interested in the state of education today, and who bring to this interest a
set of perspectives and tools developed in the disciplines known as the
'foundations' of education: philosophy, history, curriculum theory,
sociology, economics and psychology." Most of the participants are
university professors. I am a participant from time to time in this blog.
Eduwonkette.
blogs.edweek.org/edweek/eduwonkette/
Continued in article
Bob Jensen's threads on blogs are at
http://www.trinity.edu/rjensen/ListservRoles.htm
Learning Styles Sites
January 1, 2009 message from Pat Wyman
[raisingsmarterchildren@gmail.com]
Hello Bob,
Happy New Year! Your name came up through a google
alert, attached to my website and the complimentary learning styles
inventory at
http://www.howtolearn.com
It is on your page, from the community at
http://www.elearninglearning.com/learning-styles/microsoft/&query=www.howtolearn.com
I want to thank you for this is and if there is any way I can contribute
to your blog and yours to mine, articles, interviews, etc. I'd love to
connect with you.
You're doing wonderful work!
Warmly,
Pat Wyman, M.A.
-- Pat Wyman Best selling author, Learning vs.
Testing Co-Author,
Book Of The Year In the Medicine Category, The Official Autism 101 Manual
University Instructor of Continuing Education, California State University,
East Bay Founder,
http://www.HowToLearn.com and
http://wwwRaisingSmarterChildren.com
Winner, James Patterson PageTurner Award Get your copy of Learning vs.
Testing with complimentary materials at http://www.learningvstesting4.html
Get Tips For Raising A Smarter Child at
http://www.RaisingSmarterChildren.com
"There are two ways you can live your life - one as
if nothing is a miracle, and the other as if everything is a miracle."
Albert Einstein
Bob Jensen's threads on assessment and learning ---
http://www.trinity.edu/rjensen/assess.htm
Bob Jensen's threads on metacognitive learning ---
http://www.trinity.edu/rjensen/265wp.htm
Bob Jensen's threads on asynchronous learning ---
http://www.trinity.edu/rjensen/255wp.htm
Bob Jensen's threads on education technology ---
http://www.trinity.edu/rjensen/000aaa/0000start.htm
April 4, 2008 message from Carolyn Kotlas
[kotlas@email.unc.edu]
ASSESSING EFFECTIVENESS OF NEW TECHNOLOGIES
"From the perspective of instructional designers
and instructors, the decision to adopt a new technology can be exceedingly
difficult. On the one hand, we all want to create the best possible learning
environment for our students. On the other, there is the persistent fear
that integrating a new technology will be onerous in terms of integration
and only marginal in terms of impact, or worse, it may have a negative
impact."
In "How Do We Assess the Effectiveness of New
Technologies and Learning Environments?" (SLOAN-C VIEW, vol. 7, issue 2,
February 2008), Philip Ice suggests using the Community of Inquiry Framework
(CoI): "a theoretical model that seeks to explain the online learning
experience in terms of three overlapping presences: teaching, social and
cognitive." He cites two studies that support the application of CoI for
exploring the impact of new technologies in education. The article,
including links to the cited studies, is available at
http://www.aln.org/publications/view/v7n2/viewv7n2.htm
(Please note: registration is required to view some
articles; registration is free.)
Sloan-C View: Perspectives in Quality Online
Education [ISSN:
1541-2806] is published by the Sloan Consortium
(Sloan-C). Current and back issues are available at
http://www.aln.org/publications/view/
For more information, contact: The Sloan Center at Olin and Babson Colleges,
Franklin W. Olin College of Engineering, Olin Way, Needham MA 02492-1200
USA; tel: 781-292-2523; fax: 781-292-2505;
email: info@sloan-c.org ;
Web:
http://www.sloan-c.org/
Sloan-C is a consortium of institutions and
organizations committed "to help learning organizations continually improve
quality, scale, and breadth of their online programs according to their own
distinctive missions, so that education will become a part of everyday life,
accessible and affordable for anyone, anywhere, at any time, in a wide
variety of disciplines." Sloan-C is funded by the Alfred P. Sloan
Foundation.
DO STUDENTS PREFER INTENSIVE COURSES?
Researchers at the University of Texas at Austin
conducted a study to determine which was preferred by students: "regular"
courses (typical for traditional, residential institutions) or "intensive"
courses -- "those taught on a tighter than normal schedule, with more class
time each week, but fewer weeks" (typical of online courses taught at
for-profit institutions). Students rated the
intensive courses significantly higher, causing
the researchers to suggest that residential colleges may want to consider
offering more courses of this type.
Results of the study were presented at the 2008
Annual Meeting of the American Educational Research Association. An article
about the research (along with reader comments) is available:
"Students Prefer Intensive Courses"
INSIDE HIGHER ED, March 28, 2008,
http://www.insidehighered.com/news/2008/03/28/intensive
Using Field Lab Write-ups to Develop Observational and Critical Thinking Skills
---
http://serc.carleton.edu/NAGTWorkshops/structure04/activities/3856.html
Does technology have no discernable impact on learning?
I've never been a disciple of technology. For me cell
phones are multifunctional, multicolor devices that empower millions of us with
little worth saying to interrupt other millions of us who ought to have
something better to do. I don't want my car to talk to me, I don't want General
Motors to know my latitude and longitude, and I don't need a pocket-size liquid
crystal New York Times or instant access to thirty-second videos of
skateboarding dogs , , , Many American students aren't doing all that well
academically, and almost as many experts are peddling cures. Many prescribe
computers as the miracle that will rescue our kids from scholastic mediocrity.
That's why states like Michigan and Pennsylvania distributed laptops to
thousands of students. Maine led the parade by handing out laptops to every
seventh and eighth grader. Sponsors of the giveaways promised "higher student
performance." Unfortunately, the results have been disappointing. When the test
results of Maine students showed no improvement, boosters explained that it
would "take more time for the impact of laptops to show up." Inconveniently,
Maine's lackluster outcome only confirmed a rigorous international study of
student computer use in thirty-one countries, which found that students who use
computers at school "perform sizably and statistically worse" than students who
don't. Analysts warned that when computer use replaces "traditional learning
methods," it "actually harms the student." A review of California schools
determined that Internet access had "no measurable impact on student
achievement." A 2007 federal study concluded that classroom use of reading and
math software likewise yielded "no significant differences" in student
performance.
Peter Berger, "Stuck on the Cutting Edge," The Irascible Professor,
December 19, 2007 ---
http://irascibleprofessor.com/comments-12-19-07.htm
Jensen Comment
Anecdotally technology can favorably impact learning. In my own case, it's had
an enormous positive impact on my scholarship, my research, and my publishing.
Number 1 are the communications and knowledge sharing (especially from listservs
and blogs) ---
http://www.trinity.edu/rjensen/ListServRoles.htm
Number 2 is the access to enormous databases and knowledge portals ---
http://www.trinity.edu/rjensen/Searchh.htm
Number 3 is the tremendous increase in access provided by the campus
libraries for scholars who take the time and effort to determine what is really
there.
Number 4 is open courseware. The open courseware (especially shared lecture
materials and videos) from some of the best professors in our leading
universities such as 1,500 courses served up by MIT and 177 science courses
served up on YouTube by UC Berkeley are truly amazing. Critics of technology
have probably never utilized these materials ---
http://www.trinity.edu/rjensen/000aaa/updateee.htm#OKI
I think Peter Berger overlooks some of the positive outcomes of technology on
learning ---
http://www.trinity.edu/rjensen/Assess.htm#WhatWorks
More importantly look at the SCALE experiments at the University of Illinois ---
http://www.trinity.edu/rjensen/255wp.htm#Illinois
Although I always like Peter Berger's essays, this time he also overlooks
much of the dark side of technology are learning ---
http://www.trinity.edu/rjensen/000aaa/theworry.htm
Technology and learning have much more complicated interactions that are
superficially glossed over in this particular essay ---
http://www.trinity.edu/rjensen/000aaa/0000start.htm
"Beyond
Tests and Quizzes," Inside Higher Ed, December 5, 2007 ---
http://www.insidehighered.com/news/2007/12/05/mezeske
With federal and state
officials, accreditors and others all talking about the importance of
assessment, what’s going on in classrooms? Assessment, after all, takes
place every time a professor gives a test. A new volume of essays,
Beyond Tests and Quizzes: Creative Assessments in the College Classroom
(Jossey-Bass) argues that
assessments in the classroom could be more creative and more useful to the
educational process. The editors of the volume are Richard Mezeske, chair of
education at Hope College, and Barbara A. Mezeske, an associate professor of
English at Hope. In an e-mail interview, they discussed the themes of their
new book. . .
Q: Could you
share your definition of “creative assessment” and some of
your favorite examples?
A:
Creative assessment is flexible, timely, and
interesting to both the instructor and to the
student. When teachers shift instruction based on student
feedback, then they are being flexible and creative. We do
not mean that teachers should design ever more imaginative
and bizarre assessment tools, or that they should ignore
mandated curricular content. Rather, creative assessment, as
we use the term, implies focused attention to student
learning, reading the signs, engaging students, and
listening to their feedback. Creative assessment often gives
students opportunities to apply and deepen their superficial
knowledge in their discipline.
For example,
in the chapter in our book about teaching grammar, Rhoda
Janzen describes an assessment that requires students to
devise and play grammar games: They cannot do that without a
deep mastery of the principles they are learning. In another
chapter, Tom Smith describes how he grades individuals’
tests during private office appointments: He affirms correct
responses, asks students to explain incomplete or erroneous
answers, and both gives and gets immediate, personal
feedback on a student’s ability to recall and apply
concepts. In a third chapter, David Schock writes about
taking media-production skills into the community, allowing
students to demonstrate their knowledge and skills by
creating public service announcements and other media
products for an audience outside the classroom.
Q: How is
technology (the Web, etc.) changing the potential of testing
and assessment?
A:
Technology is expanding the possibilities for assessment
while at the same time complicating assessment. For example,
checking understanding of a group and individuals during
instruction is now relatively simple with electronic tools
which allow students to press a button and report what they
believe about concept X. The results are instantaneously
displayed for an entire class to see and the instructor can
adjust instruction based on that feedback. However,
technology can complicate, too. How is a teacher able to
guarantee student X working at a remote computer station on
an assessment is actually student X, and not student Y
covering for student X? Does the technology merely make the
assessment tool slick without adding substance to the
assessment? In other words, merely using technology does not
automatically make the assessment clever, substantive,
correct, or even interesting, but it can do all of
those things.
Continued in article
"The Great Debate: Effectiveness of Technology in Education," by
Patricia Deubel, T.H.E. Journal, November 2007 ---
http://www.thejournal.com/articles/21544
According to Robert Kuhn (2000), an expert in brain
research, few people understand the complexity of that change. Technology is
creating new thinking that is "at once creative and innovative, volatile and
turbulent" and "nothing less than a shift in worldview." The change in
mental process has been brought about because "(1) information is freely
available, and therefore interdisciplinary ideas and cross-cultural
communication are widely accessible; (2) time is compressed, and therefore
reflection is condensed and decision-making is compacted; (3) individuals
are empowered, and therefore private choice and reach are strengthened and
one person can have the presence of an institution" (sec: Concluding
Remarks).
If we consider thinking as both individual
(internal) and social (external), as Rupert Wegerif (2000) suggests, then "[t]echnology,
in various forms from language to the internet, carries the external form of
thinking. Technology therefore has a role to play through supporting
improved social thinking (e.g. providing systems to mediate decision making
and collective reasoning) and also through providing tools to help
individuals externalize their thinking and so to shape their own social
worlds" (p. 15).
The new tools for communication that have become
part of the 21st century no doubt contribute to thinking. Thus, in a debate
on effectiveness or on implementation of a particular tool, we must also
consider the potential for creativity, innovation, volatility, and
turbulence that Kuhn (2000) indicates.
Continued in article
Bob Jensen's threads on education technology are at
http://www.trinity.edu/rjensen/000aaa/0000start.htm
Questioning the Admissions Assumptions
And further, the study finds that all of the
information admissions officers currently have (high school grades,
SAT/ACT scores, essays, everything) is of limited
value, and accounts for only 30 percent of the grade variance in colleges —
leaving 70 percent of the variance unexplained.
Scott Jaschik, "Questioning the Admissions Assumptions," Inside Higher Ed, June
19, 2007 ---
http://www.insidehighered.com/news/2007/06/19/admit
The report is available at
http://cshe.berkeley.edu/publications/docs/ROPS.GEISER._SAT_6.12.07.pdf
Roland G. Fryer, who was hired by Schools Chancellor
Joel I. Klein to advise him on how to narrow the racial gap in achievement in
the city’s schools, made his professional name in economics by applying complex
algorithms to document how black students fall behind their white peers. But his
life story challenges his own calculations. . . . His first job, though, he
said, will be to mine data — from graduation rates to test scores to demographic
information — to find out why there are wide gulfs between schools. Why, for
example, does one school in Bedford-Stuyvesant do so much better than a school
just down the block? And he will monitor the pilot program to pay fourth- and
seventh-grade students as much as $500 for doing well on a series of
standardized tests. That program will begin in 40 schools this fall. He hopes to
find other ways to motivate students.
Jennifer Medina, "His Charge: Find a Key to Students’ Success," The New York
Times, June 21, 2007 ---
http://www.nytimes.com/2007/06/21/nyregion/21fryer.html?_r=1&oref=slogin
Jensen Comment
I suspect that SAT scores are more predictive for some college graduates than
others. For example. SAT math performance may be a better predictor of grades in
mathematics and science courses than SAT verbal performance is a predictor of
grades in literature and language courses. The study mentioned above does not
delve into this level of detail. Top universities that have dropped SAT
requirements (e.g., under the Texas Top Ten Percent Law) are not especially
happy about losing so many top SAT performers ---
http://www.trinity.edu/rjensen/HigherEdControversies.htm#10PercentLaw
SAT/ACT testing falls down because it does not examine motivation vary well.
High school grades fail because of rampant grade inflation and lowered academic
standards in high schools. College grades are not a good criterion because of
grade inflation in colleges ---
http://www.trinity.edu/rjensen/assess.htm#GradeInflation
Question
Is homework credit sometimes dysfunctional to learning?
If the instructor allows face-to-face study groups, extra-help tutorials, and
chat rooms, what is so terrible about this Facebook study group?
Answer
Apparently its the fact that ten percent course credit was given for homework
that was discussed in the study group. It seems unfair, however, to single out
this one student running the Facebook study group. If the students were
"cheating" by sharing tips on homework, they were probably also doing it
face-to-face. All students who violate the code of conduct should be sanctioned
or forgiven based on the honor code of the institution.
Ryerson U. Student Faces
Expulsion for Running a Facebook Study Group
A student at Ryerson
University, in Toronto, is facing expulsion for running a Facebook study group,
the
Toronto Star reports. Chris Avenir, a
first-year engineering student, is facing expulsion from the school on 147
counts of academic charges — one for himself, and one for every student who used
the Facebook group “Dungeons/Mastering Chemistry Solutions” to get homework
help. University officials say that running such a group is in violation of the
school’s academic policy, which says no student can undertake activity to gain
academic advantage. Students argue, however, that the group was analogous to any
in-person study group. Of course, this wouldn’t be the first
Facebook-related expulsion hearing. The
expulsion hearing is scheduled for Tuesday.
Hurley Goodall,
Chronicle of Higher Education, March 7, 2008 ---
http://chronicle.com/wiredcampus/index.php?id=2801&utm_source=wc&utm_medium=en
Jensen Comment
My approach was to assign homework for no credit and then administer online
quizzes. Students were assigned different partners each week who attested to
observing no cheating while an assigned "partner" took the online quiz. You can
read the following at ---
http://www.trinity.edu/rjensen/acct5342/acct5342.htm
Most every week beginning in
Week 2, you will be required to take an online quiz for a chapter from the
online textbook by Murthy and Groomer. This book is not in the bookstore.
Students should immediately obtain a password and print the first three
chapters of the book entitled
Accounting Information Systems: A Database Approach. You can purchase a
password at
http://www.cybertext.com/forms/accountform.shtml
You will then be able to access the book and the online quizzes at any time
using the book list at
http://www.cybertext.com/
Each week students are to take an online quiz in the presence of an assigned
student partner who then signs the attest form at
http://www.trinity.edu/rjensen/acct5342/attest.htm
The online quizzes are relatively easy if you take notes while reading the
assigned chapter. You may use your notes for each quiz. However, you may
not view a copy of the entire chapter will taking a quiz.
Bob Jensen's threads on cheating are at
http://www.trinity.edu/rjensen/Plagiarism.htm
Lawyers Don't Like Being Ranked
It's a sunny day in Seattle when two lawyers can bring
a class action suit on their own behalf -- and then see it rejected on First
Amendment grounds. That's what happened last week in the Emerald City, when
Federal District Judge Robert S. Lasnik ruled that there was no basis for
cracking down on a lawyer-rating Web site merely because some of its ratees
didn't like how they were portrayed. The site, called Avvo, does for lawyers
what any number of magazines and Web sites have been doing for other professions
for years. Magazines regularly publish stories that rank an area's doctors and
dentists. There are rating sites and blogs for the "best" hairstylists,
manicurists, restaurants and movie theaters. Almost any consumer product or
service these days is sorted and ranked.
"Judging Lawyers," The Wall Street Journal, December 24, 2007; Page A10
---
http://online.wsj.com/article/SB119846335960848261.html
Avvo Lawyer Ratings ---
http://www.avvo.com/
Jensen Comment
In fairness most of these ranking systems are misleading. For example,
physicians and lawyers who lose more often may also be willing to take on the
tougher cases having low probabilities of success. Especially note
"Challenging Measures of Success" at
http://www.trinity.edu/rjensen/HigherEdControversies.htm#BusinessSchoolRankings
And some professionals that win a lot may do so because they do so in
unethical ways. And lawyers, like physicians, have different specialties such
that in the realm of a particular specialty, maybe one that rarely call out,
from over 100 specialties, they may be outstanding.
Bob Jensen threads on college ranking controversies are at
http://www.trinity.edu/rjensen/HigherEdControversies.htm#BusinessSchoolRankings
Those conclusions come
from
a national survey of employers
with at least 25 employees and significant
hiring of recent college graduates, released
Tuesday by the Association of American
Colleges and Universities. Over all, 65
percent of those surveyed believe that new
graduates of four-year colleges have most or
all of the skills to succeed in entry-level
positions, but only 40 percent believe that
they have the skills to advance.
. .
.
In
terms of specific skills, the employers didn’t give many A’s
or fail many either. The employers were asked to rank new
graduates on 12 key areas, and the grads did best in
teamwork, ethical judgments and intercultural work, and
worst in global knowledge, self-direction and writing.
Employers Ratings of College
Graduates Preparedness on 1-10 Scale
|
Category |
Mean
Rating |
%
giving high (8-10) rating |
%
giving low (1-5) rating |
|
Teamwork |
7.0 |
39% |
17% |
|
Ethical judgment |
6.9 |
38% |
19% |
|
Intercultural skills |
6.9 |
38% |
19% |
|
Social responsibility |
6.7 |
35% |
21% |
|
Quantitative reasoning |
6.7 |
32% |
23% |
|
Oral communication |
6.6 |
30% |
23% |
|
Self-knowledge |
6.5 |
28% |
26% |
|
Adaptability |
6.3 |
24% |
30% |
|
Critical thinking |
6.3 |
22% |
31% |
|
Writing |
6.1 |
26% |
37% |
|
Self-direction |
5.9 |
23% |
42% |
|
Global knowledge |
5.7 |
18% |
46% |
To
the extent that employers give graduates mixed grades, that
raises the question of how they determine who is really
prepared. Many of the existing tools appear to be
insufficient, the poll found.
Continued in
article
Jensen Comment
This study is misleading in the sense that large employers generally hire
above-average graduates. This skews the results upward with respect to the
entire population of college graduates. Colleges have a long way to go in modern
times.
Bob Jensen's threads higher education controversies are at
http://www.trinity.edu/rjensen/HigherEdControversies.htm
Question
What factors most heavily influence student performance and desire to take more
courses in a given discipline?
Answer
These outcomes are too complex to be predicted very well. Sex and age of
instructors have almost no impact. Teaching evaluations have a very slight
impact, but there are just too many complexities to find dominant factors
cutting across a majority of students.
Oreopoulos said the findings bolster a conclusion he
came to in a previous academic paper that subjective qualities, such as how a
professor fares on student evaluations, tell you more about how well students
will perform and how likely they are to stay in a given course than do
observable traits such as age or gender. (He points out, though, that even the
subjective qualities aren’t strong indicators of student success.) “If I were
concerned about improving teaching, I would focus on hiring teachers who perform
well on evaluations rather than focus on age or gender,” he said.
Elia Powers, "Faculty Gender and Student Performance," Inside Higher Ed,
June 21, 2007 ---
http://www.insidehighered.com/news/2007/06/21/gender
Jensen Comment
A problem with increased reliance on teaching evaluations to measure performance
of instructors is that this, in turn, tends to grade inflation ---
http://www.trinity.edu/rjensen/assess.htm#GradeInflation
Question
What parts of a high school curriculum are the best predictors of success as a
science major in college?
New research by professors at Harvard University
and the University of Virginia has found that no single high school science
course has an impact beyond that type of science, when it comes to predicting
success in college science. However, the researchers found that a rigorous
mathematics curriculum in high school has a significant impact on performance in
college science courses. The research, which will be published in Science, runs
counter to the “physics first” movement in which some educators have been
advocating that physics come before biology and chemistry in the high school
curriculum. The study was based on analysis of a broad pool of college students,
their high school course patterns, and their performance in college
science.
Inside Higher Ed, July 27, 2007 ---
http://www.insidehighered.com/news/2007/07/27/qt
Jensen Comment
Now we have this when some colleges are trying to promote applications and
admissions by dropping the SAT testing requirements for admission. In Texas, the
Top 10% of any state high school class do not have to even take the SAT for
admission to any state university in Texas. Of course high schools may still
have a rigorous mathematics curriculum, but what high school student aiming for
the 10% rule is going to take any rigorous course that is not required for high
school graduation? The problem is that rigorous elective courses carry a higher
risk of lowering the all-important grade point average.
Bob Jensen's threads on higher education controversies are at
http://www.trinity.edu/rjensen/HigherEdControversies.htm
Grades are even worse than tests as predictors of success
"The Wrong Traditions in Admissions," by William E. Sedlacek, Inside
Higher Ed, July 27, 2007 ---
http://www.insidehighered.com/views/2007/07/27/sedlacek
Grades and test scores have worked well as the
prime criteria to evaluate applicants for admission, haven’t they? No!
You’ve probably heard people say that over and over again, and figured that
if the admissions experts believe it, you shouldn’t question them. But that
long held conventional wisdom just isn’t true. Whatever value tests and
grades have had in the past has been severely diminished. There are many
reasons for this conclusion, including greater diversity among applicants by
race, gender, sexual orientation and other dimensions that interact with
career interests. Predicting success with so much variety among applicants
with grades and test scores asks too much of those previous stalwarts of
selection. They were never intended to carry such a heavy expectation and
they just can’t do the job anymore, even if they once did. Another reason is
purely statistical. We have had about 100 years to figure out how to measure
verbal and quantitative skills better but we just can’t do it.
Grades
are even worse than tests as predictors of success.
The major reason is
grade inflation. Everyone
is getting higher grades these days, including those in high
school, college, graduate, and professional school. Students
are bunching up at the top of the grade distribution and we
can’t distinguish among them in selecting who would make the
best student at the next level.
We need a fresh approach. It is not good enough to feel
constrained by the limitations of our current ways of
conceiving of tests and grades. Instead of asking; “How can
we make the SAT and other such tests better?” or “How can we
adjust grades to make them better predictors of success?” we
need to ask; “What kinds of measures will meet our needs now
and in the future?” We do not need to ignore our current
tests and grades, we need to add some new measures that
expand the potential we can derive from assessment.
We appear to
have forgotten why tests were created in the first place.
While they were always considered to be useful in evaluating
candidates, they were also considered to be more equitable
than using prior grades because of the variation in quality
among high schools.
Test results
should be useful to educators — whether involved in
academics or student services — by providing the basis to
help students learn better and to analyze their needs. As
currently designed, tests do not accomplish these
objectives. How many of you have ever heard a colleague say
“I can better educate my students because I know their SAT
scores”? We need some things from our tests that currently
we are not getting. We need tests that are fair to all and
provide a good assessment of the developmental and learning
needs of students, while being useful in selecting
outstanding applicants. Our current tests don’t do that.
The rallying
cry of “all for one and one for all” is one that is used
often in developing what are thought of as fair and
equitable measures. Commonly, the interpretation of how to
handle diversity is to hone and fine-tune tests so they are
work equally well for everyone (or at least to try to do
that). However, if different groups have different
experiences and varied ways of presenting their attributes
and abilities, it is unlikely that one could develop a
single measure, scale, test item etc. that could yield
equally valid scores for all. If we concentrate on results
rather than intentions, we could conclude that it is
important to do an equally good job of selection for each
group, not that we need to use the same measures for all to
accomplish that goal. Equality of results, not process is
most important.
Therefore,
we should seek to retain the variance due to culture, race,
gender, and other aspects of non-traditionality that may
exist across diverse groups in our measures, rather than
attempt to eliminate it. I define non-traditional persons as
those with cultural experiences different from those of
white middle-class males of European descent; those with
less power to control their lives; and those who experience
discrimination in the United States.
While
the term “noncognitive” appears to be precise and
“scientific” sounding, it has been used to describe a wide
variety of attributes. Mostly it has been defined as
something other than grades and test scores, including
activities, school honors, personal statements, student
involvement etc. In many cases those espousing noncognitive
variables have confused a method (e.g. letters of
recommendation) with what variable is being measured. One
can look for many different things in a letter.
Robert Sternberg’s system of
viewing intelligence provides a model, but is important to
know what sorts of abilities are being assessed and that
those attributes are not just proxies for verbal and
quantitative test scores. Noncognitive variables appear to
be in Sternberg’s experiential and contextual domains, while
standardized tests tend to reflect the componential domain.
Noncognitive variables are useful for all students, they are
particularly critical for non-traditional students, since
standardized tests and prior grades may provide only a
limited view of their potential.
I and
my colleagues and students have developed a system of
noncognitive variables that has worked well in many
situations. The eight variables in the system are
self-concept, realistic self-appraisal, handling the system
(racism), long range goals, strong support person,
community, leadership, and nontraditional knowledge.
Measures of these dimensions are available at no cost in a
variety of articles and in a book,
Beyond the Big Test.
This
Web site has previously featured how
Oregon State University has used a
version of this system very successfully in increasing their
diversity and student success. Aside from increased
retention of students, better referrals for student services
have been experienced at Oregon State. The system has also
been employed in selecting Gates Millennium Scholars. This
program, funded by the Bill & Melinda Gates Foundation,
provides full scholarships to undergraduate and graduate
students of color from low-income families. The SAT scores
of those not selected for scholarships were somewhat higher
than those selected. To date this program has provided
scholarships to more than 10,000 students attending more
than 1,300 different colleges and universities. Their
college GPAs are about 3.25, with five year retention rates
of 87.5 percent and five year graduation rates of 77.5
percent, while attending some of the most selective colleges
in the country. About two thirds are majoring in science and
engineering.
The
Washington State Achievers program
has also employed the noncognitive variable system discussed
above in identifying students from certain high schools that
have received assistance from an intensive school reform
program also funded by the Bill & Melinda Gates Foundation.
More than 40 percent of the students in this program are
white, and overall the students in the program are enrolling
in colleges and universities in the state and are doing
well. The program provides high school and college mentors
for students. The
College Success Foundation is
introducing a similar program in Washington, D.C., using the
noncognitive variables my colleagues and I have developed.
Recent
articles in this publication have discussed programs at the
Educational Testing Service for
graduate students and
Tufts University for
undergraduates that have incorporated noncognitive
variables. While I applaud the efforts for reasons I have
discussed here, there are questions I would ask of each
program. What variables are you assessing in the program? Do
the variables reflect diversity conceptually? What evidence
do you have that the variables assessed correlate with
student success? Are the evaluators of the applications
trained to understand how individuals from varied
backgrounds may present their attributes differently? Have
the programs used the research available on noncognitive
variables in developing their systems? How well are the
individuals selected doing in school compared to those
rejected or those selected using another system? What are
the costs to the applicants? If there are increased costs to
applicants, why are they not covered by ETS or Tufts?
Until these
and related questions are answered these two programs seem
like interesting ideas worth watching. In the meantime we
can learn from the programs described above that have been
successful in employing noncognitive variables. It is
important for educators to resist half measures and to
confront fully the many flaws of the traditional ways higher
education has evaluated applicants.
A different way to think about assessment
January 26, 2007 message from Carnegie President
[carnegiepresident@carnegiefoundation.org]
A different way to think about ... assessment In
the most recent issue of
Change magazine,
I join several other authors to examine higher education's ongoing
responsibility to tell the story of student learning with care and
precision. Fulfilling this responsibility at the institutional level
requires ongoing deliberations among colleagues and stakeholders about the
specific learning goals we seek and the broad educational purposes we
espouse. What will motivate such discussions?
In this month's Carnegie Perspectives,
Lloyd Bond makes a strong case for the use of
common examinations as a powerful form of assessment as well as a fruitful
context for faculty deliberations about their goals for students. Using an
institutional example from the
Carnegie/Hewlett project on strengthening teaching
and learning at community colleges, Lloyd describes a particular example of
this principle and how it supports faculty communication and student
learning.
Carnegie has created a forum—Carnegie
Conversations—where you can engage publicly with Lloyd and read and respond
to what others have to say about this article at
http://www.carnegiefoundation.org/perspectives/january2007
Or you may respond to the author privately through
CarnegiePresident@carnegiefoundation.org
We look forward to hearing from you.
Sincerely,
Lee S. Shulman
President The Carnegie Foundation for the Advancement of Teaching
International Journal for the Scholarship of Teaching and Learning ---
http://www.georgiasouthern.edu/ijsotl/
Just-In-Time Teaching ---
http://134.68.135.1/jitt/
Just-in-Time Teaching (JiTT for short) is a
teaching and learning strategy based on the interaction between web-based
study assignments and an active learner classroom. Students respond
electronically to carefully constructed web-based assignments which are due
shortly before class, and the instructor reads the student submissions
"just-in-time" to adjust the classroom lesson to suit the students' needs.
Thus, the heart of JiTT is the "feedback loop" formed by the students'
outside-of-class preparation that fundamentally affects what happens during
the subsequent in-class time together.
What is Just-in-Time Teaching designed to
accomplish?
JiTT is aimed at many of the challenges facing
students and instructors in today's classrooms. Student populations are
diversifying. In addition to the traditional nineteen-year-old recent high
school graduates, we now have a kaleidoscope of "non-traditional" students:
older students, working part time students, commuting students, and, at the
service academies, military cadets. They come to our courses with a broad
spectrum of educational backgrounds, interests, perspectives, and
capabilities that compel individualized, tailored instruction. They need
motivation and encouragement to persevere. Consistent, friendly support can
make the difference between a successful experience and a fruitless effort.
It can even mean the difference between graduating and dropping out.
Education research has made us more aware of learning style differences and
of the importance of passing some control of the learning process over to
the students. Active learner environments yield better results but they are
harder to manage than lecture oriented approaches. Three of the
"Seven
Principles for Good Practice in Undergraduate Education" encourage
student-faculty contact, increased time for student study, and cooperative
learning between students.
To confront these challenges, the Just-in-Time Teaching strategy pursues
three major goals:
- 1. To maximize the efficacy of the
classroom session, where human instructors are present.
- 2. To structure the out-of-class time
for maximum learning benefit.
- 3. To create and sustain team spirit.
Students and instructors work as a team toward the same objective, to
help all students pass the course with the maximum amount of retainable
knowledge.
What JiTT is Not
Although Just-in-Time Teaching makes heavy use
of the web, it is not to be confused with either distance learning (DL) or
with computer-aided instruction (CAI). Virtually all JiTT instruction occurs
in a classroom with human instructors. The web materials, added as a
pedagogical resource, act primarily as a communication tool and secondarily
as content provider and organizer. JiTT is also not an attempt to 'process'
large numbers of students by employing computers to do massive grading jobs.
The JiTT Feedback Loop
The Web Component
JiTT web pages fall into three major
categories:
- 1. Student assignments in preparation
for the classroom activity: WarmUps and Puzzles.
- 2. Enrichment pages. Short essays on
practical, everyday applications of the course subject matter, peppered
with URLs to interesting material on the web. These essays have proven
themselves to be an important motivating factor in introductory service
courses, where students often doubt the current relevance the subject.
- 3. Stand alone instructional material,
such as simulation programs and spreadsheet exercises.
For detailed examples of the JiTT web
resources, please see the
JiTT
resources page.
WarmUps and Puzzles are the heart of the
JiTT web component. These are short, web-based assignments, prompting
the student to think about the upcoming lesson and answer a few
simple questions prior to class. These questions, when fully discussed,
often have complex answers. The students are expected to develop the
answer as far as they can on their own. We finish the job in the
classroom. These assignments are due just a few hours before class time.
The responses are delivered to the instructor electronically to form the
framework for the classroom activities that follow. Typically, the
instructors duplicates sample responses on transparencies and takes them
to class. The interactive classroom session, built around these
responses, replaces the traditional lecture/recitation format.
Students complete the WarmUp assignments before they receive any formal
instruction on a particular topic. They earn credit for answering a
question, substantiated by prior knowledge and whatever they managed to
glean from the textbook. The answers do not have to be complete, or even
correct. In fact, partially correct responses are particularly useful as
classroom discussion fodder. In contrast to WarmUps, Puzzle exercises
are assigned to students after they have received formal instruction on
a particular topic. The Puzzles serve as the framework for a wrap-up
session on a particular topic.
The WarmUps, and to some extent the Puzzles, are undergirded by
education research and target a variety of specific issues. The list of
targeted issues might contain: developing concepts and vocabulary,
modeling -- connecting concepts and equations, estimation- getting a
feel for magnitudes, relating technical scientific statements to "common
sense", understanding the scope of applicability of equations, etc. The
targeted issues are highly content specific. They may involve the
characteristics of a particular class (e.g. the background skills of a
particular student body).
In preparing WarmUp assignments for an upcoming class meeting, we first
create a conceptual outline of the lesson content. This task is similar
to the preparation of a traditional passive lecture. As we work on the
outline, we pay attention to the pedagogical issues that we need to
focus on when in the classroom. Are we introducing new concepts and/or
new notation? Are we building on a previous lesson, and if so, what
bears repeating? What are the important points we wish the students to
remember from the session? What are the common difficulties typical
students will face when exposed to this material? (Previous classroom
experience and teaching and learning literature can be immensely helpful
here). Once this outline has been created, we create broadly based
questions that will force students to grapple with as many of the issues
as possible. We are hoping to receive, in the student responses, the
framework on which we build the in-class experience.
The Active Learner Classroom
The JiTT classroom session is intimately linked to the electronic
preparatory assignments the students complete outside of class. Exactly
how the classroom time is spent depends on a variety of issues such as
class size, classroom facilities, and student and instructor
personalities. Mini-lectures (10 min max) are often interspersed with
demos, classroom discussion, worksheet exercises, and even hands-on
mini-labs. Regardless, the common key is that the classroom component,
whether interactive lecture or student activities, is informed by an
analysis of various student responses.
In a JiTT classroom students construct the same content as in a passive
lecture with two important added benefits. First, having completed the
web assignment very recently, they enter the classroom ready to actively
engage in the activities. Secondly, they have a feeling of ownership
since the interactive lesson is based on their own wording and
understanding of the relevant issues.
The give and take in the classroom suggests future WarmUp questions that
will reflect the mood and the level of expertise in the class at hand.
In this way the feedback loop is closed with the students having played
a major part in the endeavor.
From the instructor's point of view, the lesson content remains pretty
much the same from semester to semester with only minor shifts in
emphasis. From the students' perspective, however, the lessons are
always fresh and interesting, with a lot of input from the class.
We designed JiTT to improve student learning in our own classrooms and
have been encouraged by the results, both attitudinal and cognitive. We
attribute this success to three factors that enhance student learning,
identified by Alexander Astin* in his thirty year study of
college student success:
- increased amounts and quality of student-student interaction
- student-faculty interaction
- student study outside of class.
By fostering these, JiTT promotes student learning and satisfaction.
*Astin, Alexander:
What
matters in college? Four critical years revisited (San Francisco,
CA: Jossey-Bass Publishers, 1993).
Bob Jensen's threads on tools and tricks of the trade are at
http://www.trinity.edu/rjensen/000aaa/thetools.htm
What works in education?
Perhaps Colleges Should Think About This
"School Ups Grade by Going Online," by Cyrus Farivar, Wired News,
October 12, 2004 --- http://www.wired.com/news/culture/0,1284,65266,00.html?tw=newsletter_topstories_html
Until last year, Walt Whitman Middle School 246 in
Brooklyn was considered a failing school by the state of New York.
But with the help of a program called HIPSchools
that uses rapid communication between parents and teachers through e-mail and
voice mail, M.S.
246 has had a dramatic turnaround. The premise behind "HIP"
comes from Keys Technology Group's mission of "helping involve
parents."
The school has seen distinct improvement in the
performance of its 1300 students, as well as regular attendance, which has
risen to 98 percent (an increase of over 10 percent) in the last two years
according to Georgine Brown-Thompson, academic intervention services
coordinator at M.S. 246.
Continued in the article
September 2, 2004 message from Carolyn Kotlas [kotlas@email.unc.edu]
"CONSUMER REPORTS" FOR RESEARCH IN
EDUCATION
The What Works Clearinghouse was established in 2002
by the U.S. Department of Education's Institute of Education Sciences with
$18.5 million in funding to "provide educators, policymakers,
researchers, and the public with a central and trusted source of scientific
evidence of what works in education." The Clearinghouse reviews,
according to relevance and validity, the "effectiveness of replicable
educational interventions (programs, products, practices, and policies) that
intend to improve student outcomes." This summer, the Clearinghouse
released two of its planned reports: peer-assisted learning interventions and
middle school math curricula. For more information about the What Works
Clearinghouse and descriptions of all topics to be evaluated, go to http://www.w-w-c.org/
See also:
"'What Works' Research Site Unveiled" by
Debra Viadero EDUCATION WEEK, vol. 23, no. 42, pp. 1, 33, July 14, 2004 http://www.edweek.org/ew/ew_printstory.cfm?slug=42Whatworks.h23
"'What Works' Site Opens Dialogue on
Research" Letter to Editor from Talbot Bielefeldt, Center for Applied
Research in Educational Technology, International Society for Technology in
Education EDUCATION WEEK, vol. 23, no. 44, p. 44, August 11, 2004 http://www.edweek.org/ew/ew_printstory.cfm?slug=44Letter.h23
April 1, 2005 message from Carolyn Kotlas
[kotlas@email.unc.edu]
NEW EDUCAUSE E-BOOK ON THE NET GENERATION
EDUCATING THE NET GENERATION, a new EDUCAUSE
e-book of essays edited by Diana G. Oblinger and James L. Oblinger,
"explores the Net Gen and the implications for institutions in areas such as
teaching, service, learning space design, faculty development, and
curriculum." Essays include: "Technology and Learning Expectations of the
Net Generation;" "Using Technology as a Learning Tool, Not Just the Cool New
Thing;" "Curricula Designed to Meet 21st-Century Expectations;" "Faculty
Development for the Net Generation;" and "Net Generation Students and
Libraries." The entire book is available online at no cost at
http://www.educause.edu/educatingthenetgen/
.
EDUCAUSE is a nonprofit association whose mission
is to advance higher education by promoting the intelligent use of
information technology. For more information, contact: Educause, 4772 Walnut
Street, Suite 206, Boulder, CO 80301-2538 USA; tel: 303-449-4430; fax:
303-440-0461; email:
info@educause.edu; Web:
http://www.educause.edu/
See also:
GROWING UP DIGITAL: THE RISE OF THE NET GENERATION
by Don Tapscott McGraw-Hill, 1999; ISBN: 0-07-063361-4
http://www.growingupdigital.com/
EFFECTIVE E-LEARNING DESIGN
"The unpredictability of the student context and
the mediated relationship with the student require careful attention by the
educational designer to details which might otherwise be managed by the
teacher at the time of instruction." In "Elements of Effective e-Learning
Design" (INTERNATIONAL REVIEW OF RESEARCH IN OPEN AND DISTANCE LEARNING,
March 2005) Andrew R. Brown and Bradley D. Voltz cover six elements of
effective design that can help create effective e-learning delivery. Drawing
upon examples from The Le@rning Federation, an initiative of state and
federal governments of Australia and New Zealand, they discuss lesson
planning, instructional design, creative writing, and software
specification. The paper is available online at
http://www.irrodl.org/content/v6.1/brown_voltz.html
International Review of Research in Open and
Distance Learning (IRRODL) [ISSN 1492-3831] is a free, refereed ejournal
published by Athabasca University - Canada's Open University. For more
information, contact Paula Smith, IRRODL Managing Editor; tel: 780-675-6810;
fax: 780-675-672; email:
irrodl@athabascau.ca
; Web:
http://www.irrodl.org/
The Le@rning Federation (TLF) is an "initiative
designed to create online curriculum materials and the necessary
infrastructure to ensure that teachers and students in Australia and New
Zealand can use these materials to widen and enhance their learning
experiences in the classroom." For more information, see
http://www.thelearningfederation.edu.au/
COMPUTERS IN THE CLASSROOM AND OPEN BOOK EXAMS
In "PCs in the Classroom & Open Book Exams" (UBIQUITY, vol. 6, issue 9,
March 15-22, 2005), Evan Golub asks and supplies some answers to questions
regarding open-book/open-note exams. When classroom computer use is allowed
and encouraged, how can instructors secure the open-book exam environment?
How can cheating be minimized when students are allowed Internet access
during open-book exams? Golub's suggested solutions are available online at
http://www.acm.org/ubiquity/views/v6i9_golub.html
May 5, 2005 message from Carolyn Kotlas
[kotlas@email.unc.edu]
TEACHING, TEACHING TECHNOLOGIES, AND VIEWS OF
KNOWLEDGE
In "Teaching as Performance in the Electronic
Classroom" (FIRST MONDAY, vol. 10, no. 4, April 2005), Doug Brent, professor
in the Faculty of Communication and Culture at the University of Calgary,
presents two views of teaching: teaching as a "performance" and teaching as
a transfer of knowledge through text, a "thing." He discusses the social
groups that have stakes in each view and how teaching will be affected by
the view and group that gains primacy. "If the group that values teaching as
performance has the most influence, we will put more energy into developing
flexible courseware that promotes social engagement and interaction. . . .
If the group that sees teaching as textual [i.e., a thing] has the most
influence, we will develop more elaborate technologies for delivering
courses as online texts, emphasising the role of the student as audience
rather than as participant." Brent's paper is available online at
http://firstmonday.org/issues/issue10_4/brent/index.html .
First Monday [ISSN 1396-0466] is an online,
peer-reviewed journal whose aim is to publish original articles about the
Internet and the global information infrastructure. It is published in
cooperation with the University Library, University of Illinois at Chicago.
For more information, contact: First Monday, c/o Edward Valauskas, Chief
Editor, PO Box 87636, Chicago IL 60680-0636 USA; email: ejv@uic.edu; Web:
http://firstmonday.dk/.
......................................................................
LAPTOPS IN THE CLASSROOM
The theme for the latest issue of NEW DIRECTIONS
FOR TEACHING AND LEARNING (vol. 2005, issue 101, Spring 2005) is "Enhancing
Learning with Laptops in the Classroom." Centered on the faculty development
program at Clemson University, the issue's purpose is "to show that
university instructors can and do make pedagogically productive and novel
use of laptops in the classroom" and "to advise institutional leaders on how
to make a laptop mandate successful at their university." The publication is
available online
http://www3.interscience.wiley.com/cgi-bin/jhome/86011233 .
New Directions for Teaching and Learning [ISSN:
0271-0633], a quarterly journal published by Wiley InterScience, offers a
"comprehensive range of ideas and techniques for improving college teaching
based on the experience of seasoned instructors and on the latest findings
of educational and psychological researchers." The journal is available both
in print and online formats.
......................................................................
NEW E-JOURNAL ON LEARNING AND EVALUATION
STUDIES IN LEARNING, EVALUATION, INNOVATION AND
DEVELOPMENT is a new peer-reviewed electronic journal that "supports
emerging scholars and the development of evidence-based practice and that
publishes research and scholarship about teaching and learning in formal,
semi-formal and informal educational settings and sites." Papers in the
current issue include:
"Can Students Improve Performance by Clicking More?
Engaging Students Through Online Delivery" by Jenny Kofoed
"Managing Learner Interactivity: A Precursor to
Knowledge Exchange" by Ken Purnell, Jim Callan, Greg Whymark and Anna
Gralton
"Online Learning Predicates Teamwork: Collaboration
Underscores Student Engagement" by Greg Whymark, Jim Callan and Ken Purnell
Studies in Learning, Evaluation, Innovation and
Development [ISSN 1832-2050] will be published at least once a year by the
LEID (Learning, Evaluation, Innovation and Development) Centre, Division of
Teaching and Learning Services, Central Queensland University, Rockhampton,
Queensland 4702 Australia. For more information contact: Patrick Danaher,
tel: +61-7-49306417; email: p.danaher@cqu.edu.au. Current and back issues
are available at
http://www.sleid.cqu.edu.au/index.php .
Bob Jensen's threads on education resources are at http://www.trinity.edu/rjensen/000aaa/newfaculty.htm#Resources
Bob Jensen's threads on assessment are at http://www.trinity.edu/rjensen/assess.htm
September 2, 2004 message from Carolyn Kotlas [kotlas@email.unc.edu]
SURVEY ON QUALITY AND EXTENT OF ONLINE EDUCATION
The Sloan Consortium's 2003 Survey of Online Learning
wanted to know would students, faculty, and institutions embrace online
education as a delivery method and would the quality of online education match
that of face-to-face instruction. The survey found strong evidence that
students are willing to sign up for online courses and that institutions
consider online courses part of a "critical long-term strategy for their
institution." It is less clear that faculty have embraced online teaching
with the same degree of enthusiasm. The survey's findings are available in
"Sizing the Opportunity: The Quality & Extent of Online Education in
the U.S., 2002 and 2003" by I. Elaine Allen and Jeff Seaman, Sloan Center
for Online Education at Olin and Babson Colleges. The complete report is
online at http://www.sloan-c.org/resources/sizing_opportunity.pdf
The Sloan Consortium (Sloan-C) is a consortium of
institutions and organizations committed "to help learning organizations
continually improve quality, scale, and breadth of their online programs
according to their own distinctive missions, so that education will become a
part of everyday life, accessible and affordable for anyone, anywhere, at any
time, in a wide variety of disciplines." Sloan-C is funded by the Alfred
P. Sloan Foundation. For more information, see http://www.sloan-c.org/
Bob Jensen's threads on the dark side of distance education are at http://www.trinity.edu/rjensen/000aaa/theworry.htm
Computer-Based Assessment
Free Book on Assessment in Europe (with particular focus in computer-based
assessment) ---
http://crell.jrc.it/RP/reporttransition.pdf
The Transition to Computer-Based Assessment; New Approaches to Skills
Assessment and Implications for Large-scale Testing
by Friedrich Scheuermann & Julius Björnsson (Eds.)
European Commission Joint Research Centre
Institute for the Protection and Security of the Citizen Contact information
Address: Unit G09, CRELL TP-361, Via Enrico Fermi, 2749; 21027 Ispra (VA), Italy
E-mail: friedrich.scheuermann@jrc.it
Tel.: +39-0332-78.6111
Fax: +39-0332-78.5733
Web http://ipsc.jrc.ec.europa.eu/
http://www.jrc.ec.europa.eu/
Table of Contents ---
http://crell.jrc.it/RP/reporttransition.pdf
Introduction
6
PART I: ASSESSMENT NEEDS AND EUROPEAN APPROACHES
Assessing and Teaching 21st Century Skills Assessment
Call to Action
13
Robert Kozma
The European Coherent Framework of Indicators and Benchmarks and
Implications for Computer-based Assessment 24
Oyvind Bjerkestrand
Computer-based Assessment and the Measurement of Creativity in Education
29
Ernesto Villalba
PART II: GENERAL ISSUES OF COMPUTER-BASED
TESTING
Experiences from Large-Scale Computer-Based
Testing in the USA 39
Brent Bridgeman
National Tests in Denmark – CAT as a Pedagogic Tool 45
Jakob Wandall
Introducing Large-scale Computerized Assessment – Lessons Learned and
Future Challenges 51
Eli Moe
Large-scale Computer-based Testing of Foreign Language Competences across
Europe: Technical Requirements and Implementation 57
Jostein Ryssevik
Delivery Platforms for National and International Computer-based Surveys
63
Sam Haldane eInclusion, eAccessibility and Design-for-All Issues
in the Context of European Computer-based Assessment 68 Klaus Reich &
Christian Petter Gender differences in cognitive tests: a consequence
of gender dependent preferences for specific information presentation
formats? 75 Romain Martin & Marilyn Binkley
PART III: TRANSITION FROM PAPER-AND-PENCIL
TO COMPUTER-BASED TESTING
Risks and Benefits of CBT versus PBT in
High-Stakes Testing 83
Gerben van Lent Transformational Computer-based Testing 92
Martin Ripley 5
Reflections on Paper-and-Pencil Tests to eAssessments: Narrow and
Broadband Paths to 21st Century Challenges 99
Katherina Kikis
Transition to Computer-based Assessment:
Motivations and Considerations 104
René Meijer
Transitioning to Computer-Based
Assessments: A Question of Costs 108
Matthieu Farcot & Thibaud Latour
Shifting from Paper-and-Pencil to Computer-based Testing: Requisites,
Challenges and Consequences for Testing Outcomes - A Croatian Perspective
117
Vesna Busko
Comparing Paper-and-Pencil and Online Assessment of Reasoning Skills: A
Pilot Study for Introducing TAO in Large-scale Assessment in Hungary 120
Benő Csapó, Gyöngyvér Molnár & Krisztina R. Tóth
PART IV: METHODOLOGIES OF COMPUTER-BASED
TESTING
Computerized and Adaptive Testing in
Educational Assessment 127
Nathan A. Thompson & David J. Weiss
Computerized Adaptive Testing of Arithmetic at the Entrance of Primary
School Training College (WISCAT-pabo) 134
Theo J.H.M. Eggen & Gerard J.J.M. Straetmans
Issues in Computerized Ability Measurement: Getting out of the Jingle and
Jangle Jungle 145
Oliver Wilhelm
New Constructs, Methods, & Directions for Computer-Based Assessment
151
Patrick C. Kyllonen
Measuring Complex Problem Solving: The MicroDYN Approach 157
Samuel Greiff & Joachim Funke
Testing for Equivalence of Test Data across Media 164
Ulrich Schroeders
PART V: THE PISA 2006 COMPUTER-BASED
ASSESSMENT OF SCIENCE (CBAS)
Utilising the Potential of Computer Delivered
Surveys in Assessing Scientific Literacy
172
Ron Martin
Are Icelandic Boys really better on
Computerized Tests than Conventional ones? Interaction between Gender, Test
Modality and Test Performance 178
Almar M. Halldórsson, Pippa McKelvie & Júlíus K. Björnsson
CBAS in Korea: Experiences, Results and Challenges 194
Mee-Kyeong Lee
How did Danish Students solve the PISA CBAS items? Right and Wrong
Answers from a Gender Perspective 201
Helene Sørensen & Annemarie Møller Andersen
Computer Grading of Essays
Sociology professor designs SAGrader software for grading student essays
Student essays always seem to be riddled with the same
sorts of flaws. So sociology professor Ed Brent decided to hand the work off to
a computer. Students in Brent's Introduction to Sociology course at the
University of Missouri-Columbia now submit drafts through the SAGrader software
he designed. It counts the number of points he wanted his students to include
and analyzes how well concepts are explained. And within seconds, students have
a score. It used to be the students who looked for shortcuts, shopping for
papers online or pilfering parts of an assignment with a simple Google search.
Now, teachers and professors are realizing that they, too, can tap technology
for a facet of academia long reserved for a teacher alone with a red pen.
Software now scores everything from routine assignments in high school English
classes to an essay on the GMAT, the standardized test for business school
admission. (The essay section just added to the Scholastic Aptitude Test for the
college-bound is graded by humans). Though Brent and his two teaching assistants
still handle final papers and grades students are encouraged to use SAGrader for
a better shot at an "A."
"Computers Now Grading Students' Writing," ABC News, May 8, 2005 ---
http://abcnews.go.com/Technology/wireStory?id=737451
Jensen Comment: Aside from some of the obvious advantages such as grammar
checking, students should have a more difficult time protesting that the grading
is subjective and unfair in terms of the teacher's alleged favored versus
less-favored students. Actually computers have been used for some time in
grading essays, including the GMAT graduate admission test ---
http://www.yaledailynews.com/article.asp?AID=723
References to computer grading of essays ---
http://coeweb.fiu.edu/webassessment/references.htm
You can read about PEG at
http://snipurl.com/PEGgrade
MEDICAL- AND BUSINESS-SCHOOL ADMISSION TESTS WILL BE GIVEN BY COMPUTER
Applicants to medical and business schools will
soon be able to leave their No. 2 pencils at home. Both the Medical College
Admission Test and the Graduate Management Admission Test are ditching their
paper versions in favor of computer formats. The Association of American Medical
Colleges has signed a contract with Thomson Prometric, part of the Thomson
Corporation, to offer the computer-based version of the MCAT beginning in 2007.
The computerized version is being offered on a trial basis in a few locations
until then.The GMAT, which has been offered both on paper and by computer since
1997, will be offered only by computer starting in January, officials of the
Graduate Management Admission Council said. The test will be developed by ACT
Inc. and delivered by Pearson VUE, a part of Pearson Education Inc.The Law
School Admission Council has no immediate plans to change its test, which will
continue to be given on paper.
The Chronicle of Higher Education, August 5, 2005, Page A13
Jensen Comment: Candidates for the CPA are now allowed to only take
this examination via computer testing centers. The GMAT has been an
optional computer test since 1997. For years the GMAT has used
computerized grading of essay questions and was a pioneer in this regard.
Assessment in General
Assessment/Learning
Issues:
Measurement and the No-Significant Differences
Assessment of new technology in learning is impossible to formally evaluate
with both rigor and practicality. The main problem is the constantly changing
technology. By the time assessment research is made available, the underlying
technologies may have been improved to a point where the findings are no longer
relevant under the technologies existing at the time of the research. What
can be done for students after my university installed a campus-wide network is
vastly different than the before-network days. A classroom failure using last
year's technology may not be appropriate to compare with a similar effort using
newer technology. For example, early LCD panel projections from computers in
classrooms were awful in the early 1990s. In the beginning, LCD
panels had no color and had to be used in virtually dark classrooms. This was a
bad experience for most students and instructors (including me). Then new
technology in active matrix LCD panels led to color but the classrooms still had
to be dark. Shortly thereafter, new technologies in overhead projection
brightness allowed for more lighting in classrooms while using LCD panels.
However, many classrooms are not yet equipped with light varying controls to
optimally set lighting levels. Newer trends with even better three-beam
projectors and LCD data projectors changed everything for electronic classrooms,
because now classrooms can have normal lighting as long as lights are not aimed
directly at the screen. The point here is that early experiences with the first
LCD panel technology are no longer relevant in situations where the latest
projection technology, especially in fully equipped electronic classrooms, is
available. Unfortunately, there is a tendency among some faculty to be so
discouraged by one or two failed attempts that they abandon future efforts using
newer technologies.
One of the most creative attempts to evaluate effectiveness from a Total
Quality Management (TQM) perspective is reported by Prabhu and Ramarapu (1994).
This is an attempt to measure learning using a TQM database that can be used to
compare alternative teaching methods or entire programs. [Prabhu, S.S. and
N.K. Ramarapu (1994). “A prototype database to monitor course effectiveness: A
TQM approach,” T H E Technological Horizons in Education, October,
99-103.]
It is easy to become discouraged with first efforts using older technologies.
Many faculty and students became highly frustrated with the early complexities
of using the Internet and/or campus networks that were not user friendly. Unless
they took the time and trouble to become well versed in UNIX programming and
became experienced hackers, the Internet turned into a totally discouraging
nightmare. Now with the WWW and many other user-friendly innovations in campus
and international networking, the need to become an experienced hacker is vastly
reduced.
"When Coaching and Testing Collide," by Lee S. Shulman, Carnegie
Perspectives," May 2008 ---
http://www.carnegiefoundation.org/perspectives/sub.asp?key=245&subkey=2598
It's a scene we have watched dozens of times in the
movies. A young man or woman of modest talent tries out for the baseball or
football or basketball team under the tutelage of a gruff, demanding coach
who expresses initial doubts about the likelihood that the kid will prove
himself or herself worthy of a spot on the team. The coach is tough and
persistent, setting high standards and then mercilessly driving all his
charges to meet them. In the climactic scene at the season's end, the good
guys or gals are losing by several baskets, or runs, or a
touchdown—depending on the sport. "Send me in, coach," pleads our young
hero/ine, which coach reluctantly does. The kid scores the winning points,
and the team wins. The coach turns out to have a heart of gold, and the
reasons for his seeming cruelty become apparent.
What exactly is it that the coach provides the
aspirant? Let me propose five processes associated with both the coach and
mentor roles: 1) technique, learned through endless drill; 2) strategy, that
allows the person who is coached to become capable of a conception of the
work that will turn out to be pivotal in their eventual victory; 3)
motivation, which produces a "Rocky-like" level of commitment that will help
them exceed their own and others' expectations; 4) vision, where players
come together in a new vision of the process and their capabilities for
success; and 5) identity, whereby the protagonist not only wins, but is
transformed, with an internalized new sense of self.
In sports there is always a clear line between the
coaching situation and the performance context. When the final jump shot is
made from the three-point line by the basketball player, the coach can't
jump onto the court and give the ball the extra momentum or spin it might
need. I prefer to call such typical relationships between a coach/mentor and
player/protégé examples of unmediated mentoring. No separate product comes
in the middle between the coaching and the performing that renders the
relative contributions of the coach and the coached inherently ambiguous
because the entire performance is visible and is itself the basis for
evaluating success or failure.
There is, however, an entire genre of mediated
mentoring. The performance is not directly observed and has yielded a
product which is the focal point of competition and evaluation. Thus in the
case of mediated performances, the respective roles of coach and performer
are inherently invisible. Although the five processes are in place and just
as transformative, there is inherently no way to discern how much of the
work was done independently by the candidate, by peers or by advisors.
Whenever mentoring is mediated by a product whose
actual authoring processes are not directly observable, as is the case with
literature, objects of architectural or mechanical design, scholarly
publications, doctoral dissertations, and even paintings, assessment of
individual competence is problematic. But are these problems of educational
measurement or a new set of realities regarding the conditions of expert
performance? Stanford education professor Sam Wineburg and others point out
that the crux of the problem may not be measurement error but rather the
inherently social and interactive character of the performances whose
competence is assessed. Writing is and should be critiqued and edited, as
should painting, the designs for buildings and the research performed in
scientific laboratories. To avoid mentoring merely to ensure the legitimacy
of individual test scores might even be judged a form of malpractice! So we
are faced with an essential tension between the inherently social character
of most forms of complex human performance and the psychometric imperative
to estimate a "true score" for ability or any other personal trait using the
individual as the unit of analysis.
In an education setting, the distinction between
the scores that a student earns on any test-like event—multiple choice test,
essay exam, portfolio or senior sermon in a seminary—and their underlying
"true" capability is a reflection of the distinction, borrowed perhaps from
the field of linguistics, between competence and performance. Psychometrics
rests on the claim that the observed performance is a valid indicator if it
tracks the underlying competence faithfully. But what if mentored or coached
performances actually track underlying competence more validly than
measurement of students working alone? What if the composition written by a
student in the presence of his editing team is a better indicator of his
future writing competence than having him write alone?
That is what sits at the heart of the puzzle.
My proposal for "getting over" this essential
tension is three-fold: making changes in the processes of assessment, making
explicit the parameters of mentoring, and developing a clear code of ethical
principles for both assessment and mentoring. At the heart of these
proposals is the principle of transparency. Everything possible must be done
to ensure that the roles of mentors, peers and students be transparently
clear in any mediated mentoring activity. There should be ways of reporting
on the character of coaching for test performance that make the efforts of
the coach entirely transparent to assessment.
I have often written that collaboration is a
marriage of insufficiencies; that students can work together in ways that
scaffold and support each others' learning, and in ways that support each
others' knowledge. Now I call for a marriage of sufficiencies to overcome
the essential tensions between individual work and collaborative
performance, coaching support and independent assessment, the mentor as an
agent of zealous advocacy and the mentor as a steward of the commons.
As Dewey observed, we will not solve this problem,
we will get over it. It is built in to the psychometric paradox: Our
measurement models are psychometric but our assessment needs are often
sociometric, requiring the measurement of socially scaffolded and joint
productions.
Carnegie Perspectives ---
http://www.carnegiefoundation.org/perspectives/
|
From The Wall Street
Journal Accounting Weekly Review on November 17, 2006
TITLE: Colleges, Accreditors Seek Better Ways to Measure Learning
REPORTER: Daniel Golden
DATE: Nov 13, 2006 PAGE: B1
LINK:
http://online.wsj.com/article/SB116338508743121260.html?mod=djem_jiewr_ac
TOPICS: Accounting
SUMMARY: The article discusses college- or university-wide
accreditation by regional accreditation bodies and reaction to the
Spellings Commission report. Questions extend the accreditation
discussion to AACSB accreditation.
QUESTIONS:
1.) What is accreditation? The article describes university-wide
accreditation by regional accrediting bodies. Why is this step
necessary?
2.) Does your business school have accreditation by Association to
Advance Collegiate Schools of Business (AACSB)? How does this
accreditation differ from university-wide accreditation?
3.) Why are regional accrediting agencies planning to meet with
Secretary Spellings?
4.) Did you consider accreditation in deciding where to go to college
or university? Why or why not?
5.) Do you think improvements in assessing student learning are
important, as the Spellings Commission argues and accreditors are now
touting? Support your answer.
SMALL GROUP ASSIGNMENT: Find out about your college or university's
accreditation. When was the last accreditation review? Were there any
concerns expressed by the accreditors? How has the university responded
to any concerns expressed?
Once these data are gathered, discuss in class in groups:
Has this information been easy or difficult to find? Do you agree
with the assessment of concerns about the institution and/or the
university's responses?
Reviewed By: Judy Beckman, University of Rhode Island
TITLE: Colleges, Accreditors Seek Better Ways to Measure Learning
REPORTER: Daniel Golden
DATE: Nov 13, 2006 PAGE: B1
LINK:
http://online.wsj.com/article/SB116338508743121260.html?mod=djem_jiewr_ac
At the University of the South, a
highly regarded liberal-arts college in Sewanee, Tenn., the dozen
professors who teach the required freshman Shakespeare course design
their classes differently, assigning their favorite plays and
writing and grading their own exams.
But starting next fall, one question
on the final exam will be the same across all of the classes, and
instructors won't grade their own students' answers to that
question. Instead, to assure more objective evaluation, the
professors will trade exams and grade each other's students.
The English department adopted this
change -- despite faculty grumbling about losing some classroom
independence -- under pressure from the Southern Association of
Schools and Colleges. The association, one of the six regional
groups that accredit nearly 3,000 U.S. colleges, told the University
of the South that, to have its accreditation renewed, it would have
to do a better job of measuring student learning. Without such
accreditation, the school's students wouldn't qualify for federal
financial aid.
The shift "does cut into the
individual faculty member's autonomy, and that's disturbing," says
Jennifer Michael, an associate professor. "On the other hand, it's
making us think about how do we figure out what students are
actually learning. Maybe having them take and pass a course doesn't
mean they've learned everything we think they have."
Regional accreditors used to limit
their examinations to colleges' financial solvency and educational
resources, with the result that well-established schools enjoyed
rubber-stamp approval. But now they are increasingly holding
colleges, prestigious or not, responsible for undergraduates' grasp
of such skills as writing and critical thinking. And prodded by
regional accreditors, colleges are adopting various means of
assessing learning in addition to classroom grades, from electronic
portfolios that collect a student's work from different courses to
standardized testing and special projects for graduating seniors.
The accreditors aren't moving fast
enough for the Bush administration, though. In the wake of a
federally sponsored study published in 2005 that showed declining
literacy among college-educated Americans, Secretary of Education
Margaret Spellings and a commission she appointed on the future of
higher education want colleges to be more accountable for -- and
candid about -- student performance, and they have criticized
accreditors as barriers to reform.
Congress sets the standards for
accreditors, and the Education Department periodically reviews
compliance with those standards. Congress identified "success with
respect to student achievement" as a requirement for accreditation
in 1992, and then in 1998 made it the top priority. That imperative,
along with the advent of online education, has spurred accreditors
to rethink their longtime emphasis on such criteria as the number of
faculty members with doctorates. Since 2000, several regional
accreditors have revamped their rules to emphasize student learning.
"Accreditors have moved the ball
forward," says Kati Haycock, a member of the Spellings commission
and the director of the nonprofit Education Trust in Washington,
D.C., which seeks better schooling for disadvantaged students. "Not
far enough, not fast enough, but they have moved the ball forward."
An issue paper written for the
commission by Robert Dickeson, a former president of the University
of Northern Colorado, complained that accreditation "currently
settles for meeting minimum standards," and it called for replacing
regional accreditors with a new national foundation. "Technology has
rendered the quaint jurisdictional approach to accreditation
obsolete," Mr. Dickeson wrote.
The commission didn't endorse that
recommendation, but its final report last month cited "significant
shortcomings" in accreditation and called for "transformation" of
the process. In a Sept. 22 speech marking the release of the report,
Secretary Spellings said that accreditors are "largely focused on
inputs, more on how many books are in a college library than whether
students can actually understand them....That must change."
David Ward, a commission member and
the president of the American Council on Education, a higher
education advocacy group, declined to sign the report, in part
because he objected to its criticism of accreditors as overly
simplistic.
Russell Edgerton, president emeritus
of the American Association for Higher Education, says "there's no
question that American colleges are underachieving," but he argues
that accreditors are rising to the challenge. "Ten years ago, I
would have said that regional accreditors are dead in the water and
asleep at the wheel," he says. But "there's been a kind of
renaissance within accreditation agencies in the past five to six
years. They're helping institutions create a culture of evidence
about student learning."
Mr. Edgerton also thinks the federal
government's emphasis on new accountability measures is flawed
because it bypasses the judgment of traditional arbiters like
faculty and accreditors. "The danger is that the standardized
testing approach in K-12 would slop over into higher education," he
says. "Higher ed is different."
Jerome Walker, associate provost and
accreditation liaison officer for the University of Southern
California, agrees that the administration's attacks on accreditors
are unfair. The Western Association of Schools and Colleges, which
accredits USC, "has been extremely sensitive" to student learning,
he says.
According to the Western Association's
executive director, Ralph Wolff, the group revamped its standards in
2001 to require colleges to identify preparation needed by entering
freshmen and the expectations for student progress in critical
thinking, quantitative reasoning and other skills. Its accreditation
process now takes four years, up from 1½, and it features a
detailed, peer-reviewed proposal for improvement and two site
visits, including one devoted to "educational effectiveness."
Historically, research universities
like USC "used to blow off" accreditation, Mr. Wolff says. "Now this
has become a real challenge for them in a good way."
Encouraged by Mr. Wolff, USC last year
assigned the same two essay questions -- one about conformity,
another based on a quotation from ethicist Robert Bellah -- to
freshmen in a beginning writing course and juniors and seniors in an
advanced course. A group of faculty then evaluated the essays
without knowing the students' names or which course they were
taking. The reassuring outcome, according to Richard Fliegel,
assistant dean for academic programs, was that juniors and seniors
"demonstrated significantly more critical thinking skills" than
freshmen, and that advanced students who had taken the first-year
course outperformed transfer students who hadn't taken beginning
writing at USC.
Because the writing initiative is
tailored to USC's curriculum, the results -- while helpful to
administrators and accreditors -- wouldn't necessarily help the
public compare USC to other schools. That is a big drawback as far
as the Bush administration is concerned. "I have two kids in college
now," says Vickie Schray, deputy director of the Spellings
commission. "It's a huge expense. Yet there's very little
information on return of investment or ability to shop around for
the greatest value."
She adds, though, that it is a
"misconception" to think that the administration wants to have "one
standardized test for all institutions" or to extend the testing
requirements of the "No Child Left Behind" law for K-12 schools to
higher education.
Even so, one standardized test of
critical thinking, the Collegiate Learning Assessment, is becoming
popular. It adjusts for students' scores on the SAT and ACT
college-entrance exams, potentially allowing more meaningful
comparisons of the value added by colleges. The number of schools
using the assessment has soared from 54 two years ago to 170 this
year. Among those using the test this fall: the University of Texas
at Austin, Duke University, Arizona State University and Washington
and Lee University.
Roger Benjamin, president of the
nonprofit Council for Aid to Education, which sponsors the test,
says state officials and university administrators have been the
principal forces behind its increasing use. "Accreditors are coming
to the party, but a bit late," Mr. Benjamin says.
Meanwhile, Secretary Spellings plans
to meet with accreditors in late November to discuss how to
"accelerate the focus on student achievement," Ms. Schray says.
Accreditors say they welcome the opportunity to tout their progress.
"We have made a lot of reforms," says the Western Association's Mr.
Wolff. "We'd like to bring the secretary up-to-date on the
significance of these reforms and the impact they're already having
on institutions."
|
As David Bartholomae observes, “We make a huge
mistake if we don’t try to articulate more publicly what it is we value in
intellectual work. We do this routinely for our students — so it should not be
difficult to find the language we need to speak to parents and legislators.” If
we do not try to find that public language but argue instead that we are not
accountable to those parents and legislators, we will only confirm what our
cynical detractors say about us, that our real aim is to keep the secrets of our
intellectual club to ourselves. By asking us to spell out those secrets and
measuring our success in opening them to all, outcomes assessment helps make
democratic education a reality.
Gerald Graff, "Assessment Changes
Everything," Inside Higher Ed, February 21, 2008 ---
http://www.insidehighered.com/views/2008/02/21/graff
Gerald Graff is professor of English at the University of Illinois at Chicago
and president of the Modern Language Association. This essay is adapted from a
paper he delivered in December at the MLA annual meeting, a version of which
appears on the MLA’s Web site and is reproduced here with the association’s
permission. Among Graff’s books are Professing Literature, Beyond the
Culture Wars and Clueless in Academe: How School Obscures the Life of the Mind.
The consensus report, which was approved by the
group’s international board of directors, asserts that it is vital when
accrediting institutions to assess the “impact” of faculty members’ research on
actual practices in the business world.
"Measuring ‘Impact’ of B-School Research," by Andy Guess, Inside
Higher Ed, February 21, 2008 ---
http://www.insidehighered.com/news/2008/02/22/impact