Journal of Effective Teaching

Peter W. Schuhmann
Department of Economics and Finance
University of North Carolina at Wilmington
601 South College Rd.
Wilmington, NC 28403-3297
Phone: (910) 962-3417
Fax: (910) 962-7464
Email: schuhmannp@uncwil.edu

KimMarie McGoldrick
Department of Economics
University of Richmond
Richmond, VA 23173
Phone: (804) 289-8575
Fax: (804) 289-8878
Email: kmcgoldr@richmond.edu

School pride is often borne of direct competition between universities. The most obvious manifestation of this is in collegiate sports. Yet there is no reason to presume that similar feelings of regard for one's institution (or disregard for another’s) could not be created in an academic setting. To explore this possibility and attempt to capitalize on the impact it could have on student learning and interest in economics, the authors devised a simple web-based method for allowing students at the University of Richmond (UR) and the University of North Carolina at Wilmington (UNCW) to "compete" with students at the other university in a manner that applied course content.[1] By providing the opportunity to improve their grades and incorporating rivalry incentives, we hope to encourage student interest in the subject beyond that which would be attained otherwise. The assignment detailed in this paper is used primarily for pre-exam review; hence we set a timeline for the assignment to ensure that students are engaged in the material well in advance of the exam. Further, we construct the format of the assignment in such a way as to give students exposure to working in groups, writing and answering original questions, critically thinking about the process of grading, and using computer software and the Internet outside of the classroom.

At the beginning of the fall 2000 semester, a WebBoard discussion group was established using server space at the University of Richmond. Similar to UseNet, WebBoard is an interface that allows the user to post and reply to written messages using a web browser. Approximately three weeks prior to the first exam, an out-of-class tutorial was conducted at each institution to introduce students to the rules of the competition and demonstrate the WebBoard software.[2] Notably, the majority of students at UNCW had never been exposed to WebBoard while most students at UR had. Another benefit of this project then, at least for some students, is exposure to a software program that they otherwise might not encounter.

Following the tutorial, interested students in principles of microeconomics courses at the University of Richmond and the University of North Carolina at Wilmington were put into groups of 2 or 3 and given user names and passwords to access the site. Class sizes at UNCW were approximately 60 while those at UR were around 25. Placing students in groups of different sizes (larger groups at UNCW) allowed for a matching of team numbers across the two schools. By requiring them to work with other students, we hope to encourage the development of group studying skills and cooperation.

Two weeks prior to each exam, the web board was opened for student postings. Each student was required to create and post one question pertaining to the material to be covered on the upcoming exam, and to answer one question posted by a student group at the other institution. Additionally, the question-writer was required to grade the answer to their question, and grade the question that they answered. The format for questions was left up to the students, but we required that all answers be constructed in essay form. In this way, students are given the opportunity to further develop their writing skills and “practice” writing about a topic before being graded by the instructor. Further, this method may be particularly important for larger classes because it creates an opportunity for the writing of essays that might otherwise be hindered by the grading burden that it will place on the instructor. By setting specific time deadlines for the posting of questions and answers, and grading, we ensure that students begin thinking about the exam material at least one week prior to the exam itself. In this way, we hope to get students to recognize the value of preparing further in advance than the common behavior of “cramming” at the last minute, and hopefully develop more effective study skills.

A combination of the student and instructor grades given for the questions and answers generated a final score for each group. By requiring that students not only pose and answer questions, but also grade questions and answers from other students, we foster critical thinking about the issues under study and the grading process itself. For each round of questions and answers, the “winner” of the round from each school (the group with the highest score) had 8 bonus points added to their exam score. The second place group was awarded 5 bonus points and 3 bonus points were awarded for third place from each school. Everyone participating in the session was awarded 1 bonus point, with all participants from the school with the highest average score receiving an additional 2 bonus points.

While most students showed initial enthusiasm for the project, participation rates varied across the two schools and over the three periods during which these sessions occurred. Sixty-seven percent of students in the UR classes participated in the first bonus session while the participation rate of UNCW students was 39 percent. Participation dropped in the second round to 33 percent participation by the UR students and 25 percent by the UNCW students.[3] In the final round, participation dropped slightly for UR students (to 30 percent) while it rose among UNCW students (to 30 percent). As participation at UNCW was lower than at UR during the first two rounds, we suspect that more self-selection took place at UNCW. This may indicate that the caliber of students that participated was higher than average at UNCW, and as a result there was a better match with academically stronger UR students in terms of academic ability.

Demographic information about students that participated in at least one of the three potential rounds during the semester indicate that 25.7 percent belonged to either a fraternity or sorority, 16 percent were active on a sports team, and nearly 9 percent were involved in student government. On average, students gave themselves a rating of 3.33 out of 5 (= expert) on their computer abilities. Finally, the average GPA for students participating was 2.94 with no significant difference between UR and UNCW students.

As suggested above, each student was required to contribute to the grading of both questions and answers. In order to help minimize the subjectivity in student grading, a list of important components of good questions and answers was provided. Perhaps more importantly, the detailed grading criteria and accompanying graded example (both provided in the instructional handout) sent a direct message to students about how to properly answer questions. By grading questions themselves and following the criteria, students are more apt to realize the instructor’s perspective in grading and use this information to construct complete answers in their own work. Hence beyond the studying of specific course material, this procedure provides a valuable lesson to students for future courses where essay questions are used.

· Does the question require critical thinking/application of problem solving methods or merely memorization? (0-20 points)

Note that the above criteria yield a score between 1 and 25 for each question and answer. The instructor also graded the questions and answers to each question using the same criteria. The student’s score and the instructor’s score were then added to form the full grade to each answer and each question. As a result, each student group received a score out of 100 points. In order to provide an incentive for honesty in grading, if the instructor’s score of the question or answer was greater than the student’s score, the difference was deducted from the student’s score. Student grades were submitted to the instructor via email.

The quality of student work varied, as can be expected with any assignment. Below is an example of a typical student posting.

For a given firm, assume that the price of capital rises. Explain how this affects its use of both labor and capital. How does the rise in the price of capital relate to the Law of Demand?

Capital and labor complement each other in the fact that if you add better capital to labor then you are going to increase productivity all together. However, labor and capital can also be substituted for each other if labor becomes expensive then various other things could take its place. In this case, when the price of capital increases the amount of labor will decrease. Associating this with the equimarginal principle: MPc/Pc=MPl/Pl...if you increase Pc then you decrease MPl. This enforces the Law of Demand by showing that as price increases, quantity demanded decreases where price is for capital and quantity demanded is number of workers.

· Background information: 3--information given about the equimarginal principle doesn't seem to apply, though they included the explanation of the law of demand.

· Clarity and Focus: 4--was understandable, though they used "amount of labor" instead of "quantity demanded of labor"

· Completeness/Correctness: 3--we were looking specifically for them to mention the substitution effect and output effect and explain what each of them means and then apply that to the law of demand where as the price of capital goes up, the quantity demanded of capital goes down.

· Background information: 5-- The answer identified that the inputs two can be either substitutes or complements.

· Clarity and Focus: 4-- The answer was linked to the principles necessary to answer the question and contained no extraneous information. However, the link between increases in Pc and decreases in MPl was not clearly explained.

· Relevance: 4-- The application of the equimarginal principle can be used to explain why the increase in the price of capital leads to a decreased in the quantity demanded of capital.

· Completeness/Correctness: 6-- The answer did contain an error in that when the price of capital increases we would expect the substitution effect to increase the amount of labor used while the output effect suggests a decrease in the amount of labor used. Also, the law of demand clearly does not apply to changes in the price of capital leading to changes in the quantity demanded of workers.

Note that in this case, the instructor’s grade was 4 points higher than that given by the students, hence the UR student group received a 4-point deduction from their score.

Effectiveness

A student survey was administered at the end of the semester to gauge the effectiveness and perceived difficulty of the web board competition. Students who participated in at least one session were asked to rate the level of difficulty and effectiveness of the project in fostering learning on a scale of 1 to 5. Students were also asked to provide an approximation of the time they spent on each session. These results are shown in Table 1. Note that UNCW students generally spent more time on the assignment (nearly double the minutes as UR students) and reported it to be more effective for their learning. We suspect that this may again be due to the more apparent self-selection at UNCW. Both groups of students rated it moderately difficult and found it to be a worthwhile expenditure of their time. As no definition of “worthwhile” was provided, this may simply be an expression of student satisfaction with the extra points received for the amount of time spent completing the assignment.

Because the competition took place across two different universities there were some logistical difficulties that arose during its implementation. First, in order to ensure that students at each school had the same background materials prior to each session, both instructors’ courses had to be closely matched. This was accomplished in a pre-semester meeting where a common syllabus was developed, a textbook was agreed upon, lecture content was discussed, and a schedule for exams and the web board sessions was established. It was necessary to have exams on the same days at the two universities so that the session dates for questions and answers were identical across classes.

Beyond the initial set-up, a few difficulties were encountered during the implementation of the procedure itself. The grading of questions and answers, while not more burdensome than grading of traditional answers, presented some difficulties because each question had to be matched up with an answer (and vice versa) from the other institution. Three different scenarios were used to share the grading burden between the instructors. For the first exam, each instructor graded his or her own student’s work. For the second, each instructor graded the other instructor’s student’s work. For the last exam, one instructor graded all the questions, while the other graded all the answers. During each session, both instructors graded a random sample of the other’s questions and/or answers to ensure comparable grading criteria. Student and instructor grades for questions and answers were matched for each student group and penalties were assessed for harsh grading in a common spreadsheet that was shared between the instructors via email.

Another difficulty was that some students only partially completed the requirements for participation in one of the sessions. Most incidences of this were students answering a question from the other university without posting a question of their own. Some groups posted questions but did not answer one, and still others posted questions and answers, but failed to complete the grading. All of these resulted in the group being disqualified from the session and earning zero bonus points. However, because of the logistical difficulties that incomplete participation created during the first two sessions (for example, not enough questions being posted to allow for an answer to be posted by all students at the other university), a 2-point penalty for starting but not finishing the session was imposed on the final session. This penalty was an effective deterrent – all groups who started the last session completed all the required tasks. However, few students were deterred from participating at all. Notably, participation at UR only fell from 33 percent during exam 2 to 30 percent during exam 3. Participation at UNCW actually increased.

Finally, some students learned that they could at least partially influence their own grades and the probability that their school would win the competition. This is because the penalty for low scores in grading other students’ work was relatively severe (the direct loss of points from the student’s score based on the difference between the student’s grade and the instructor’s grade), but there was little penalty for assigning high scores (the students only risked the contribution of the other team’s score to the opposing school’s average, and the corresponding effect on the probability that the other school would win). Hence in the final round we observed a much higher average grade given by students and a much smaller variance around that average.

Although much of our evidence is anecdotal, we believe that this competition achieved our goals of helping students develop more effective writing and study skills, exposing them to a new computer software, and critically thinking about the process of grading. Students were motivated to participate in an exercise that required them to understand the economics they were learning in class sufficiently to construct and ask economic questions, reply to other students' questions and evaluate questions and answers. Student responses to a post participation survey suggest that they found the exercise to be worthwhile. When asked “What is the primary benefit for students of the WebBoard project?” students replied with comments such as: “studying early,” “type of study guide/review for tests,” “reinforce topics discussed in class and makes us apply theory to real world situations,” and “increased interest in economics and learning through competition.” In general, student comments indicated that one of the more beneficial aspects of the competition was that, because of the timing of the sessions, they were forced to begin critical thinking about the material well in advance of the exam date. As a result students had to do less last-minute studying and were better prepared for the exam. Students also indicated that the browsing of questions and answers posted by other groups was a valuable study aid.

Acknowledging that because of the voluntary nature of the assignment, self-selection may be an issue, we note that pre-bonus exam scores were on average 7.7 and 7.9 points higher for those who participated in the corresponding web board session than those who did not at UNCW and UR respectively.[4] Further, for students who participated in some but not all of the sessions, the pre-bonus scores on the exams for which they participated were approximately 3.4 points higher on average at UNCW and 3.5 points higher at UR.

One unique aspect of this project is the potential adaptability to other environments. Competition between students does not have to be at the inter-university level to be effective. In fact, competition between sections for a single professor should prove to be just as effective in motivating students, and may present fewer of the logistical difficulties noted above. In addition, competitions across professors at the same institution should present similar opportunities.

This assignment is voluntary and used to generate bonus points for exam grades. You may volunteer for each and any of the three windows of bonus point opportunities (one prior to each exam). Students in principles of microeconomics classes at University of Richmond (UR) and the University of North Carolina at Wilmington (UNCW) will use a "WebBoard" online discussion group to pose and answer questions. Both questions and answers will be graded by the students and the instructors.

The WebBoard is a user interface that allows the user to post and reply to written messages using a web browser. Students will be given user names and passwords to access the site. Twelve days prior to each exam, the web board will be opened up for student postings for a 9-10 day period. Each student group will be required to post one question and answer one question posted by a student group at the other institution. Additionally, the question-writer will be required to grade the answer to their question, while the student who answered the question will be required to grade the question itself.

For each round of questions and answers, the “winner” of the round from each school (those with the highest score) will have 8 bonus points added to their exam score. 5 bonus points will also be awarded for second place and 3 bonus points will be awarded for third place from each school. Everyone participating will earn at least 1 bonus point for each completed session.

· All questions must be one of the following three formats: essay, multiple choice, true false. See examples below.

· No question should come from any textual source that you might have access to. If questions are not original 10 points will be deducted from your exam grade as this is considered an honors violation.

· There will be a 4-5 day window for question posting (although answers may be posted at any time once a question is posted up until the web board session ends).

2. Identify the topics and chapters from the text associated with your question.

4. Grade answer to question once it is posted. E-mail the grade along with a copy of the answer to the instructor. Be sure to include a justification for each component of your grade. See grading directions below.

· All answers must be essay format, regardless of the format of the question. Thus, if you are answering a multiple-choice question your answer must provide a discussion of why the answer is correct, etc. For further details see grading direction below.

· Answers must be in your own words. Any answers that use sources without proper citation and quotations will lead to 10 points being deducted from your exam grade as this will be considered an honor violation.

· Once a question is posted it is available to be answered by any group at the opposing university. (While questions will be posted only during the first 5 days of the session, answers can be posted up to three days prior to the exam.) It is up to you as an respondent to determine whether you wish to answer a question as it becomes posted or take a chance that a better questions will be posted at a later date.

1. Grade the question according to the directions outlined below. E-mail the grade along with a copy of the question to the instructor. Be sure to include a justification for each component of your grade.

· Does the question require critical thinking/application of problem solving methods or merely memorization? Be sure to take into account the level of difficulty of the question. (0-10 points)

The above criteria yield a score between 1 and 25. The instructor will also grade the questions and answers to each question. Your score and the instructors score will be added to form the full grade to each answer and each question. In order to provide an incentive for honesty in grading, if the instructor’s score of the question or answer is greater than your score, than the difference will be deducted from your score. For example, suppose UR group 1 posts a question that is replied to by UNCW group3 and UR group 1 answers a question posted by UNCW group 2. UR group 1 grades (harshly) the question posted by UNCW group 2 as a 20 (out of 25). The instructor grades (fairly) the same question as a 24 (out of 25). Obviously this is an unfair transaction for UNCW group 2. Thus, the grading process has a built in incentive to grade fairly. In this case, the difference between the instructor grade of 24 and the student grade of 20 will be deducted from the grade of UR group 1 who graded harshly.

The total possible points for any given group per session is 100. There are five components to this grade.

Carefully explain the idea of opportunity costs. Be sure to include definitions and an example in your answer.

Opportunity cost is the value of what is sacrificed when a choice is made. From page 2 of the Case and Fair text, opportunity cost is defined as “That which we forego, or give up, when we make a choice or decision.” Opportunity cost is a result of the fact that resources such as time and money are scarce. For example, suppose that you are hungry and are deciding with your friends where to go to lunch. You have the following options: pizza, burgers, salad bar, or sushi. You vote and decide on pizza. The opportunity cost of this choice is giving up burgers, salad bar and sushi.

Question is not particularly well focused. Opportunity cost is a concept that can be applied across a wide spectrum of scenarios such as individuals, firms and societies. Thus, it is hard to know when to begin in our answer because few guidelines for constructing our answer were provided.

Economics is the study of choices and scarcity. Therefore opportunity cost is fundamental to the study of economics.

· Does the question require critical thinking/application of problem solving methods or merely memorization? 9 of 10 points

Two reasons: first it asks for an explanation as opposed to simply the definition and then it further asks for a demonstration of our understanding through an example.

Question 1 is graded by instructor as a 22 out of 25. The difference is based on the clarity of focus. The instructor gave a grade of 8 of 10 points since they felt that the question did focus on a definition, understanding, and application of a particular topic.

They defined opportunity cost in their own words and provided a properly cited quote from the test. Further, they also provided the reason that opportunity cost arises, “scarcity.”

The example provided was clearly incorrect. It is obvious that the opportunity cost of choosing pizza is only one of the other alternatives, namely the next best alternative, and not all the alternatives.

These will include a one-hour evening session for demonstration and student inquiry

[1] It should be noted that there is a strong athletic rivalry between these two schools. Both schools compete in the Colonial Athletic Association in NCAA division 1 basketball. In the past two years (1999, 2000) these teams met in the finals of the CAA basketball tournament, with the victor (UR in 1999 and UNCW in 2000) advancing to the NCAA tournament while the loser did not. R

[2] Instructions were provided to students at this time, see Appendix A for an abbreviated version of these. Also, students were provided a set of reminder points just prior to the first session. (A copy of these is available upon request.) R

[3] Each of these differences in participation is significant at the one percent level. R

[4] The average number of bonus points earned for those participating in a WebBoard session was 3.3 at UNCW and 2.5 at UR. For those participating in every session the average number of bonus points earned was 3.6 at UNCW and 2.5 at UR. R

	UR	UNCW	Significant Difference
Mean effectiveness of project (5 max effectiveness)	2.679	4.000	1%
Mean difficulty of assignment (5 very difficult)	3.000	3.200	Insignificant
Mean time spent (in minutes)	67.105	102.300	2%
Worthwhile? (yes/no)	100%	89%	2%


	The Journal of Effective Teaching an online journal devoted to teaching excellence