Setting the Test's Minimum Passing Level (MPL) or Passing Mark

About this video

• Video Title: Setting the Test's Minimum Passing Level (MPL) or Passing Mark
• Channel: Darbs Santi
• Speakers: Darbs Santi
• Duration: 00:16:09

Overview

This video explains how to determine the minimum passing level (MPL) for examinations. It covers two main systems for setting passing marks: the norm-referenced system and the criterion-referenced system. The video then details three specific methods within the criterion-referenced system: the Nedelsky, Angoff, and Ebel methods, providing practical examples for each.

Key takeaways

• Norm-Referenced System: Scores are interpreted based on comparison to the performance of other test-takers. The MPL is typically set at a point below the mean, often 0.5 to 1 standard deviation.
• Criterion-Referenced System: Scores are interpreted based on a predetermined standard or mastery of course objectives. This system requires examiners to consider the perspective of a "minimally competent student."
• Definition of Minimally Competent Student: This student possesses a good foundation of information, can recall facts, make simple connections, understands common conditions, and can perform basic diagnostic and management tasks, but struggles with complex problems, rare conditions, or integrating multiple concepts.
• Nedelsky Method: Applicable to multiple-choice questions, this method involves determining the acceptability index for each item based on how many choices a minimally competent student can eliminate. The MPL is calculated by summing these indices and dividing by the total number of questions.
• Angoff Method: This method involves faculty members ranking each item based on the likelihood a minimally competent student will answer it correctly (1.0 for likely, 0.5 for 50/50, 0 for unlikely). The average rank for each item is then used to calculate the overall MPL. An alternative approach assesses the percentage of borderline students who can answer an item correctly.
• Ebel Method: This method categorizes questions by both relevance (e.g., essential, important, acceptable) and difficulty (easy, medium, hard). Pre-assigned scores or percentages are given to each category, and a table is used to calculate the final MPL by multiplying the number of items in each category by their assigned MPL.

The video discusses two primary systems for setting examination passing marks: the Norm-Referenced System and the Criterion-Referenced System.

Norm-Referenced System: This system interprets scores by comparing an individual's performance to that of a group who took the same test. The underlying principle is that a student's achievements are evaluated against the performance of their peers. To set the Minimum Passing Level (MPL) using this method, scores are gathered, a frequency distribution, mean, and standard deviation are calculated. The MPL is then established at a point below the mean, typically ranging from 0.5 to 1 standard deviation below the mean, depending on the teacher's leniency.

Criterion-Referenced System: This system interprets scores based on how well an individual performs against a predefined standard or set of objectives. The rationale is that if knowledge and skills are clearly defined, a minimum performance level for success can be determined logically. A crucial aspect of this system is defining the "minimally competent student." This student is characterized as having a good grasp of information, recalling facts, making simple connections, understanding common diseases, and being able to perform basic diagnostic and management tasks. However, they would struggle with complex problems, rare conditions, comparing similar disease states, integrating concepts, or interpreting lab tests. The video emphasizes that examiners should consider the perspective of such a student when setting the MPL.

Within the Criterion-Referenced System, three specific methods are detailed:

• Nedelsky Method:

  • This method is specifically for multiple-choice questions.
  • For each item, examiners identify the options a minimally competent student could reject as incorrect.
  • An "acceptability index" (AI) is calculated for each item as 1 divided by the number of choices not rejected.
  • The MPL for the entire test is computed by summing the AIs for all items, dividing by the total number of questions, and multiplying by 100.

• Angoff Method:

  • This method involves faculty members ranking each item based on the probability that a minimally competent student will answer it correctly. A rank of 1.0 means the student is likely to answer correctly, 0.5 means a 50/50 chance, and 0 means unlikely.
  • The rankings from multiple faculty members for each item are averaged to determine the item's MPL.
  • Alternatively, examiners can determine the percentage of approximately 100 borderline students who would be able to answer the item correctly. For instance, if 60% of borderline students would answer correctly, the MPL for that item is 0.6.
  • The overall MPL for the exam is calculated by averaging the individual item MPLs.

• Ebel Method:

  • This method requires categorizing questions based on both their relevance (e.g., essential, important, acceptable, questionable, or must know, useful to know, nice to know) and their difficulty level (easy, medium, hard).
  • Scores or percentages are assigned to each category combination (e.g., an easy, essential question might get 100%, while a hard, questionable question might get a lower percentage).
  • Examiners then classify each question in the test according to these categories and count the number of items in each cell of the relevance-difficulty matrix.
  • A table is created to multiply the number of items in each category by their assigned MPL. The sum of these products is then divided by the total number of items in the test to get the final MPL.

The video concludes by summarizing the two systems and the three criterion-referenced methods, highlighting that the criterion-referenced system is widely used but has disadvantages, such as being difficult to set reasonable standards and potentially leading to a high failure rate.

I cannot generate snapshots or images, but I can provide the equations used for each method as described in the transcript.

Nedelsky Method:

1. Acceptability Index (AI) for an item: $AI = \frac{1}{\text{Number of choices not rejected}}$

2. MPL for the entire test: $MPL = \left( \frac{\sum_{i=1}^{N} AI_i}{N} \right) \times 100$ Where:

   • $AI_i$ is the acceptability index for item $i$.
   • $N$ is the total number of questions.

Angoff Method:

• Item MPL (Method 1: Probability-based): $MPL_{item} = \text{Average rank assigned by faculty}$ (Ranks are 1.0 for likely, 0.5 for 50/50, 0 for unlikely)

• Item MPL (Method 2: Percentage of borderline students): $MPL_{item} = \frac{\text{Percentage of borderline students who can answer correctly}}{100}$

• MPL for the entire test (for both Angoff methods): $MPL = \frac{\sum_{i=1}^{N} MPL_{item_i}}{N} \times 100$ Where:

  • $MPL_{item_i}$ is the MPL for item $i$.
  • $N$ is the total number of questions.

Ebel Method:

1. Calculate the total weighted score: $\text{Total Weighted Score} = \sum (\text{Number of items in category} \times \text{MPL for category})$

2. MPL for the entire test: $MPL = \frac{\text{Total Weighted Score}}{\text{Total Number of Items}}$ (This is often expressed as a percentage)

   The video shows this calculation using a table:

   Relevance	Difficulty	Number of Items	MPL for Category (%)	Product (Col 3 x Col 4)
   Essential	Easy	94	100	9400
   Important	Easy	106	90	9540
   Acceptable	Easy	(example value)	80	(example value)
   Questionable	Easy	(example value)	70	(example value)
   ... and so on for Medium and Hard difficulties
   Total		500		37130

   Then, $MPL = \frac{37130}{500} = 74.26%$ or 74.