The term content validity refers to how well a survey or test measures the construct that it sets out to measure.
For example, suppose a professor wants to test the overall knowledge of his students in the subject of elementary statistics. His test would have content validity if:
- The test covers every topic of elementary statistics that he taught in the class.
- The test does not cover unrelated topics such as history, economics, biology, etc.
A test lacks content validity if it doesn’t cover all aspects of a construct it sets out to measure or if it covers topics that are unrelated to the construct in any way.
When is Content Validity Used?
In practice, content validity is often used to assess the validity of tests that assess content knowledge. Examples include:
Example 1: Statistics Final Exam
A final exam at the end of a semester for a statistics course would have content validity if it covers every topic discussed in the course and excludes all other irrelevant topics.
Example 2: Pilot’s License
An exam that tests whether or not individuals have enough knowledge to acquire their pilot’s license would have content validity if it includes questions that cover every possible topic discussed in a pilot’s course and exclude all other questions that aren’t relevant for the license.
Example 3: Real Estate License
An exam that tests whether or not individuals possess enough knowledge to get a real estate license would have content validity if it covers every topic that needs to be understood by a real estate agent and excludes all other questions that aren’t relevant.
In each situation, content validity can help determine if a test covers all aspects of the construct that it sets out to measure.
How to Measure Content Validity
In a 1975 paper, C.H. Lawshe developed the following technique to assess content validity:
Step 1: Collect data from subject matter experts.
Lawshe proposed that each subject matter expert (SME) on a judging panel should respond to the question:
“Is the skill or knowledge measured by this item ‘essential,’ ‘useful, but not essential,’ or ‘not necessary’ to the performance of the job?”
Each SME should provide this response to each question on a test.
Step 2: Calculate the content validity ratio.
Next, Lawshe proposed the following formula to quantify the content validity ratio of each question on the test:
Content Validity Ratio = (ne – N/2) / (N/2)
where:
- ne: The number of subject matter experts indicating “essential”
- N: The total number of SME panelists
If the content validity ratio for a given question falls below a certain critical value, it’s likely that the question is not measuring the construct of interest as well as it should.
The following table shows the critical values based on the number of SME panelists:
The content validity index, denoted as CVI, is the mean content validity ratio of all questions on a test. The closer the CVI is to 1, the higher the overall content validity of a test.
The following example shows how to calculate content validity for a certain test.
Example: Measuring Content Validity
Suppose we ask a panel of 10 judges to rate 6 items on a test. The green boxes in the following table shows which judges rated each item as an “essential” item:
The content validity ratio for the first item would be calculated as:
Content Validity Ratio = (ne – N/2) / (N/2) = (9 – 10/2) / (10/2) = 0.8
We could calculate the content validity ratio for each item in a similar manner:
From the critical values table, we can see that an item is considered to have content validity for a panel of 10 judges only if it has a CVR value above 0.62.
For this particular test, only three of the items pass this threshold.
Lastly, we can also calculate the content validity index (CVI) of the entire test as the average of all the CVR values:
CVI = (0.8 -0.2 + 1 + 0.8 + 0.6 + 0) / 6 = 0.5
This CVI value is quite low, which indicates that the test likely doesn’t measure the construct of interest as well as it could.
It would be recommended to remove or modify the items that have low CVR values to improve the overall content validity of the test.
Content Validity vs. Face Validity
Content validity is different from face validity, which is when a survey or test appears valid at face value to both the individuals who take it and the individuals who administer it.
Face validity is a less technical way of assessing the validity of a test and it’s often used just used as a quick way to detect whether or not a test should be modified in some way before being used.