These suggestions of sampling were approved in November of 2015 by the Learning Outcomes Committee. Due to changes in assessments practices, assessing ALL students, the Learning Outcomes Committee is working to come up with innovative ways to meet the needs of the ACCJC and Glendale College.
Rationale: Why should we sample?
Sampling facilitates the assessment process when programs have large numbers of students and it is not feasible to assess all students. Furthermore, sampling may be useful when assessing artifacts that take a long time to review.
For the Student Learning Outcomes (SLO), we must assess artifacts that reflect the desired outcomes. You may use a subjective artifact from a course (e.g. the research project in the capstone course or a paper) for a SLO measure only if you have a faculty panel evaluate the artifacts independent of the course instructor.
There might be too many students or too many artifacts for a panel to evaluate each semester or year; therefore, only some of the artifacts would be effectively and efficiently evaluated each year. The portion evaluated is the sample of the entire population.
Census vs. Sampling
Explanation: For programs that are small, assessing the entire population may yield a more accurate measure of student learning. Assessing the entire population is called a census whereas assessing only part of the population is called a sample.
Example of Using a Census: Whole populations should be considered for the assessment and evaluation of an SLO when the population is rather small. For example, some degree programs only have 5 or 10 students. Those programs could and should include all the students in the assessment and evaluation of an SLO.
Example of Using a Sample: It is difficult and time-consuming to evaluate all students’ assignments for an SLO if there are over 100 students in a program, especially when the artifacts are long and complicated such as a 20-page research paper.
Before evaluating artifacts or data for the SLO, you must:
- Decide whether you will use a sample or the whole population.
- Choose an appropriate sample size based on percentage, artifact size and complexity, and faculty panels.
- Choose an appropriate sampling method.
Determining Sample Sizes
Whether or not to sample and the size of the sample depend on three factors, all of which must be kept in mind when making sampling decisions:
- The length and complexity of the assignments/artifacts.
- The number of students enrolled in the program.
- The number of faculty members serving on the faculty panel (the artifact evaluators).
Choosing a Sample Based on a Percentage
Explanation: If you have a large program (over 100 students), you may not have the people and time to evaluate 100 artifacts. Therefore, you would choose a specific percentage of students or artifacts (e.g. 20%).
Example: Programs with over 100 students and only 2 faculty evaluators may choose to evaluate and report on a sample that represents 10% of the population (10 students) or 20% (20 students).
Caution: Your percentage should be based on the three factors mentioned above.
- The length and complexity of the assignments
- If the assignment or artifact is over 50 pages (e.g. class action research project), then a smaller percentage of students should be chosen for the sample. Since the university standard is 10% or 10 students, whichever is greater, then you would sample 10% of your population (10% of 100 students = a sample size of 10 students).
- The number of students enrolled in the program
- If the program has less than 50 students, then you should consider using a larger percentage or the entire population. According to the university sampling policy, you would choose 10 students minimum.
- The number of faculty members serving on the faculty panel
- If the program only has three faculty members on the faculty panel, then a smaller sample size would be more appropriate depending on the complexity of the assignment. However, programs with over 10 faculty members and short assignments could have a much larger sample size since there are many more people available to evaluate the artifacts.
Choosing a Sample Based on Length and Complexity of the Assignment/Artifact
Explanation: Programs that have a very long, complex artifact should use a smaller sample size. Programs that have a short, simple artifact should use a larger sample size.
Example #1: A large program with 200 students and a 50-page artifact (a research paper) for its SLO would have a sample size of 5% of the population (total number of students enrolled in the program), which would yield ten 50-page artifacts. It would be much more feasible for a faculty panel of 2-3 members to assess ten 50-page artifacts than for them to assess two-hundred 50-page artifacts.
Example #2: A large program with 200 students and a 5-page artifact (an analytical paper) for its SLO should have a sample size of 10% of the population (total number of students enrolled in the program), which would yield 20 5-page artifacts. It would be feasible for a faculty panel of 2-3 members to assess 20 5-page artifacts.
Choosing a Sample Based on the Size of the Faculty Panel
Explanation: The number of faculty raters on the faculty panel affects the number of artifacts that can be evaluated each semester or year. Programs with a small faculty panel (4 members or less) should evaluate a smaller number of artifacts. Programs with a large faculty panel (5 or more faculty members) have the resources to evaluate a larger number of artifacts.
Example #1: A large program with 200 students and a 50-page artifact (e.g. the research project) for its SLO should have a small sample of 5% (10 students) if the faculty panel only has 3 members.
Example #2: A large program with 200 students and a short, oral artifact (e.g. 10-minute presentation) and 3 faculty panel members may have a large sample of 20% (20 students).
Common Types of Sampling
There are a variety of sampling methods. Simple random, stratified, systemic, and cluster sampling are examples of four common and appropriate sampling methods for institutional assessment activities.
Simple Random Sampling
Explanation: Randomly select a certain number of students or artifacts.
Example: You have 100 students in your program who have completed the mandatory artifact. You want to sample 20% of your artifacts (20 artifacts). Therefore, you randomly select the 20 students or artifacts without any order or plan. Random sampling can be done with a random numbers table, by random number generators (computerized), or by selecting from a hat.
Explanation: Students are sorted into homogenous groups and then a random sample is selected from each group. This is useful when there are groups that may be underrepresented.
Example: In a program that has few female students, it may be desirable to ensure they are represented in the sample. Therefore, all students are sorted by gender and a sample is selected from each group.
Explanation: You select the nth (e.g. 7th, 9th, 20th) student or artifact from a list.
Example: You have an alphabetical listing of all 100 students who have just completed your program. You want to sample 20% of your student population (20 students). Therefore, you go through the list of 100 students and pick every 5th student as you move down the list.
Explanation: You randomly select clusters or groups (e.g. classes or sections), and you evaluate the assignments of all the students in those randomly selected clusters or groups.
Example: The artifact that represents an SLO in your program is a paper that is produced in the highest level course of your program, and there are 15 sections offered a semester. Each section has 30 students. You would like your sample to be 20% (90 students) of your overall student population (450 students) for one semester. Therefore, you randomly select 3 sections, and you evaluate the assignments of all 90 students enrolled in those 3 sections.