1. Project Title and Proposal #:
Assessing Student Success in Basic Skills with eLumen Software
2. Author: Alice Mecom
3. Project Director: Tina Andersen-Wahlberg, Alice Mecom
Key Personnel: Sarah McLemore, Charlotte Schulten, Kirk Vaughn, Kathy McNeese
4. Project Goals:
a. To measure GCC Basic Skills student performance in terms of quantitative data
b. To implement GCC-created rubrics and common finals within eLumen to allow for software-generated assessment reports on student achievement for each learning criterion
c. To develop a core group of faculty who will serve as a pilot group for eLumen and who will train and support other faculty and staff in its use
d. To make use of this newly-purchased software for the purposes of meeting the accreditation standards which require institutionalized assessment cycles of student learning on course, program, and institutional levels.
This project will move our college in new directions. Most faculty and staff have embedded cycles of teaching and testing in their classrooms, yet do not have the time, the interest, or the “know-how” to go beyond this cycle by analyzing the results of student performance on classroom activities, projects, and tests. This analysis enables us to identify specific trends, both positive and negative, in our students’ learning. We can use this information in a variety of ways that will benefit the students, the instructors, counselors, staff, and the college as an institution. The students benefit most directly from these cycles because their learning experiences are actively being evaluated and improved by those faculty and staff who provide them. There are some faculty and staff that do run analyses of assessments, but GCC can only document that these analyses occur in 5% of our courses, even less in our programs, and not at all as an institution. Therefore, we need to increase our frequency of analyzing assessments campus-wide. In addition to course and program assessment, eLumen also allows us to assess and act on student achievement in the seven core competencies, which reflect basic skills such as communication, quantitative reasoning, and personal responsibility. Therefore, eLumen will provide data about students’ basic skills even when the students are not enrolled in courses typically considered to be “basic skills”, such a biology and history, for example. It is common knowledge that students’ lack of basic skills impedes their progress in such courses. With eLumen’s reporting ability on learning needs, we can better address these concerns that have eluded many of us professionals.
The College has purchased the eLumen software program because it is a powerful and efficient tool that will allow us to analyze student data within our time, manpower, and fiscal constraints. This project will “kick start” our implementation of eLumen and begin assimilating our faculty and staff into a culture of doing assessment cycles.
Rationale for doing assessment cycles:
1. By identifying student learning needs, we can plan strategies to improve and/or maintain successful learning experiences. Identifying learning needs goes a step beyond assigning grades to student work. The cycles of analysis and planning benefit faculty and staff professionally while also benefiting the students who learn from them.
2. The data that we gather from eLumen will direct college planning and budgeting so that decisions are made based on student needs.
3. Accreditation requires that we institutionalize assessment cycles on course, program, and institutional levels. We don’t have the choice not to analyze assessments and report these data to both our divisions and to the institution via program review and budgeting requests.
This project directly addresses three goals of GCC’s Basic Skills Action Plan:
1. Monitor Outcomes for Basic Skills Students: The primary function of the eLumen software is to electronically store student learning outcomes and assessment tools that our faculty and staff have created, and to provide reports on students’ levels of achievement on these outcomes. The reports give us quantitative information about our students’ weaknesses and strengths. We can evaluate students’ individual and group performances both formatively and summatively.
2. Explore and develop a second level of assessment for basic skills students: Using eLumen is indeed an explicit second level of assessment for basic skills students. The information gathered here will be unique and invaluable.
3. Provide specific training to faculty teaching basic skills classes: The instructors in the pilot group will learn how to use innovative assessment software to improve the cycle of teaching and learning, and will in turn train others at GCC to do the same.
This project directly and indirectly utilizes the effective practices from the “Basic Skills as a Foundation for Student Success in California Community Colleges” manual. The project directly utilizes B2 and D9:
B2 Regular program evaluations are conducted, results are disseminated widely, and data are used to improve practice.
B.2.1 Developmental education course content and entry/exit skills are regularly reviewed and revised as needed.
B.2.2 Formative program evaluation activities occur on a regular basis.
B.2.3 Summative program evaluation activities occur on a regular basis.
B.2.4 Multiple indices exist to evaluate the efficacy of developmental education courses and programs.
B.2.5 Data obtained from course/program evaluation are disseminated and used for future planning.
D9 Faculty and advisors closely monitor student performance
D9.1 Mechanisms exist to frequently and consistently provide course performance feedback to students.
The project focuses on staff development, and incorporates the suggestions offered in C3:
C3 Staff development programs are structured and appropriately supported to sustain them as ongoing efforts related to institutional goals for the improvement of teaching and learning.
C.3.1 Developmental education staff development activities are clearly linked to department, program, and/or institutional goals.
C.3.2 Developmental education staff development activities are not based around “one-shot” workshops; rather, staff development activities are comprehensive and ongoing.
C.3.3 Staff development activities are adequately funded, funding is ongoing, and development activities are coordinated by specific designated staff as part of their core responsibilities.
The project indirectly relates to D7. Once we know how to use eLumen to its potential, we will have quantitative data that will direct us in aligning programs and entry/exit skills.
D7 Programs align entry/exit skills among levels and link course content to college-level performance requirements.
D7.1 Developmental education course entry/exit standards are regularly reviewed and revised as needed.
D.7.2 The entire trajectory of developmental course sequences (including entry by placement instruments) is periodically reviewed and aligned to ensure appropriate student progression through sequential levels.
The project will require two leaders, Tina Andersen-Wahlberg and Alice Mecom, and 3-4 faculty members from Basic Skills disciplines. This group will serve as the pilot group which will learn how to use the eLumen software:
eLumen will run the reports for the participants to analyze
The pilot group will identify “glitches” and special needs that GCC would like from the eLumen software
The pilot group will improve and clarify the sets of instructions for general use for all faculty and staff
The pilot group will offer trainings to other faculty and staff members
Fall 2009: The pilot group will begin inputting data into eLumen using draft sets of GCC-created instructions. The group will identify “glitches” and special needs.
Spring 2010: The pilot group will train colleagues to use eLumen software
Fall 2010: The pilot group will implement changes to their instruction and/or services and continue eLumen trainings and workshops.
Spring 2011: The pilot group and newly trained faculty and staff will implement changes to their instruction and/or services and continue eLumen trainings and workshops.
The College has already committed to using eLumen by purchasing the software for
$ 27,000 and maintaining it yearly for approximately $9500. We must train our faculty and staff to use the software so that we can generate assessment reports. Accreditation standards require that the college perform assessment cycles on course, program, and institutional levels, so we do not have a choice but to institutionalize assessment cycles. Per the accreditation standards, the college is building a program review and budgeting process that uses student achievement data to direct college planning and spending. The college is relying on faculty and staff to provide the data, and a great many faculty and staff are relying on eLumen to make this process efficient and meaningful, though it should be noted that faculty and staff are not required to use eLumen to run assessment cycles.
Once the pilot group of trainees becomes fluent in the software, the group members will train their colleagues in its use through workshops and training sessions. They will address questions and concerns regarding the software, and will show how the software can be a worthwhile tool that brings to life all of the outcome and assessment design work that we have created over the past few years. It would not be surprising if other campuses state-wide and beyond seek the insight we gain from this project.
Using eLumen as a storage and reporting tool will replace the current institutional process of submitting “assessment reports” and “alignment grids” to the SLO Chair and to the Office of Research and Planning.
8. Evaluation and Assessment Methods:
This project centers around training faculty and staff to manage student learning data with eLumen. This ability allows us to identify specific learning needs that our students have that we may not be able to identify so clearly otherwise. With this information, faculty and staff can make informed decisions that will most enhance their instruction and services. Therefore, there is no evaluation method that can be used on the students themselves in the early stages of the project. The evaluation and assessments will have to be done with the faculty and staff who participate in the project. Prior to and after the experience, we will survey the pilot group (and any additional faculty and staff) on their attitudes towards eLumen, including their expectations and concerns. We will ask if and how eLumen maximizes the value of our written outcomes and assessment tools (ie, rubrics, tests, surveys, etc), and if and how it can improve our teaching. If faculty and staff make adjustments to their courses and/or services as a result of the data reports, we will ask if and how these adjustments positively affected student learning. We anticipate the collection of faculty and staff responses to be positive and encouraging, and we will use the results of our surveys to inform and motivate others as we implement eLumen and assessment cycles around campus. The results of this project will undoubtedly interest other campuses in and outside of California who have also adopted this software.
9. Estimated Project Cost:
|Tina Andersen-Wahlberg, project leader:
|Alice Mecom, project leader:
|Sarah McLemore, English:
|Charlotte Schulten, Math:
|Kirk Vaughn, ESL:
|Kathy McNeese, Nursing:
*no stipend is necessary if RTEP is renewed for SLO Chair (20%) and eLumen (20%)