Date of Award
2002
Document Type
Dissertation
Degree Name
Doctor of Education
Department
Programs for Higher Education
Advisor
Sonia J. S. Crandall
Committee Member
Richard E. Hoehlein
Committee Member
Alan K. Cohen
Keywords
academic performance, construct validity, convenience sample, correlation, criterion validity, data analysis, evaluation tool, ex post facto research, face validity, final portfolio scores, grade point average, inter-rater reliability, Kaplan NCLEX Diagnostic Test, Mosby's RN AssessTest, nonexperimental research, nursing education, nursing portfolios, Old Dominion University, Pearson correlation, portfolio evaluation, prelicensure nursing students, program evaluation, regression analysis, research questions, role transition clinical performance appraisal, School of Nursing, statistical analysis, student assessment, validity
Abstract
A portfolio evaluation tool had been created by undergraduate nursing faculty at Old Dominion University School of Nursing to enable student portfolios to be used as an adjunct method of program evaluation. While the tool was previously judged to have face validity and high inter-rater reliability, the lack of evidence to show that portfolio scores were comparable to other measures of academic performance used to determine the attainment of program outcomes was a problem. Thus, the purpose of this project was to investigate the extent to which portfolio evaluation scores correspond with other measures of academic performance.
There were four research questions for this study:
I. Is there a relationship between prelicensure senior nursing students' final portfolio evaluation scores (FPS) and role transition clinical performance appraisal (RT-CPA) scores?
2. Is there a relationship between prelicensure senior nursing students' final portfolio evaluation scores and graduating grade point average (GPA)?
3. Is there a relationship between prelicensure senior nursing students' final portfolio evaluation scores and Mosby's RN AssessTest© scores?
4. Is there a relationship between prelicensure senior nursing students' final portfolio evaluation scores and Kaplan's NCLEX Diagnostic Test scores?
An ex post facto, non-experimental research methodology was used to conduct this study. The study utilized a convenience sample of 120 pre-licensure senior nursing students who completed portfolios prior to graduation in 2000 and 2001. Data were gathered from sources used by the school of nursing to compile departmental evaluation reports. Student identification numbers were used to match the data then expunged from the database.
The STAT A® (version 6.0, College Station, TX) statistical software program was used to guide the treatment of data. Data analysis involved correlational procedures using Pearson's r, as well as simple regression procedures.
The results of this study suggest that prelicensure senior nursing students' FPS scores do have a relationship, albeit a weak one, to GPA and Mosby RN AssessTest© scores, supporting rejection of the corresponding null hypotheses. Additionally, the results of this study suggest that prelicensure senior nursing students' FPS scores have no relationship to RT-CPA or Kaplan NCLEX Diagnostic Test scores, thereby failing to reject the corresponding null hypotheses.
Although the findings of this study do not provide the level of evidence needed to support a conclusion of criterion- and construct-related validity of the portfolio evaluation tool, they do provide direction for enhancing the comparability of portfolio evaluations scores through the refinement of criteria. It is recommended that the portfolio evaluation tool be revised, and the study repeated. Further efforts to establish validity of the portfolio evaluation tool should include exploring the use of other measures for comparison, as well as determining the feasibility of correlating scores for individual criteria common among the measures.