Julian J. Meimban III
Edwin A. Benigno
University Research Center (URC)
Luzale D. Henson
Audrey Lyle dM. Diego
College of Engineering and Architecture (CEA)
Melanie C. Pacho
College of Arts and Sciences (CAS)
How to Cite:
Meimban, J. J., III, Benigno, E. A., Henson, L. D., dM. Diego, A. L., & Pacho, M. C. (2017). Satisfaction survey of a university’s commencement exercises. NEU Knowledge Journal: A Compilation of Researches of New Era University Faculty, Staff, Students, and Administrators, 1(1), 44–53. https://doi.org/10.64303/neu-urc-kj2017-001SaSuUCE
ABSTRACT
A study was conducted to determine the satisfaction rating of 11 groups (graduates from nine Colleges, parents, and faculty) who attended a University’s Commencement Exercises. A six-item, 1-5 Likert scale (1-Very Poor, 2-Poor, 3-Satisfactory, 4-Good, 5-Excellent) questionnaire was given to 275 randomly selected respondents (based on power analysis). The first five items pertained to accessibility/cost (ACCESS), staff performance (AIDES), orderliness (ORDER), formality (FORMAL), and discipline/conduct (CONDUCT); the sixth item about recommendation (RECOMD). The instrument met validity test and reliability standards (Cronbach’s Alpha=0.84).
One-sample T test showed a highly statistically significant overall rating beyond satisfactory level (3 on the scale). To test the hypothesis of no difference in satisfaction rating among groups, parametric ANOVA was tried initially, but because normality and homogeneity of variance assumptions were not met, Kruskal-Wallis (K-W) test was used instead, using composite score. Results revealed significant group differences. Post hoc Mann-Whitney U (M-W) test with Bonferroni correction was made to find significant pairwise comparison. Chi-Square test showed moderately strong relationship (Contingency Coefficient= 0.34) between group affiliation and RECOMD. Almost all of the groups overwhelmingly recommended holding the University’s next year graduation again at the same venue.
Keywords: Satisfaction survey, Kruskal-Wallis, Mann-Whitney U, Bonferroni correction, Chi-Square
INTRODUCTION
Attendees of Commencement Exercises usually expressed varying degree of satisfaction not only on their overall experience on the event but also on particular part of it. For instance, some attendees expressed dissatisfaction on venue’s proximity, accessibility, and ambience; program’s orderliness and duration; and expenditure, to name a few. Others because of much dissatisfaction even suggested alternative venues for future graduations. On the contrary, not a few attendees intimated favorable overall experience. In this light, a satisfaction survey study was made to assess the level of satisfaction of graduates, parents, and faculty who attended a University’s Commencement Exercises using inferential statistics.
RESEARCH QUESTIONS
Question 1: What is the overall satisfaction rating of the attendees (categorized into 11 groups), α=0.05?
Ho: Overall satisfaction rating is 3 or Satisfactory (μ=3)
Ha: μ not equal to 3
Question 2: Is there a significant difference in the satisfaction rating between groups, α=0.05? The groups referred to were the following: Accountancy (ACC), Arts & Sciences (AS), Business Administration (BA),Communication (COM), Computer Studies (CS), Education (EDU), Engineering & Architecture (EA), Graduate Studies (GS), Branch-College (BC), Faculty (FAC), and Parents (PAR).
Ho: There is no significant difference in the satisfaction rating of groups (or all μ’s or population means are equal)
Ha: Not all μ’s are equal
Question 3: Does group membership have something to do with whether one recommends or not holding the University’s next year graduation again at the same venue, α=0.05?
Ho: Group membership is independent of the kind of recommendation
Ha: Group membership and the kind of recommendation are related
METHODOLOGY
Sample Size
Sample size was calculated based on fixed effects, omnibus, one-way ANOVA involving 11 groups, 5 measurements, 80% power, 0.25 effect size (medium), and 5% significance level. For this specification, a total sample size of 275 (or 25 cases per group) was needed as recommended by Cohen (1988) as mentioned in the G*Power 3.1.9.2 program.
Survey Questionnaire and Distribution
The 6-item questionnaire concerned about (1) accessibility of the venue (ACCESS), (2) performance of graduation staff (AIDES), (3) orderliness of the graduation procedure (ORDER), (4) formality/solemnity of ceremony (FORMAL), (5) discipline/conduct of graduating students (CONDUCT), and (6) recommendation (RECOMD). The first five items were rated using a 5-point Likert scale, where: 5 – Excellent, 4 – Good, 3 – Satisfactory, 2 – Poor, and 1 – Very Poor. On the other hand, item 6 was rated Yes or No. Graduate respondents were randomly selected from the area where they were designated to occupy. Parents and faculty respondents were picked randomly based on their seat number in the Arena.
Eleven student-volunteers were trained, instructed, and tasked to administer the distribution of questionnaire to graduating students. Surveyors were also instructed to distribute the questionnaire only when the graduating students had returned to their assigned seats after they had gone on the stage. Respondents were told not to confer or communicate by any means to anyone while answering the questionnaire. To ensure strict compliance of survey instructions, four assistant researchers were also tasked to supervise the distribution and assist filling out of the questionnaires.
Validity and Reliability
Before the questionnaire was distributed, it was tested for validity and reliability. For content validity, the questionnaire was assessed by three professionals in test construction. Their comments and suggestions were taken into consideration in finalizing the survey questionnaire. For reliability, internal consistency reliability was estimated using 226 respondents. The Cronbach’s Alpha obtained was 0.84, greater than the 0.70 recommended minimum acceptable value for responses to be considered reliable or consistent (Bohrnstedt and Knoke, 1988).
Statistical Analysis
To address Research Question 1, one-sample T test was used. Test of normality assumption was met as evidenced by the almost equality of mean, median, and mode (3.98, 4.00, and 4.00 respectively). In addition, skewness, although slightly negatively skewed, had a value of -0.84 which is less than twice its standard error (0.16).
To address Research Question 2, Kruskal-Wallis (K-W) test was used for the omnibus test and Mann-Whitney U (M-W) as post-hoc test following a significant K-W test.
Moreover, the high or strong inter-correlation of variables, which ranged from 0.40 to 0.62, suggested that a composite score could be formed and used to represent an index of satisfaction rating. Thus, a composite variable (SCORE) was created, which was the total score of a respondent in the first five items in the survey questionnaire (i.e., SCORE = ACCESS + AIDES + ORDER + FORMAL + CONDUCT). The variable SCORE was then used in comparing the overall satisfaction rating of groups.
Attempt was made to use one-way ANOVA using SCORE as the dependent variable. However, normality and homogeneity of variance tests were not satisfied. For normality test, Kolmogorov-Smirnov tests for BA (p=0.030), CS (p=0.013), PBC (p=0.001), FAC (p=0.024), and PAR (p=0.044) were significant at α=0.05. For homogeneity of variance test, Levene Statistic LS(10,219)=3.12, p<0.001 was highly significant at α=0.05. For this reason, K-W test, a nonparametric test, was used in lieu of one-way ANOVA to test the null hypothesis of no significant group differences.
For Research Question 3, Chi-Square test was used to determine if there was significant relationship between the variables GROUP and RECOMD, and Contingency Coefficient was computed to estimate the strength of relationship.
RESULTS AND DISCUSSION
Respondents
Out of the 275 survey questionnaires distributed and returned, 231 to 233 turned out usable because some survey forms were found improperly filled out or had missing entries. Thus, this study was based on 84.0% to 84.7% response rate. The number of respondents who rated the graduation ceremony Good or Excellent is evidently far greater than those who rated it Poor or Very Poor (Table 1). The percentage of respondents under combined Good and Excellent rating, which ranges from 57.9% to 84.5%, is overwhelmingly greater than that under combined Poor and Very Poor rating, which ranges only from 2.6% to 9.0% (Table 2). Thus, majority of the respondents rated the graduation ceremony Good to Excellent. A far smaller number rated it Poor to Very Poor.
Answers to Research Questions
Research Question 1: What is the overall satisfaction rating of the attendees (categorized into 11 groups), α=0.05?
Ho: Overall satisfaction rating is 3 or Satisfactory (μ=3)
Ha: μ not equal to 3
A highly statistically significant result was obtained in favor of rating beyond satisfactory level (i.e., > 3 on the Likert scale), mean=3.98, t(229)=22.39, p<0.001, effect size (d)=1.48 (large according to Cohen, 1988). The large value of effect size (mean difference/standard deviation) estimated can be explained by the large mean difference (0.98) relative to a low variability (0.66). A 95% CI of mean difference ranges from 0.90 to 1.07. This interval suggests that when sampling is done 100 times, and 95% CI of mean difference is computed each time, 95 of such CIs have positive lower and upper limits. This finding indicates that the population mean satisfaction rating is above 3.0. In this particular study, the 95% CI of mean difference suggests that the population mean satisfaction rating ranges from 3.90 to 4.07, rated as Good on the Likert scale. In fact, 90.87% of the respondents rated the graduation beyond satisfactory level (3.20 to 5.00)(Table 3).
Research Question 2: Is there a significant difference in the satisfaction rating between groups, α=0.05?
Ho: There is no significant difference in the satisfaction rating of groups (or all μ’s or population means are equal)
Ha: Not all μ’s are equal
When composite score (SCORE) was used as the dependent variable, K-W test showed statistically significant group differences, Chi-Square (10, N=230) = 25.48, p=0.005. Group-pairs found significantly different by post hoc M-W test with Bonferroni correction (α=0.05/55=0.001) are shown in Table 4. Specifically, College of Education (EDU) rated the graduation ceremony more favorably (mean rank=28.59) than College of Accountancy (ACC) did (mean rank=16.41), p=0.001, effect size=0.48 (medium to large). Similarly, EDU expressed greater degree of satisfaction (mean rank=28.86) than College of Communication (COM) did (mean rank=16.14), p=0.001, effect size=0.50 (large). Other possible group-pair comparisons were found not statistically significant.
Research Question 3: Does group membership have something to do with whether one recommends or not holding the University’s next year graduation again at the same venue, α=0.05?
Ho: Group membership is independent of the kind of recommendation
Ha: Group membership and the kind of recommendation are related
Sample evidence suggested that group membership was related to RECOMD, Likelihood Ratio (10, N=229)=30.68, p=0.001 and the strength of relationship (Contingency Coefficient =0.34, p=0.001) was considered moderate (Salkind, 2000). In other words, whether or not one recommends holding the Univesity’s next year graduation again at the same venue depends on what group he or she belongs. For instance, a high percentage of respondents of the following groups recommended holding next year graduation again at the same venue: ACC (72.7%), AS (81.8%), BA (77.3%), COM (86.4%), CS (100.0%), EDU (95.5%), EA (94.4%), BC (86.4%), GS (85.7%), and PAR (63.6%). However, only 47.6% of the respondents from the FAC (Faculty) favored such recommendation.
CONCLUSION
Overall, the graduation attendees show satisfaction beyond satisfactory level. Almost all of the groups overwhelmingly recommend holding the University’s next graduation again at the same venue.
BIBLIOGRAPHY
Bohrnstedt, George W. and David Knoke. (1988). Statistics for Social Data Analysis. 2nd Edition. USA: F. E. Peacock Publishers, Inc.
Cohen, Jacob. (1988). Statistical Power Analysis for the Behavioral Sciences. 2nd Edition. USA: Lawrence Erlbaum Associates, Inc.
G*Power 3.1.9.2 program.
Salkind, Neil J. (2000). Exploring Research. 4th Edition. USA:Prentice-Hall, Inc.
