A Multi-Level Assessment of the Impact of Orientation Programs on Student Learning

Abstract

The purpose of this study was to investigate the influence of orientation programs on student academic and social learning. Moving beyond previous studies, we examined how participation in orientation programming affected student learning and how the impact of these programs on learning varied by organizational characteristics (i.e., institutional control, size of undergraduate enrollment, sponsoring division, and whether the institution has an office designated for managing orientation programs), student entry characteristics (i.e., gender, race, transfer status), and student experiences (i.e., perceived quality of orientation program in helping student transition and in meeting students’ expectations, positive experiences with orientation staff, and perceptions of orientation programs and their efficacy in helping students navigate resources and in providing useful campus-based information). Hierarchical linear analysis was used to analyze these cross-level effects. Results demonstrated that having a designated office for orientation programs on campus was important for narrowing the academic learning gap between new-first year and transfer students. Implications for researchers and practitioners were discussed.

Appendix

Fully Unconditional Models

For academic and social learning, the fully unconditional models can be expressed using a similar equation,

$$ Y_{ij} \, = \beta_{0j} + r_{ij}. $$

Y _ij is the dependent variable (i.e., academic or social learning); β_0j is the institution mean for institution j; and r _ij is the deviation from the institution mean for students ij. Results of the fully unconditional model are used to estimate the proportion of variance that exists between and within colleges. In this case, the proportion of variance explained by institutional differences was approximately .053 for academic learning and .030 for social learning.

Level-1 Models

The Level-1 models can be represented as,

$$ \begin{gathered} Y_{ij} \, = \, \beta \, _{0j} + \, \beta \, _{ 1} \, \left( {Transfer*} \right) \, + \, \beta \, _{ 2} \, \left( {Male*} \right) \, + \, \beta \, _{ 2} \left( {Transgendered} \right) \, + \, \beta \, _{ 3} \, \left( {No \; reported \; gender} \right) \hfill \\ + \, \beta \, _{ 4} \, \left( {African \; American} \right) + \, \beta \, _{ 5} \left( {Asian \; American} \right) \, + \, \beta \, _{ 6} \, \left( {Hispanic \; American} \right) \; + \, \beta \, _{ 7} \, (Native \hfill \\ American) \, + \, \beta \, _{ 8} \, \left( {Mutiracial} \right) \, + \, \beta \, _{ 9} \, \left({Non{\text{-}}US \; Citizen} \right) \, + \, \beta \, _{ 10} \, \left( {Other} \right) \, + \, \beta \, _{ 1 1} \, (No \; reported \hfill \\ race/ethnicity) \, + \, \beta \, _{ 1 2} \, \left( {Academic\;expectation} \right) \, + \, \beta \, _{ 1 3} \left( {Academic \; Transition} \right) \, + \, \beta \, _{ 1 4} \hfill \\ \left( {Positive \; experiences \, with \, staff} \right) \, + \, \beta \, _{ 1 5} \, \left( {Navigating \; campus\;resources} \right) \, + \, \beta \, _{ 1 6} (Usefulness \hfill \\ of \; information) \, + \, r_{ij} , \hfill \\ \end{gathered} $$

where Y _ij (i.e., academic or social learning) is a function of the average academic learning at an institution (β_0j ) based on the effect of being a transfer student (β₁), the effect of gender (β₂–β₃)_, the effect of race (β₄–β₁₁), the effect of perceived ac academic, social, and functional experiences (β₁₄–β₁₆), and error (r _ij ).

*Although data structures for the Level-1 models for academic and social learning were similar, the process of centering variables was not. Based on preliminary results, for the academic learning model, variances for variables marked with an asterisk (i.e., transfer status and for indicator variables comparing men and women) were not constrained due to their likelihood of being influenced by Level-2 predictors. For the social learning model, variances for all Level-1 independent variables were constrained.

Level-2 Intercept-only Models

Level-2 intercept-only models for academic and social learning included the following representations:

$$ \begin{gathered} \beta_{0j} = \, \gamma_{00} + \gamma_{{01}} (\% \, Private) + \gamma_{{ 02 }} \left( {Average \; undergraduate \; enrollment} \right)+ \gamma_{{03 }} (\% {\text{ Housed\; in }} \hfill \\ {\text{Academic \;Affairs}}) + \gamma_{{04}} \left( {\%\;{\text{Housed \;in \;Student \;and \;Academic \;Affairs}}} \right) + \gamma_{{ 05 }} (\% {\text{ Housed\;in }} \hfill \\ {\text{Enrollment\; Management}}) + \gamma_{{ 06 }} \left( {\% {\text{\;Designated \;orientation \;office}}} \right) \, + \, u_{0 j,} \hfill \\ \end{gathered} $$

where the average academic or social learning at the institution (β_0j ) was a function of institutional control (γ₀₁), institutional size (γ₀₂), the sponsoring division of orientation programming (γ₀₃–γ₀₅), whether or not the institution had a designated office for orientation programs (γ₀₆), and deviations from the institutional average (γ₀₀), plus error (u _0j ). This function is the same for the academic and social learning models; however, we added an additional equation to predict slopes in the academic learning model.

Level-2 Final Slope-as-Intercepts Model for Academic Learning

For academic learning, we were interested in explaining the academic learning gaps between transfer students and those originating at the institution. To do this, we included an additional model representation:

$$ \beta_{ 1j} = \, \gamma_{10} + \gamma_ {11} \left( {\%\;{\text{Designated orientation office}}} \right) \, + \, u_{ 1j,} $$

where (β_1j ) indicated the relationship between transfer status and academic learning for each institution. This relationship (slope) is a function of whether or not the institution had a designated office for orientation programs (γ₁₆), deviations from the institutional average (γ₁₀), plus error (u _ij ).

For ease with data interpretation, the reference group (i.e., institution average) for these data represents students enrolled in public institutions, with an average undergraduate enrollment, where orientation is housed in Student Affairs only, and that do not have a designated orientation office. This means coefficients representing γ₁₁ should be interpreted as the incremental change that having a designated orientation office on campus contributes to explaining the relationship between academic learning and transfer status.