Development and validation of a measurement scale to assess nursing students’ readiness for the flipped classroom in Sri Lanka

Background/rationale

Contemporary nursing education aims to prioritize student-centred learning, which is perceived as high-order, flexible, and individualized [1]. Recent advances in technology have accelerated educational innovations such as blended learning by providing easy access to information [2,3]. Blended learning is a novel student-centred pedagogical approach that includes technology-mediated online and face-to-face (F2F) learning [2]. The flipped classroom (FC) is one of several modern blended learning strategies [3]. In FC, teachers use technology to share pre-class learning material to activate low-order learning, and students study the material before attending the F2F classroom. Teachers design the F2F classroom as an interactive educational environment by using student-centred teaching strategies that allow the students to apply or evaluate the learnt concepts [1,2].

The FC has entered into use in undergraduate nursing education and has been in the spotlight in discussions of the implementation of nursing curricula [2,3]. With an increasing emphasis on the FC, the available empirical evidence on the usage and efficacy of the FC in nursing education mainly refers to its effects on students’ academic achievements and does not take into account other aspects of educational effectiveness [1,2]. Thorndike [4] in 1932 outlined a law of learning according to which students’ readiness to learn is an indispensable factor for measuring the degree of success of academic achievement. Readiness for the FC can be conceived of as a concept describing the ability of an individual to benefit from blended learning [5]. Moreover, readiness refers to a level of mental and physical preparedness among learners when taking part in the FC [6]. Assessing students’ readiness is a preliminary step for implementing the FC [3]. However, there is limited research investigating students’ readiness for the FC educational process in the context of nursing education. Therefore, it would be valuable for educational and research purposes to develop a measurement instrument to investigate students’ readiness for the FC.

Objectives

The aim of this study was to develop and validate a tool to measure nursing students’ readiness for the FC, namely the Nursing Students’ Readiness for Flipped Classroom (NSR-FC). Content validity, construct validity, and reliability tests were done to validate the measurement scale.

Ethics statement

The study was approved by the University of Auckland Human Participants Ethics Committee (reference no., 024079). Participants were provided with an information sheet before the anonymized questionnaire was administered. Participants were clearly informed that the voluntary return of the questionnaire to the collection box indicated their consent to participate in the anonymized survey.

Study design

This was a psychometric study to validate the measurement scale based on experts’ opinion and survey results for the scale.

Participants

Five experts participated in validity testing of the scale, and 265 undergraduate nursing students (in the 1st and 2nd academic years) from 3 state universities (Colombo University [n=141], University of Peradeniya [n=73], and Eastern University, Sri Lanka [n=51]) in Sri Lanka participated in exploratory factor analysis (EFA). Responses from 90 different undergraduate nursing students (3rd and 4th academic years) (University of Peradeniya [n=42] and Eastern University, Sri Lanka [n=48]) were employed for confirmatory factor analysis (CFA).

Setting

This cross-sectional study was conducted in 2 steps. The first step involved the development of the NSR-FC. The second step was to investigate the psychometric properties of the NSR-FC.

Step 1: Developing a scale for measuring nursing students’ readiness for the flipped classroom

The NSR-FC was drafted after reviewing the literature pertaining to learner readiness. The following existing inventories were used to generate items for the NSR-FC: E-Learning Readiness [7]; Online Learning Readiness Scale [6], and the ICT literacy scale [8]. However, none of the existing inventories had been developed specifically for nursing education. Therefore, 18 items (Q1–3, Q5–6, Q15–17, Q21–24, Q26, Q28–29, and Q33–35) were generated specifically to assess FC readiness by our research team based on our experience in teaching and learning in clinical and nursing education. Furthermore, 19 items were modified from the existing inventories and included in the NSR-FC. As a result, 37 preliminary items were included in the NSR-FC. A 5-point Likert scale was used, with responses ranging from 1 (strongly disagree) to 5 (strongly agree) as scale response anchors (Supplement 1).

Step 2: Exploring the psychometric properties of the NSR-FC

Study size

In accordance with the recommendation made by Lynn [15], 5 content experts were recruited for content validation. In total, 365 students participated in the study. Cattell [16] in 1978 recommended 3 to 6 samples per variable when conducting EFA. Therefore, we randomly selected students in their 1st and 2nd academic years ([35*6]<[n=265]) for EFA and the reliability test. The remaining sample (n=90) was used for CFA, which was sufficient to produce good agreement between the sample and population solutions (K value=0.92) [17].

Statistical methods

Descriptive statistics were employed to calculate CVI measures using Microsoft Excel (Microsoft Corp., Redmond, WA, USA). PCA was applied using IBM SPSS ver. 26.0 (IBM Corp., Armonk, NY, USA) to explore the factor structure. Cronbach α coefficients were computed using IBM SPSS ver. 26.0 (IBM Corp.). The CFA goodness-of-fit indices were calculated using AMOS ver. 26.0 (IBM Corp.) [8,10].

Content validation

At the item level, 91.9% of the NSR-FC’s items (n=34) had an I-CVI greater than or equal to 0.90. Two items of the NSR-FC (E1 & E2) received an I-CVI less than 0.78, namely “I can discipline myself to follow flipped learning” (I-CVI=0.75) and “I am committed to using flipped learning” (I-CVI=0.45). In addition, reviewers reported that E1 and E2 duplicated existing items, so these 2 items were excluded from the questionnaire. The S-CVI/average of the questionnaire achieved an acceptable level of 0.94 (Supplement 1).

Exploratory factor analysis of the NSR-FC

The NSR-FC was explored using PCA to determine the optimal model that best represented the data. The KMO value of the NSR-FC was 0.873, suggesting that the samples were adequate for PCA. The Bartlett test of sphericity further affirmed the suitability of the data for PCA with orthogonal factor rotation (χ²=4,717.18, P<0.001). The PCA revealed that NSR-FC could be reduced to 5 factors, namely technological readiness, environmental readiness, personal readiness, pedagogical readiness, and interpersonal readiness. Parallel analysis confirmed the 5 factors in the NSR-FC. The percentage of variance explained by the rotated factor matrices ranged from 4.29% to 25.42% per factor, with the 5 factors explaining 55.25% of the overall variance. Factor loading after the rotation of each item is shown in (Table 1, ^{Dataset 1}).

Table 1.

Results of principal component analysis with varimax rotation

Accordingly, technological readiness (factor 1) included 12 items (Q6–16 and Q18). The factor had a cumulative eigenvalue of 8.89 and accounted for 25.42% of the total variance. Environmental readiness (factor 2) comprised 5 items (Q20–21 and Q23–25), had a cumulative eigenvalue of 4.38, and accounted for 12.51% of the total variance. Personal readiness (factor 3) contained 6 items (Q1–4, Q17, and Q19). The factor had a cumulative eigenvalue of 2.81, and accounted for 8.02% of the total variance. Pedagogical readiness (factor 4) encompassed 7 items (Q22, Q26–29, and Q34–35), had a cumulative eigenvalue of 1.75, and accounted for 5.01% of the total variance. Interpersonal readiness (factor 5) included 4 items (Q30–33) and had a cumulative eigenvalue of 1.50, accounting for 4.29% of the total variance. Item Q5 obtained a factor extraction value of 0.25, which was deemed to be low; this item was therefore excluded from the NSR-FC.

Internal consistency of the NSR-FC

The Cronbach α values of the NSR-FC’s factors ranged from 0.76 to 0.93, and the Cronbach α for the scale as a whole was 0.90, indicating excellent reliability. The intraclass correlation coefficients (ICCs) for the NSR-FC’s factors showed satisfactory results for each subscale (ICCs ranging from 0.76 to 0.92; P<0.01), and the overall ICC was 0.90, exhibiting an acceptable level of reliability. The corresponding data are presented in (Table 2, ^{Dataset 1}).

Table 2.

Results of internal consistency

Confirmatory factor analysis of the NSR-FC

Model 1 of the CFA denotes the baseline model, with 34 items as identified by PCA. In reference to model 1, the goodness-of-fit indices did not achieve an acceptable level, except for χ²/df. Therefore, model 2 was created by excluding 5 items (Q7, 10, 16, 22, and 33) that showed loading values less than 0.4; however, the goodness-of-fit values of model 2 were still below the cut-off level (Table 3, ^{Dataset 2}).

Table 3.

Goodness-of-fit indices from confirmatory factor analysis to test the suitability of the Nursing Students’ Readiness for Flipped Classroom

In addition, all items in factor 5 did not obtain an acceptable loading value. Therefore, it was decided to exclude factor 5 from model 3. Model 3 was found to be the model with the best fit in the current study (Fig. 1). According to SRMR and an adjunct discrepancy-based fit index (χ²/df), model 3 achieved an acceptable level of fit (SRMR=0.08 and χ²/df=1.57). In reference to RMSEA, model 3 demonstrated reasonable fit (RMSEA=0.08). The CFI of model 3 (0.87) came close to meeting the cut-off level (Table 3).

Fig. 1.

Model 3 with factor loadings for the 20-item Nursing Students’ Readiness for Flipped Classroom.

Interpretation

Student readiness is recognized as a valuable factor for determining pedagogical effectiveness [4]. Therefore, the present study investigated the development and psychometric properties of a scale used to measure nursing students’ readiness for the FC. More specifically, the study explored the construct validity and reliability of the NSR-FC.

The method used to construct the NSR-FC was similar to the procedure used for the development of student readiness scales in other disciplines, and included proposing constructs, item generation, analysis of the content, item reduction, and validation of the newly developed instrument [6,7]. PCA revealed a potential 5-factor structure for the NSR-FC. However, CFA was only able to confirm a 4-factor model as determined by an inspection of the fit indices. The 4 factors that best fit the data with respect to the NSR-FC included technological readiness, environmental readiness, personal readiness, and pedagogical readiness.

The 4 best-fitting factors of the NSR-FC were comparable with existing inventories used to measure students’ readiness for blended learning in school-level education. Technological readiness is a common subset, identified as technology self-efficacy in many other studies [6,18]. Technological readiness denotes an individual’s willingness to leverage novel technologies for carrying out tasks [19]. Items relating to environmental readiness in the NSR-FC were seen to load together with the technological factor in the E-Learning Readiness Scale [7]. Personal readiness was recognized as a factor in the NSR-FC. Since FC is a student-centred pedagogical approach, students play a significant role in teaching-learning activities. Therefore, it is necessary to estimate students’ individual willingness to engage in the FC. Personal readiness was identified as “learner control,” with some variation, in the Online Learning Readiness Scale [6]. Pedagogical readiness in the NSR-FC describes students’ willingness to embrace learning content through FC pedagogy. A few items in the pedagogical readiness factor were correlated with the subscale of “online communication self-efficacy” in the Online Learning Readiness Scale [6] and the “content factor” in the E-Learning Readiness Scale [7].

In the reliability assessment, the NSR-FC demonstrated acceptable internal consistency. All 4 factors of the NSR-FC generated Cronbach α and ICC values greater than 0.7, confirming internal consistency within the domains and the ability of the NSR-FC to generate reproducible results. Therefore, the 4-factor model of NSR-FC is a valid and reliable tool that can be used to measure students’ readiness.

Limitation/generalizability

This instrument can be used by nursing educators and curriculum planners to evaluate the effectiveness of FC pedagogy. However, the results of factor analysis may have been sample-specific, and the generalizability of these results is subject to the similarity of respondents to the sample. Since the study was conducted in Sri Lanka, the NSR-FC could be used to assess nursing students’ readiness for the FC in the context of South Asia. Moreover, the CFI value of the NSR-FC did not meet the acceptable level. This may have been due to the limited sample size and is acknowledged as a limitation of the study. Thus, future research is encouraged to perform CFA of the NSR-FC with a larger sample size. Lastly, the study was conducted through a self-reported questionnaire survey, which may have resulted in reporting biases such as social desirability bias [20]. Qualitative research will likely be instructive in terms of enabling a deeper understanding of the phenomena underlying nursing students’ readiness for the FC.

Conclusion

These findings indicate that the NSR-FC is an acceptable instrument for measuring nursing students’ readiness for the FC in reference to its construct validity and reliability within the Sri Lankan nursing education context. The 4 readiness subscales were found to be technological readiness, environmental readiness, personal readiness, and pedagogical readiness. This finding may provide a good platform and frame of reference for nursing curriculum researchers and educational designers regarding the necessity of assessing students’ readiness for gaining actual educational achievements through the FC.