This study’s data were collected from five middle schools in China through convenience sampling, with the objective of evaluating adolescent mental health. With a response rate exceeding 95%, students filled out a demographic survey and the CES-D [5] during the initial assessment phase. The final sample comprised 1,042 students: 521 girls, 505 boys, and 16 with unspecified gender. Of these participants, 525 were from rural areas and 517 from urban settings. The mean age was 16.27 years, with a standard deviation of 0.90 years (M = 16.27, SD = 0.90).

Three months after the initial survey, a follow-up assessment was conducted in the same schools, with 967 students participating again. This follow-up sample consisted of 490 girls, 462 boys, and 15 students with unspecified gender. Among these, 491 were from rural areas and 476 from urban settings. At the time of the follow-up, the average age of the participants was 16.27 years (M = 16.27, SD = 0.91), indicating a relatively narrow age range.

The CES-D (Center for Epidemiological Studies Depression Scale; Radloff, 1977) is a self-report instrument comprising 20 items, designed to measure depressive symptoms experienced in the preceding week. Respondents rate each item using a 4-point Likert scale, where 1 signifies ‘rarely or less than 1 day’ and 4 represents ‘most or all of the time, 5–7 days.’ This results in a cumulative score range from 20 to 80. Previous studies have established the CES-D’s robust internal consistency [25,26]. In our study, the Cronbach’s alpha values were 0.942 and 0.945 for the first and second waves, respectively, confirming its excellent reliability.

This study employed CFA and invariance testing across rural and urban contexts as well as longitudinally. The analysis was performed using R software. We employed weighted least squares means and variance-adjusted estimates, a choice informed by the four-category response format of the questionnaire items [27].

The analysis comprised three steps. Initially, structural validity was assessed through CFA on the first wave data (n = 1,042) to establish the CES-D’s factor structure. We compared five distinct models (detailed in Table 1). Model 1, reflecting Radloff’s initial conceptualization identified a four-factor structure, with depressed affect (DA), somatic complaints (SC), positive affect (PA), and interpersonal problems (IP), has shown satisfactory fit in various international studies [8,9]. Model 2 proposed a two-factor structure differentiating positive from negative items, validated across diverse populations [13,14]. Model 3, was further recognized by Wang et al. in their study focusing on adolescents from mainland China [18], segmented the CES-D into three factors: a four-item positive affect factor, a nine-item somatic complaints factor, and a seven-item depressed affect factor. Model 4 suggested a three-factor model grouping positive affect, interpersonal problems, and a combined factor for depressed affect and somatic complaints [19]. Finally, Model 5 suggested a three-factor framework, combining positive affect into two novel factors that evolved from the original components of depressed affect, interpersonal issues, and somatic complaints [15].

Consistent with standard procedures, the fit of the model was assessed using several criteria: chi-square (χ²), root mean square error of approximation (RMSEA), comparative fit index (CFI), and Tucker–Lewis index (TLI) [15,28]. Consistent with recommendations for structural equation modeling, we reported the 90% confidence interval for RMSEA (90% CI), providing a more precise assessment of model fit. This interval indicates the range in which the true RMSEA value likely falls, with an acceptable model criteria set at RMSEA ≤ 0.08 and CFI and TLI values ≥ 0.90 [15]. Given the distinct structures of the models under comparison, we opted for the Bayesian information criterion (BIC) over chi-square differences. Following Raftery’s guidelines, a lower BIC value denotes a more favorable model fit. Specifically, a ∆BIC exceeding 10 points is indicative of a substantial improvement in the model [29].

In the subsequent stage, having determined the most suitable CES-D model, our focus shifted to examining its factorial invariance in different residential settings, specifically between rural and urban areas, using data from wave 1 (n = 1,042) and wave 2 (n = 967). In line with established methodologies [15,30], we initially assessed the model’s fit for each residential context separately. This was followed by multigroup invariance testing, which encompassed configural invariance (uniform factor structures across groups), metric invariance (consistent factor loadings), and scalar invariance (equivalence in factor loadings and thresholds). This approach aligns with standard practices in factorial invariance testing [31–33]. For nested model comparisons, Cheung and Rensvold conducted a study using data of unequal values and demonstrated that among various parameters, ΔCFI was the most effective indicator of model variance [34]. The ΔCFI test was superior to the Δχ² test for variance in nested models [35]. Therefore, we used ΔCFI for nested model comparisons. The establishment of measurement invariance is deemed credible when ∆CFI does not exceed 0.01. A range of 0.01 to 0.02 in ∆CFI implies a decrease in model fit, yet the significance of this difference cannot be conclusively determined. If ∆CFI surpasses 0.02, the model is considered to be differentially structured [36,37].

Thirdly, our study proceeded with longitudinal measurement invariance, utilizing the follow-up sample (n = 967). This step was crucial to determine if the CES-D reliably gauges the same construct over time. The standard sequence of invariance tests–configural, metric, and scalar–was applied, ensuring equality of the respective parameters across both assessment periods. For these tests, we adhered to the criteria of ∆CFI < 0.01.

Fourth, as part of the longitudinal invariance analysis for the CES-D, we extracted factor loadings for each item from waves 1 and 2. We then calculated the average factor loadings for each dimension. Using SPSS 26.0, we determined the average variance extracted (AVE) at both time points to assess the convergent validity of the factor structure. Furthermore, to assess the CES-D’s internal consistency reliability, we calculated Cronbach’s alpha coefficients.

Table 2 presents the fit indices for the five structural models evaluated in this research. Each model achieved an acceptable fit, as indicated by CFI and TLI values exceeding 0.90, and RMSEA values falling below 0.08. Notably, both the initial four-factor model (Model 1) and Model 4 showed the most favorable alignment with the data, as reflected in their lowest BIC values (∆BIC > 10) compared to the others. Considering the slight difference in fit between these models (∆BIC = 0.605) and the widespread validation of the original four-factor structure in diverse racial/ethnic populations [38,39], including numerous Asian countries [40,41], our study further substantiates the four-factor structure’s applicability to Chinese adolescents.

Table 3 outlines the goodness-of-fit indices for both the independent models and those testing invariance between rural and urban settings. For wave 1, the four-factor model exhibited a satisfactory fit in both rural and urban groups. Specifically, for the rural group, the fit indices were χ² (164) = 549.188, p < 0.001, CFI = 0.966, TLI = 0.961, RMSEA = 0.067. Similarly, for the urban group, the indices were χ² (164) = 527.973, p < 0.001, CFI = 0.977, TLI = 0.973, RMSEA = 0.066. Moreover, in the comparison of nested models–namely configural, metric, and scalar invariance–the model fit did not show any notable decline. This observation is supported by the change in CFI (∆CFI), where the fit indice exceeded the recommended benchmark.

For wave 2, the four-factor model continued to show a satisfactory fit for both rural and urban groups. Specifically, the fit indices for the rural group were χ² (164) = 554.491, p < 0.001, CFI = 0.964, TLI = 0.958, RMSEA = 0.070. For the urban group, they were χ² (164) = 553.198, p < 0.001, CFI = 0.975, TLI = 0.971, RMSEA = 0.071. Assessing the nested models, which include configural, metric, and scalar invariance, we observed no substantial reduction in model fit, as the ΔCFI values consistently stayed above the advised thresholds. This result corroborates the configural, metric, and scalar invariance of the four-factor model in both rural and urban contexts.

Table 4 shows that the four-factor model maintained an acceptable fit across both assessment waves. For wave 1, the fit indices were χ² (164) = 954.698, p < 0.001, CFI = 0.969, TLI = 0.964, RMSEA = 0.068; For wave 2, the indices were χ² (164) = 972.683, p < 0.001, CFI = 0.968, TLI = 0.963, RMSEA = 0.071. The tests for longitudinal invariance, encompassing configural, metric, and scalar invariance, all received support. Remarkably, the most stringent model, positing equal factor loadings and item thresholds, demonstrated robust fit indices: χ²(380) = 1,665.144, CFI = 0.974, TLI = 0.974, RMSEA = 0.059. In comparing nested models, ΔCFI was 0.012 when assessing configural vs. metric invariance. A ΔCFI within the range of 0.01 to 0.02 indicates a moderate but not significant deterioration in model fit, which does not provide sufficient evidence to reject invariance [36,37]. When comparing metric invariance and scalar invariance, the ΔCFI equals 0.007. Since this value is below 0.01, it indicates a good fit of the model. Thus, the longitudinal measurement invariance of the CES-D’s four-factor model was maintained.

In the analysis of the CES-D’s longitudinal invariance model, we examined the factor loadings for each item at both wave 1 and wave 2. As detailed in Table 5, during wave 1, item loadings across the four dimensions of the CES-D–Depressed Affect (DA), Positive Affect (PA), Somatic Complaints (SC), and Interpersonal Problems (IP)–varied between 0.542 (“I had crying spells”) and 0.911 (“I felt that people disliked me”). At wave 2, these loadings ranged from 0.545 (“I had crying spells”) to 0.892 (“I felt that people disliked me”). Additionally, the mean factor loadings for all dimensions exceeded 0.7 at both waves, indicating reliable measurement of the latent constructs. Additional psychometric evaluations revealed that the average variance extracted (AVE) for all factors exceeded 0.5 in both waves. This indicates that more than 50% of the variance in the observed items was explained by the latent variables, thus affirming adequate structural validity [42]. Furthermore, the composite reliability (CR) values were above 0.85 in both assessments, reflecting outstanding reliability. Moreover, all domains recorded values above 0.74, reflecting consistent internal consistency ranging from acceptable to excellent across the two waves. In conclusion, the factor analysis conducted on the CES-D scale indicates that it maintains a range from satisfactory to excellent in terms of reliability and validity when used to assess adolescent samples across the two waves.

This research distinctively employed categorical confirmatory factor analysis (CFA) using the weighted least squares means and variance-adjusted estimator. This approach marks a departure from numerous prior studies that have analyzed CES-D items as continuous variables [9,17,39]. This approach allows for a more nuanced investigation of the CES-D’s factor structure. As far as we are aware, this research represents the inaugural effort to evaluate the rural-urban invariance of the CES-D, constituting a notable advancement in this area of study. By examining both rural-urban and longitudinal invariance, our study provides a comprehensive illustration of the CES-D’s measurement properties.

The use of convenience sampling from five schools may limit the representativeness of our sample. Future studies could improve generalizability by adopting a stratified sampling approach for a more diverse representation of the Chinese adolescent population. Our analysis was based on only two time points. Subsequent research could benefit from including additional time points to provide a more detailed temporal picture. Considering the 3-month interval used in our study, future research should explore the impact of different time intervals as potential confounding factors.