Revisiting the Defeatist Performance Belief Scale in Adults With Schizophrenia and Youth at Clinical High-Risk for Psychosis: A Comprehensive Psychometric Analysis

Abstract

Background and Hypothesis

In accordance with the Cognitive Model of Negative Symptoms, defeatist performance beliefs (DPBs) are an important psychosocial mechanism of negative symptoms in schizophrenia-spectrum groups. DPBs are also mediators of negative symptom improvement in clinical trials. Despite the clinical significance of DPBs and their inclusion as a mechanism of change measure in clinical trials, the psychometric properties of the DPB scale have not been examined in any schizophrenia-spectrum group.

Study Design

This study evaluated the factor structure, reliability, and validity of the DPB scale in 943 schizophrenia and 250 clinical high-risk for psychosis (CHR) participants from multiple US sites. Confirmatory factor analyses tested competing factor structures: a unidimensional model—consistent with how DPBs are currently assessed—and multifactorial models with up to 4 factors identified with exploratory factor analyses.

Study Results

Models with 3 and 4 factors provided superior fit compared to the unidimensional model, with an advantage for the 3-factor model. The 3-factor model, consisting of Overvaluing Success, Overvaluing Failure, and Overvaluing Social Evaluation factors, demonstrated good replicability, temporal stability, and measurement invariance in schizophrenia and CHR samples. Convergent validity was demonstrated via significant correlations with negative symptoms and functioning, but limited associations were present with neurocognition. Discriminant validity was supported by low correlations with positive symptoms.

Conclusions

Findings support the validity and reliability of the 3-factor structure of the DPB scale across phases of psychosis. Use of a 3-factor structure may clarify the most critical DPB targets for negative symptom treatment and early prevention and intervention.

Introduction

Negative symptoms significantly predict poor occupational and social functioning in people with schizophrenia (SZ).^1,2 Although pharmacological treatments have demonstrated limited efficacy,^3,4 psychosocial treatments—especially Cognitive Behavior Therapy (CBT)—may successfully improve negative symptoms. Recent meta-analyses indicate that CBT for negative symptoms yields medium effect size improvements,⁵ with effects that persist posttreatment.^6–8

The theoretical framework underlying CBT for negative symptoms is Dr. Aaron T. Beck’s Cognitive Model.⁹ In this model, dysfunctional beliefs are the principal mechanisms underlying the development and maintenance of psychopathology. Specifically, negative symptoms stem from beliefs that involve the incapability of behavioral initiation, persistence, and value.⁹ Defeatist performance beliefs (DPBs) are a central concept in this model (e.g., “It is not worth trying because I will only fail”).¹⁰ DPBs involve themes of potential failure that are trans-situational, definitive, and personally costly. DPBs are associated with limited behavioral output for even simple activities, presumably through attempts to protect oneself from seemingly evitable failures. The resulting lack of attempts to engage in activities provides few opportunities to counteract these beliefs, leading DPBs to become strengthened and inactivity to become entrenched.

Three lines of research have bolstered support for the Cognitive Model of Negative Symptoms. First, relative to healthy controls, DPBs are elevated in psychometrically defined schizotypy,¹¹ clinical high-risk for psychosis (CHR),¹² early psychosis,¹³ and more prolonged SZ samples.^10,14–16 Second, increased DPBs are significantly associated with greater negative symptom severity, poorer functioning, and neurocognitive impairment (see¹⁷). Finally, CBT-related improvements in negative symptoms are mediated by DPB changes.^18–21 This converging evidence provides strong support for the Cognitive Model of Negative Symptoms.^9,22,23

Given the significance of DPBs for negative symptoms and their common inclusion as a psychological mechanism of change measure in clinical trials, it is vital that DPB measures are valid and reliable for SZ-spectrum groups. However, the primary DPB scale used has not been rigorously psychometrically evaluated in SZ-spectrum samples. This DPB scale was originally derived from the Dysfunctional Attitudes Scale (DAS²⁴), a putative unidimensional scale created to assess dysfunctional attitudes underpinning depression. However, subsequent factor analyses in depression, general population, and college student samples identified two subscales: one capturing DPBs and the other related to need for approval.^25–27 Consistent with the Cognitive Model of Negative Symptoms,⁹ DPBs (but not need for approval) are uniquely associated with negative symptoms, poor functioning, and cognitive impairment in SZ.^10,28 Although the DPB scale has been assumed to be unidimensional, a rigorous examination of the measure’s factor structure has not been conducted. Testing the factor structure of the DPB scale could identify more discrete and personalized mechanistic targets for psychosocial treatment. Indeed, if there are specific DPBs (or subscales) most central to negative symptoms, then measuring DPBs with a total score may obfuscate treatment effects and lead to inaccurate conclusions regarding treatment efficacy.

The current study evaluated the factor structure, reliability, and validity of the DPB scale in 943 SZ and 250 CHR participants recruited from multiple sites throughout the United States. We tested whether DPBs were best characterized by a unidimensional model—consistent with current DPB scoring methods—or a multifactorial structure. The measurement invariance of the most appropriate factor structure was tested across SZ and CHR samples. Given that the DPB items appear to reflect different clusters of content, we hypothesized a multifactor structure would better characterize DPBs than a unidimensional model. The preferred factor structure was hypothesized to yield strong fits and be invariant across the SZ samples. We hypothesized some measurement invariance between CHR and SZ samples, as DPBs may not be as entrenched in CHR youth. The convergent validity of the most appropriate factor(s) was examined through associations between DPBs and negative symptoms, functioning, and neurocognition. Discriminant validity was assessed through associations between domains not theorized to be associated with DPBs: positive and disorganized symptoms.²⁹

Methods

Participants

Data was used from several research programs interested in a mechanistic understanding of negative symptoms or the efficacy of Cognitive Behavioral Social Skills Training,²³ pharmacological treatments, Recovery-Oriented Cognitive Therapy,²² or a mobile intervention targeting reward processing³⁰ (baseline data are reported from treatment studies). SZ participants (N = 943 total) were recruited from five different sites: University of Pennsylvania (Penn) (n = 269), University of California San Diego (UCSD) (n = 343), the University of Georgia (UGA) (n = 89), Indiana University-Purdue University Indianapolis (n = 56), and the Maryland Psychiatric Research Center (n = 186). SZ participants were recruited in outpatient mental health clinics and via paper and online advertisements. SZ participants met DSM-IV or DSM-5 criteria for schizophrenia or schizoaffective disorder based on the Structured Clinical Interview for DSM-IV or DSM-5.³¹

CHR participants (n = 250 total) were recruited as part of the Clinical Assessment of Psychosis Risk (CAPR) consortium study (n = 195),³² which includes 7 recruitment sites: University of Maryland Baltimore, Northwestern University, Temple University, UGA, Emory University, University of Califonia Irvine, and Yale University. Outside of CAPR, additional (n = 55) CHR participants were recruited at UGA. CHR participants were recruited through specialized clinics focusing on the evaluation of emerging psychotic symptoms, print and online advertisements, and through community mental health center referrals. All CHR participants met criteria for a CHR for psychosis syndrome based on the Structured Interview for Prodromal Syndromes.³³

Demographically matched healthy controls who completed the DPB scale were available for the SZ UGA (n = 82) and Maryland Psychiatric Research Center (n = 115) samples. Additional healthy controls for the CAPR and UGA CHR samples (n = 139) also completed the DPB scale.

Table 1 presents sample demographics (see supplement for sample demographic differences).

Procedures

In addition to the DPB scale, all participants completed validated measures of negative symptoms, positive symptoms, functioning, neurocognition, and depression; these measures assessed convergent and divergent validity. To examine how risk for a psychotic disorder was related to DPBs, the North American Prodrome Longitudinal Study-2³⁴ cross-sectional conversion risk prediction score³⁵ was also calculated for CHR participants. The supplementary materials contain detailed measure information. All participants provided written informed consent at their respective sites.

Measures

Defeatist Performance Belief Scale.

The 15-item defeatist performance belief subscale¹⁰ from the Dysfunctional Attitudes Scale (DAS)²⁴ contains items indexing overgeneralized negative beliefs about a person’s capacity to perform tasks (e.g., “If I do not do well all the time, people will not respect me.”). Items are rated on a 7-point Likert scale from “Agree Completely” to “Disagree Completely.” Items are scored so higher scores reflect greater DPBs.

Data Analysis

We aimed to identify the optimal factor structure of the DPB scale using several steps. We first used the Penn sample as a large training sample, and then cross-validated the preferred factor structure in the UCSD sample and remaining SZ samples (referred to hereafter as GMI (Georgia, Maryland, and Indiana samples); these sites were combined so that sample sizes were more equivalent). We then evaluated the identified structure in the CHR sample. In each sample, we first estimated exploratory factor analyses (EFA) extracting 1 to 4 factors; 4 was chosen as the upper limit given the number of scale items (15). The EFAs were an important first step for examining the viability of multifactor models (e.g., 2–4) sans clear guiding theory or prior evidence delineating specific subdomains of DPBs. EFAs were conducted using the oblique Quartimin rotation. EFA models were examined using pattern loadings and goodness-of-fit indices to more objectively evaluate and compare EFA models. To more definitively test the latent structure of the DPB scale and test competing hypotheses about the DPB structure (unidimensional or mutifactorial), CFA was used to test and directly compare a unidimensional/1-factor model and 2, 3, and 4-factor models identified in the EFA in each of the samples. Testing models in these separate, large samples allowed for testing whether identified factor solutions replicated and converged across different independent samples. Estimators included: a weighted least squares estimator with standard errors and mean- and variance-adjusted χ² test that uses a full-weight matrix (WLSMV) and the maximum likelihood with robust standard errors model estimation (MLR). CFA estimations leveraged model modification indices to evaluate select fixed parameters and to determine if freely estimated fixed parameters would improve any poor-fitting model. All factor analyses were conducted in Mplus version 5.0.

Model fit was evaluated using several indices,^36–41 including the χ² test, comparative fit index (CFI), Tucker–Lewis index (TLI), root mean square error of approximation (RMSEA), the standardized root mean squared residual (SRMR), and the weighted root mean squared residual (WRMR); these helped to evaluate absolute fit. Fit was estimated using standard cutoffs for fit indices, including a nonsignificant χ² test, RMSEA values ≤ to 0.08, CFI and TFI values ≥ to 0.95, SRMR values ≤ 0.08, and WRMR values ≤ 1.00. Of note, the χ² test is sensitive to sample size, leading to the potential rejection of good-fitting models. Thus, alternative fit indices are generally preferred for estimating model fit.

To compare alternative models, the log-likelihood, Akaike information criterion, Bayesian information criteria, and the sample size–adjusted Bayesian information criteria were used; lower values indicate superior models except for the log-likelihood, where higher values (less negative) correspond to better fit. These information criteria are relative fit indices of model parsimony that consider model complexity based on degrees of freedom.

Multigroup CFA was used to evaluate the measurement invariance of the DPB preferred factor structure across the SZ and CHR samples. Sequential tests of configural, metric, scalar, and residual invariance were conducted.^42,43 Configural invariance indicates that items load on the identical factor in all samples. Metric invariance is present when factor loadings are equivalent across samples. Scalar invariance indicates that both factor loadings and intercepts are equivalent across samples. Residual invariance is present when factor loadings, intercepts, and residual variance are equivalent across samples. Invariance models were evaluated through an examination of change in chi-square (χ² diff), CFI (ΔCFI), TLI (ΔTLI), and RMSEA (ΔRMSEA) estimates as constraints are successively imposed at each step of invariance testing. Invariance is indicated when ΔCFI ≤ −0.01, ΔTLI ≤ −0.01, and ΔRMSEA ≤ 0.015.

The reliability and validity of the preferred factor structure were then tested in several steps. Reliability was tested via test-retest-reliability in a subsample of participants with available observational longitudinal data (n = 139) and through factor replicability indices. Replicability indices meeting the following criteria were adjudged as indicating good reliability: factor determinacy ≥ 0.8, H-index ≥ 0.8, and Omega ≥ 0.75.

The validity of the preferred factor structure was tested using correlational analyses. Convergent validity analyses involved associations with negative symptoms, functioning, and neurocognition. Discriminant validity was tested using positive and disorganized symptoms. Correlations with depression were also examined. Criterion validity was tested by comparing the identified factors between SZ and CHR and their respective control groups and between CHR and SZ. To summarize effects across different sites and measures, meta-analyses were conducted using the psychmeta package in R to synthesize the convergent and discriminant validity effect sizes.

Results

Factor Structure—EFA

EFA was first used to extract up to 4 factors in each of the SZ samples and the CHR sample. Results are summarized in Supplementary Tables 1–5. The CFI, TLI, and RMSEA values indicated the 1-factor model was a poor fit in all 3 SZ samples and the CHR sample, with all CFI values below 0.88, almost all TLI values below 0.94, and all RMSEA values above 0.13. Although the 2-factor model showed incremental improvements in these fit statistics, it similarly was a mediocre fit to the data in all samples, and the loading patterns were inconsistent across samples. The EFA 3- and 4-factor models were superior fits to the data, with CFI, TLI, RMSEA, and SRMR values above or in line with cutoffs for a good fit. Information criteria largely favored the 3- and 4-factor models over the 1- and 2-factor models in all samples. The BIC largely favored the 3-factor model, while the other information criteria generally favored the 4-factor model. Both the 3- and 4-factor structures showed consistent loading patterns in the UCSD and GMI samples. These loading patterns were not extracted in the individual Penn and CHR samples but did emerge when all study samples were combined. When only SZ samples were combined, the 3-factor loading pattern was replicated. In the 4-factor solution, Item 4 showed a very low saturation, and Item 8 switched factors across samples.

Factor Structure—CFA

Given inconsistencies in loading patterns obtained in the EFAs, we further investigated the factor structure of DPBs using a CFA approach. We fitted the 3- and 4-factor solutions that most consistently emerged in the EFAs and compared these to a 1-factor and 2-factor solution (See Supplementary Table 6 for the competing factor structures). The availability of four independent samples allowed us to estimate these competing models in a designated calibration sample (Penn) and verify the preferred model across three cross-validation samples (UCSD, GMI, and CHR). In the Penn sample, none of the indices of absolute fit met acceptable thresholds for the 1-factor and 2-factor models save the TLI for the 2-factor model (0.95) (see Table 2; Supplementary Table 7). The 3- and 4-factor models had TLI and WRMR estimates that exceeded the threshold for adequate fit; however, both the CFI and RMSEA failed to reach the pertinent 0.95 or 0.08 threshold. The information criteria favored the 3-factor over the other three models.

A post hoc examination of the modification indices (see Supplementary Table 8) to identify possible model misspecifications flagged a cross-loading of Item 9 across two factors and substantial residual covariances between some item pairs as potentially contributing to model misfit. In the Penn calibration sample, these residual correlations were between Items 2 & 3, Items 6 & 7, and Items 5 &15 (see Table 3 for items). These parameters were subsequently estimated sequentially in respecified 3- and 4-factor models. The respecified models substantially improved overall goodness-of-fit with CFI and RMSEA now close to acceptable thresholds in the Penn sample.

A cross-validation of the 3- and 4-factor models in the UCSD, GMI, and CHR samples showed results similar to that of the Penn sample with CFI and RMSEA that failed to exceed conventional thresholds for the 3- and 4-factor solutions (see Table 2; Supplementary Table 7). The one exception was that the CFI was above 0.95 in the CHR sample for both the 3- and 4-factor respecified models. Free estimation of select residual covariances and a cross-loading of Item 9 in the UCSD and GMI sample improved goodness-of-fit. Further, the TLI and WRMR estimates exceeded thresholds for all 3- and 4-factor respecified models in all samples.

Aided by modification indices, we believe there is adequate rationale to freely estimate the identified residual covariances: (1) There is arguably conceptual overlap in the content of the item pairs that were flagged by high modification indices and freely estimated; (2) Modification indices flagged these residual covariances in multiple samples; and (3) These item pairs frequently produced high correlations across the study samples. Save the CHR sample, the cross-loading of Item 9 on the same factor was similarly flagged across samples. Supplementary Table 8 identifies the fixed parameters that were freed, modification index value, and expected parameter change.

The information criteria favored the 3- and 4-factor models (original or respecified) over the 1- and 2- factor models in all samples (see Table 2). Unsurprisingly, the respecified models produced lower (or less negative for LL) information criteria than the original 3- and 4-factor models. However, the 3-factor respecified model was favored by all information criteria over the 4-factor respecified model in all samples except the GMI sample where the 4-factor respecified model was favored. This suggests that although both the 3- and 4-factor respecified models were valid across samples, the 3-factor model appeared slightly stronger.

Factor Interpretation

Both the 3- and 4-factor solutions had a factor capturing Overvaluing Success (see Table 3; Supplementary Table 6, 9, and 10), which included beliefs about success and high levels of achievement being a key source of value. Both solutions also had a factor of Overvaluing Social Evaluation, which included items about being judged by others if success was not obtained. The factor solutions differed in structure of the remaining items: for the 3-factor solution, the remaining items comprised a factor assessing Overvaluing Failure; this included items about heightened sensitivity to failure. This was broken into two factors for the 4-factor solution: Overvaluing Autonomous Performance—or beliefs that asking for help is a sign of weakness—and Overvaluing Potential Failure, which included items capturing the idea that failure is a catastrophe.

Measurement Invariance

We tested the factorial invariance among the 3 SZ and 1 CHR samples for the 3-factor and 4-factor solutions by running the measurement invariance models in the full sample combining all SZ and CHR samples. Table 4 contains the fit indices of the measurement invariance models. For the configural model, the fit statistics indicated that 3-factor and 4-factor models held across all SZ and CHR samples. CFI and TFI values were at 0.95, and RMSEA fell slightly below the 0.08 threshold. Metric invariance or the presence of equivalent factor loadings across all SZ and CHR samples was also found for both the 3- and 4-factor solution, with ΔCFI, ΔTLI, and ΔRMSEA all falling within accepted thresholds. Evidence for scalar invariance (equivalence of factor loadings and intercepts) was less robust in both the 3-factor and 4-factor models, as CFI decreased by more than 0.01 and RMSEA values increased by more than 0.015 and no longer met invariance thresholds. Modification indices suggested that item 4 particularly had a higher intercept in the SZ samples whereas items 6 and 7 had a higher intercept in the CHR samples. This suggests that the SZ samples endorsed item 4 more strongly than the CHR sample, while the CHR sample endorsed items 6 and 7 more strongly than the SZ samples. When these items were freely estimated, the scalar models improved remarkably and met threshold for invariance. Finally, residual invariance was supported, as indicated by CFI, TLI, and RMSEA change values that met thresholds and decreased less than 0.01 from the scalar partial model.

Reliability

The stability of both the 3- and 4-factor solutions was assessed using replicability indices (see Supplementary Table 11). Since item 9 had relatively lower factor loadings than other items, cross-loaded on two factors in both the 3- and 4-factor models in several samples, and is the only reverse-scored item (potentially necessitating more cognitive processing), replicability indices were calculated with and without item 9. The Overvaluing Failure factor in the 3-factor solution was stable across all SZ and CHR samples based on the H, Omega, and factor determinacy scores. Factors Overvaluing Success and Overvaluing Social Evaluation in the 3-factor solution were stable across all SZ and CHR samples except the Penn SZ sample, where H and Omega scores were below desired thresholds, but factor determinacy scores were above the desired threshold. For the 4-factor solution, the results were more variable. The Overvaluing Potential Failure factor met stability thresholds for all three indices in all SZ and the CHR samples. The Overvaluing Success Factor generally met stability criteria (except for the Penn sample), but the stability of the Overvaluing Autonomous Performance and Overvaluing Social Evaluation factors were mixed for H and Omega values. Notably, in none of the SZ or CHR samples were all stability thresholds met for both the Overvaluing Autonomous Performance and Overvaluing Social Evaluation factors in the 4-factor solution. Removing item 9 only led to slight declines in stability indices for both factor solutions. Thus, the 3-factor solution with item 9 retained was identified as the most stable factor solution.

Test-Retest Reliability

Significant positive correlations were observed between baseline and 6-month follow-up scores in the available SZ Penn sample (n = 139) for each of the 3 factors (range: r = 0.42 to 0.57) (see Supplementary Table 12).

Four-Factor vs Three-Factor Solution

Since there was support for both the 3- and 4-factor solution in the EFA and CFA analyses, we present the pooled results in Supplementary Tables 13 and 14 to aid with identifying the optimal factor solution. While the EFA fit statistics and information criteria largely favored the 4-factor model, the CFA fit statistics were split, but the CFA information criteria favored the 3-factor model. Further, the interpretability of the factors as well as the replicability indices favored the 3-factor model. Recent work has also highlighted the importance of examining model characteristics beyond model fit statistics to identify the optimal factor solution.^44,45 Taken together, the 3-factor solution appears to be the most parsimonious factor solution for DPBs. Thus, the remaining analyses focus on the 3-factor solution, but analyses with the 4-factor solution are reported in the supplement.

Criterion Validity

Both CHR and SZ groups reported significantly greater DPBs compared to their respective CN on all 3 factors (see Supplementary Table 15). Effect sizes were medium to large. CHR and SZ did not significantly differ on Overvaluing Success and Failure factors; however, CHR had trending greater scores on the Overvaluing Social Evaluation factor (P = .05, d = 0.14).

Convergent Validity.

Correlations between negative symptoms and functioning for each sample are reported in Supplementary Table 16. Individual study level correlations for negative symptoms and functioning ranged from small to medium (negative symptoms: range −0.02 to 0.35; functioning: range −0.03 to −0.31) but were generally small.

The meta-analytic cumulative effect sizes across samples between the 3 factors and the convergent validity measures are reported in Table 5 (and Supplementary Table 17 for the 4-factor solution). The Overvaluing Success and Overvaluing Failure factors were significantly related to all negative symptom domains and factors scores and the total score; overall meta-analytic effect sizes were small. The Overvaluing Social Evaluation factor was only significantly related to alogia, blunted affect, expressive negative symptoms factor, and total negative symptoms. The Overvaluing Failure factor was more strongly or trending towards being more strongly related to asociality (P = .07), motivation and pleasure negative symptoms factor (P = .04), expressive negative symptoms (P = .06), and total negative symptoms (P = .02) than the Overvaluing Social Evaluation factor (see Supplementary Table 18). No other significant differences were present for the strength of the associations between negative symptoms and the 3 DPB factors. The strength of the associations between negative symptoms and the Overvaluing Failure factor and the single DPB factor score were statistically similar and most similar in magnitude. In addition, all 3 factors and the single DPB factor score were significantly associated with functioning in a similar magnitude; effects sizes were small.

The correlations between cognition and DPB factors were small in the individual samples (see Supplementary Table 19) and at the meta-analytic level (see Table 5; Supplementary Table 17). Almost all associations between processing speed, working memory, and overall cognition at the meta-analytic level failed to reach significance. The 3 factors and the single DPB factor score did not show differential associations with cognition (see Supplementary Table 18).

An additional consideration is whether DPBs play a role in risk for transitioning to a psychotic disorder in CHR youth. Greater DPBs on each factor were significantly associated with greater probability of conversion to a psychotic disorder (see Supplementary Table 16); effects for DPB factors and the single DPB factor score were statistically similar in magnitude.

Discriminant Validity.

The correlations between positive and disorganized symptoms were generally small in each sample (see Supplementary Table 20). The meta-analytic effect sizes between positive symptoms and the 3 DPB factors across samples were nonsignificant or trending (see Table 5; Supplementary Table 17); disorganized symptoms showed small but generally significant overall associations with each factor. There were no significant differences in the strength of the associations between positive and disorganized symptoms and the 3 DPB factors and single DPB factor score (see Supplementary Table 18).

Depression.

Depression evidenced small to moderate associations with the DPB factors in individual samples (see Supplementary Table 20) and small effect sizes at the meta-analytic level across samples (see Table 5; Supplementary Table 17). The Overvaluing Success factor was less strongly associated with depression than the Overvaluing Social Evaluation factor, Overvaluing Failure factor, and the single DPB factor score (see Supplementary Table 18).

Discussion

The Cognitive Model of Negative Symptoms⁹ posits that DPBs are a key mechanism driving the emergence and maintenance of negative symptoms in those at risk for and with SZ.⁹ This model forms the basis of CBT for negative symptoms, which in contrast to pharmacological treatments,^3,4 yields significant negative symptom improvments.⁵ Thus, the measurement of DPBs is vital for ensuring mechanism engagement and change in clinical trials. However, no prior study to our knowledge has examined the psychometric properties of the DPB scale in SZ-spectrum groups. The aim of this study was to evaluate the factor structure, reliability, and validity of the DPB scale in 943 SZ and 250 CHR participants from multiple US sites.

Across SZ and CHR samples, EFA and CFA results consistently found that the unidimensional model provided a poor fit to the data across all fit statistics. Thus, the underlying DPB factor structure is not optimally represented by a 1-factor solution as currently used in the literature. In contrast, the 3- and 4-factor models provided superior fit. Although the fit statistics demonstrated a slight preference for the 4-factor model, the replicability indices, interpretability of the factors, and parsimony indicated the 3-factor model was a better fit. We therefore recommend the single total score no longer be used for the DPB scale. Instead, we recommend using the 3-factor solution: Overvaluing Success (items 1, 11–14), Overvaluing Failure (items 5–10, 15), and Overvaluing Social Evaluation (items 2–4). Given the similarity of factor results across CHR and SZ samples, the 3-factor structure appears appropriate for use across the SZ-spectrum. Both the 3- and 4-factor solutions showed remarkable invariance across the SZ and CHR samples. Thus, even though we observed some significant demographic differences between the SZ samples in terms of race, age, and sex (as noted in the supplement), the 3- and 4-factor CFA solutions were largely invariant across these samples. However, the multigroup CFA provided only partial support for scalar invariance, identifying that the CHR and SZ groups differed in their degree of endorsement of some DPB items.

Reliability of the 3 DPB factors was demonstrated via reliability indices and temporal stability. Reliability indices indicated the 3 factors were stable across the CHR sample and all but one SZ sample; notably, the 4-factor structure had lower stability than the 3-factor solution in all samples. In addition, the temporal stability across 6 months was good—ranging from r = 0.42 to 0.57—for the 3 factors in the SZ sample with available data. Although these scores might appear low, they are not unexpected given the longer test-retest period and suggest DPBs fluctuate over time. Indeed, these “lower scores” align with recent findings suggesting that negative symptoms and associated mechanisms dynamically vary across time.^46,47

Evidence also largely supported the convergent validity of the 3 DPB factors. In accord with the Cognitive Model of Negative Symptoms and past findings,^9,17 the Overvaluing Success and Overvaluing Failure factors were significantly related to all negative symptom domain, dimension, and total scores; correlations were small in magnitude at the meta-analytic level. However, the Overvaluing Social Evaluation factor was only significantly related to expressive and overall negative symptoms. The Overvaluing Failure factor was more strongly related to negative symptoms (MAP dimension, total score) than the Overvaluing Social Evaluation factor. No other DPB factors evidenced significantly stronger associations with negative symptoms, and the magnitude of the associations with negative symptoms was similar among both the Overvaluing Failure factor and the DPB single-factor score. The Overvaluing Failure factor may therefore be most closely tied to negative symptoms and the most important type of DPB to target in CBT for negative symptoms. This is important as DPBs as currently measured with a single score cover a range of overgeneralized general beliefs about performance, and it leaves clinicians with little guidance about the specific types of DPBs to prioritize in treatment. In addition, associations with cognition—in contrast to our hypotheses—were not significantly associated with any DPB factor at the meta-analytic level. Previous cognition and DPB associations have been more variable than negative symptom and functioning associations.^13,48 DPBs specific to cognitive task performance may identify more consistent associations. Alternatively, DPBs may only impact cognition once DPBs reach a certain threshold, or sample characteristics (e.g., illness length) may influence the magnitude of the association.

Although the magnitude of the correlations between the DPB factors and negative symptoms and functioning was less than one might expect for evidence of convergent validity, these results are consistent with prior meta-analytic effect sizes observed with the single DPB factor.¹⁷ The small associations suggest that DPBs may be one of several factors important for these domains. Multicomponent treatments targeting DPBs and additional components related to negative symptoms (e.g., asocial beliefs,⁴⁹ reward-processing,⁵⁰ and environmental processes⁵¹) are likely needed to yield sustained negative symptom improvements. Alternatively, although the DPB scale from the DAS has been invaluable to the development of CBT for negative symptoms and has been a foundational tool to measure DPBs, the observed small effect sizes and limited differential relationships between the 3 factors, single DPB factor, and negative symptoms may also suggest that a revision to the DPB scale from the DAS is needed. Indeed, clinical trials examining CBT for negative symptoms have repeatedly demonstrated that targeting DPB successfully improves negative symptoms in people with SZ.^18,19,21,52 However, prior meta-analytic evidence examining the single DPB factor¹⁷ and our work examining novel DPB factors in a large SZ sample both suggest that only small associations are observed between DPBs as measured by the DAS and negative symptoms. This disconnect may suggest that revisions to the scale are needed. Indeed, the differential timepoints assessed by the DPB DAS scale and negative symptom ratings may obscure the true strength of the association. Similarly, the overgeneralized, trait-like nature of the DPB items on the DAS may be too general to adequately capture the key beliefs that contribute to negative symptoms. Our recent work using state-like DPB items found stronger, temporally precise associations between negative symptoms.⁵³ Other work has suggested more positive, adaptive beliefs related to energy, capability, control, and connection,²² or beliefs related to fears of being judged negatively in social situations may be essential beliefs to target in order to improve negative symptoms.⁵⁴ Thus, it may be that more state-like or domain-specific measures of DPBs or novel conceptualizations related to positive beliefs might capture beliefs that are more closely tied to negative symptoms.

Notwithstanding, our findings offer support for the discriminant validity of the 3 DPB factor solution as currently measured with the DAS. Specifically, negligible to small correlations that were smaller than the negative symptoms and functioning associations were observed between the 3 DPB factors and positive and disorganized symptoms. This bolsters support for the specificity of these beliefs for assessing mechanisms underlying negative symptoms and functioning. Nevertheless, we also observed slightly larger associations between the DPB factors and depression. However, meta-analytic effect sizes were still small, with the largest associations being between depression and the Overvaluing Failure and Overvaluing Social Evaluation factors. Associations with DPBs and depression are also not entirely surprising. Cognitive models of depression^55,56 posit a key role of dysfunctional thinking patterns in the emergence and maintenance of depression. Indeed, the DAS was originally created to measure such beliefs.²⁴ Depression and mood disorders are also common among SZ and especially CHR youth and contribute to secondary negative symptoms.^57,58 Correlations between depression and DPBs have also been found to be as high or higher than negative symptoms.^10,59 Thus, as to be expected with a population with high rates of comorbid mood disorders, in addition to primary negative symptoms, the DPB scale may assess psychological mechanisms underlying secondary negative symptoms related to depression. Findings may therefore suggest that targeting especially the DPB Overvaluing Failure and Overvaluing Social Evaluation factors could also reduce these secondary negative symptoms and in turn depression in people accross the SZ-spectrum.

Our criterion validity analyses also helped to clarify when during the illness course DPBs develop. Mirroring past findings with the single-factor DPB score,^10,12,13 all 3 factors were significantly elevated in SZ and CHR compared to CN. In an extension of prior work, we also compared DPB levels between CHR and SZ. Notably, fewer differences emerged when comparing the CHR and SZ groups, with the only difference being that CHR youth more strongly endorsed the Overvaluing Social Evaluation factor than SZ. This may be due to the greater focus on social relationships for CHR adolescents and young adults and the greater challenges that may occur due to emerging psychopathology. More broadly, these findings suggest that DPBs become elevated even prior to the onset of a full psychotic disorder and thus are not the result of stigma, antipsychotic use, or other factors associated with a SZ diagnosis. Findings bolster support for Cognitive Model theory, which suggests DPBs develop prior to illness onset and contribute to declines in motivation, social engagement, and the experience of pleasure.⁹ Further, increased DPBs on all 3 factors were associated with increased probability of conversion to a psychotic disorder on a cross-sectional psychosis risk calculator. This suggests that DPBs, a negative symptom mechanism, may be valuable for predicting which CHR cases will transition. Longitudinal data collection is ongoing, allowing for testing whether CHR youth who develop a psychotic disorder have differential DPB elevations.

While study strengths were the sample size and geographically diverse recruitment sites that allowed for confirmation of replication of the identified superior factor structure, several limitations should be considered. This study did not evaluate measurement invariance for other potential sources of DPB heterogeneity, including sex and racial groups. Indeed, there were significant site/group differences between the 3 SZ samples in age, race, and sex. However, despite these differences, the 3- and 4-factor DPB solutions were invariant across the samples. Nevertheless, systematically examining the measurement invariance across race and sex is a useful next step for validation but is beyond the scope of the current study.

Overall, our findings point to two key implications. First, our results suggest that the 3-factor DPB structure on the DAS may help to isolate specific DPB factors or types of DPBs that might be most important to target in negative symptoms treatments. The superior fit of the multifactorial structure in both SZ and CHR youth suggests that use of a single DPB factor score as currently measured with the DAS may be statistically unjustified across phases of psychotic illness. Use of a single-factor score in clinical trials may result in underspecified treatment effects or clarity around the mechanism of therapeutic action. It is possible that treatments may have differential effects or engagement on each factor, thus potentially obfuscating effects that are DPB factor-specific if a single-factor score is used. Indeed, we isolated one factor, the Overvaluing Failure factor, as being the most closely linked to negative symptoms, suggesting that if successfully targeted, this factor may be driving treatment effects. However, at the same time, the observed associations between the Overvaluing Failure factor and negative symptoms were small. Thus, our second key implication is that despite the advances the DPB DAS scale has helped to facilitate in the treatment of negative symptoms, a revision to the DPB scale may be needed to better capture more precise types of DPBs that are contributing to the development and maintenance of negative symptoms. Future efforts examining more state-like and domain-specific DPBs could help to clarify the association between DPBs and negative symptoms and critically, clarify the key DPBs that should be prioritized to improve negative symptoms. Indeed, as currently measured, it is possible that prior clinical trials have impacted DPBs, and changes in DPBs were the primary factor facilitating negative symptom change, but this effect was undetectable when measuring DPBs more globally as is done with the current scale or was undetectable at a single-factor DPB level. In summary, although this work began the process of isolating the primary DPBs that may be most essential to target to improve negative symptoms across different stages of psychosis, it also points to a need for novel measures that may better capture the most critical DPB targets for negative symptom treatment and early prevention and intervention efforts.

Funding

This work was supported by the following National Institutes of Health grants: R01 MH091057 (E.G.), R01 MH120090 (J.M.G.), R01 MH112613 (L.M.E.), R01 MH120091 (L.M.E.), R01 MH120092 (G.P.S.), R01 MH116039 (G.P.S./V.A.M.), R21 MH119438 (G.P.S.), R01 MH112545 (V.A.M.), R01 MH1120088 (V.A.M.), U01 MH081988 (E.F.W.), R01 MH120090 (J.A.W.), R01 MH112612 (J.S.), R21 MH112925 (GPS); K23 MH092530 (G.P.S.); R21 MH122863 (G.P.S.), R01 MH120089 (P.R.C/S/S.W.W.W). Additionally, this work was supported by the Department of Veterans Affairs, Veterans Health Administration, Rehabilitation Research & Development Service (E4876R; E.G.) and the Biomedical Laboratory Research & Development Service (1I01CX000810; E.G.). This work was also supported by the William and Dorothy Bevan Scholarship from the American Psychological Foundation (L.L.), an Indiana Clinical and Translational Sciences Institute predoctoral award (ICTSI) (L.L.), the ICTSI Clinical Research Center (UL1TR001108), a Distinguished Investigator Award from the National Alliance for Research on Schizophrenia and Depression and by grants from the Heinz Foundation and the Barbara and Henry Jordan Foundation.

Conflicts of Interest

Dr Gregory Strauss is one of the original developers of the Brief Negative Symptom Scale (BNSS) and receives royalties and consultation fees from Medavante-ProPhase LLC in connection with commercial use of the BNSS and other professional activities; these fees are donated to the Brain and Behavior Research Foundation. Dr Strauss has received honoraria and travel support from Medavante-ProPhase LLC for training pharmaceutical company raters on the BNSS. In the past 2 years, Dr Strauss has consulted for and/or been on the speaker bureau for Minerva Neurosciences, Acadia, Lundbeck, Sunovion, Boehringer Ingelheim, and Otsuka pharmaceutical companies. Dr Ahmed has consulted for Minerva Neurosciences and Boehringer Ingelheim. Dr Grant has received royalties from the Guilford Press. Dr Granholm has consulted for Boehringer Ingelheim and has received royalties from Guilford Press. All other authors have no relevant disclosures to report.

References