Adaptation and Validation of Version B of the Edinburgh Cognitive and Behavioural ALS Screen for the Portuguese Population

Abstract

Objective

This study aims to adapt and provide psychometric support for the validation of version B of the Edinburgh Cognitive and Behavioural ALS Screen (ECAS) for the Portuguese population, addressing the need for consistent cognitive evaluations in amyotrophic lateral sclerosis (ALS). A second culturally adapted ECAS screen facilitates the accurate characterization of ALS progression, mitigates learning effects, and supports tailored care management.

Methods

The adaptation process included forward-backward translation, cultural adaptation, and cognitive debriefing on a prospective sample of 193 ALS patients and 106 controls. A multiple regression analysis identified predictors relevant for establishing ECAS cut-off scores. Psychometric evaluations, including reliability assessments and tests of convergent, construct, and criterion validity, were conducted. Additionally, version A's psychometric properties were reevaluated with complementary analyses and a larger sample.

Results

Version B demonstrated good internal consistency with Cronbach's alpha of 0.802, comparable to the previously established version A. Moderate inter-item correlations further supported reliability, reflecting internal coherence. Equivalence testing between the Portuguese versions supported convergent validity, confirming version B's alignment with version A's theoretical framework. Exploratory factor analysis provided preliminary support for construct validity, and receiver operating characteristic analyses established cut-off values for both versions, revealing moderate sensitivity with a tendency toward false negatives, and higher specificity.

Conclusions

This study provided evidence for the cultural suitability, reliability, and validity of the Portuguese ECAS B. As evidence supports the equivalence of the Portuguese ECAS versions, they can be used for flexible screenings and applied with the calculated cut-off values to enhance diagnostic accuracy.

INTRODUCTION

Amyotrophic lateral sclerosis (ALS) is a neurodegenerative disease with heterogenous presentations ranging from pure motor manifestations to frontotemporal dementia (FTD) (De Marchi et al., 2021; Gregory et al., 2010). ALS incidence is reported to be rising, with prevalence estimates ranging from 1.57 to 9.62 per 100,000 people globally (Xu et al., 2020). Certain European countries, such as Portugal, however, report higher rates—around 10.32 per 100,000 (Conde et al., 2019; Wolfson et al., 2023). Approximately 50% of these patients develop cognitive-behavioral impairment, impacting quality of life, treatment compliance, caregiver burden, survival, and disease progression (Temp et al., 2021, Temp et al., 2022; Yang et al., 2021). Early and frequent screenings to detect these changes are strongly recommended (Taule et al., 2022).

The Edinburgh Cognitive and Behavioural ALS Screen (ECAS) (Abrahams et al., 2014) is a multidomain tool regarded as the gold standard for screening cognitive impairments associated with ALS (Pinto-Grau et al., 2017; Poletti et al., 2016). It has a score range of 0 to 136 points, with 85% sensitivity and specificity for detecting ALS-specific cognitive impairment (Niven et al., 2015; Pinto-Grau et al., 2017). It is concise, requiring 15–20 min to administer, and is specifically adapted to accommodate the motor limitations of ALS patients (Es et al., 2017; Gray et al., 2024; Hodgins et al., 2020).

Specifically, the ECAS focuses on executive functions, prioritizing inhibition, dual-attention processes, concept formation and abstraction (Abrahams, 2023; De Marchi et al., 2021; Katerelos et al., 2024). It also evaluates language and verbal fluency, particularly through word retrieval and strategy formation (Abrahams, 2023; De Marchi et al., 2021; Jellinger, 2023). Social cognition and theory of mind are included, as these, along with the other processes, tend to be impaired in ALS (Abrahams, 2023; De Marchi et al., 2021). Scores below established cut-offs in language, verbal fluency and executive domains indicate ALS-specific dysfunction, and are suggested to be biomarkers for pathological change (Abrahams, 2023; Gregory et al., 2020; Pinto-Grau et al., 2017). Many of the mentioned tasks rely on working memory, which can deteriorate as ALS progresses and damages the frontal and temporal lobes (Abrahams, 2023; Abrahams et al., 2000; Kobeleva et al., 2021; Vellage et al., 2016). The ECAS also includes memory and visuospatial scores to detect other neurodegenerative diseases like Alzheimer's (Kourtesis et al., 2020). If overall cognitive functions are below cut-off values, results suggest FTD, especially if accompanied by behavioral symptoms included in the screening interview (Crockford et al., 2018; Hodgins et al., 2020).

Other cognitive screening tools validated for ALS include the ALS-Brief Cognitive Assessment (ALS-BCA), ALS-Cognitive Behavioral Screen (ALS-CBS), Frontal Assessment Battery (FAB), and Mini-Mental State Examination (MMSE) (Gosselt et al., 2020). These tools generally show moderate to high sensitivity and specificity, although the MMSE has poorer psychometric properties (Gosselt et al., 2020; Taule et al., 2020). Most screenings either are or can easily be adapted to accommodate mobility difficulties, with the exception of the MMSE and FAB (Gosselt et al., 2020). In terms of domain coverage, all assess executive functions, fluency, and memory, while language and social cognition are only evaluated by the ECAS (Beeldman et al., 2021; Gosselt et al., 2020).

Due to its high clinical utility, the ECAS is increasingly required in clinical trials, studies, and routine practice, necessitating repeated evaluations (Beswick et al., 2021; Crockford et al., 2018; Crockford et al., 2018; Simão et al., 2024; Taule et al., 2022). With a documented learning effect within six months, a minimum of two versions are required to monitor disease progression, with implications for consent capacity and end-of-life care planning (Burkhardt et al., 2017; Crockford et al., 2018). These alternate forms allow repeated assessments for ALS patients, preserving the construct characteristics and difficulty level of the original version A while minimizing practice effects (Crockford et al., 2018). In Portugal, however, only version A has been adapted and supported by validity evidence (Tomsic Masa, 2015).

This study aims to adapt and provide psychometric support for the validation of version B of the ECAS for the Portuguese population. This will be achieved by assessing internal consistency and coherence using Cronbach's alpha, item-total and inter-item correlations, as well as construct validity through exploratory factor analysis to assess the underlying cognitive domains, and criterion validity via receiver operating characteristic (ROC) analyses to determine diagnostic accuracy and cut-off values. Additionally, we aim to verify convergent validity through equivalence testing between versions B and A (Tomsic Masa, 2015), to ensure that version B measures the intended constructs enhancing assessment flexibility (Crockford et al., 2018; Es et al., 2017).

The study will also investigate predictors of cognitive-behavioral performance, such as age, education level, and sex, addressing inconsistencies found in previous research (Beeldman et al., 2016; Consonni et al., 2021; Rhodes et al., 2023; Tomsic Masa, 2015; Yang et al., 2021) and further characterize version A with a larger sample size, offering additional evidence of its psychometric robustness and clinical utility.

METHODS

Study Design and Participants

The study employed a cross-sectional, observational, quantitative design. Clinical and cognitive data were collected from a sample of 299 participants (193 ALS patients and 106 healthy controls) for the adaptation of the version B of the ECAS from March 2022 to May 2024. The sample size was determined based on comparable studies in the literature (Albertyn et al., 2022; Kacem et al., 2022; Mora et al., 2018; Niven et al., 2015; Poletti et al., 2016; Soliman et al., 2023) and the maximum feasible number of participants that could be recruited during the study period, considering the importance of minimizing participant burden, particularly for ALS patients.

All patients had ALS meeting the Gold Criteria for ALS (Shefner et al., 2020) and were recruited consecutively at the neurology department of Santa Maria Hospital (Academic Medical Centre). The healthy control group was recruited from senior universities, day-care centers, local associations, worker unions, and among patient caregivers. All interviews were conducted in a quiet, controlled, and private setting to ensure confidentiality and minimize distractions for participants.

ALS patients who were unable to speak and write, or who had premorbid cognitive deficits, were excluded. Exclusion criteria for both groups included a history of Axis I psychiatric illnesses (Diagnostic and Statistical Manual of Mental Disorders, 2013), other neurological diseases (except ALS in patients), and non-Portuguese native speakers. This exclusion was designed to minimize confounding effects of cognitive impairments unrelated to ALS, following the methodologies of prior ECAS validation studies (Abrahams et al., 2014; Albertyn et al., 2022; Lulé et al., 2015; Mojtabavi et al., 2021; Niven et al., 2015; Poletti et al., 2016; Soliman et al., 2023; Taule et al., 2022; Tomsic Masa, 2015).

Procedures and Measurements

Translation of ECAS Version B

The translation and adaptation process followed established guidelines (R. Hambleton & De Jong, 2003; Hambleton & Li, 2005; Van de Vijver & Poortinga, 2005). Authorization to adapt was obtained from Sharon Abrahams and Thomas Bak. Translation and back-translation were performed by two bilingual translators at the Laboratório de Estudos de Linguagem, Centro de Estudos Egas Moniz, Lisbon Faculty of Medicine. A consensus version was developed through an expert meeting and a literature review to confirm that the adapted items accurately represent the intended construct. Cross-cultural and linguistic reviews were conducted to maintain semantic, idiomatic, experiential, and conceptual consistency with the original English version B of the ECAS.

The language sub-scores were adapted to reflect ALS-related impairments in verb and noun processing, object naming (living, non-living, manipulable, and natural objects), and compound words (Abrahams et al., 2014; Bak & Hodges, 2001; Pinto-Grau et al., 2021). Additionally, Portuguese lexical frequency and everyday relevance were used to align the spelling sub-score with Portuguese version A (Davies, 2012; Maria & Nascimento, 2006; Quaresma, 2008; Tomsic Masa, 2015). For the verbal fluency sub-scores that target phonemic fluency and rapid executive functioning, Portuguese lexical frequencies guided letter selection to approximate difficulty levels in English version B and Portuguese version A (Abrahams et al., 2004; Davies, 1990, 2012; Maria & Nascimento, 2006; Tomsic Masa, 2015). The visuospatial and executive function sub-scores retained the original content, except for sentence completion. This sub-score was culturally adapted to remain relevant for Portuguese speakers while assessing inhibitory control, a known deficit in ALS patients, without altering cognitive demands (Abrahams, 2023). For memory sub-scores, the original short-story structure was maintained to assess immediate and delayed recall, along with delayed recognition, so that episodic memory evaluation remained equivalent across both versions (Abrahams et al., 2014; Pinto-Grau et al., 2017; Tomsic Masa, 2015).

As content and construct validity were established by the original authors (Abrahams et al., 2014; Abrahams & Bak, 2013), version B's items were reviewed for construct coverage and understandability after translation. The pretesting interviews included cognitive debriefing with 10 healthy controls. Participants completed version B of the ECAS and provided open-ended feedback on the clarity, understandability, and difficulty of the items. Participants discussed their thoughts, strategies used during tasks, and how well the items measured cognitive functions (e.g., language, visuospatial skills, memory). The feedback was documented and percentages of incorrect answers for more challenging items were analyzed. No major comprehension issues were identified, and participants confirmed that the translated version was clear and accurately measured the intended constructs.

A pilot study was conducted to assess difficulty, response frequency, and the inappropriateness or redundancy of items. Version B was applied to 32 healthy controls and 23 ALS patients, that were evaluated with both ECAS versions (A and B) in a counterbalanced design to test equivalence, neutralizing habituation effects.

The recorded data included age, sex, and education level. Cognitive data were derived from the ECAS (Abrahams et al., 2014) scale raw scores (available in the supplemental material) validated for Portuguese (Tomsic Masa, 2015).

After testing, some items were adapted, resulting in an improved version B. In total, 84 participants (58 healthy controls and 26 ALS patients) were tested using an independent groups design, with half completing version A and half completing version B. The study design shifted from dependent to independent samples after the pilot study due to patient feedback about fatigue. To lessen the burden, subsequent data collection involved each participant completing either version A or B of the scale. Demographic variables were matched between the two groups.

Ethical Considerations and Data Availability

The study was approved by the Ethics Committee of the Medical Academic Centre of Lisbon. All patients provided informed consent before enrolment, and databases were anonymized to protect privacy.

Statistical Analysis

A multiple regression analysis was employed to evaluate predictors of the ECAS total score, with age and education treated as continuous variables and sex and group (ALS patients vs. healthy controls) treated as categorical variables. The psychometric evaluation began with reliability analyses, using Cronbach's alpha to assess internal consistency, as well as item-total and inter-item correlations to evaluate each item's contribution to the overall scale and the consistency between items.

Following reliability, validity was assessed across multiple dimensions. Convergent validity was evaluated through confidence interval equivalence testing, bootstrapped for non-parametric data. A 90% confidence interval was constructed, while a 95% confidence interval was used for the overall analysis. Both dependent and independent samples were matched to test the equivalence between ECAS versions, with the equivalence margin (Δ) set at −1 to +1 for the dependent sample and −5 to +5 for independent samples.

Construct validity was investigated with exploratory factor analysis, employing principal-axis factoring with Promax rotation and Kaiser normalization to examine the scale's underlying factor structure and assess construct validity.

Criterion validity was evaluated through ROC analyses to derive the cut-off values across four demographic groups. Youden's index was used to determine the optimal balance of sensitivity and specificity. To address skewed data in this analysis, natural and base-10 logarithmic transformations, along with custom power transformations were applied.

The assumptions for each analysis were confirmed: linearity, multicollinearity, normality of residuals, homoscedasticity, and absence of autocorrelation for the multiple regression analysis; appropriate equivalence margins and two-sided testing for equivalence testing; sampling adequacy, suitability of the correlation matrix, linearity, and absence of multicollinearity for the exploratory factor analysis; and binary outcome, normalized data, and consistent classification across samples for ROC analyses. Independence of observations was confirmed for all analyses.

All statistical tests were performed using IBM Statistics SPSS, V.28 and Python, V.3.12.

RESULTS

Verbal Fluency Indexes

The adaptation of version B included developing verbal fluency performance intervals based on a Portuguese-speaking sample to allow for appropriate scoring (Abrahams & Bak, 2013). The two sub-scores—unrestricted and restricted—include both spoken and written response modalities to accommodate patients with orofacial muscle weakness or fine-motor loss (Beeldman et al., 2014; Loeffler et al., 2016). Separate scoring tables were developed for each modality to allow for precise assessment across patient groups.

A total of 30 healthy participants, (mean age 60, SD 11.96; mean education 12.46, SD 6.31; 16 women) were assessed to calculate the spoken verbal fluency index (VFI), and another 30 participants, (mean age 59.87, SD 10.74; mean education 12.73, SD 4.66; 16 women), were assessed to calculate the written VFI. These demographic traits were chosen due to the peak age at ALS onset (58–63 years [Kiernan et al., 2011]) and the mandatory education level in Portugal (12 years [Marôco, 2021]). To ensure representation, 15% of each group had only a 4^th grade education level, reflecting the low education rates reported among Portuguese elders (Paulo et al., 2011).

The Shapiro–Wilk test confirmed the age distribution was parametric for both the spoken (W = 0.959, p = .293) and written groups (W = 0.965, p = .413), while education was not normally distributed. Age comparability was confirmed using an independent-samples t-test, t (58) = 0.045, p = .964, with Levene’s test indicating homogeneity of variances (F (1,58) = 1.951, p = .168). The 95% confidence interval for the difference in means ranged from −5.74 to 6.01, with an effect size (Cohen’s d) of 0.012, suggesting a negligible effect. Education groups were shown to be equivalent through a Mann–Whitney test (U = 438.50, Z = −0.171; p = .864).

The verbal scores for the spoken modality of the unrestricted exercise were calculated using the original authors' preferred method, based on mean and standard deviation (Abrahams & Bak, 2013; Pinto-Grau et al., 2017). Details are provided in Table 1 of the supplementary material. For the written recall of unrestricted words, and both spoken and written recall of restricted words, percentiles were used due to wider standard deviations, as recommended (Abrahams & Bak, 2013).

Although spoken and written VFIs are usually comparable (Abrahams & Bak, 2013), these modalities were analyzed using the independent sample's confidence interval equivalence testing method, bootstrapped for non-parametric data. Both verbal fluency exercises were compared across modalities and their sum, with no significant differences found. Further details can be found in Table 1.

Pilot Study

In line with the study's objective of evaluating convergent validity, we conducted equivalence testing between version A and the newly adapted version B. This analysis was performed on a sample of 32 healthy controls (mean age 62.75, SD 12.40, mean education 11.88, SD 5.39), and 23 ALS patients (mean age 61.04, SD 9.72; mean education 10.35, SD 4.39). Version B was easier in naming and sentence completion but more challenging in immediate recall, delayed recognition, and spelling, which prompted adjustments. These differences are detailed in Table 2 of the supplementary material.

Psychometric Evaluation

A comprehensive psychometric evaluation was conducted, beginning with reliability assessments using Cronbach's alpha, item-total and inter-item correlations to evaluate internal consistency and coherence. Validity was then assessed across multiple dimensions: convergent validity by testing the equivalence between ECAS version B and the previously established version A, construct validity through exploratory factor analysis to examine the underlying factor structure, and criterion validity using ROC analyses to derive cut-off scores for distinguishing ALS patients from controls.

Internal Consistency and Coherence

From a sample of 30 controls (mean age 56.77, SD 9.64; mean education 14.53, SD 5.01; 63.3% women), Cronbach's alpha for version B was 0.802 for the 15 sub-scores and 0.858 for the 8 overall scores. For comparison, version A, based on a sample of 74 controls (mean age 59.81, SD 11.48; mean education 13.39, SD 5.80; 70.3% women), showed a Cronbach's alpha of 0.816 for the 15 sub-scores and 0.875 for the 8 scores. These values indicate good internal consistency for both versions, providing evidence for their reliability.

For version B, removing scores such as language (0.824), verbal fluency (0.744), executive (0.856), ALS-specific (0.963), memory (0.470), ALS non-specific (0.601), and ECAS total (1) would decrease the alpha, suggesting these scores contribute positively to internal consistency. Only visuospatial (0.563) would slightly increase the alpha if deleted, suggesting that visuospatial abilities may represent a more distinct cognitive domain. Similar patterns were observed for version A, removing any of the scores, except for visuospatial (0.585) would decrease the alpha, reinforcing their role in the scale's overall consistency.

Item-total correlations were calculated for scores (score-total correlations), revealing similar patterns in both versions. Correlations were high to very high, further supporting their strong contribution to the overall scale. Except for memory in version B, which showed an acceptable contribution of 0.470, suggesting that it may be less aligned with the overall scale. Across versions, ALS-specific had the highest correlation (0.963 for versions B and A), followed by the executive score (0.856 for version B and A) reinforcing their central role in ECAS. Non-specific ALS also shows high correlations (0.601 for ECAS version B and 0.776 for ECAS version A), suggesting that although both ALS-specific and non-specific contribute to the ECAS total, the non-specific ALS domain captures a more distinct cognitive aspect. These values are presented in Table 2.

Examining the item-total correlations for sub-scores for a more granular view, dot counting in version B had a negative item-total correlation (−0.009) and a low correlation in version A (0.233), suggesting it might weaken coherence. Removing dot counting would slightly increase the alpha in both versions. Similarly, delayed recall in version B had a low item-total correlation (0.093) and removing it would slightly increase the alpha. In version A, comprehension had a low item-total correlation (0.296), though removing it would decrease the alpha. These values are presented in Table 3.

Inter-item and inter-score correlations (found in Tables 3–6 of the supplementary material) were calculated to explore relationships between different scores and sub-scores. In version B, naming had a high correlation with social cognition (0.837), possibly due to eyesight difficulties in older participants, a pattern not seen in version A, likely because of its larger sample size. In both versions, mostly moderate correlations within the ALS-specific domain suggest that these sub-scores capture related and complementary aspects of the same construct. In contrast, non-specific ALS scores further reinforce their measurement of more distinct aspects, as does dot counting, which showed negative correlations with most sub-scores. Lastly, moderate to high correlations between naming, comprehension and sentence completion, as well as between recall, spelling and verbal fluency, suggest these cognitive skills are interrelated and jointly contribute across sub-scores, highlighting coherence within these areas while supporting the distinctiveness of others.

Regarding inter-score correlations, the executive score was the highest contributor to ALS-specific (0.912 in version B and 0.910 for version A), underscoring its central role in the ALS-specific domain. Memory was highly correlated to non-specific ALS (0.975 for version B and 0.990 for version A) indicating it is a critical component of this domain. Visuospatial showed the lowest correlations with other scores but correlated notably with the executive score in both versions (0.693 for version B and 0.555 for version A).

Overall, both versions demonstrate strong internal consistency, with high Cronbach's alpha and score-total correlations supporting reliability across cognitive domains. Each score, except visuospatial, enhances consistency, and moderate inter-item correlations highlight coherence, especially in ALS-specific measures. A larger sample could further corroborate version B's reliability findings.

Convergent Validity

Following the adjustments made after the pilot study, equivalence between version A and B was re-tested to confirm that they measure the same constructs. 84 participants were divided into two groups with 29 controls and 13 ALS patients each. Group 1 completed version A of the ECAS (mean age 58.14, SD 11.70; mean education 13.74, SD 4.77; 57% women) and group 2 completed version B of the ECAS (mean age 58.12, SD 11.04; mean education 13.85, SD 5.74; 59% women). The groups were matched in age, education, and sex. The Shapiro–Wilk test confirmed that the age distribution was parametric for both group 1 (W = 0.970, p = .326) and group 2 (W = 0.973, p = .407). Non-parametric tests (Mann–Whitney U and Chi-Square) showed no significant differences in education (U = 838.0, Z = −0.398; p = .690) and sex (χ² (1, N = 84) = 0.049, p = .825).

Bootstrapping for non-parametric data generated 90% confidence intervals for predefined equivalence margins (Δ) of ±5. These wider margins were used due to the independent samples design, contrasting with the narrower ±1 margin used for intra-participant analysis. All subtests fell entirely within the predefined equivalence margin, as shown in Table 4.

These results demonstrate equivalence between the Portuguese ECAS versions, indicating that version B aligns with the theoretical framework of version A, contributing to its convergent validity.

Construct Validity

Construct validity was investigated through exploratory factor analysis using principal axis factoring with Promax rotation and Kaiser normalization to examine the underlying factor structure of both ECAS versions (details in Tables 7–9 of the supplementary material). Promax rotation was chosen because the ECAS sub-scores were expected to be interrelated, a relationship confirmed by the factor correlation matrix, which showed moderate correlations between factors, ranging from 0.467 to 0.673.

The decision to retain three factors was based on the eigenvalue >1 criterion, supported by the scree plot, which showed a clear elbow at the third factor. Factors beyond the third had eigenvalues below 1 and contributed less explanatory power, further supporting the decision to retain only the first three factors. This aligns with the theoretical expectation that the ECAS measures distinct but related cognitive domains. The KMO measure of sampling adequacy was 0.901, indicating excellent suitability for factor analysis, and Bartlett's test of sphericity was significant (χ² = 1718.06, df = 105, p < .001), confirming that the data was appropriate for factor analysis. Our sample included 299 participants – 257 completed version A and 42 completed version B – comprising 106 controls (mean age 58.42, SD 11.55; mean education 13.82, SD 5.59; 68% women) and 193 ALS patients (mean age 62.91, SD 12.21; mean education 10.40, SD 5.46; 60% men).

The extraction communalities reflect the variance in each item explained by the factors, with extraction communalities ranging from 0.314 to 0.601. While most items are well-represented by these factors, items like digit span (0.368) and delayed recall (0.314) are less well-represented.

The three factors extracted had eigenvalues of 6.105, 1.391, and 1.052, explaining 40.70%, 9.27%, and 7.01% of the variance, respectively, for a total of 56.98%, suggesting that the ECAS captures a substantial portion of the underlying cognitive domains. After extraction, the factors explained 46.86% of the variance, with factor 1 explaining 37.19%, factor 2 explaining 5.93%, and factor 3 explaining 3.74%. This three-factor solution supports the interpretation that the ECAS measures distinct but interrelated cognitive domains.

The pattern matrix shows strong loadings (above 0.4) for most items on their respective factors, confirming that these items align with the cognitive domains they are intended to measure. The factors reflect relationships observed in the inter-item and inter-score correlations. Factor 1 represents language comprehension and naming skills, with items like naming, comprehension, and sentence completion loading highly. Factor 2 represents memory recall and attentional sub-scores, with items like immediate recall, spelling, and unrestricted verbal fluency loading highly. Factor 3 represents spatial abilities and executive function, with items like dot and cube counting loading highly. The factor correlation matrix also indicates the factors were related yet distinct, with correlations of 0.673 between factors 1 and 2, 0.467 between factors 1 and 3 and 0.561 between factors 2 and 3. This supports the multidimensional structure of the ECAS, reflecting both coherence and distinctiveness within the cognitive domains.

This analysis provides preliminary support for the construct validity of the adapted ECAS version B, indicating that the ECAS scales effectively measure distinct cognitive domains relevant to ALS. This approach is well-suited when the factor structure is not yet well-established, particularly for newly adapted tools. Confirmatory factor analysis, which typically requires a stronger theoretical basis or prior studies, is recommended for future research in the Portuguese population to confirm the structure identified here.

Criterion Validity

This study used ROC analyses to determine cut-off values and evaluate sensitivity and specificity, using the same sample of 299 participants described in the construct validity section.

The Portuguese adaptation of version A of the ECAS previously relied on education-based cut-offs from the 10^th percentile of control group scores (Tomsic Masa, 2015), however, other adaptations also found age to be a relevant predictor (Niven et al., 2015; Pinto-Grau et al., 2017). To enhance diagnostic accuracy, we tested both age and education as predictors.

A stepwise-multiple regression analysis was conducted, resulting in a significant model (F (3.295) = 86.781, p < .001) that explained 68.5% of the variance in ECAS total (R² = 0.463). Age (β = −0.233; p < .001), education (β = 0.491; p < .001) and group (controls and patients) (β = −0.171; p < .001) were significant predictors, while sex was not.

ROC analyses were then performed to select optimal cut-off points for segmenting groups according to age and education, based on discriminatory ability. The final age groups were ≤ 50 (AUC =0.683) and > 50 (AUC =0.727), while the education groups were ≤ 12 (AUC =0.689) and > 12 (AUC =0.705). Consequently, group 1 (age ≤ 50, education ≤12) included 27 participants (17 patients, 10 controls), group 2 (age ≤ 50, education >12) included 35 participants (17 patients, 18 controls), group 3 (age > 50, education ≤12) included 153 participants (113 patients, 40 controls), and group 4 (age > 50, education >12) included 84 participants (46 patients, 38 controls).

A ROC analysis was then applied to each group to derive cut-off values for ECAS using Youden's index as the selection method. These cut-off values are shown in Table 5, with additional analyses details in Table 6. The AUCs for all ECAS scores are presented in Figs. 1 to 8 of the supplementary material.

As depicted in Table 5, discrepancies between the sum of sub-score cut-off values and the ECAS total cut-off values were observed. The discrepancy arose from the independent optimization of each score and sub-score to maximize sensitivity and specificity for diagnostic accuracy.

When considering the AUCs of the total and specific ALS scores depicted in Table 6, groups 1 and 3 suggest poor to acceptable discriminatory ability with moderate to high uncertainty, while groups 2 and 4 indicate acceptable to good discriminatory ability with some uncertainty. Overall, the sensitivity and specificity results suggest a susceptibility to false negatives, as models detected controls better. However, groups 2 and 3 demonstrated a more balanced detection of false positives and false negatives.

These results support the criterion validity of the Portuguese ECAS adaptations, with ROC analyses providing suitable cut-off values by age and education, though a tendency toward false negatives indicates a need to improve sensitivity.

DISCUSSION

This study aimed to adapt and provide psychometric support for the validation of version B of the ECAS for the Portuguese population, ensuring cultural and linguistic suitability. Pilot testing identified items requiring adjustment based on participant feedback and item analysis, which improved comprehension and aligned the difficulty level of the Portuguese version B with version A. Subsequent testing confirmed the effectiveness of these revisions.

Good internal consistency supports the ECAS' reliability, with Cronbach's alpha for version B at 0.802 for sub-scores and 0.858 for scores, similar to version A, which shows a slightly higher alpha, likely due to its larger sample size. Both versions share a similar structure, with high score-total correlations, and most scores contributing meaningfully to scale consistency in “alpha if deleted” analyses, reinforcing reliable measurement of the intended constructs.

Coherence is supported by strong score-total and inter-score correlations, with particularly strong alignment in ALS-specific and executive scores. Moderate inter-item correlations within the ALS-specific domain indicate complementary relationships between sub-scores, further supporting reliability.

These analyses also showed that the non-specific ALS domain measures a more distinct cognitive aspect, consistent with the ECAS's theoretical framework, as shown by an increase in Cronbach's alpha if visuospatial is deleted and low inter-item correlations in dot counting (Abrahams, 2023).

Validity was also supported through multiple analyses. Equivalence testing showed that all sub-scores fell within predefined margins, supporting convergent validity by demonstrating that version B measures the same constructs as version A. The comparability of written and spoken VFIs for both unrestricted and restricted verbal fluency reinforced the scale's adaptability for ALS patients. Exploratory factor analysis provided preliminary evidence for construct validity, indicating that the ECAS measures distinct yet interrelated cognitive domains, also reflected in inter-item and inter-score correlations. Sampling adequacy (KMO measure) and Bartlett's test of sphericity confirmed factor analysis reliability. Most items showed high extraction communalities, indicating good representation by the factors, though digit span and delayed recall were less representative. The three-factor solution suggests that the ECAS captures a substantial portion of the underlying cognitive domains, with most items loading strongly on their respective factors confirming alignment with the intended domains. This further supports the scale's multidimensional structure, in which interrelated but distinct cognitive domains are measured. Criterion validity was supported by cut-off values derived for age and education, variables that significantly predicted the Portuguese ECAS total score, similar to the findings with the English ECAS (Pinto-Grau et al., 2017). This enhanced diagnostic precision for cognitive impairment in Portuguese ALS patients. ROC analyses showed acceptable to good discriminatory ability in higher education groups, and poor to acceptable ability in lower education groups. Optimized sensitivity and specificity improved clinical interpretability, although a tendency toward false negatives was observed, particularly in groups with lower discriminatory performance. While AUC values were moderate, a larger and more balanced sample could strengthen findings.

Regarding limitations, discriminant and predictive validity, along with test–retest reliability, were not assessed due to the cross-sectional design, limiting the evaluation of version B's long-term effectiveness. The use of a single rater for all assessments introduces potential bias, as inter-rater reliability was not analyzed. Additionally, the strict exclusion criteria may reduce the representativeness of our sample by excluding participants with psychiatric and neurological comorbidities. Although this approach aligns with previous ECAS validation studies (Abrahams et al., 2014; Albertyn et al., 2022; Lulé et al., 2015; Mojtabavi et al., 2021; Niven et al., 2015; Poletti et al., 2016; Soliman et al., 2023; Taule et al., 2022; Tomsic Masa, 2015), these comorbidities are prevalent in both the general population and ALS patients (Solmi et al., 2022; Zucchi et al., 2019). The moderate AUC values from the ROC analyses suggest some imprecision, which could affect diagnostic accuracy in clinical settings.

The study results are likely generalizable to Portuguese ALS patients without significant psychiatric or neurological comorbidities, particularly in similar clinical settings, given the consecutive sampling of ALS patients and careful matching of controls. However, applying ECAS version B across various Portuguese hospitals and testing its construct validity and reliability with a larger and more inclusive sample may enhance generalizability by better capturing the diversity of the ALS population (DiBernardo & Cudkowicz, 2006).

This study also expanded upon the Portuguese adaptation of ECAS version A by comparing it with version B through detailed psychometric analyses. Version A was initially described using a sample of 41 healthy controls, assessing various aspects of validity and reliability (Tomsic Masa, 2015). This study provided further evidence of its psychometric strength using a larger sample size and complementary tests. We introduced item-total and inter-item correlations to assess item consistency, along with ROC and factor analyses, which were supported by a strong KMO measure of sampling adequacy. These analyses confirmed that both versions can be used interchangeably while maintaining robust psychometric properties.

Moving forward, future studies could expand upon these findings by evaluating additional psychometric properties, such as discriminant and predictive validity, as well as test–retest and inter-rater reliability. More detailed intra-item analysis, exploring item difficulty, discrimination indices, item response theory and response bias, could offer significant insights into response patterns. Confirmatory factor analysis is also recommended to further validate the factor structure of the scale. Lastly, developing and testing a remotely administered version of the ECAS for the Portuguese population could benefit patients with mobility difficulties and those in rural areas (Gray et al., 2024).

CONCLUSION

The Portuguese versions A and B of the ECAS, now supported by gathered evidence for validation and shown to be equivalent, enable more flexible screenings. The alternate ECAS versions were designed to avoid the practice effects found with repeated use of version A and have been shown to do so in the English versions (Crockford et al., 2018). Further studies are needed to determine whether the Portuguese version B similarly addresses practice effects. In addition, research is required to develop reliable measures of change to accurately track performance variance in the Portuguese population, ensuring that observed differences reflect true changes in cognition. The ECAS versions can now be applied using the calculated cut-off values and are best implemented early in the disease, following diagnosis, to provide comprehensive information on disease progression and patient-specific needs (Costello et al., 2021; Crockford et al., 2018).

Implications

The successful adaptation and psychometric evaluation of version B of the ECAS for the Portuguese population offers a reliable, evidence-based screen for assessing cognitive-behavioral changes in ALS patients, enhancing diagnostic accuracy and enabling more targeted interventions (Hodgins et al., 2020). Having two versions with validation support mitigates the learning effect associated with repeated use, which is crucial for monitoring disease progression and adjusting patient care plans (Crockford et al., 2018). The availability of these tools also supports ongoing and future research by providing standardized cognitive screenings. Leveraging them enables healthcare providers, researchers, and policymakers to better monitor and manage the cognitive-behavioral aspects of ALS, improving patient outcomes and advancing the field.

FUNDING

This work was supported by the project “Brainteaser—Bringing Artificial Intelligence home for a better care of amyotrophic lateral sclerosis and multiple sclerosis” funded by the European Union's Horizon 2020 research and innovation program under the grant agreement No GA101017598.

ACKNOWLEDGEMENTS

We thank Maša Tomšič for adapting ECAS version A to the Portuguese population and Professor Abrahams and Bak for their translation guidelines. We also thank the participants, the Department of Neurology at Santa Maria University Hospital, and the experts at Laboratório de Estudos de Linguagem, Centro de Estudos Egas Moniz, Lisbon Faculty of Medicine for their support and contributions.

AUTHOR CONTRIBUTIONS

Sara Aguiar Simão (Conceptualization, Data curation, Formal analysis, Investigation, Methodology, Software, Validation, Visualization, Writing—original draft, Writing—review & editing), Lucas Wilfried Lopes Naumann (Methodology), Mamede de Carvalho (Conceptualization, Funding acquisition, Project administration, Resources, Supervision, Writing—review & editing), Miguel Santos (Resources), and Isabel Pavão Martins (Conceptualization, Resources, Supervision, Writing—review & editing)

REFERENCES