Establishing Convergent Validity of the FACE-Q Aesthetics Module Scales

Abstract

FACE-Q Aesthetics is a patient-reported outcome measure (PROM) designed to evaluate outcomes following both surgical and minimally invasive facial aesthetics procedures.^1,2 This PROM has 4 key domains (ie, facial appearance, health-related quality of life, naturalness, and adverse effects of treatment) and consists of 37 independently functioning scales and 6 checklists.^3-5

Although the FACE-Q Aesthetics module has been found to be reliable and responsive to change, establishing the validity of this PROM remains an iterative process.^1,6,7 To date, the convergent validity of these scales has yet to be established, representing a gap within the current literature. “Convergent validity” refers to how closely a scale or test relates to another tool designed to measure the same or similar construct (ie, the concept being studied). Specifically, if 2 scales designed to measure similar constructs (eg, facial appearance) have a moderate or high correlation, this can be viewed as evidence of convergent validity.⁸ Ultimately, this aspect of validity gives clinicians and researchers confidence that the scale being used accurately measures what it was intended to measure.

Typically, convergent validity is established through hypothesis testing, where investigators make a priori predictions about the strength of the correlation between 2 related measures. The evidence supporting an instrument's convergent validity becomes stronger as more predefined hypotheses are tested and determined to be correct.⁸

To establish the convergent validity of the FACE-Q Aesthetics module, it can be compared with another validated measure of facial aesthetics commonly used within the literature such as the MERZ Aesthetics (Raleigh, NC) scales. This measure consists of photonumeric scales designed to evaluate the appearance of the upper, middle, and lower face.⁹ To date, the MERZ Aesthetics scales have been validated for use by clinician/expert raters and have been shown to have good reliability.^10-12

The primary aim of this study was to establish the convergent validity of FACE-Q Aesthetics scales vs the MERZ Aesthetics scales in an online, international sample of participants.

METHODS

This online cross-sectional survey was coordinated at McMaster University (Hamilton, ON, Canada) and received ethics approval through the Hamilton Integrated Research Ethics Board (#13603; Hamilton, ON, Canada).

Participant Recruitment

This study was completed as a part of a larger planned analysis to validate new FACE-Q Aesthetics “natural” scales and item libraries for satisfaction with face and psychological well-being.⁷ A detailed description of participant selection and recruitment are reported elsewhere.⁷

Following a pilot survey of 144 individuals, a larger sample of participants from the United States and Canada were invited to complete the survey in December 2022, and the United Kingdom in August 2023, through the online crowd-sourcing platform Prolific (London, UK). Participants were compensated the equivalent of 10.80 GBP per hour for their time.

Participants were eligible to participate in this survey if they self-selected that: (1) they were ≥20 years old; (2) lived in Canada, the United Kingdom, or the United States; and (3) had been to a dermatology or a plastic surgery clinic in the past 12 months to receive one of 14 facial aesthetic treatments (Supplemental Table 1). Figure 1 demonstrates a flow diagram of participant selection.

Figure 1.

Flow diagram of participant selection.

Open in new tabDownload slide

Scales

Participants were required to complete 11 FACE-Q Aesthetics appearance scales and 12 MERZ Aesthetics scales. We selected these scales a priori by because they were deemed to measure similar constructs. The selected scales are shown in Table 1.

FACE-Q Aesthetics scales are scored by converting raw ordinal scores into a continuous score between 0 and 100, where higher scores correspond to higher satisfaction with appearance or better health-related quality of life. The description of individual scales can be found in the FACE-Q Aesthetics User's Guide (https://qportfolio.org/face-q/aesthetics/).⁴

The MERZ Aesthetics scales are scored ordinally between 0 to 3 or 0 to 4 and correspond to a spectrum of photographs for a selected aspect of the face. For example, the MERZ Jawline—At Rest can be scored between 0 (no sagging) to 4 (very severe sagging). Higher scores correspond to less favorable appearance for all MERZ Aesthetics scales except for Upper Lip Fullness—At Rest and Lower Lip Fullness—At Rest, for which higher scores correspond to a more favorable appearance.⁹

Sample Size

As per COSMIN guidelines for convergent validity, we aimed to have a minimum of 100 participants provide responses for each scale being examined.¹³

Data Analysis

To assess convergent validity, hypotheses regarding the correlation between the FACE-Q Aesthetics and MERZ Aesthetics scales were generated a priori (Table 2). Hypotheses were adapted from the “COSMIN Methodology for Systematic Reviews of Patient-Reported Outcome Measures User Manual.”¹⁴

Given that the selected FACE-Q Aesthetics scales include items that evaluate participant perceptions of their appearance (eg, how bothered they are by the area being evaluated) and the MERZ scales use standardized photographs for participants to match their appearance (eg, degree of jawline sagging), it was hypothesized that these scales likely measure related, but dissimilar constructs for the purpose of hypothesis testing.

Spearman's rank correlation coefficient (ρ) was used to calculate correlation. Convergent validity is generally considered adequate if >75% of hypotheses are correct or if a correlation with an instrument measuring the same construct is ≥0.50 (Hypothesis A).^8,15 An additional precondition was added by the study authors that all Hypotheses A and/or B should be true for a scale to demonstrate adequate convergent validity.

An analysis of variance (ANOVA) was performed to determine if statistically significant differences exist between related FACE-Q Aesthetics scale scores (ie, continuous data) and MERZ Aesthetic scale scores (ie, categorical data). Statistical significance was considered to be a P-value ≤.05. Missing data were handled using an available-case analysis. All analyses were performed with SPSS version 26.0 for Windows (IBM Corporation, Armonk, NY).

RESULTS

Following completion of this cross-sectional survey, n = 1259 participants were included in this analysis. The mean [standard deviation] age of the sample was 42.6 [11.9] years (range, 20-75 years), with 913 (72.5%) female, 332 (26.4%) male, and 14 (1.1%) gender diverse/other. Most survey respondents self-identified as Caucasian (76.9%) and lived in the United States (49.9%) or the United Kingdom (42.9%). Additional sample demographics are reported in Table 3.

Convergent Validity

The results of hypothesis testing are reported in Supplemental Table 2 and summarized in Table 4. Specifically, the FACE-Q: Lines Overall scale was found to have adequate convergent validity. This scale had moderate correlations with all chosen MERZ Aesthetics photonumeric scales except for Upper/Lower Lip fullness and Upper/Lower Cheek fullness, which were hypothesized to have weaker correlations because these scales do not measure the presence of lines. Overall, 100% of hypotheses were correct.

Additionally, the region-specific FACE-Q Aesthetics appearance and lines scales (ie, Lower Face and Jaw, Appraisal of Lines—Forehead/Between Eyebrows/Crow's Feet/Nasolabial Folds/Marionette/Lips, and Lips) were found to have adequate convergent validity. These scales demonstrated moderate correlations with MERZ scales that measured related constructs and weaker correlations with MERZ scales that measured unrelated constructs. Overall correct hypotheses ranged from 83.3% to 100%.

The FACE-Q scales that did not demonstrate convergent validity through this analysis were the Face Overall and Cheeks scales. Notably, the Face overall scale was hypothesized to have moderate correlation with all MERZ scales but was only weakly correlated with all scales. Thus both Hypotheses B and C were incorrect. Furthermore, the Cheeks scale was hypothesized to have moderate correlation with the MERZ: Upper/Lower Cheek fullness scales but was found to be weakly correlated with all MERZ scales. As such, Hypothesis B for both these 2 scales was incorrect, resulting in inadequate convergent validity, despite 83.3% of the hypotheses being correct overall.

ANOVA

Table 5 demonstrates the results of the ANOVA and the mean FACE-Q Aesthetics scale scores for moderately correlated MERZ scales. All FACE-Q scale scores were statistically significantly different between MERZ photonumeric scale scores. For example, patients scoring 0 (no sagging) on the MERZ: Jawline at rest scale had a mean FACE-Q: Lines overall score of 73.5 [21.9], while patients scoring 4 (very severe sagging) on the scale MERZ scale had a mean FACE-Q: Lines overall score of 25.5 [17.7].

DISCUSSION

FACE-Q Aesthetics was designed to measure outcomes that are important to patients undergoing surgical or nonsurgical cosmetic facial interventions. These scales have been broadly validated to demonstrate various properties of validity, reliability, and responsiveness, with a subset of the most frequently used scales qualified for use by the FDA as Medical Device Development Tools.^1-3,5,16-18

The concept of validity as it applies to measurement instruments is defined by the COSMIN group as “the degree to which [a PROM] measures the construct(s) it purports to measure.”¹⁹ Within the domain of validity, there exist 3 specific properties: content validity (ie, does the content of the PROM reflect the construct to be measured?), criterion validity (ie, does the score of the PROM adequately reflect the scores of the “gold standard”?), and construct validity (ie, does the PROM adequately measure the intended construct?).^19,20

In the present study, we assessed the convergent validity of FACE-Q Aesthetics with the MERZ Aesthetics scales and provide evidence of convergent validity for 9 out of 11 scales tested, namely, Lines Overall, Lower Face and Jaw, Appraisal of Lines—Forehead/Between Eyebrows/Crow's Feet/Nasolabial Folds/Marionette/Lips, and Lips scales. Although >75% of a priori defined hypotheses were correct for all scales, we concluded that convergent validity could not be sufficiently supported for the Face Overall and Cheeks scales because hypotheses suggesting a moderate correlation (ie, Hypothesis B) for these scales were incorrect—a precondition at the onset of the study.

There are several hypotheses for why the Face Overall scale may not have been convergent with the MERZ scales. First, the Face Overall scale evaluates the entire face as a whole; therefore satisfaction/dissatisfaction about specific areas of their face may not be reflected in their Face Overall scores.⁴ Taking the example of the MERZ scale with the lowest correlation, Lower Lip Fullness—At Rest, raters who believe they have thin lips do not necessarily have the same degree of dissatisfaction about their overall facial appearance. Notably, the Lines Overall scale was found to have convergent validity despite also evaluating the entire face. However, in the Face Overall scale, the questions ask the rater to evaluate holistic aspects of their appearance, such as symmetry, balance, freshness, etc.⁴ In contrast, the Lines Overall scale asks specifically about static and dynamic lines on the face.⁴ These questions therefore more closely reflect the various lines (forehead, glabellar, nasolabial folds, etc) depicted in the MERZ scales.⁹

The FACE-Q Cheeks scale was also not convergent with its respective MERZ Upper/Lower Cheek Fullness—At Rest scales. This may be because the FACE-Q scale asks the rater to assess multiple domains of their cheek appearance (symmetry, smoothness, attractiveness, contour, fullness).⁴ In the MERZ scales, symmetry and smoothness cannot be graded because they remain constant for all photographs.⁹ As such, it could be argued that the Cheeks scale and its respective MERZ scales measure dissimilar constructs, explaining its poor convergent validity.

Overall, this study provides evidence of convergent validity for the majority of FACE-Q Aesthetics scales used in this analysis and may be relevant in an eventual systematic review of the measurement properties of the FACE-Q Aesthetics module, which in turn can be used to make evidence-based recommendations for clinical and research use.¹⁴ The study findings are in line with other studies investigating the measurement properties of the FACE-Q Aesthetics scales, which show favourable results.^1,6,7 Additionally, this study demonstrates mean FACE-Q scores for its corresponding MERZ Aesthetics photonumeric scale. Ultimately, these results may be used to aid in clinicians' and researchers' interpretation of how these scales relate to one another. In a clinical setting, clinicians may use the study findings to provide a visual estimation of projected improvement on the MERZ Aesthetics photonumeric scale, based on the expected improvement in FACE-Q scores. However, although the 2 instruments show evidence of convergent validity, the MERZ Aesthetics photonumeric scales should not be used to replace the FACE-Q scale and vice versa in clinical research. These tools measure similar, but not identical constructs and each adds important elements to the overall assessment of an aesthetic outcome.

This study has several limitations. First, given that the selected FACE-Q Aesthetics scales include items which evaluate participant perceptions of their appearance (eg, how bothered they are by the area being evaluated) and the MERZ scales use standardized photographs for participants to match their appearance (eg, degree of jawline sagging), it was hypothesized that these scales likely measure related, but dissimilar, constructs for the purpose of hypothesis testing. As such, the authors did not generate any hypotheses about strong correlations that would better support the convergent validity of the FACE-Q scales. Second, the MERZ scales had previously been validated for use by clinicians and expert raters only.^10-12 In our study, MERZ scales were completed directly by the participants based on our assessment of the face validity of the MERZ scales for direct participant use. Especially in an online survey format, differences in picture quality between participants may also have affected the ratings. Third, the sample only included English-speaking participants who were mostly from two countries (United States, United Kingdom), and mostly Caucasian, limiting the generalizability of study findings. Fourth, participants self-selected to enroll in the study and were provided monetary compensation, potentially leading to volunteer bias, and therefore the sample may not be fully representative of the study population. Finally, the data provided by participants were self-reported and therefore could not be independently verified—which may limit the accuracy of the data.

CONCLUSIONS

This study demonstrates evidence of adequate convergent validity of 9 FACE-Q Aesthetics scales (Lines Overall, Lower Face and Jaw, Appraisal of Lines—Forehead/Between Eyebrows/Crow's Feet/Nasolabial Folds/Marionette/Lips, and Lips). Further research should be conducted comparing the FACE-Q Aesthetics scales to other PROMs measuring similar constructs to strengthen the available evidence supporting the construct validity of these scales. Additionally, this study provides estimates of FACE-Q scale scores based on their corresponding MERZ scales that can be used to aid in the interpretation of scores for both instruments.

Supplemental Material

This article contains supplemental material located online at www.aestheticsurgeryjournal.com.

Disclosures

Memorial Sloan Kettering Cancer Center (New York, NY) holds the copyright of the FACE-Q and its translations. Dr Cano, Dr Pusic, and Dr Klassen are co-developers of FACE-Q Aesthetics and receive a share of any license revenue associated with its use by “for-profit” organizations only.

Funding

The authors received no financial support for the research, authorship, and publication of this article.

REFERENCES

Author notes