Process Evaluation of the NYU Caregiver Intervention-Adult Child

Abstract

Purpose of the Study

A noted limitation of dementia caregiver intervention research is a lack of focus on the mechanisms of successful programs. The purpose of this study was to conduct a process evaluation of the New York University Caregiver Intervention-Adult Child (NYUCI-AC) to describe its delivery and determine which of its components were associated with key outcomes (caregiver stress and well-being; care recipient residential care admission).

Design and Methods

Adult child caregivers randomly assigned to receive the NYUCI-AC intervention (n = 54) were included. Detailed data on the frequency and duration of each intervention component received, 4-month review checklists, and regular caregiver assessments were collected. Quantitative (descriptive, logistic regression, Cox proportional hazards models, growth curve models) and qualitative thematic analyses were performed.

Results

Adult children receiving the NYUCI-AC completed a mean of 5.19 individual and family counseling sessions; it took on average a little over 11 months to do so. All NYUCI-AC counseling components were generally well-received and improved caregivers’ management of care-related stress. The individual counseling sessions’ clinical benefits in reducing primary subjective stress were most apparent in the first year of the intervention. Caregivers who experienced negative outcomes over time used more family sessions.

Implications

A key mechanism of benefit for adult child caregivers in the NYUCI-AC was the frequent use of individual counseling sessions. The qualitative and quantitative results emphasize the value of the NYUCI-AC’s flexibility in allowing adult child caregivers to choose the timing and use of specific intervention components.

The extensive financial and human costs of providing family care to someone with Alzheimer’s disease or a related disorder (ADRD) are well documented. Many clinical approaches now exist to help alleviate the stress of family caregiving and improve the well-being of both family caregivers and persons with dementia (Gaugler, Jutkowitz, Shippee, & Brasure, 2017). A noted limitation of existing dementia caregiver intervention research is a lack of focus on the process and mechanisms of successful programs (Burgio & Gaugler, 2016; Gitlin & Hodgson, 2015; Gitlin, Marx, Stanley, & Hodgson, 2015; Wethington & Burgio, 2015). Although randomized controlled trials remain the “gold standard” when establishing the effectiveness of a given intervention, such designs may not assist practitioners in determining how to best to replicate these interventions in the context of their organizations, or whether and for whom they will be able to achieve the benefits of the original trial (Moore et al., 2015; Oakley et al., 2006). The main objective of the current project was to conduct a process evaluation of the New York University Caregiver Intervention-Adult Child (NYUCI-AC), which was the treatment in a randomized controlled, multi-component psychosocial intervention for adult child caregivers of persons with ADRD (Gaugler, Reese, & Mittleman, 2013, 2015, 2016) to help inform potential implementation efforts in practice contexts.

The NYU Caregiver Intervention and the NYU Caregiver-Adult Child Intervention

The original study of the NYU Caregiver Intervention (NYUCI) was among the first to evaluate the effects of comprehensive counseling for ADRD caregivers. This randomized controlled evaluation included spousal or partner caregivers of people with Alzheimer’s disease in the New York City region. The NYUCI began in 1987 and collected up to 18 years of data on caregiver and care recipient outcomes. The NYUCI consists of three components: individual and family counseling, support groups, and ad hoc counseling. Six individual and family sessions are provided in the first 4 months of participation: one individual session with the spousal caregiver; four sessions that include the spousal caregiver and at least one other family member; and a second individual session with the spousal caregiver. The major aims of the counseling sessions include improving social support from family members and friends, education, promotion of communication, problem solving, strategies for responding to care recipient behavior, and concrete planning. Following the initial 4-month counseling period, spousal caregivers in the original study were referred to weekly support groups. In addition, NYUCI counselors are available to caregivers and families over the telephone or in-person (i.e., “ad hoc counseling”) to help them deal with crises and the changing nature of the person with dementia’s symptoms. NYUCI support continues through key care recipient transitions (e.g., institutionalization, bereavement). Control caregivers in the original evaluations of the NYUCI received routine services provided at the NYU Alzheimer’s Disease Center, including ad hoc counseling. The nearly two and a half decades of published results demonstrate that the NYUCI reduces the primary caregiver’s stress and depression, improves and enhances social support, improves caregiver health, and delays nursing home placement for persons with ADRD (Mittelman & Bartels, 2014). The NYUCI is a recognized, evidence-based intervention that is now in translation in various community and clinical settings in the United States and beyond (Mittelman & Bartels, 2014).

A major goal of the NYUCI-AC study was to examine whether the substantial benefits of the NYUCI extended to adult child caregivers. The NYUCI-AC included the same three clinical components as the original NYUCI. The NYUCI-AC has demonstrated similar benefits related to residential care placement (admission or entry to nursing homes, assisted living/assisted living memory care, or group homes) and caregiver stress and well-being for up to a 3-year period (Gaugler et al., 2013, 2015, 2016), although the strength and extent of effects were stronger in the original NYUCI (e.g., the original 3-year evaluation of the NYUCI delayed institutionalization by 329 days, whereas the NYUCI-AC was estimated to delay residential care placement by 228 days; the NYUCI had strong, significant long-term effects in reducing depressive symptoms overall, whereas the NYUCI-AC exerted significant effects over time on a subscale of depressive symptoms).

Process Evaluation Framework and Focus

The 2015 Medical Research Council’s (MRC) framework for complex interventions was utilized as a post hoc guide for this process evaluation (Moore et al., 2015). The framework begins with a description of the intervention and its causal assumptions; this then influences the implementation of the intervention. Implementation includes implementation process (how delivery of the intervention is achieved) and a description of what is delivered (fidelity, frequency and duration of intervention components delivered, adaptations to the intervention). Implementation then influences mechanisms of impact, or, participant responses to the intervention, as well as unexpected pathways and consequences and/or possible mediators of intervention effects. Throughout the intervention process, implementation and mechanisms are influenced by the overall intervention context (i.e., how the given intervention context influences implementation and mechanisms of impact, and vice versa; Moore et al., 2015).

The most effective dementia caregiver interventions, based on results of multi-site, randomized controlled evaluations, are complex multi-component programs (Gitlin & Hodgson, 2015). These state-of-the-art interventions have been adopted in community translations and implementations; however, a missing “link” in the evolution from research to practice is a lack of understanding of how and why certain interventions are effective, and for who (Wethington & Burgio, 2015). It is often unclear whether certain components of multicomponent interventions are more beneficial than others (for an exception, see Czaja, Schulz, Lee, Belle, & REACH Investigators, 2003). To help address this gap, the current project analyzed NYUCI-AC process data to: (a) elucidate the causal mechanisms underlying the effects of the NYUCI-AC; (b) describe the process of delivery of the NYUCI-AC; (c) examine adult child caregivers’ response to NYUCI-AC components; and (d) determine whether the use of each intervention component was associated with demonstrated NYUCI-AC benefits, including residential care placement of the person with ADRD, primary subjective stressors as reported by adult child caregivers, and global outcomes such as depressive symptoms and quality of life of adult child caregivers.

Process Evaluation Design

Causal Mechanisms

When the stress process model developed by Pearlin and associates emerged in 1990 (Pearlin, Mullan, Semple, & Skaff, 1990), it was adopted as the conceptual framework underlying the effectiveness of the NYUCI and later guiding the evaluation of the NYUCI-AC. The stress process is guided by the mechanism of proliferation, or the spread of stress from primary objective domains (e.g., care demands) to primary subjective stressors (perceptions of being overwhelmed with care tasks-role overload, or the sense that care responsibilities are an involuntary obligation and thus entrap the caregiver-role captivity). As primary, care-specific stress accumulates and intensifies, the stress process model postulates that this stress will spread and negatively influence other life domains and role responsibilities. The proliferation of stress eventually influences global outcomes such as caregiver depression and care recipient residential care placement. Various resources, both external (social support) and internal (coping, appraisal) may operate to stem the proliferation of stress. Within the stress process framework, the NYUCI improves resources, primarily family caregivers’ social support, to help alleviate stress and negative health outcomes associated with dementia care provision. In this regard, the NYUCI is conceptually grounded in the stress process model but clinically driven by elements of family systems theory. A main tenet of the family systems perspective is that spousal or adult child caregivers have resources within themselves, their families, and their communities that can be harnessed to reduce or solve problems related to care provision of a relative suffering from ADRD (Schulz, Gallagher-Thompson, Haley, & Czaja, 2000).

Design

A prospective, single blind, randomized controlled study design was utilized to evaluate the NYUC-AC. Participants included 107 adult child caregivers (n = 54 in the treatment condition) of persons with ADRD who lived in the 7-county Minneapolis/St. Paul region or outlying areas. Eligible participants included adult children who self-identified as the primary caregiver of care recipients who had a physician diagnosis of ADRD and lived at home in the community at the time of the initial assessment, who lived with the care recipients or visited them at least weekly, and had not utilized professional counseling for problems related to dementia caregiving in the 12 months prior to enrollment. Participants were enrolled from January 2006 to August 2009. In-person, telephone, or mail assessments were administered to caregivers by a rater blinded to group assignment at baseline, every 4 months in Year 1, and every 6 months thereafter for up to 3 years. More details about the recruitment and enrollment procedure of the NYUCI-AC are available in Gaugler and colleagues (2013, 2015, 2016).

The process evaluation of the NYUCI-AC utilizes a parallel convergent mixed methods design (QUAN + qual; see Creswell & Plano-Clark, 2010). The point of interface between the quantitative and qualitative strands of data occurred in the interpretation of findings, where the qualitative data were utilized to expand upon and explain the empirical results.

Data Collection

Process/Implementation Data

Records of process indicators, or frequency and types of counseling contact (e.g., individual/family counseling, support group attendance, ad hoc contacts/provision of ongoing telephone or in-person counseling on an as-needed basis, e-mail exchanges), were collected and maintained by the two NYUCI-AC study counselors. At each follow-up, adult child caregivers were asked to evaluate each component of the intervention by completing up to three 21-item review checklists and providing open-ended feedback (i.e., “Please add any other ways that the individual or family counseling/support groups/ad hoc counseling has been helpful to you, or any other comments that you have”). The item-review checklists focused on how well caregivers perceived that they were helped by the NYUCI-AC to navigate family conflict, learn more about ADRD, obtain help, and manage symptoms of ADRD (see below). Items were scored on a 5-point scale (anchored at 1 = not at all helpful and 5 = very helpful).

Context of Care

Background and sociodemographic information about the caregiver and care recipient were collected in each baseline assessment survey (Table 1); caregivers indicated their and their parent’s race, age, years of education, marital status, number of children, and income. Caregiver employment status and care recipient Medicaid status and living arrangement were also provided. Care recipient disposition (whether admitted to residential care or died and dates of these events) was collected during each follow-up assessment.

Primary Objective Stressors

The severity of dementia of the person with ADRD was categorized with the Global Deterioration Scale (Reisberg, Ferris, de Leon, & Crook, 1982). The frequency of care recipient behavior problems was measured with the Revised Memory and Behavior Problems Checklist (RMBPC; α = .78; Teri et al., 1992). The number of specific care tasks provided to the care recipient during a typical week was summed (Rice et al., 1993).

Primary Subjective Stressors

A four-item scale assessed the involuntary aspects of the caregiving role (i.e., role captivity, α = .76; Pearlin et al., 1990) and a three-item scale measured caregivers’ feelings of emotional and physical fatigue (i.e., role overload, α = .79; Pearlin et al., 1990). Caregivers’ overall reaction to care recipient behavior (α = .85) was assessed with the RMPBC. The 14-item Perceived Stress Scale (α = .90; Cohen et al., 1983) was used to measure chronic perceived stress.

Resources

The size and composition (i.e., number of friends/relatives and whether they were considered close) of caregivers’ social networks were measured with The Stokes Social Network List (Stokes, 1983). Three indicators of social support were also included: how satisfied the adult child caregiver was with her/his overall social network, the emotional support provided by her/his social network, and the assistance offered by her/his social network. Service utilization was assessed by asking caregivers to identify the number of community-based and psychosocial services they were currently using.

Global Outcomes

The Geriatric Depression Scale (GDS, α = .97; Yesavage, Rink, Rose, & Aday, 1983) was used to assess depressive symptoms of caregivers. Adult child caregivers were also asked to rate their overall quality of life at the present time on a 100-point Cantril ladder (Cantril, 1965).

Analysis

Descriptive quantitative analyses were initially utilized to examine process indicators of NYUCI-AC implementation. A series of correlations were also performed to examine whether baseline context of care indicators and primary objective stressors were associated with the implementation process of the NYUCI-AC (Moore et al., 2015).

Mechanisms of NYUCI-AC impact were ascertained by first summarizing the empirical results of NYUCI-AC review checklists and then conducting thematic analyses of the qualitative data from these checklists. The first author developed coding categories together with descriptors and a shared coding scheme that reflected the primary themes of the qualitative data available in the checklists (Morse & Field, 1995; Morse & Niehaus, 2009). Through repetition of this procedure and the use of nVivo 10, a consensus perspective with the second author on appropriate themes was developed to identify the benefits and challenges of NYUCI-AC receipt. In instances of disagreement, the first two authors discussed any divergences in interpretation until a consensus was reached.

Level 2 individual growth curve models were then performed to determine whether individual NYUCI-AC process indicators were empirically associated with statistically significant changes in primary subjective stressors and/or global outcomes. The multi-wave data collection approach allowed for intensive longitudinal analyses of change in various caregiving domains when compared to traditional 2-wave (i.e., “pretest vs. posttest”) or lagged designs. The utilization of only two waves of data often fails to capture the nature of long-term change. The individual growth curve model postulates that development, or change, is a function of an individual growth curve plus random error. A strength of the individual growth curve approach is its flexibility in handling timing of assessments in longitudinal data. The multiple data points available in the NYUCI-AC allowed for parameterization of individual growth trajectories, including initial status and linear slope of change. Additional detail on growth curve models conducted for the NYUCI-AC is available in Gaugler et al., 2015.

As there was a considerable drop-off in observed data points following the 24-month follow-up interval (adult child caregivers had 6.21 data collection points/completed assessments available, on average: SD = 1.81), the growth curve models considered change in primary subjective stressors and global outcomes at 1-year, 18-month and 24-month intervals, controlling for whether the care recipient died or was placed in a residential care setting during the evaluation.

Bivariate logistic regression and Cox proportional hazards models were also conducted to determine whether pre-placement process indicators were predictive of subsequent residential care admission and time to placement for persons with ADRD in the NYUCI-AC treatment condition. The Cox proportional hazard survival model permits an event-history analysis in which the dependent variable is a combination of time and whether the event (time to residential care placement) occurs or not. Each logistic regression model included a pre-placement process indicator; the binary outcome variable in the logistic regression models indicated whether residential care placement of any kind occurred for persons with ADRD during the 3-year follow-up study period. Each Cox proportional hazard survival model included a pre-placement process indicator to determine whether more frequent use of a given NYUCI-AC intervention component was associated with delays in time to residential care placement.

Process Evaluation Findings

Implementation: Frequency, Type, and Duration of NYUCI-AC Counseling

Participants in the NYUCI-AC completed their individual and family counseling sessions an average of 11.10 months following baseline (SD = 8.95). Adult child caregivers who received the NYUCI-AC participated in an average of 5.19 (SD = 1.54) counseling sessions, of which 3.96 were individual sessions (SD = 1.86) and 1.22 were family sessions (SD = 1.56). On average, each individual counseling session lasted 99.41 min (SD = 27.17); family sessions lasted a mean of 107.5 min (SD = 32.14). Twenty-five participants (46.3%) did not utilize any family sessions. Among those who did use family counseling, approximately three individuals (including the caregiver) attended family sessions (M = 2.91; SD = 1.58). Adult child caregivers in the NYUCI-AC received 145 ad hoc sessions (M = 2.65 ad hoc sessions, SD = 3.22; M = 59.21 min each, SD = 36.25). Of the 1,678 e-mails exchanged between study counselors and adult child caregivers (M = 31.07; SD = 24.17), approximately one-quarter involved counseling-related issues with the remainder involving administrative issues. Univariate analyses of the process indicators demonstrated high skewness; therefore quartiles, tertiles, or dichotomous cut points were used to result in a more equitable distribution in these key variables (Table 2).

Correlational analyses found several significant associations between context of care, primary objective stressors, and NYUCI-AC process indicators (shown in the Supplementary Material). Adult child caregivers who either cared for relatives with greater cognitive (Global Deterioration Scale r = .30, p < .05) and functional needs (number of care tasks provided r = .32, p < .01) generally participated in more individual and family counseling sessions, whereas those who lacked social support tended to participate in more individual counseling sessions (e.g., general satisfaction with social network r = .41, p < .001; higher scores indicated less satisfaction). In addition, adult child caregivers who were women (r = .26, p < .05), had more children (r = .42, p < .01), and had more close relatives (r = .40, p < .01) appeared more likely to participate in family sessions or NYUCI-AC support groups.

Mechanisms of Impact

Caregiver Response to the NYUCI-AC

The empirical data presented here is derived from the checklists completed at the first 4-month follow-up (which is when the individual and family counseling sessions were to be completed per the original NYUCI protocol). As shown in Table 3, adult child caregivers indicated that the individual/family counseling (M = 4.55), the support group (M = 4.36), and ad hoc counseling (M = 4.00) components were all “helpful” to “very helpful” on the final item/overall rating of helpfulness.

There was, however, heterogeneity in how adult child caregivers appraised the specific aspects of each of the counseling components. For example, all NYUCI-AC components were rated as helpful in allowing adult child caregivers to discuss problems related to caring for someone with ADRD. Support groups and ad hoc counseling were more helpful in providing information on ADRD and how to manage parents’ behavioral issues. Both individual and family counseling sessions and support groups were helpful when assisting caregivers to identify where to go for additional assistance. Individual and family counseling sessions appeared to help adult child caregivers cope with emotional problems, whereas support groups were helpful in facilitating participants’ management of functional issues and changes that occurred during a relative’s dementia. The NYUCI-AC intervention components did not help in arranging for legal or financial assistance, but this was not within the clinical scope or expertise of the NYUCI-AC counselors. Adult child caregivers were also neutral about whether the NYUCI-AC counseling sessions helped them identify someone else to provide care.

A total of 56 open-ended comments were provided by 25 adult child caregivers (M = 2.24 comments; SD = 1.51). Analysis of the open-ended feedback provided on all review checklists identified four themes: How Counseling was Helpful; What the Counselor Did that Helped, What was Learned/Achieved Due to Counseling; and Underutilization. The How Counseling Was Helpful theme provided insights into the mechanisms of benefit associated with participating in the NYUCI-AC intervention:

“What a great program. I would have thrown in the towel had it not been for my counselor. The agitated behavior at one point was a rage that was pointed directly at me. The counselors own shared experience and guidance got me through that horrible experience. The access to the counselor is better than what I could have hoped for and I am never made to feel that it is an imposition. The follow-up call or email to me after some serious situations was also very beneficial.” (referring to ad hoc counseling)

NYUCI-AC participants were also pleasantly surprised that the support groups helped them realize that their own issues related to caregiving were more manageable than perhaps originally thought:

“I was surprised by how supportive the group was. We spent so much time talking about another family member that needs help too but it made my issues with mom feel easier as a result.” (referring to support group)

Caregivers pointed to several counselor activities and skills that made the NYUCI-AC useful, such as counselors’ ability to encourage them to discuss challenging issues related to behavioral management or coping with stressful aspects of care. Counselors appeared to be understanding of issues related to family dynamics as well:

“<Name of counselor> has been available to us and patient with the challenges we’ve had getting the four of us sisters together to discuss issues related to caring for our mother.” (referring to ad hoc counseling)

Adult child caregivers noted that having a counselor who has had caregiving experiences of his or her own was invaluable:

“Knowing the counselor had been through a similar situation with caregiving and lost a parent helped me tremendously because then much of his words resonated with what I was going through.” (referring to ad hoc counseling)

When commenting on individual and family counseling sessions, several adult child caregivers suggested that counseling also offered the “opportunity to validate current situation(s) and choices.”

Caregivers also reflected upon positive outcomes, or What Was Learned. Several caregivers identified ways in which the individual and family counseling sessions aided them in asking for help (“She has offered ideas such as asking for specific help, which I have been doing”), how NYUCI-AC support groups resulted in improved feelings of self-efficacy (“I always leave with an uplifted attitude that ‘I can do this’”), and that the ad hoc counseling sessions assisted them in acquiring skills to handle care recipient behavior problems:

“I have learned that I’m not responsible for how my mom behaves and don’t need to be controlled by her guilt. I now know how to handle her moods and help make choices in my life to help cope with her caregiving. I do what I can to make sure she is safe and fed. I now can leave and feel good about what I have done for my mom. Guilt cannot control my caregiving for her.” (referring to ad hoc counseling)

Underutilization and its causes were also mentioned. Several adult child caregivers noted that they under-utilized the counseling and support provided by the intervention (“I think I may be underutilizing the counseling program;” referring to individual and family counseling), often due to the time constraints of dementia care:

“I’m sorry that I have not attended group. By that time I’m very tired from the day. My mind can no longer function. I’m the primary caretaker. Things have been changing fast for me. We decided it was time to check out (sic) nursing home.” (referring to support group)

Empirical Associations Between Process Indicators and Key Outcomes

The Supplementary Material presents the significant results of level 2 growth models that examined the effects of individual process indicators on primary subjective stressors and global outcomes over a 12-, 18-, and 24-month period. The most consistent effects occurred during the first year of the treatment period, where participation in more individual counseling sessions was associated with statistically significant decreases in role captivity (B = 0.67, SE = 0.24, p < .01; higher scores represent less role captivity) and overall perceived stress per the PSS (B = −1.83, SE = 0.75, p < .05). These findings appeared similar, although less pervasive, in the 24-month period (role captivity B = 0.73, SE = 0.34, p < .05). In contrast, adult child caregivers who were more likely to indicate statistically significant increases in primary subjective stress and depressive symptoms over 12-months (e.g., role captivity B = −0.87, SE = 0.37, p < .05) participated in more family sessions. No process indicators were significantly associated with whether placement occurred or time to placement in the bivariate logistic and Cox regression models, respectively.

Implications of Process Findings

The MRC process evaluation framework emphasizes that the context of an intervention can influence implementation (Moore et al., 2015), and this was apparent in the NYUCI-AC. The original NYUCI, which was designed for and delivered to spouses of persons with ADRD, is based on the completion of 6 individual and family counseling sessions within a 4-month period. During the evaluation of the NYUCI-AC, the research team found that adult children generally: (a) did not want to participate in as many as six sessions in 4 months; (b) viewed counseling sessions as a resource they could use per their needs; (c) wished to join support groups within the first 4 months of participation; and (d) were reluctant to include other family members or simply did not have any available to participate in the counseling sessions. For these reasons, the ultimate structure of the NYUCI-AC was considerably different from the parent NYUCI.

Previously published results (Gaugler et al., 2013, 2015, 2016) and the outcomes of the process evaluation suggested that the NYUCI-AC was beneficial and well received. All components of the intervention were generally well regarded. A strength of this process evaluation was the integration of qualitative and quantitative data via the parallel convergent mixed methods design, which allowed for more in-depth descriptions of why the various NYUCI-AC counseling components were perceived as beneficial by adult child caregivers. For example, counseling sessions as well as support groups were viewed as helpful, particularly when discussing problems related to care provision or coping with the emotional distress that is common during dementia care. Ad hoc counseling was also viewed as helpful when grappling with the behavioral changes of ADRD. The NYUCI-AC counselors encouraged caregivers to seek instrumental support, but this aspect was viewed as less useful by participants. Caregivers appreciated the time during counseling sessions to discuss issues weighing on them, as well as the various strategies they had acquired to cope, seek information, or manage their dementia care situations. They valued the counselors’ patience and empathy.

Analysis of process indicators and longitudinal outcomes suggested that much of the NYUCI-AC’s clinical benefit in reducing primary subjective stress occurred by the end of the first year of participation. Interestingly, role captivity decreased over 1 year among those who utilized individual counseling sessions more frequently. This is noteworthy, as role captivity is among the most consistent predictors of institutionalization among persons with dementia (Gaugler, Yu, Krichbaum, & Wyman, 2009) and often has high empirical associations with more general measures of caregiver well-being such as depression. Anecdotal information from the blinded rater suggested that caregivers were most likely to exhibit an emotional response when asked the items on the role captivity measure (e.g., “I wish I could just run away”). It is possible the initial decrease in role captivity demonstrated in the NYUCI-AC helped to drive the positive overall effects of the NYUCI-AC on residential care placement and caregiver well-being (Gaugler et al., 2013, 2015).

All participants were offered six individual and family sessions. Adult child caregivers who chose to utilize family sessions appeared to experience increases in stress over time. Another study that evaluated an intervention that only included 4 sessions of family counseling without any other components of the NYUCI found no evidence that family sessions reduced depression or anxiety for dementia caregivers (Joling et al., 2012). It is possible that adult child caregivers in the NYUCI-AC who chose to have family sessions did so because they viewed them as a source of help in instances when disrupted family dynamics exacerbated stress. Decisions to use family sessions may thus have been crisis-driven rather than a resource to prevent caregiver distress (see the case summary in the Supplementary Material for an example). Selection bias may also explain the relationship between use of family sessions and stress: those who experienced more stress may have been less likely or motivated to participate in family sessions, but rather elected to use individual sessions instead.

Frequency of use of the NYUCI-AC components (examined separately) was not associated with likelihood of or time to placement. Given that the NYUCI-AC was linked to prevention of or delays in placement in the original outcome evaluation (Gaugler et al., 2013), it is reasonable to assume that examination of the effects and use of each of the counseling components separately did not capture the full effect of the multi-component intervention. The interactive effects of the NYUCI-AC components may be particularly germane to an outcome such as residential care placement that can generally be affected only by an intensive, sustained intervention. In addition, context of care, unmeasured stressors and resource variables may have directly influenced or interacted with/moderated NYUCI-AC use to influence residential care entry. Thus, although our process evaluation can provide insight into the program elements and mechanisms that were linked to specific benefits, a more complex analysis of sufficient statistical power would be required to evaluate the interactions of the multiple components and their influence on residential care placement or other outcomes. This is relevant not only to the NYUCI-AC, but to other process evaluations of multi-component dementia caregiving interventions.

There are important limitations to note. The sample is small, largely Caucasian and well-educated, and almost entirely female which hinders its generalizability. Moreover, placement rates in the NYUCI-AC treatment condition were low (Gaugler et al., 2013), perhaps further influencing power and interpretation of residential care placement as an outcome in this process evaluation. Due to the rolling enrollment procedure, some adult child caregivers were followed from up to 2 years to over 3 years; although the analyses utilized here can adapt to various lengths of follow-up, the reduced sample during later intervals may have further influenced the findings. The qualitative data collected were derived from the NYUCA-AC review checklists; a more robust mixed methods study design that incorporated semi-structured interviews during or following the randomized controlled evaluation (i.e., an embedded mixed methods experimental design; see Creswell & Plano-Clark, 2010) might have yielded greater information regarding the NYUCI-AC’s mechanisms of benefit by linking quantitative analyses of outcome (e.g., the growth curve models) with qualitative findings. The empirical analyses were exploratory and included multiple tests.

This process evaluation illustrates the challenges of balancing internal validity and fidelity to the original study design with implementing dementia caregiving interventions that more fully engage participants. This point has relevance for practitioners. In the case of the NYUCI-AC, adult child participants were essentially allowed to choose when they were to receive counseling sessions; the individual and family counseling sessions were administered in a somewhat ad hoc manner in that they were provided on demand. The research team incorporated this flexibility because it became clear early in the study that adult children were participating with a decidedly consumer-oriented mindset, and would not passively adhere to an a priori counseling structure. On the one hand, this raises internal validity issues in the classic sense as the NYUCI-AC intervention did not conform to the preset counseling session structure. Alternatively, it could be argued that the NYUCI-AC’s positive results demonstrate the importance of incorporating caregiver preferences when adapting dementia caregiver interventions into community or practice contexts. Adult child caregiver engagement via participant selection of the timing and dosage of the intervention may have enhanced the overall efficacy of the NYUCI-AC; similarly, when translating or implementing complex, multi-component protocols for dementia caregivers, the incorporation of caregiver perspectives to effectively tailor the program for diverse cultural, familial, or socioeconomic contexts may be similarly warranted (Wethington & Burgio, 2015).

The results underscore the value of process evaluations for ADRD caregiver and other complex gerontological interventions (Burgio & Gaugler 2016; Moore et al., 2015; Oakley et al., 2006). Both “decomposing” interventions and examining how the components interact will not only advance the scientific state-of-the-art but also better inform practice efforts by illuminating clinical mechanisms of benefit (Czaja et al., 2003). Such advances would also allow providers to identify which program elements may be most appropriate for translation and implementation, and for whom. This can only occur, however, when the true “processes” of intervention approaches are described as opposed to cursory descriptions of individual components (Burgio & Gaugler, 2016). The general lack of robust process evaluation serves as a barrier to translation (Wethington & Burgio, 2015). Process evaluations that identify how and under what circumstances a given dementia caregiver intervention is effective can help guide provider agencies when considering more efficient approaches to implementing evidence-based programs, such as obtaining an understanding of which clients are most likely to benefit via improved targeting of services (Gitlin et al., 2015). Using process evaluations to identify program components that are effective and amenable to tailoring to local or cultural contexts would also contribute to advancing the practice of dementia caregiver interventions.

In terms of the specific influence of this process evaluation on subsequent implementation of the NYUCI-AC, the findings imply that adult children may require greater flexibility in how this or similar multi-component interventions are delivered. Adult child caregivers were insistent on “saving” their counseling sessions beyond the initial 4 months of participation. Moreover, most adult child caregivers in this study used individual sessions rather than family sessions. For ADRD caregivers of varying kin relationship, some flexibility in how counseling sessions are provided (de-emphasizing the consolidation of sessions in a brief period) and who attends such sessions (allowing adult child caregivers the option to not include family members if they prefer) may result in greater uptake of an intervention and more positive outcomes during the caregiving trajectory. Such results clearly have implications for translation and implementation of the NYUCI for adult children, as they emphasize caregiver choice when planning and delivering the individual, family, and ad hoc counseling sessions.

Supplementary Material

Supplementary data are available at The Gerontologist online.

Funding

This research was supported by grant R01 AG01022066. This project was also supported by Grant Number 1UL1RR033183 from the National Center for Research Resources (NCRR) and by Grant Number 8 UL1 TR000114-02 from the National Center for Advancing Translational Sciences (NCATS) of the National Institutes of Health (NIH) to the University of Minnesota Clinical and Translational Science Institute (CTSI).

Acknowledgments

The authors would like to thank William Haley, Steve Zarit, Kristie Kellis, Cynthia Epstein, Mary Boldischar, Vanessa Berglund, Katie Wocken, Sarah Wooley, Mary Dang, Ronit Notkin, Olanta Barton, and June Aaronson for their clinical and scientific expertise, data management, and/or data entry on this project. The authors would especially like to acknowledge the families who contributed their valuable time to this study.

Conflict of Interest:

The author declare no conflict of interest.

References