Efficacy and sustainability of dialectical behaviour therapy for inpatient adolescents: a follow-up study

Present study

This current study is a follow-up for Tebbett-Mock et al’s study and examined the next cohort of patients who received DBT over the same seasonal span of 8 months to explore if findings were maintained. For this study, we hypothesised that adolescents on an acute care psychiatric inpatient unit who received DBT in our follow-up group (ie, DBT Group 2) would not have statistical difference (ie, greater or fewer) of the following compared with the first group of patients who received DBT (ie, DBT Group 1) on the same inpatient unit the year prior: (1) CO hours for suicidal ideation, self-injury and aggression while hospitalised; (2) incidents of SA, self-injury and aggressive behaviour while hospitalised; (3) restraints while hospitalised; (4) seclusions while hospitalised; (5) days hospitalised; (6) times readmitted to the unit within 30 days of discharge.

Participants

Adolescents in the current study were aged 12 to 17 and hospitalised on a 22-bed coeducational, acute care inpatient unit within a private psychiatric hospital in the Northeast. Patients in DBT Group 1 were those who were hospitalised during an 8-month period of time following implementation of DBT on the unit. Historical controls (ie, TAU group) included all patients who were hospitalised on the same unit during the exact same seasonal span of 8 months the year prior to DBT implementation and received TAU. DBT Group 2 included all patients who were hospitalised on the same unit during the exact same seasonal span of 8 months the year after DBT Group 1. Refer to Tebbett-Mock et al for further detailed information on participants.6

Treatment

The inpatient unit comprised a multidisciplinary treatment team including two attending psychiatrists and psychiatry trainees (ie, residents and child and adolescent fellows), two psychologists and psychology trainees (ie, externs, interns, fellows) who were supervised by licensed psychologists, two social workers, two psychiatric rehabilitation specialists, three nurses per shift and three to four mental health workers per shift.

Historical control patients on the unit received TAU consisting of a token economy system, 3 to 4 cognitive–behavioural therapy (CBT) skill groups per week, 10 activity groups per week and intensive psychotherapy including approximately 3 individual sessions per week and 1 to 2 family/collateral therapy sessions per week.

DBT Group 1 and DBT Group 2 received DBT treatment that included all functions and components of Linehan’s original model1 as well as adaptations for adolescents made by Miller et al.2 16 For both DBT groups, patients on the unit received DBT milieu treatment including DBT coaching and a token economy including an egregious behaviour protocol, 9 DBT skill groups per week, daily therapeutic and leisure groups, and intensive psychotherapy including approximately 3 individual sessions per week and 1 to 2 family/collateral therapy sessions per week. Approximately half of the patients on the unit at any given time were assigned to DBT individual and family therapy provided by psychology staff, and the other half of the patients received either CBT or supportive psychotherapy provided by social work or psychiatry staff. For both groups, a multidisciplinary team met weekly for consultation team. Refer to Tebbett-Mock et al for further detailed information on TAU and DBT treatments.6

Although the DBT milieu, groups and individual and family therapy were comparable between DBT Group 1 and DBT Group 2, there were notable differences between the consultation team and staff trainings for these two groups. For DBT Group 1, a multidisciplinary team participated in the DBT intensive training by Behavioral Tech, as well as a strategic day of planning with a Behavioral Tech trainer to facilitate treatment adaptations for an acute care, adolescent psychiatric inpatient unit. In DBT Group 1, this multidisciplinary team included two attending psychiatrists, two licensed psychologists, two psychiatric rehabilitation specialists, one nurse manager and two nurses. These nine multidisciplinary team members who participated in DBT training by Behavioral Tech met weekly for consultation team. For DBT Group 1, consultation team members provided training, supervision and didactic series to all staff across the three work shifts on the unit to facilitate implementation of and fidelity to DBT. All nursing staff, mental health workers and psychiatry trainees received a 3-hour didactic series including an overview of DBT modes of treatment and dialectical philosophy, skills and milieu coaching. At the beginning of DBT Group 2, there were changes to the consultation team. Both psychiatric rehabilitation specialists left the team, and two new psychiatric rehabilitation therapists joined the team. These team members participated in Behavioral Tech’s foundational training, not the intensive training attended by the original members of the consultation team. Additionally, there was considerable turnover with respect to front-line staff, including nurses and mental health workers. A full cohort of these staff members is approximately 35. During DBT Group 2, 11 front-line staff left the unit and 7 new front-line staff were hired who did not receive the 3-hour didactic series that was provided to front-line staff during DBT Group 1 or any DBT-specific training.

Procedures and variables

Data were extracted from the archived electronic medical records database by the quality management team of the hospital. Data included the following variables: age; gender; diagnosis; number of incidents of SA, self-injurious behaviour, aggression patient to patient, aggression patient to staff while hospitalised; number of CO hours for indications of suicide, self-injurious behaviour and aggression while hospitalised; number of restraints and seclusions while hospitalised; number of days hospitalised and times readmitted to the unit within 30 days of discharge from the unit. Refer to Tebbett-Mock et al for detailed information on procedures and variables.6

Data analysis

The χ² tests and a one-way analysis of variance (ANOVA) test were conducted as preliminary analyses to assess differences between groups (DBT Group 1, DBT Group 2 and TAU) on diagnosis, gender and age. Kruskal-Wallis H tests were used for outcome analyses as it is a rank-based non-parametric test that can be used to determine if there are statistically significant differences between two or more groups (DBT Group 1, DBT Group 2 and TAU) of an independent variable on dependent variables that are continuous, but not normally distributed. The Kruskal-Wallis H test determines if the medians (Mdn) of the groups are different. It is considered the non-parametric alternative to the one-way ANOVA, and an extension of the Mann-Whitney U test conducted in the prior study to allow the comparison of more than two independent groups. The non-parametric effect size Epsilon square (E²) was calculated for each outcome analysis.17–19 Mann-Whitney U tests were used as post-hoc analyses to further examine the differences between groups (DBT Group 1 and DBT Group 2, and TAU and DBT Group 2). Mann-Whitney U tests were used as it is a non-parametric test and compares differences between two independent groups (eg, DBT Group 1 and DBT Group 2) on dependent variables that are continuous, but not normally distributed.

Sample description

A total of 1194 patients were included in our analyses. Patients in DBT Group 1 included 425 adolescents aged 12 to 17.92 years (mean (SD)=15.67 (1.44)), and 66.4% were female (n=282). They had a mean of approximately 8 days hospitalised (mean (SD)=8.36 (8.09)). Patients in DBT Group 2 included 393 adolescents aged 12.04 to 17.96 years (mean (SD)=15.78 (1.43)), and 62.8% were female (n=247). They had a mean of approximately 12 days hospitalised (mean (SD)=11.84 (11.36)). Historical control participants who received TAU included 376 adolescents aged 12 to 17.92 years (mean (SD)=15.59 (1.54)), and 62.8% were female (n=236). They had a mean of approximately 11 days hospitalised (mean (SD)=10.74 (10.57)). See table 1 for race/ethnicity information for all three groups. Twenty-four patients were excluded from the analyses (figure 1), 16 from DBT Group 1, 5 from TAU and 3 from DBT Group 2, due to moderate to severe intellectual disability such that it was not reasonable to consider these patients to benefit from talk psychotherapy (ie, DBT).

Request permission

Adolescents presented with a range of primary diagnoses on discharge, which were collapsed and categorised according to disorder (see table 1). There were no statistical differences between TAU, DBT Group 1 and DBT Group 2 for primary diagnosis or gender based on a χ² analysis, χ²(22)=33.22, p=0.059 and χ²(2)=1.49, p=0.474, respectively, or for age based on a one-way ANOVA (F(2,1191)=1.62, p=0.198).

Differences in clinical outcomes

Table 2 shows the descriptive statistics for the comparison variables for patients who received DBT (Group 1, Group 2) and TAU. Table 3 shows the results of Kruskal-Wallis H tests. Table 4 shows the results of post hoc Mann-Whitney U tests.

A Kruskal-Wallis H test showed that there was a statistically significant difference in CO hours for self-injury between the different treatment groups, χ²(2)=9.80, p=0.007, with a mean rank score of 608.92 for TAU, 591.98 for DBT Group 1 and 592.54 for DBT Group 2, with a small effect size of 0.01. Statistically significant differences were not found between DBT Group 1 and DBT Group 2 for CO hours for self-injury. The number of CO hours for self-injury was significantly fewer for patients who received DBT in Group 2 (Mdn=0) than for patients who received TAU (Mdn=0), U=71 855.5, p=0.017.

A Kruskal-Wallis H test showed that there was a statistically significant difference in incidents of SA between the different treatment groups, χ²(2)=7.64, p=0.022, with a mean rank score of 603.12 for TAU, 592.00 for DBT Group 1 and 598.07 for DBT Group 2, with a small effect size of 0.01. The number of incidents of SA was significantly fewer for patients who received DBT in Group 1 (Mdn=0) than for patients who received DBT in Group 2 (Mdn=0), U=82 662.5, p=0.037. Statistically significant differences were not found between DBT Group 2 and TAU for the number of SAs.

A Kruskal-Wallis H test showed that there was a statistically significant difference in incidents of self-injury between the different treatment groups, χ²(2)=54.87, p<0.001, with a mean rank score of 582.15 for TAU, 563.97 for DBT Group 1 and 648.45 for DBT Group 2, with a small effect size of 0.05. The number of incidents of self-injury was significantly fewer for patients who received DBT in Group 1 (Mdn=0) than for patients who received DBT in Group 2 (Mdn=0), U=71724.5, p<0.001, and significantly fewer for patients who received TAU (Mdn=0) than for patients who received DBT in Group 2 (Mdn=0), U= 65649.0, p<0.001.

A Kruskal-Wallis H test showed that there was a statistically significant difference in restraints between the different treatment groups, χ²(2)=7.07, p=0.029, with a mean rank score of 609.50 for TAU, 587.19 for DBT Group 1 and 597.16 for DBT Group 2, with a small effect size of 0.01. Statistically significant differences were not found between DBT Group 1 and DBT Group 2 for restraints or between DBT Group 2 and TAU.

A Kruskal-Wallis H test showed that there was a statistically significant difference in the number of days hospitalised between the different treatment groups, χ²(2)=33.20, p<0.001, with a mean rank score of 600.65 for TAU, 529.54 for DBT Group 1 and 667.99 for DBT Group 2, with a small effect size of 0.03. Statistically significant differences were not found between DBT Group 1 and DBT Group 2 or between DBT Group 2 and TAU for the number of days hospitalised.

There were no statistically significant differences in CO hours for suicidal ideation or aggression, incidents of aggression patient to patient or patient to staff, seclusions or readmissions between the different treatment groups.

Main findings

Findings from this study were consistent with the findings from the previous study by Tebbett-Mock et al, in which the number of CO hours for self-injury, incidents of SA and self-injury, restraints and days hospitalised remained statistically significant in the overall model, supporting DBT treatment over TAU.6 More specifically, patients in DBT Group 2 were comparable to patients in DBT Group 1 for the number of CO hours for self-injury, restraints and days hospitalised, indicating that results were maintained for these variables. However, patients in DBT Group 2 had a significantly greater number of incidents of SA than patients who received DBT in Group 1, and these patients were comparable to patients in the TAU group. Further, patients in DBT Group 2 had a significantly greater number of incidents of self-injury than patients in DBT Group 1 and TAU.

Implications and limitations

The increase in SA and incidents of self-injury in DBT Group 2 compared with DBT Group 1 was concerning, and there are several possible explanations for this. First, there was a lack of training in DBT for front-line staff. In our previous study, we hypothesised milieu treatment as a primary mode impacting improvements in safety-related variables. Indeed, nursing staff primarily play a vital role in the milieu as they have the most frequent patient contact and are the most available for DBT skills coaching and generalisation. Specifically, during DBT Group 2, 11 front-line staff left the unit all of whom participated in the 3-hour DBT training, and 7 new front-line staff were hired on the unit during the time of DBT Group 2 but did not receive any DBT training. This total number of staff was substantial given that the entire cohort of front-line staff was 35. Indeed, staff retention is consistently reported as a main organisational factor that inhibits implementation and sustainability of DBT across behavioural health settings, such as ours.14 Relatedly, failure to protect time needed to deliver treatment and training coupled with competing staff roles prevents sustainability. This was the case for our consultation team as we were unable to train front-line staff in DBT Group 2 as we did for DBT Group 1 due to increased clinical case loads and a focus on training new members of the consultation team. There was a strong effort paid by onboard front-line staff to their traditional unit responsibilities, leading to no time dedicated to DBT training.

Second, during DBT Group 2, our consultation team experienced notable turnover and related challenges that we hypothesised impacted our ability to effectively treat our patients as we had in DBT Group 1. There were 2 staff additions to the consultation team during DBT Group 2. In DBT Group 1, all consultation team members participated in the intensive training by Behavioral Tech, which included two 5-day training segments separated by a 6-month self-study and trial implementation, practice and homework exercises to target team building and mutual responsibility for learning and implementing DBT, a substantial set of contingency management procedures and content and coaching on how to use DBT strategies to target barriers to full implementation and maintenance of DBT.20 This likely fostered our ability to achieve the goals of a DBT consultation team, namely motivation to deliver effective treatment, enhancing clinical skills and monitoring fidelity to the treatment model, which, in turn, likely positively impacted our patients. In DBT Group 2, our new consultation team members did not complete a rigorous training, as they completed the foundational training which was a 5-day training along with self-study. This training model may not have equally fostered team cohesion and goals that the team seemingly experienced in DBT Group 1. Research on effective methods for training clinicians in evidence-based interventions is lacking in the field. The DBT intensive training was developed by Linehan in response to the demand for DBT training and recognised the need for facilitating DBT implementation in community settings. While foundational training is the training recommended by Behavioral Tech for new members joining existing teams, there are no known studies examining the difference in the two types of training. While the core content may be similar, the increased quantity of training as well as the reinforcement of prior learning and team building inspired by the week 2 of intensive training may be important differences. Thus, it is possible that less intensive training impacted our fidelity to treatment, and thus the greater number of incidents of SA and self-injury happened on the unit. In addition to concerns about the level of training, our consultation team experienced a shift on our focus. Time was spent reviewing homework for new DBT team members and providing orientation for the new team members. It is possible that this detraction from the fidelity of our consultation team hampered consulting providers in ensuring adherence to treatment.

Third, previous findings suggest that staff skills in implementing EBTs should regress to baseline in as little as 3 months following training when not adequately reinforced within the agency setting.11 13 The sustainability of EBTs, such as DBT, relies on continued resources such as ongoing training for all milieu staff as well as training and consultation for the entire team.11 Our team’s experience was that, while efforts were successful in securing time and funding for initial training, programme development and DBT implementation, it was much more difficult to continue securing these resources for ongoing training and efforts to maintain fidelity to the treatment for all staff. Many empirical questions remain about the duration and content such training should take, how frequently these trainings should happen and how to handle ongoing training for all staff. Emerging literature supports online training in DBT as a high-quality, easily accessible and affordable option to traditional in-person, lengthy training methods.21 22 Nonetheless, institutional ‘buy in’ is essential to sustain the efficacy of DBT within an inpatient setting, regardless of the format and length of DBT training.

Fourth, an alternative possibility is that our patients experienced an increase in the number of incidents irrespective of treatment and staff training. This would be consistent with notable rising rates of self-injury and suicide in the general population of adolescents in the USA.23 24 Relatedly, this phenomenon may be a contributory factor influencing the number of incidents in our patient population. Another possibility is that with DBT becoming the recognised and articulated formal treatment model and culture of the unit by the time of DBT Group 2, our team increased in recognition, assessment and documentation of suicidal behaviour and NSSI. Similarly, it is possible that our patients increased in their reporting of these behaviours, rather than in the number of actual incidents, due to comfort of disclosing to staff, willingness to seek staff support and skills coaching and lack of punitive consequences (eg, taking away personal belongings), which were more likely to be enacted in TAU.

Fifth, by the time of DBT Group 2, our programme had been in existence for approximately 2 years and thus had grown in local reputation, as well as in requests for presentations and consultations to hospitals, schools and local mental health organisations in other area. As such, it is possible that we received patients presenting with more severe symptomatology and greater propensity to engage in suicidal and non-suicidal self-injurious behaviours. While there were no treatment group differences on diagnosis, diagnosis alone does not capture symptom severity or risk, and it is possible there were unknown and unmeasured differences between DBT Group 1 and DBT Group 2 in this regard.

The question remains as to why patients in DBT Group 2 were comparable to patients in DBT Group 1 for the number of CO hours for self-injury, restraints and days hospitalised, indicating that results were maintained for these variables, despite not being maintained for SA and NSSI. It may be that these variables were most greatly targeted by individual and family therapy and DBT skill groups, rather than milieu treatment, and individual and family therapy were comparable between DBT Group 1 and DBT Group 2.

Future directions

Further research on DBT adapted for an acute care adolescent inpatient unit is highly warranted given our findings coupled with international recognition of the severity of suicide rates for youth, and serious need for treatment and prevention strategies.25 Randomised controlled trials should be conducted to rigorously explore the efficacy of DBT within this setting. It is essential to examine staff-level variables in conjunction with treatment outcome variables given the high rate of staff turnover and possible drift from integrity to the treatment, as well as other staff-related variables including skills knowledge, integrity to treatment and burnout given the inherent challenges of inpatient treatment. Given the paucity of literature in this area, there are a number of important areas for additional study. Outcomes typically examined in outpatient DBT research may not be sensitive to change in the short duration of an acute care hospitalisation. As such, one valuable question relates to the most appropriate ways of measuring the efficacy of DBT in an inpatient setting. The absence of readily available measures of fidelity of DBT implementation26 and the paucity of literature describing evaluative approaches to the training and implementation processes27 present additional challenges to community agencies and hospital settings who wish to ensure the effectiveness of their training efforts and treatment outcomes for their patients.11

Given that DBT was originally developed as an outpatient treatment, application within other settings, including inpatient units and other milieus, can be quite challenging. The conditions necessary for successful implementation of DBT outside of controlled settings are not well understood or easily met15 nor are the conditions for the maintenance and ongoing fidelity and adherence to treatment. The programme Change Model developed by Simpson and expanded by Simpson and Flynn emphasises the implementation process and divides it into four crucial features that are characteristic of DBT’s intensive training: (1) exposure to new treatment via training, including didactic information and hands-on practice with feedback and rewards for progress, realistic views of skill requirements and limitations, team building, peer support and empirical evaluation of results, (2) adoption including a trial process of implementation involving decision-making and action taking, (3) implementation, and (4) practice improvement which implies full implementations and focuses on outcomes, services and budgets.28 29 Despite the comprehensiveness of this model, it fails to address common system barriers, particularly those of an inpatient hospital programme, namely staff overturn. Training front-line care providers is challenging, but multimodal approaches such as ongoing online training, expert-led DBT webinars, self-study of the Skills Training Manual and self-study of Linehan’s manual may be promising.26

In summary, this study continues to support the clinical benefits of DBT in an acute care psychiatric inpatient unit for adolescents with a variety of diagnoses. However, ongoing training and fidelity monitoring efforts are likely imperative for the maintenance of safety-related variables on the unit.