Development of the Life Gatekeeper suicide prevention training programme in China: a Delphi study

The Delphi method

The Delphi technique involves a group of experts making independent ratings of agreement with a series of statements through an iterative, multi-stage process. This systematic approach draws on the expertise of people working in specific areas and is applicable to provide guidance in a particular context. Delphi studies have commonly been used for the content development of mental health training programmes, including culturally appropriate mental health first aid,47 suicide postvention guidelines for secondary schools,48 and dos and don’ts in designing school-based awareness programs for suicide prevention.49

In this study, the Delphi process was conducted to identify the statements that should be included for the school-based suicide prevention gatekeeper training programme regarding the training content, training delivery form, training goals and training materials. Panel members were invited to review and rate their agreement with a range of initial statements and suggest any relevant information that could be added to each section. Informed consent was obtained from all participants, who were aware of their right to withdraw from the study at any time. Additional statements suggested by panel members in round 1 were included in round 2 for all members to rate. After obtaining the data from round 1, panel members received personalised feedback, including their rating and the overall rating for each item in round 1. Subsequently, they had the opportunity to change or maintain their original rating in round 2 based on this feedback for re-rating items that did not achieve consensus earlier. Panel members were compensated for their time participating in the study. Ethical approval was obtained prior to the start of this study.

Literature review

A literature search was conducted to identify information about the main content of the training programme, and search keywords were determined based on a previous study.17 The keywords ‘school-based’, ‘curriculum-based’, ‘suicide prevention’, ‘suicide education’, ‘gatekeeper’, ‘teacher’, ‘staff’, ‘parent’ and their various synonyms were investigated in the databases of Google Scholar, Web of Science, PubMed and Chinese National Knowledge Infrastructure. Studies on effective elements of gatekeeper training,50 development and evaluation of gatekeeper programmes12 51 52 and theoretical models of gatekeeper behaviour20 were consulted when conceptualising the development of the initial statements.

Questionnaire development and adaptation

The content of the Delphi survey was developed in two phases: first, by reviewing existing literature relevant to school-based gatekeeper training for suicide prevention, and second, by focus group discussion with school counsellors and experts in suicide prevention for practical insights in intervention development before the Delphi study began. The Literacy of Suicide Scale53 and guidelines from WHO1 were also consulted in developing these statements. As a result, the contents of the Life Gatekeeper training programme are original, created by the research team members, rather than by direct translation from existing programmes.

Statements for the content of the programme were organised into eight general modules: (1) identifying the urgency of suicide among adolescents and common feelings of persons at risk for suicide, (2) establishing understanding about suicide, (3a) recognising risk factors associated with suicide, (3b) identifying warning signs for suicide; (4) comprehending recommended ways to communicate suicide risk; (5) assessing suicide risk; (6) making a safety plan; (7a—for teachers’ training only) instructing teachers how to communicate with parents about their children’s suicide risk and find help for them, (7b—for parents’ training only) teaching parents how to express support for their children and find resources for help; and, finally, (8) identifying barriers for adolescents when seeking help or receiving assistance from teachers or parents. These eight modules were designed so that in addition to psychoeducation on suicide, some parts of the training also corresponded to the TPB model. For example, in the second module, statements included commonly encountered stigmas and misunderstandings related to suicide in an attempt to improve the attitude of trainees. Furthermore, in order to increase the perceived behavioural control of trainees, the final module pre-empted potential barriers of help-seeking by at-risk adolescents and provided support to teachers and parents. Moreover, statements within the fourth to seventh modules included detailed, step-by-step practical tips on what the trainees should and should not do when communicating with at-risk students, potentially increasing the perceived behavioural control of participants who will attend the training programme.

The first six modules of the programme shared similar components with existing gatekeeper programmes such as ASIST and QPR,12 15 whereas the latter two sections were locally developed with innovative features of the Life Gatekeeper programme. The teacher-specific and parent-specific modules were developed in response to the call for family–school partnerships in suicide prevention, and the final module, which encourages trainees to discuss potential barriers, was in line with the TPB,21 25 aiming to promote positive attitudes and increased perceived behavioural control regarding the performance of gatekeeper behaviours.

After that, the research team members set up a working group, including experts in mental health intervention project development and suicide prevention. The working group met regularly to discuss each possible statement extracted from the preliminary content that may be applicable to this training programme. We revised the statements to ensure that they could be understood by teachers and parents who lacked background knowledge of suicide prevention and were suitable for implementation. After several rounds of discussion and modification, the statements applicable to this intervention programme constituted the initial Delphi study questionnaire.

Panel formation

Panel members were invited to participate in this study if they were specialists with relevant suicide prevention or intervention experience. The experts were individually invited if they met any of the following inclusion criteria:

• a member of the Crisis Intervention Committee of the Chinese Association for Mental Health

• a psychiatrist/psychotherapist working in medical institutions for more than 5 years with clinical experience in suicide intervention

• a professor engaged in teaching and psychology research (psychological crisis intervention) in a college or university

• a school counsellor who regularly counselled students and was involved in suicide crisis management

• a crisis line operator with more than 5 years of working experience in answering calls and managing a crisis hotline

Data collection and analysis

At the beginning of each round of the Delphi study, an online link for participation in the survey was sent to all of the experts on the list. One week later, the experts who had not completed the questionnaire received an email reminder. Each round of the study lasted for 2 weeks; responses that exceeded the time limit were not collected or included in the data analysis. A flowchart of panel member recruitment, engagement and number of statements for each Delphi round is shown in figure 1.

Request permission

Panel members completed two rounds of questionnaires using the web-based survey platform Wenjuanxing. They were informed that their participation in the Delphi study would lead to the development of a gatekeeper training programme for suicide prevention that was culturally and contextually appropriate for teachers and parents in China.

The structured questionnaire used to collect expert feedback consisted of five sections: training content, training methods, achieving the training objectives, training materials, and general comments. For the round 1 and round 2 questionnaires, panel members were instructed to rate each statement according to one aspect of the criteria on a 5-point Likert scale, including the importance of the training content, the feasibility of the training method, the achievement of the training objectives, and the appropriateness of the training materials. For example, in the training content section, the instruction before the items to be assessed was, ‘Please rate the importance of including the content of the following items in the training programme’; options included 1=least important, 2=unimportant, 3=unsure/depends, 4=important and 5=essential. In the training methods and the achieving the training objectives sections, the instruction before the items was, ‘Please rate the following training methods/achieving the following training objectives according to their feasibility’; options included 1=not at all feasible, 2=not feasible, 3=unsure/depends, 4=feasible and 5=completely feasible. In the section ‘training materials’, options included 1=not at all appropriate, 2=not appropriate, 3=unsure/depends, 4=appropriate and 5=completely appropriate.

In addition to the statements and rating scales, there were three open-ended questions at the end of the above four sections during the round 1 questionnaire. (1) ‘Do you think it is necessary to add other items or content in this section? Please provide your suggestions for supplementation.’ (2) ‘Which items in this section do you think are inappropriate or need to be modified? Please provide your suggestions for modification.’ (3) ‘What other suggestions do you have for this section?’. In the fifth section, there were three different open-ended questions. ‘In general, how likely do you think this intervention will cause harm to the trainees? Please explain what you think may cause harm during the intervention and provide suggestions for how you think such harm could be reduced’; ‘Overall, what additional content or intervention techniques do you think are needed for this intervention to help trained teachers or parents develop a better understanding of suicide intervention and apply the gatekeeper behaviour techniques?’; ‘In order to design a localised intervention, we used data from relevant national studies, designed cases that fit the local context, conducted interviews with crisis intervention hotline workers and psychologists in China, and emphasised the need for home and school cooperation when protecting children at risk of suicide. Do you think these can meet the criteria for a localised intervention? If not sufficient, what other modifications could we make to strengthen the localisation of this intervention?’. Open-question responses were then discussed among the authors to generate new statements.

After each round, responses were analysed to calculate the percentage of the panel who rated an item as 4 or 5. According to previous similar studies,44 54 the criteria for consensus were defined as 80% or more of the panels scoring an item as necessary (≥4). Statements that were endorsed by 80% or more of the panel members were included in the training guide immediately. Statements rated by 70%–79% of the panel members as necessary were re-evaluated in the following round. Statements that were rated by less than 70% of the panel members as necessary were immediately excluded.

Following round 1, all panel members were provided with a summary report that included a comparison of their own ratings against the overall response for each item. In the round 2 questionnaire, panel members were asked to re-rate statements which were endorsed by 70%–79% of panel members and to rate the new statements created from the open-ended questions from round 1 for inclusion in the training programme.

Expert panel information

In round 1, 34 of the 40 invited potential expert panel members agreed to participate in the study and completed the survey. Thirty-one of them participated in round 2 (retention rate=91%). The mean (standard deviation (SD)) ages of experts participating in round 1 and round 2 were 42.4 (8.5) and 41.5 (8.2) years, respectively, and the mean (SD) years of suicide prevention and intervention working experience were 13.7 (8.1) and 12.7 (7.2), respectively. There was no significant difference in age (t=0.43, p=0.670) or duration of suicide prevention and intervention working experience (t=0.52, p=0.610) between experts who completed two rounds of Delphi versus those who only completed one round of the questionnaire. All panel members were currently working in China and were recruited from 13 provincial regions, including Beijing (35%), Hubei (15%), Hunan, Shanxi (Central provinces), Jilin, Liaoning, Inner Mongolia, and Tianjin (Northern provinces), Shanghai, Zhejiang, and Shandong (Eastern provinces), and Fujian and Guangdong (Southern provinces). The composition of participants represented various professional backgrounds. Most of them were school or university teachers (38.2%), psychiatrists (35.3%), psychotherapists (26.5%), psychological counsellors (23.5%), academics (26.5%) and hotline operators (17.6%); see table 1 for more details.

Endorsed statements

Figure 1 shows the number of statements to be endorsed, re-rated, rejected and newly added in each Delphi round. Of the 157 items included in the first round, 151 were endorsed, one was excluded and five needed to be re-rated (see online supplemental table 1 for statements and their ratings in round 1). All of the 50 new statements developed from panel members’ comments collected through the round 1 survey were endorsed in the round 2 rating process. Of the 55 statements included in round 2, all of the five re-rated statements were rejected and excluded (see online supplemental table 2 for statements and their ratings in round 2). After two survey rounds, 201 of the 207 statements assessed by the expert panel were endorsed for final inclusion in the gatekeeper training programme (see online supplemental table 3 for a full list of these included statements).

Examples of the new statements generated according to the open-ended text comments after each section are shown in online supplemental table 4. Table 2 shows examples of statements endorsed for inclusion in the training programme.

According to the results of two rounds of the Delphi questionnaire, we revised the training programme’s content. The first section was used for content development based on the importance of each section and specific items to be included in the programme. The second section was used to adjust the delivery methods of the training. According to the survey results, four initial training methods, such as video display (both animated videos and those by real actors), group discussion and role-play were endorsed and included, as well as adding the training format of an online question and answer (Q&A) session. The third section was used to confirm the experts’ evaluation of whether the existing training content could achieve the desired objectives. In this part, all of the six original items were highly endorsed by the panellists, and there were no new suggestions. For example, the majority of experts (97.1%) agreed that presenting statistics, common misconceptions, risk factors and warning signs related to suicide in the form of animated videos is feasible for enhancing trainees’ ability to understand and retain the relevant information. Based on the results, survey consensus on the feasibility of the proposed training method to meet the objectives was reached, increasing confidence that the intervention will be effective in achieving its goals. The fourth section was used to evaluate the form and content framework of the training materials. The final form of training materials was the same as the original version, including standardised intervention videos, manuals, appendices, training presentations and materials for at-risk students and parents. Six specific statements explained the purpose of the various training materials in detail. For example, one statement says, ‘The manual is an exercise booklet for use during training in which trainees can take notes, follow the prompts for group discussions or role plays, and refer to examples of role-play exercises.’ Two items added information about post-training testing and available online materials. All of the items were endorsed by the panel members and consensus was achieved.

In the fifth section, at the end of the questionnaire, panel members were asked to indicate their overall evaluation of the training programme, whether they considered the programme to be culturally suitable for local use, and if they foresaw any potential harmful effects. According to their comments, new items were generated to modify the existing training form and content the overall prerequisites, and the ways of providing the training programme. Among the 34 experts participating in the round 1 survey, 27 (79.4%) chose the option that the intervention would not cause any harm at all or was unlikely to cause harm, while six experts were uncertain and an expert chose the option that it is likely to cause harm to trainees. Based on the suggestions, two items synthesised for reducing harm to trainees were included. The informed consent form highlights that the training is voluntary and that teachers or parents with previous trauma experience or who perceive death as a taboo topic could choose whether to attend the training or not. Also, if during the training any teacher or parent feels uncomfortable, they can leave at any time. Twenty-five of the 34 (73.5%) experts considered the intervention sufficient to meet localisation standards. Three new items were generated and adopted, including conducting interviews with parents and students, surveying parents from different backgrounds and adding resources about referrals in the training materials.

Main findings

This is the first Delphi study conducted as part of developing a school-based suicide gatekeeper programme systematically and locally designed in China. Overall, expert consensus was achieved on the training content, the feasibility of training delivery method, the possibility of achieving training goals and the appropriateness of training materials. Findings from this study provide a preliminary evidence base for the systematic development of the Life Gatekeeper programme, a timely initiative given the continued lack of localised school-based suicide prevention development in China. Eight modules ranging from improving suicide literacy to pragmatic techniques of gatekeeper behaviour were established, and delivery methods of the gatekeeper training programme included both animated and role-play demonstration videos, group discussion and triad role-play.

When developing statements for the current study, research literature from English and Chinese databases were consulted. Thus, we drew inspiration from various sources: evidence-based international gatekeeper programmes (eg, ref 16 55), Chinese literature that provided insights on sociocultural factors relevant to the development of a localised gatekeeper training programme in China, and national guidelines for suicide prevention created by the Chinese government. Sometimes modifications of Western protocols were necessary. For example, existing gatekeeper training programmes such as ASIST or QPR commonly require gatekeepers to refer at-risk students to school counsellors for support.12 15 However, this might not be feasible in China due to the current limited availability of mental health professionals within schools, especially in rural areas.56 57 As such, it is inevitable that Chinese school teachers must undertake some responsibilities of school counsellor when acting as gatekeepers by learning how to communicate with at-risk students in an empathic and compassionate way, to explain the principle of confidentiality regarding disclosure of the suicide risk, and then to collaboratively develop a safety plan with them. They also need to communicate with parents effectively and support them in seeking timely medical support for the vulnerable student.

To optimally equip teachers with the necessary knowledge and skills for these challenging tasks and to empower them to perform gatekeeper duties, statements included in this Delphi study reflected the thorough consideration and nuanced design of the Life Gatekeeper programme. The programme content included detailed information about suicide risk assessment, templates for questions and safety plans, grounding techniques that could help students to calm down if they feel overwhelmed by emotions during conversation, and communication skills with parents. Moreover, this Delphi study also established the benefits of providing a variety of training materials, such as the training manual that has instructions on the training procedure and note-taking space that encourages engagement, an appendix that contains key information for training exercises and references for actual intervention, leaflets that contain psychoeducational information specifically for at-risk students or their parents, and a one-page summary of local and national crisis services and hospitals with mental health services. Furthermore, consensus was reached by experts that triad role-play consisting of a teacher, an at-risk student and an observer could be helpful for trainees to practise communication skills; previous research has shown that the observer role of watching others praticising skills facilitates further learning. 58 Lastly, as agreed upon and advised by panel members, at the end of the training, a Q & A session will be offered by experienced clinicians to address any unanswered questions, thereby further empowering trainees by increasing their confidence in applying gatekeeper skills.

Thirty-four panellists completed the initial consensus round and 31 completed the second round, yielding a high retention rate (91.2%); both rounds had more than 23 experts. Thus, it is likely that the results produced will remain stable over time.59 Moreover, the current Delphi study benefited from diverse backgrounds of expert panel members as they were recruited from several provinces across the country and had divergent roles (eg, researchers, clinicians or crisis frontline workers), educational levels and years of suicide prevention expertise. Their endorsed statements reflected consensus achieved from a multidimensional understanding of the topic, thus, increasing the generalisability of the findings.

Compared with existing gatekeeper training programmes, a module unique to the Life Gatekeeper programme is the teacher-training on how to communicate with parents about their child’s suicidal risk. As mentioned previously, the Ministry of Education of the People’s Republic of China6 has emphasised the school–family partnership in crisis intervention for students with suicidal risk. It has specifically stated that schools should assist parents in promptly seeking professional support for at-risk students. This strategy has been supported by Chinese researchers, for the psychological crisis experienced by an adolescent potentially reflects a larger, systemic problem in their ecological system wherein school teachers and parents play essential roles in protecting the child’s safety and supporting recovery.60 For example, the Delphi study statement ‘Teachers should communicate and update parents regularly about their children’s safety and what help the family may need’ was endorsed by 97.1% of the panel experts.

Considering that parents may experience strong emotional reactions of fear, overwhelm, anger, helplessness and worry when informed of their child’s suicidal risk,61 Chinese teachers may be challenged to deliver the message in a safe, contained way to upset parents.(In Western countries such messages are likely to be delivered by school counsellors who are trained to manage difficult feelings). Thus, to support teachers in their role of gatekeepers and decrease their perception of this challenge as a barrier to taking action, we included statements of ‘When communicating with parents about their child’s suicide risk, teachers need to focus on parents’ emotions and inform them that suicide is largely preventable to keep them from becoming overly anxious’ and ‘Teachers should talk to parents about the support available at school and the medical resources they can utilise in order to help alleviate excessive worry’ as guidance for teachers when communicating with parents. Both of these statements were endorsed by 100% of the panel members. Furthermore, since youth suicidality has been found to be associated with family factors such as lack of parental warmth,62 impaired family functioning,63 perceived authoritarian parenting and negative family climate,64 during the Life Gatekeeper training programme, teachers will also practise explaining to parents appropriate approaches for communicating with their at-risk child.

Additionally, panel members achieved consensus on all the statements regarding the parents’ version of training. Statements in this section were about what and how parents should communicate with their children about their suicidal risks and how they should seek further support from schools, hospitals and local resources on behalf of their children. Furthermore, based on the feedback from experts, a statement of life education—‘Parents need to educate their children about life and encourage them to discuss the value of life together’—was added in the second round, and it was endorsed by 93.5% of panel members. Indeed, it has been found that having meaning and purpose in life significantly impacts attitudes towards suicide among university students.65

Limitations

The feasibility of the Life Gatekeeper programme as perceived by school teachers, parents, at-risk adolescents and their peers has yet to be assessed due to practical challenges in identifying and recruiting participants. Furthermore, the sustainability and efficacy of a suicide intervention programme for adolescents may be affected by staff turnover and insufficient training time.66 Intensive training may help enhance intervention skills and produce a positive impact, but the appropriate frequency of delivering this gatekeeper training remains unclear (ie, whether it should only be delivered once or multiple times with booster sessions). Further investigation is warranted to evaluate the effectiveness of the Life Gatekeeper training programme in evoking gatekeeper behaviour after it has been delivered to school teachers and parents, emphasing the need for follow-up measures to investigate the long-term effectiveness of this prevention strategy.

Study implications

This Delphi study provides an evidence basis for further development of the Life Gatekeeper programme (eg, feasibility or randomised controlled trial studies). The originality of the study, starting from the conceptual stage of designing initial statements, has been demonstrated, for example, by considering the limited resources available to teachers and parents to support students at risk of suicide in China. In addition, some statements that reached consensus in this study could also be used for developing other types of suicide prevention programmes (eg, psychoeducational programmes) or similar gatekeeper programmes for other at-risk populations, such as for older adults living in rural areas of China.67 Furthermore, since suicide risk is often associated with mental illness,68 future research could also focus on early identification and intervention programmes for mental disorders, such as depression,69 for upstream suicide prevention.