Introduction
Acquired brain injury (ABI) is a major cause of death and long-term disability among the pediatric population. ABI is defined as damage to the brain that occurs after birth and is not related to a congenital disorder or a degenerative disease; damage may be caused by a traumatic injury or by a non-traumatic cause (Brain Injury Association of America, n.d.). In USA, the annual estimate of traumatic brain injury (TBI) alone in children (aged 0-14) is approximately 448000 (Rutland-Brown et al, 2006). Recovery from pediatric ABI which coincides with the natural development may lead to highly variable outcomes in areas such as physical, cognitive, and social. Even with the mild injuries, some children may still experience long-term sequelae (Mami and Nance, 2008). Because of the vastly variable nature, conducting methodologically sound longitudinal studies which include children and adolescents with the whole spectrum of severity has been found to be a challenging endeavour.
Success in longitudinal research relies on minimizing the loss to follow-up and retaining a representative group of participants (Hunt and White, 1998; Bell et al, 2008), because inadequate retention may bias the research findings (Marcellus, 2004; Woolard et al, 2004; Barry, 2005; Bell et al, 2008; Gul and Ali, 2010). In a general research setting, retention often refers to the participant completion of study while attrition means the loss of participants during the study. It is also important to examine the differences, if any, between those who are lost to follow-up and those who remain until the end. For instance, if the demographics and the injury information are similar between the two groups, the bias to the study validity would be less concerned (Woolard et al, 2004). Straus et al (2005) suggested a ‘5 and 20’ rule for prognosis studies: that is, a less than 5% loss to follow-up is probably of little concern, while a greater than 20% loss would most likely threaten the study validity. Marcellus (2004) published a review on participant attrition in longitudinal studies, and found that reported attrition rates ranged from 5% to 70%. Sifers and colleagues (2002) reviewed 260 articles published during 1997 in pediatric psychology and child development journals, and found that the attrition rates were significantly underreported. There is no universally agreed upon standard for the acceptable attrition rate, and this issue clearly requires more attention.
Some have acknowledged that the retention of children and adolescents and their families in the field of health research is quite challenging (Marmor et al, 1991; Ryan and Hayman, 1996; Motzer, Moseley and Lewis, 1997; Frank et al, 2003; Ely and Coleman, 2007; Williams et al, 2008; Anderson et al, 2009; Anderson et al, 2012). The attrition rate at 5 years after TBI in the study by Anderson et al (2009) was 35.7% and this was considered lower than most pediatric longitudinal studies with children with TBI. In another TBI study by Anderson et al (2012), only 42% of children were remained at the 10-year mark. Simkin et al (2000) were only able to retain 73% of their study cohort at 18 months, and this was considered as an achievement even though the school setting is believed to provide a ‘captive audience’ sample. Retaining families with school-aged children and youth is even more complicated because the ‘captive audience’ component is no longer in effect. In a 13-year longitudinal study about children’s activity and nutrition which followed them from preschool to high school years, the final retention rate of 53% was considered a success (Frank et al, 2003). In a recent substance use prevention study by Bruzzese et al (2009) for adolescents and their parents, the retention rate was found to be over 87% at the two-year post-intervention assessment. Bruzzese et al (2009) contributed their success to the deliberate implementation of many recruitment and retention strategies which required a significantly high level of resources. This latest finding suggested that even though the retention of the pediatric population and their parents could be challenging, it is still achievable.
A few researchers in the substance use and human immunodeficiency virus literature have started to address the retention issue in the pediatric longitudinal research (Williams et al, 2008; Bruzzese et al, 2009). Unfortunately, this is not the case yet for children and adolescents with ABI and their families. In a recent article by Bell et al (2008), the authors discussed the importance of participant recruitment and retention based on their experience as collaborators in a multi-centre longitudinal study. While Bell et al (2008) shed some light about the retention challenges in the field of traumatic brain injury, the participants enrolled in their study needed to be 16 years of age or older (Dijkers et al, 2010). As is commonly observed in the ABI literature, children are not included. Therefore, their experience might not be suitable to apply directly on the pediatric ABI population.
Locating and retaining research participants for follow-up studies have been identified as two common challenges for the longitudinal research (Lyons et al, 2004). Intuitively, the longer study would face more challenges in retaining its participants over time (Gul and Ali, 2010). Traditionally, researchers tend to ‘react’ to the issue of attrition by adopting statistical methods to address missing data (Marcellus, 2004). It may be more proactive to anticipate attrition and invest on retention for a change. This paper is based on our research team’s years of experience in conducting longitudinal studies with families after ABI. The aims of this paper were to describe the retention strategies adopted in these studies, to explore the variables related to attrition, and to evaluate the possible impact of attrition on study outcomes.
Methods and Procedures
Study Design and Participants
Our cohort of children and adolescents with ABI and their families was first recruited in our earlier study entitled ‘Transitions experienced by children and their families after acquired brain injury’, or ABI Transitions Study for short. The inclusion criteria were children and adolescents (aged 5-18) who were admitted to McMaster Children’s Hospital (Hamilton, Ontario, Canada) with a diagnosis of ABI between November 2001 and December 2003. Participating families were followed after discharge for up to two years at five key transition points, such as return to school. At the end of this two-year ABI Transitions Study, all of the participating families were notified to ‘stay tuned’ for the possibility of another follow-up study.
In the fall of 2005, the research team received funding to continue following the same cohort from the ABI Transitions Study for another four years. This latter study was entitled ‘Trajectories and consequences: Long-term follow-up of children and youth and their families after acquired brain injury’, or the ABI Trajectories Study for short. Since children and adolescents with brain tumour often experienced more complicated prognoses, this particular sub-group was excluded from the subsequent ABI Trajectories Study. By the end of this four-year study, participating families had in total (including the time of the ABI Transitions Study and the gap between two studies) seven years at most and five years at least of follow-up after ABI. The objective of the ABI Trajectories Study was to evaluate the long-term effect of sustaining an ABI in childhood, and the families were only asked to complete the assessment on a yearly basis. Both studies were conducted with approval of the McMaster University Health Sciences’ Ethics Review Board, Hamilton, Ontario, Canada.
Measures
In the ABI Transitions Study, the families were assessed on a variety of measures which included health status, participation, academic performance, and family function at five key transition points (ie, discharge, return-to-school, eight-month, 12-month, and two-year post ABI). In the subsequent ABI Trajectories Study, the families were assessed on similar measures once a year on the child’s ABI anniversary date.
Procedures — Strategies Adopted to Enhance Retention
Keep Participants in the Know
- It is important to keep open communication with the participants as the participants have the right to know the expectation of the study in details. Given et al (1990) reported that such open communication enhanced retention as it not only allows the participants to make informed decision but also helps in building the rapport.
- In the longitudinal studies with multiple follow-ups, the participants might miss once or twice for various reasons. It is important to remind them that this does not necessarily suggest that they have to withdraw all at once (Bell et al, 2008).
Building Rapport with Participants
- Relationship building with the participating families is essential to successful retention (Ely and Coleman, 2007). The participants should not feel they are ‘just another number’. The research coordinator maintained a log book in which she documented the summary of each phone conversation, and this log book served as a great memory-boosting tool for any future correspondences. Ryan and Hayman (1996) utilized the index card system for similar purpose.
- Participants’ inquiries should always be responded promptly (Given et al, 1990; Marmor et al, 1991). As per the study protocol, a clinical referral would also be made if the family expressed some sort of concerns around the child’s health status.
- A cover letter always accompanied the mail-out questionnaires. The letter served as another reminder (Given et al, 1990) and it also provided an opportunity to reiterate the importance of participating in the study (Hunt and White, 1998). To be consistent with the rapport building, the letter was always personalized rather than using the generic ‘dear parent’ or ‘dear participant’.
Accommodating Participants’ Schedule
- The phone log book can help the research coordinator observe a pattern of contact attempts. While Lyons et al (2004) recommended against contacting participants in the evening and weekends, our experience and that of Adubato et al (2003) suggested otherwise.
- A reminder postcard was mailed out and up to three subsequent reminder calls would be made if the participating family did not return the completed questionnaires on time. A month after the last reminder call was placed, a letter would be sent. This persistent monitoring strategy was also found to be successful by Williams et al (2008) and Bruzzese et al (2009).
Using Incentives that Would Interest Participants
- Similar to Given et al (1990) and Ely and Coleman (2007), incentives such as fridge magnets, were provided to the participating families. Moreover, in the ABI Trajectories Study, an annual lottery draw of a digital audio player (ie, iPod nano) was held to enhance the overall assessment completion rate, especially targeting children and adolescents.
- Developing regular newsletter also serve to retain the participants (Given et al, 1990; Woolard et al, 2004; Bell et al, 2008), because it helps to remind them about the study, and the unsuccessful delivery can alert the research team about the change in the participant’s contact information. In the ABI Trajectories Study, we sent out annual newsletter with study updates and education materials as information can also be considered as an alternative incentive.
Keep in Touch with Participants
- During the recruitment, the research team also collected the family physician’s information. However, this information alone was not sufficient as the family physician’s office often did not have the updated contact information. It is therefore important to obtain multiple telephone numbers and addresses, including the participant’s relatives and friends as backup (Hunt and White, 1998; Lyons et al, 2004; Woolard et al, 2004; Bell et al, 2008). The other possible avenue that we tried was through consulting public database, such as Canada411 (www.canada411.ca). However, similar to Lyons et al (2004), this strategy was found to be more effective when the participant’s last name was not common.
Statistical Analysis
Statistical analysis was performed with the Statistical Package for the Social Sciences, SPSS 17.0 (SPSS Inc., Chicago, IL). To evaluate the possible impact of attrition, the first step was to examine whether there were any differences in demographics (ie, gender) and the ABI related variables (ie, age at the time of ABI, cause of ABI, and severity) between those who were lost to follow-up and those who remained in the study. In both studies, multiple assessments were required throughout the years, and not every family completed all of the scheduled assessments. Therefore, it would also be informative to find out whether the completion rate (ie, the percent of completed assessments) was different between the two groups. To examine the differences between groups, a chi-square test was used for nominal variables; an independent two-sample t-test was used for ratio variables that are normally distributed; a Mann-Whitney U test was used for ordinal variables and ratio variables that are not normally distributed.
Results
Retention
Figures 1 and 2 illustrate the retention statistics in the ABI Transitions Study and the ABI Trajectories Study, respectively. In the ABI Transitions Study, 192 families consented to participate, and 175 families (91%) remained at the end. In the ABI Trajectories Study, children and adolescents (n = 14) with brain tumour were excluded due to their more complicated prognoses. In addition, more than one year had passed since our last contact with the participating families in the ABI Transitions Study, and we were unable to locate 24 of them. Fifty families did not consent to participate in the subsequent study, and ‘too busy’ was the most commonly given reason. Hence, the number of overall participating families was reduced from 175 to 87 at the onset of the ABI Trajectories Study. Seventy-eight of the 87 families (90%) remained by the end of the ABI Trajectories Study. The characteristics of the study cohort at the onset and the end of both studies are displayed in Table 1.
Figure 1. Retention in the ABI Transitions Study
Figure 2. Retention in the ABI Trajectories Study
Table 1. Characteristics of the Study Cohort at the Onset and the End of the ABI Transitions Study and the ABI Trajectories Study
Impact on Attrition
At the end of the ABI Transitions Study, no statistically significant difference in demographics and the ABI related variables (ie, gender, age at the time of ABI, cause of ABI, and severity), and the completion rate (ie, percent of completed assessments) was found between those who were lost to follow-up (n = 17) and those who remained (n = 175). At the recruitment stage of the ABI Trajectories Study, no statistically significant difference in demographics and the ABI related variables was found between those who did not consent (n = 50) and those who consented to participate (n = 87). Also, no statistically significant difference in demographics and the ABI related variables was found between those who could not be located (n = 24) and those who consented to participate (n = 87). However, those who consented to continually participate in the ABI Trajectories Study had much higher completion rate back in the ABI Transitions Study than those who refused to participate (83% versus 57%, p < 0.001), and again much higher rate than those who could not be located (83% versus 44%, p < 0.001). At the end of the ABI Trajectories Study, no statistically significant difference in demographics and the ABI related variables, and the completion rate was found between those who were lost to follow-up (n = 9) and those who remained (n = 78).
Discussion
The primary aim of this paper was to describe the strategies employed to enhance the retention of children and adolescents with ABI and their families in two longitudinal studies. Many of our adopted retention strategies have also been found helpful in the literature. For instance, the persistent monitoring strategy was also found to be a successful retention strategy by Williams et al (2008) and Bruzzese et al (2009). The only exception is that we found that merely collecting family physician’s information as the backup contact is not sufficient enough. The current knowledge in this topic echoes our findings.
Our second aim was to explore factors related to attrition. Our results suggested that there is no predictable pattern in participants who are lost to follow-up and those that remain. There was a lack of statistically significant difference in demographics and ABI related variables. However, the findings showed that those who continued to participate in the subsequent ABI Trajectories Study had significantly higher completion rate in the first study, than those who did not consent to participate and those who cannot be located. It could be speculated that for those who were well engaged from the commencement, they would be more likely to continue participating over the years. The retention strategies reviewed in this paper could all be implemented starting from the onset of the project. It might be more effective for future research projects to take a proactive approach by implementing the retention strategies as early as possible.
The third aim was to review the possible impact of attrition in these studies. As noted in other pediatric health research literature (Motzer, Moseley and Lewis, 1997; Frank et al, 2003; Bruzzese et al, 2009), this is a rather difficult group when it comes to follow-up because the participants are out there in the community. Moreover, since there is no routine clinical follow-up for most children with ABI, particularly those with mild ones, there are no service providers who see the children and adolescents regularly that could serve as the liaison to maximize the retention. Despite these challenges, the results of this paper showed that the study cohort was well maintained — over 90% of the study cohort was retained by the end of each respective project. These were both within the ‘5 and 20’ rule suggested by Straus et al (2005). The lack of statistically significant difference in demographics and ABI related variables further demonstrated that the impact of attrition could be minimal (Woolard et al, 2004). Since no pattern of attrition was established (eg, we did not find that children with mild injury were most likely to drop out from the study or vice versa), we can be confident that the study results were not biased or confounded by any of these independent variables.
Study Limitations
Despite our endeavours to share the team’s experience in adopting retention strategies in pediatric ABI research, this paper is not without its limitations. It is important to note that both studies were not originally designed to evaluate the adopted retention strategies. While we are able to share our experiences in what worked and what didn’t, these retention strategies were not implemented and tracked systematically to allow for more sophisticated analyses. Even though we are unable to recommend the best retention strategies for the population of pediatric ABI, our success in retention appears to support the use of these strategies. On the other hand, despite the successful retention in each respective study, the large reduction in the number of participating families (ie, from 175 to 87) between the two studies requires additional attention. The one-year time gap between the studies was inevitable due to funding interruption. The participating families were merely asked to ‘stay tuned’ for the possibility of a follow-up study at the end of the ABI Transitions Study. In hindsight, the research team should have tried to maintain some sort of contact with the participating families to bridge the two studies as suggested by Lyons et al (2004) to minimize the loss.
Conclusion
Our results showed that there were no particular variables that would predict the pattern of retention in our studies. The only exception was that those who had significantly higher completion rate from the first study were more likely to continue their participation into the subsequent study. Moreover, the interruption between the two studies might have led to further reduction in the sample size in the subsequent study. The retention strategies are resource intensive, but they appear to contribute to successful retention. More work will be needed to systematically implement and evaluate these strategies. Every effort should be made to proactively ensure the participant retention, thus limiting threats to study validity.
Acknowledgements
The research team would like to thank all families for their years of participation. This manuscript is based on the ABI Transitions Study and the ABI Trajectories Study, which were funded by Ontario Neurotrauma Foundation (01046-O), and Canadian Institutes of Health Research (MOP-77618), respectively.
References
Adubato, S., Alper, R., Heenehan, M., Rodriguez-Mayor, L. & Elsafty, M. (2003). “Successful Ways to Increase Retention in a Longitudinal Study of Lead-Exposed Children,” Health and Social Work, 28 312-31.
Publisher – Google Scholar
Anderson, V., Catroppa, C., Morse, S., Haritou, F. & Rosenfeld, J. V. (2009). “Intellectual Outcome from Preschool Traumatic Brain Injury: A 5-Year Prospective, Longitudinal Study,” Pediatrics, 124 (6) e1064 -e1071.
Publisher – Google Scholar
Anderson, V., Godfrey, C., Rosenfeld, J. V. & Catroppa, C. (2012). “Predictors of Cognitive Function and Recovery 10 Years after Traumatic Brain Injury in Young Children,” Pediatrics, 129 (2) e254 -e261.
Publisher – Google Scholar
Barry, A. E. (2005). “How Attrition Impacts the Internal and External Validity of Longitudinal Research,” Journal of School Health, 75 267-270.
Publisher – Google Scholar
Bell, K. R., Hammond, F., Hart, T., Bickett, A. K., Temkin, N. R. & Dikmen, S. (2008). “Participant Recruitment and Retention in Rehabilitation Research,” American Journal of Physical Medicine & Rehabilitation, 87 330-338.
Publisher – Google Scholar
Brain Injury Association of America (2013). “What is the Difference between an Acquired Brain Injury and a Traumatic Brain Injury?,” [Online], [Retrieved July 17, 2013], http://www.biausa.org/FAQRetrieve.aspx?ID=43913&A=SearchResult&SearchID=2079221&ObjectID=43913&ObjectType=9http://biausa.fyrian.com/FAQRetrieve.aspx?ID=43913&A=SearchResult&SearchID=1711888&ObjectID=43913&ObjectType=9
Publisher
Bruzzese, J.- M., Gallagher, R., McCann-Doyle, S., Reiss, P. T. & Wijetunga, N. A. (2009). “Effective Methods to Improve Recruitment and Retention in School-Based Substance Use Prevention Studies,” Journal of School Health, 79 400-407.
Publisher – Google Scholar
Dijkers, M. P., Harrison-Felix, C. & Marwitz, J. H. (2010). “The Traumatic Brain Injury Model Systems: History and Contributions to Clinical Service and Research,” Journal of Head Trauma Rehabilitation, 25 (2) 81-91.
Publisher – Google Scholar
Ely, B. & Coleman, C. (2007). “Recruitment and Retention of Children in Longitudinal Research,” Journal for Specialists in Pediatric Nursing, 12 199-202.
Publisher – Google Scholar
Frank, G. C., Nader, P. R., Zive, M. M., Broyles, S. L. & Brennan, J. J. (2003). “Retaining School Children and Families in Community Research: Lessons from the Study of Children’s Activity and Nutrition (SCAN),” Journal of School Health, 73 51-57.
Publisher – Google Scholar
Given, B. A., Keilman, L. J., Collins, C. & Given, C. W. (1990). “Strategies to Minimize Attrition in Longitudinal Studies,”Nursing Research, 39 184-186.
Publisher – Google Scholar
Gul, R. B. & Ali, P. A. (2010). “Clinical Trials: The Challenge of Recruitment and Retention of Participants,” Journal of Clinical Nursing, 19 227-233.
Publisher – Google Scholar
Hunt, J. R. & White, E. (1998). “Retaining and Tracking Cohort Study Members,” Epidemiologic Reviews, 20 57-70.
Publisher – Google Scholar
Lyons, K. S., Carter, J. H., Carter, E. H., Rush, K. N., Stewart, B. J. & Archbold, P. G. (2004). “Locating and Retaining Research Participants for Follow-up Studies,” Research in Nursing and Health, 27 63-68.
Publisher – Google Scholar
Mami, A. G. & Nance, M. L. (2008). “Management of Mild Head Injury in the Pediatric Patient,” Advances in Pediatrics, 55 385-394.
Publisher – Google Scholar
Marcellus, L. (2004). “Are we Missing Anything? Pursuing Research on Attrition,” Canadian Journal of Nursing Research, 36 82-98.
Publisher – Google Scholar
Marmor, J. K., Oliveria, S. A., Donahue, R. P., et al (1991). “Factors Encouraging Cohort Maintenance in a Longitudinal Study,” Journal of Clinical Epidemiology, 44 531-535.
Publisher – Google Scholar
Motzer, S. A., Moseley, J. R. & Lewis, F. M. (1997). “Recruitment and Retention of Families in Clinical Trials with Longitudinal Designs,” Western Journal of Nursing Research, 19 314-333.
Publisher – Google Scholar
Rutland-Brown, W, Langlois, J. A., Thomas, K. E. & Xi, Y. L. (2006). “Incidence of Traumatic Brain Injury in the United States,” Journal of Head Trauma Rehabilitation, 21 544-548.
Publisher – Google Scholar
Ryan, E. A. & Hayman, L. L. (1996). “The Role of the Family Coordinator in Longitudinal Research: Strategies to Recruit and Retain Families,” Journal of Family Nursing, 2 325-335.
Publisher – Google Scholar
Sifers, S. K., Puddy, R. W., Warren, J. S. & Roberts, M. C. (2002). “Reporting of Demographics, Methodology, and Ethical Procedures in Journals in Pediatric and Child Psychology,” Journal of Pediatric Psychology, 27 19-25.
Publisher – Google Scholar
Simkin, L. S., Hirsch, L., Radosh, A., Middlestadt, S. E., Kaiser, J. & Santelli, J. S. (2000). “Strategies to Maximize Retention of a Sample of Young Adolescents in a Longitudinal Evaluation of Healthy & Alive!,” Journal of School Health, 70 286-291.
Publisher – Google Scholar
Straus, S. E., Richardson, W. S., Glasziou, P. & Haynes, R. B. (2005). Evidence-Based Medicine: How to Practice and Teach EBM, 3rd ed. Toronto: Elsevier Churchill Livingstone.
Publisher – Google Scholar
Teasdale, G. & Jennett, B. (1974). “Assessment of Coma and Impaired Consciousness. A Practical Scale,” Lancet, 2 (7872), 81—84.
Publisher – Google Scholar
Williams, P. L., Van Dyke, R., Eagle, M., et al. (2008). “Association of Site-Specific and Participant-Specific Factors with Retention of Children in a Long-Term Pediatric HIV Cohort Study,” American Journal of Epidemiology, 167 1375-1386.
Publisher – Google Scholar
Woolard, R. H., Carty, K., Wirtz, P., et al. (2004). “Research Fundamentals: Follow-up of Subjects in Clinical Trials: Addressing Subject Attrition,” Academic Emergency Medicine, 11 859-866.
Publisher – Google Scholar
