An Evaluation of an E-learning Intervention to Update Social Work Practice
Authors: Campbell, Dan; Campbell, Ruth; Campbell, Tim; Ramos, George; Sullivan, Daniel
Abstract
This paper presents an evaluation of an e-learning intervention for social work clinicians designed to teach practice skills useful with substance use disorders. The evaluation compares outcomes for two groups of trainees in clinical social work practice. Outcomes of the group taking the training with interactive components demonstrate higher clinician satisfaction and lower retention as compared with the control group. Additionally, the evaluation examines outcomes of the training on social workers’ self-confidence, attitude, knowledge, and skill concerning the use of Cognitive–Behavioral Therapy in practice with substance use disorders. Outcomes suggest advantages of developing e-learning modes for conveying clinical skills.
E-learning has been defined as “an approach to teaching and learning […] based on the use of electronic media and devices […] for improving access to training, communication and interaction and that facilitates the adoption of new ways of understanding and developing learning” (Sangra, Vlachopoulos, & Cabrera, 2012). Over the past two decades, social work education has made greater use of e-learning in the classroom, as well as in continuing education opportunities (Ayala, 2009; Reeves & Reeves, 2008). Integrating healthcare information into social work education efforts has been challenging and important (Gant, Benn, Gioia, & Seabury, 2009), and is particularly important for members of the social work profession at this time. Social workers have been called to regard themselves as part of a “treatment team” (U.S. Department of Health and Human Services, 2016) in response to the increase in the use of prescription and non-prescription opiate drugs—the “opioid epidemic.” E-learning modes of teaching practical clinical skills to social workers can provide an appropriate and timely response to this crisis because of documented success of e-learning in improving healthcare training (George et al., 2014) and the improved availability of e-learning modes compared to face-to-face coursework and training. This paper examines outcomes of an e-learning interactive intervention designed to teach practice skills useful with clients using substances to clinical social workers. It will discuss implications for developing clinical social work education.
Current Social Work Education in Substance Use
According to a 2006 survey of the members of the National Association of Social Workers (NASW), 71% of respondents reported diagnosing or treating patients with substance use disorders (SUDs) in the past 12 months, though only 47% of them reported specific training in SUD assessment and treatment (Smith, Whitaker, & Weismiller, 2006). Despite the prevalence of SUDs among clients treated by social workers, substance abuse specialties and coursework are limited in schools of social work. A 2007–2009 survey of a random sample of the 187 then-accredited schools of social work found that none of them offered a substance abuse specialization and 52% of them did not offer a substance abuse-specific course (Bina et al., 2008). Clinicians in practice also cite a shortage of SUD training availability (Amodeo, 2000; Campbell, 2012).
The quality of social work education concerning substance use disorders also suffers. Researchers (e.g., Miller et al., 2006) note the gap between empirically-based treatment interventions for substance use disorders and clinical practice in the field. The expansion of conceptual understanding of mental disorders to include more medically-related information, as exemplified in the fifth edition of Diagnostic and Statistical Manual of Mental Disorders (DSM-5; American Psychiatric Association, 2013), has introduced social work clinicians to a medically-oriented approach to treatment. That approach demands a better grounding in medical information than most social worker clinicians receive.
The lack of initial coursework in substance use disorders in social work education, the lack of SUD training availability post-graduation, and the research–practice gap predict that clinicians currently working with SUDs may be unprepared to identify and treat opiate disorders.
Personal barriers. In addition to systemic barriers, such as the lack of social work coursework and training in substance use disorders, some barriers that inhibit assisting clients affected by the opiate epidemic are personal e.g., low self-efficacy in practice performance with clients with substance use disorders or negative attitude toward clients who use substances.
Social Cognitive Theory (Bandura, 1986) suggests an individual must have confidence in his or her own ability in order to perform an action. Researchers (Glanz, Lewis & Rimer, 1997; Grol & Wensing, 2004) find that confidence, or self-efficacy, is related to desired health behavior. Confidence has been related directly to social work clinicians’ implementation of cognitive–behavioral therapy (CBT), an empirically supported practice for clients with substance use disorders (Campbell, 2012).
Research has noted that stigmatizing or negative attitudes of clinicians toward a person with SUDs affects the treatment in negative ways (Hayes et al., 2004; van Boekel, Brouwers, van Weeghel, & Garretsen, 2013). For example, clinicians may be skeptical of the ability of a client who misuses substances to take advantage of what treatment the clinician has to offer, and thus respond by not offering the treatment. According to governmental agencies that promote the treatment of substance use disorders (Center for Substance Abuse Treatment, 2005), the conceptualization of substance use as a moral failing has stigmatized those seeking treatment for opiate use disorders.
Current Study
The authors, members and associates of Best Practice Trainers, Inc., a not-for-profit organization with a research and training mission, proposed to study social work clinician response to an e-learning interactive intervention. The proposed e-learning intervention conveyed practice skill information and video case presentation modeling opportunities concerning the use of CBT with clients with substance use disorders. Authors specifically wanted to evaluate an interactive mode of delivery because interactive modes have proven to have greater efficacy than didactic modes (Bhoophati & Sheoran, 2006; Gentry et al., 2016; George et al., 2014) for improving mental health or healthcare practice. Outcomes of clinical social worker attitude, knowledge, skill, and satisfaction with the experience were selected as important because they have proved important in evaluation for e-learning interventions (Gentry et al., 2016; George et al., 2014).
Authors hypothesized that the e-learning training with interactive elements would significantly increase (a) social work clinician self-confidence in performance of CBT in an SUD context, (b) attitude toward SUD clients, (c) knowledge about CBT in an SUD context, (d) implementation of CBT with SUD clients, (e) satisfaction with the e-learning interactive intervention, and (f) retention in the training compared to outcomes for clinicians who didn’t have access to the interactive elements in the intervention.
Method
Participants
Participants were all members of the NASW. A representative sample was produced by offering participation in a free NASW-approved training to the 8,876 members of the NASW who were involved in active clinical practice. Members residing in the six states that did not offer continuing education credit (CEUs) for NASW-approved courses (ID, NY, NJ, MI, OR, and WV) were excluded from the study. Participants were informed that the NASW had given prior approval to the course materials and the post-test, had approved the study protocol, and had neither developed nor endorsed the study. Participants were informed of protections offered to them as research participants, which included confidentiality, group assignment, and other protections. Hummingbird Institutional Review Board in Massachusetts was noted as the IRB of record. Three hundred and twenty-nine clinical social workers chose to participate in the study.
Measures
All outcome measures were formulated from the questions on the post-test questionnaire for each participant. Specific questions for each measure, with answer sets, can be found by contacting the corresponding author.
Dependent variables.
Implementation. Clinician implementation of CBT with their SUD clients was measured as the percentage of clients with SUDs on the clinicians’ caseloads with whom the clinician utilized CBT in the past week. Initially, the measure used a month-long interval. At post-test, the measure was changed to a week-long period in order to improve self-report capability.
Satisfaction. Satisfaction with the participant’s e-learning experience was a dichotomized measure, derived from a question concerning respondent rating of the trainings. The original question had a five-item Likert scale answer set and was created for this study.
Self-confidence. Clinician confidence in the delivery of CBT within an SUD context was conceptually derived from Social Learning Theory (Bandura, 1986) and formulated as a prorated scale with nine items. The CBT-SUD_Confidence scale was created for this study.
Attitude. Clinician attitude was influenced by the idea that clinicians may stigmatize an SUD client (van Boekel, Brouwers, van Weeghel, & Garretsen, 2013), and was formulated by dichotomizing the Likert scale answers to a confidence item gauging a client’s ability to benefit from clinician CBT interventions. This measure was created for this study.
Knowledge. Clinician knowledge of CBT in an SUD context was formulated as a prorated 13-item scale of multiple-choice questions that were rated as correct or incorrect. The final measure, termed CBT-SUD_Knowledge scale consisted of a 9-item prorated scale created for this study.
Completion. The electronic database automatically noted participants’ progress through the training. Participants had to open every page in the course in order to access the post-test. When participants completed the post-test, the system noted that completion. Completion is a dichotomous variable that notes whether the respondent completed the training and post-test.
Independent variable. The independent variable in this study was the type of group into which the trainee was randomly assigned at the beginning of the study. There were two groups into which a trainee could be assigned: the experimental group or the control group.
Design. Authors used an experimental pre-test–post-test design with randomization to examine the effects of the online e-learning interactive intervention on outcomes. Paired-samples t-tests were used to specifically examine the effects of the training itself.
Procedures
After the NASW had approved the coursework and the invitation, and the Hummingbird Institutional Review Board had approved all protocols, Infocus Marketing arranged an email ‘blast’ to 8,876 NASW members in June 2017. The e-mail blast offered a free online training on the use of cognitive–behavioral therapy with clients with SUDs. Six CEUs were granted after clinicians signed the informed consent, completed the “Essential CBT for Substance Use Disorders” training, and completed a post-test with at least 80% accuracy. Five hundred and ninety NASW clinicians responded to the invitation, were provided an access code, and were randomly assigned to either the experimental or the control group. Of these, 329 respondents took the required pre-test for the CBT training. One hundred and fifty-three respondents completed the training and post-test within the three-month period.
Experimental group. Experimental group clinicians began the online training after they could access the training site. On the “home page,” each individual could access a pre-test button, a course account bar graph that informed them of the percentage of the course they had completed, and a CBT post-test button. The training used didactic material and ten illustrative video case presentations to convey a clinical understanding of CBT use with SUDs. The training site continuously informed individuals about the number of days they had to complete the training. Trainees were given contact information in case of problems. Clinicians in the experimental group received weekly newsletters with news about substance use or mental health topics. They also had access to a set of “skill challenges”—video case presentations with small quizzes attached. The newsletters encouraged trainees to access the skill challenges. Control group. Clinicians in the control group had access to the training site, the CBT training with videos, the pre-test, and the post-test. They were given weekly reminders that they had a certain number of days left in which to complete the training. They had no access to the skill challenges with quizzes, or the newsletters..
Results
The experimental condition was an e-learning interactive intervention composed of an online training concerning the use of CBT with substance use disorders, weekly newsletters, and digital skill challenges. At the end of the study, 153 NASW clinicians had completed the training and taken the posttest.
Demographic characteristics for the entire sample of social work clinicians are presented in Table 1. Table 1 also compares characteristics of social workers who were working with clients with substance use disorders with characteristics of social work clinicians who did not work with clients with SUDs in order to determine factors that might contribute to outcome differences. As seen in Table 1, social work clinicians in the sample (N = 329) averaged 55.0 (SD = 11.7) years of age, were generally female (88.9%), Caucasian (76.6%), and employed in outpatient treatment or private practice (81.7%). Almost all (98.1%) had attained a Master of Social Work degree, averaging 15.4 (SD = 10.8) years of practice with SUDs. There were three interns in the study with one year of experience with SUDs. Most clinicians (n = 263; 83.8%) did not have a specialized substance use certificate, and most had not had previous CBT training (n = 198; 63.5%). Most clinicians stated that they derived support that was useful to their practice from e-learning modes (52.3%) and conferences (21.3%).
In this sample, 49.4% (n = 156) of the clinicians stated they were offering SUD assessment, education, or treatment by answering yes to a question concerning current SUD work. As seen in Table 1, social work clinicians who were working with clients with SUDs differed significantly in years of SUD practice, the possession of a special SUD certificate, practice type, and previous CBT training from other clinicians. SUD clinicians had been in SUD practice for an average of 16.8 (SD = 10.5) years, significantly greater than the average number of SUD practice years of those in the sample who were not working with clients with SUDs (M = 13.2; SD =11.0; t (233) = 2.5, p <.02). Social workers who were working with SUDs were more likely to hold a special SUD certificate (27.6%) than other clinicians (5.2%), c2 (1, n = 305 ) = 28.6, p <.001; and more likely to work in a private practice setting (87.8%) than other clinicians (74.8%), c2 (1, n = 307) = 8.56, p < .01; and more likely to have had previous CBT training (50.7%) than others in the sample (20.8%), c2 (1, n = 306) = 29.78, p <.001. [t] Table1 near here.[/t]
Effect of the intervention. Results of bivariate tests on demographic characteristics of participants in the experimental group compared with the control group showed no significant differences. As shown in Table 2, results of bivariate tests of the experimental intervention with outcome variables demonstrated that the experimental condition had a significant effect only on trainee satisfaction and trainee retention in the system. The experimental condition had no significant effect on any other outcome as measured in this study.
Completion. One main effect of the experimental condition was on retention in the system. The Norwegian company Adeptio Consult had designed the training system with page-tracking capability. NASW clinicians’ use of the CBT training system ranged from 1 to 383 pages (M = 119.3, SD = 76.95; n = 241). Since the CBT training itself was 125 pages in length, the mean of the pages opened indicated that many users did not complete the training. A total of 241 people began the training by taking the pre-test, but only 153 completed the training. There were significantly more dropouts in the experimental condition (46.6%) than in the control condition (29.2%), χ2 (1, n = 241) = 7.67, p < .01 (see Table 2). This is discussed further in the discussion section.
Satisfaction. Overall, the satisfaction by the clinicians with the e-learning interactive intervention was high. Most (93.4%) of the 152 clinicians who answered the question on satisfaction with the training rated the e-learning interactive intervention good or excellent. Most (94.7%) of the 76 SUD clinicians who completed the training and answered the question rated the e-learning interactive intervention good or excellent. However, as shown in Table 2, a significantly greater percentage of those who completed in the experimental condition (100%) rated the training good or excellent compared to the percentage of clinicians in the control condition who rated the training good or excellent (89.7%), χ2(1, n = 152 ) = 6.07, p < .05. [t]Table 2 near here.[/t]
Effect of the e-learning training. Post-test outcomes were compared to pre-test levels of variables of interest with the use of paired-samples t-tests to determine the effect of the training itself. As shown in Table 3, outcomes demonstrated that all clinicians had significantly increased levels of self-confidence in the performance of CBT in an SUD context, increased positive attitude toward SUD clients, increased levels of knowledge about CBT in an SUD context, and increased implementation of CBT with SUD clients three months post-training compared with pre-training levels. All outcome measures for SUD clinicians, determined separately, demonstrated similar increases at three months post-training.
Self-confidence. There was a significant increase in NASW respondents’ self- confidence after the training (M = 38.0, SD = 5.20) compared with pre-test levels (M = 33.2, SD = 7.66), t(152) = 10.06, p < .001. There was also a significant increase in SUD clinicians’ self-confidence after the training (M = 39.1, SD = 4.98) compared with pre-test levels of confidence (M = 35.0, SD = 4.98), t(77) = 6.05, p < .001. This demonstrates that the e-learning training was effective in increasing all clinicians’ confidence in performing CBT with SUD clients. This is consistent with Bandura’s understanding of self-efficacy or confidence as an essential part of the learning process.
Knowledge. There was a significant increase in NASW respondents’ knowledge after the training (M = 0.88, SD = 0.09) compared with pre-test levels (M = 0.72, SD = 0.13) t(47) = 8.27, p < .001. There was also a significant increase in in SUD clinicians’ knowledge after the training (M = .89, SD = .075) compared with pre-test levels (M = .76, SD = .08), t(21) = 6.83, p < .001. This increase demonstrated that the e-learning training was effective in increasing all clinicians’ knowledge in performing CBT with SUD clients.
Implementation. Paired-sample t-tests were conducted to compare the implementation of CBT with SUD clients (CBT used with % SUD clients) before the training and after the training. Only the clinicians who said that they worked with clients with SUDs were considered in this calculation. There was a significant increase in SUD clinicians’ implementation of CBT with their SUD clients after the training (M = 87.0, SD = 27.42) compared to their implementation of CBT with SUD clients before the training (M = 49.1, SD = 38.0), t(44) = 6.26, p < .001. Clinicians increased their use of CBT with clients with substance use disorders from 5 out of 10 clients to 9 out of 10 clients with SUDs.
Attitude toward SUD clients. A chi-square analysis was conducted to compare the attitude of clinicians toward SUD clients before the training and after the training. At post-test, of the 147 NASW clinicians who answered the question, 49.7% reported a positive attitude toward SUD clients’ ability to benefit from CBT interventions, a significant increase over the 38.0%, χ2 (1, n = 147) = 7.48, p <.01, who reported a positive attitude toward SUD clients’ capabilities at the beginning of the training. There was also a significant improvement in SUD clinicians’ attitude toward their SUD clients. At post-test, of the 76 SUD clinicians who answered the question, 57.9 % reported a positive attitude toward SUD clients’ ability to benefit from CBT interventions, a significant increase over the 31.6%, χ2 (1, n = 76) = 4.72, p < .05, who reported a positive attitude at the beginning of the training. [t]Table 3 near here.[/t]
Discussion
Social work has neglected educating its own practitioners about substance use disorders. That trend should be reversed, not only because of the current call to become part of the “treatment team,” but also because of our professional ethical commitments to our own competence and, more broadly, to the general welfare of society.
Improving Social Work Education.
The promotion of education pertaining to treatment of SUDs seems imperative when one considers social work’s core ethical commitments. The lack of coursework and training in CBT for SUDs demonstrated by this sample of social work clinicians is striking. Only 36.5% of clinicians in the representative sample of NASW active clinicians in this study had received such training. Even among the SUD clinicians in the study, only one out of every two (50.7%) had had previous training with the use of CBT in SUD treatment. This is a low percentage, given the fact that CBT is an empirically supported practice, useful with many substance use and mental health disorders, and is often used as an adjunct to Medication Assisted Treatment in opiate addiction.
Many clinicians in this study did not understand that psychosocial treatments alone were not effective against opiate use. For instance, on one question in the Knowledge section of the post-test that pertained to opiate addiction, 43% of clinicians in the whole sample who answered the question chose an answer that suggested that CBT as a stand-alone treatment would be successful against opiate addiction, despite evidence to the contrary (Amato, Minozzi, Davoli, & Vecchi, 2011), emphasized in the training. The overestimation of the usefulness of CBT with opiate addiction by social workers who work with SUDs would suggest that trainings in opiate treatment would be useful for social workers in this age of the opioid epidemic. All clinicians should become acquainted with screening tools for assessing opiate use, preventive strategies to ward off or deal with opiate overdose, have referral phone numbers at hand for crises, and know what options exist in their area for treatment.
Another factor contributing to the crisis is the negative attitude clinicians have toward those who fall prey to substance dependence. Of the social workers who responded to the offer of training by taking a pre-test (n = 297), 44.4% had low to moderate confidence that their SUD clients would benefit from the CBT they offered.
E-Learning Methods
E-learning has been useful at times or in places in which the health or mental health care is challenged by overwhelming numbers and the need for updated training (George et al., 2014). E-learning modes can provide the advantage of accessibility and rapid assessment of effectiveness. In this study, the e-learning training reached clinicians in every state (except six) across the US with relative ease. The training was completed even in Houston, Texas during Hurricane Harvey. E-learning interventions has the potential to advance ethical aims of the profession by providing much-needed education to social workers who would not otherwise receive it.
From the results of this study, it is clear that e-learning interventions can update clinical practice skills for social workers. Results of this study demonstrate substantial improvement of social work practice skills with exposure to an e-learning intervention. Clinicians had improved significantly on outcomes of confidence in the performance of a practice skill, knowledge about practice with SUDs, implementation of a practice with clinical populations, and attitude toward substance-using clients three months after having participated in an e-learning intervention. Social workers’ increase in implementation of an empirically supported practice with clients with substance use disorders is a successful outcome of the project and adds to the evidence concerning the ability of e-learning modes to convey clinical skills.
The results of the interactive part of the e-learning intervention are more challenging to understand. The experimental design of this study separated and clarified the effects of the interactive part of the e-learning intervention. Unlike many of the interactive e-learning successes in healthcare (Bhoopathi & Sheoran, 2006; Gentry et al., 2017; George et al., 2014), which document change in attitude, skill, and/or knowledge as the effect of the interactive intervention, the interactive part of the intervention in this study had no effect on those outcome measures. The specific effects of the interactive part of the e-learning intervention in this study, shown in Table 2, were to decrease clinician retention in and increase satisfaction with the entire e-learning experience. That paradox invites further investigation. The interactive part of the intervention was separated from, rather than integrated with, the training part of the intervention in this study. The separation of the skill challenges from the main part of the training may have been part of the reason that the effect of the interactive part of the intervention was to decrease retention in the training. Perhaps the encouragement to use the skill challenges led to the diversion of trainees from the training to the game-like skill challenges.
One interesting fact is that only a little over 6% of the 8,876 clinicians who were offered the e-learning training responded by clicking on the link. Fewer still completed a pretest. Why did those clinicians participate while 94% did not? Perhaps the answer lies in how comfortable the clinicians were with e-learning and technology. Support for that idea may rest in the clinicians’ answer to “What do you find most useful to support your practice?” Over 50% of the clinicians who participated reported that they received most of their practice support from e-learning interventions. That answer may reveal the technological skill level of participants, and may be an important factor in the dissemination of e-learning. To date, the research that applauds the success of e-learning interventions does not mention skill level with internet-capable devices as a factor that affects outcome. It may be that those in the experimental condition who were retained were more comfortable with the interactive interventions. More investigation into what is appealing to clinical people and what increases retention would be worthwhile in the development of clinical e-learning interventions.
Limitations and Future Research
This study followed the experimental protocol well, using a representative sample and randomly assigning trainees who signed up into experimental and control groups. For that reason, results are generalizable to clinicians who are members of the NASW. Generalization to other groups should be done only with caution.
Measures in this study have not been validated, so authors cannot report the validity of observations on confidence and knowledge, attitude, and satisfaction. The interval over which the implementation measure was taken was changed from a month at the pre-test measure to a week at the post-test measure. This was done because the answers of clinicians seemed to vary so widely on the pre-test measure that authors felt that reducing the timeframe would increase the accuracy of self-report. All else being equal, this change should have decreased the percentage of clients with which a clinician used any practice. However, the outcome demonstrated the opposite. One way of explaining the outcome is that the accuracy of self-report may have increased. Another way of explaining the outcome is that the training intervention was the source of the increase in implementation.
Future research. The future holds an explosion of e-learning trials and models. E-learning modes have proved to be successful in training healthcare personnel in other countries. In this country, the opioid crisis has made apparent how a crisis in healthcare can overwhelm our resources. Possibly this experience will open the door for more e-learning interventions to integrate empirically supported practices into clinical social work education. The encouraging results of the training are that, with relatively low cost, e-learning trainings can improve the availability of educational or practice updates for clinicians throughout the U.S. They also hold the promise of increasing the level of implementation of empirically supported practices such as CBT. In this time of an opioid crisis, a definitive response by social workers who are capable of using proven practices to identify, assess and treat substance use disorders would be reassuring, to say the least. It would also go some way toward discharging the ethical obligations of social workers to “promote the general welfare of society, . . . and the development of people, their communities, and their environments” (National Association of Social Workers, 2017).
References
Amato, L., Minozzi, S., Davoli, M., & Vecchi, S. (2011). Psychosocial combined with agonist maintenance treatments versus agonist maintenance treatments alone for treatment of opioid dependence. Cochrane Database of Systematic Reviews, 2011(10). doi:10.1002/14651858.CD004147.pub4.
American Psychiatric Association. (2013). Diagnostic and statistical manual of mental disorders (5th ed.). Washington, DC: Author.
Amodeo, M. (2000). The therapeutic attitudes and behavior of social work clinicians with and without substance abuse training. Substance Use and Misuse, 35, 1507–1536.
Ayala, J. S. (2009). Blended learning as a new approach to social work education. Journal of Social Work Education, 45, 277–288.
Bandura, A. (1986). Social foundations of thought and action. Englewood Cliffs, NJ: Prentice-Hall.
Bhoopathi, P. S., & Sheoran, R. (2006). Educational games for mental health professionals. Cochrane Database of Systematic Reviews, 2006(2). doi:10.1002/14651858.CD001471.pub2.
Campbell, R. (2012). Substance use disorder treatment: Best practice use and social work. Saarbrucken, Germany: Lambert Academic Publishing.
Center for Substance Abuse Treatment. (2005). Medication-assisted treatment for opioid addiction in opioid treatment programs. Treatment Improvement Protocol (TIP) Series 43. HHS Publication No. (SMA) 12-4214. Rockville, MD: Substance Abuse and Mental Health Services Administration. Retrieved from https://www.ncbi.nlm.nih.gov/books/NBK64149/
Gant, L., Benn, R., Gioia, D., & Seabury, B. (2009). Incorporating integrative health services in social work education. Journal of Social Work Education, 45, 407–425.
Gentry, S., L’Estrade Ehrstrom, B., Gauthier, A., Alvarez, J., Wortley, D., van Rijswijk, J., . . . Zary, N. (2016). Serious gaming and gamification interventions for health professional education. Cochrane Database of Systematic Reviews, 2016(6). doi:10.1002/14651858.CD012209.
George, P. P., Papachristou, N., Belisario, J. M., Wang, W., Wark, P. A., Cotic, Z., . . . Car, J. (2014). Online eLearning for undergraduates in health professions: A systematic review of the impact on knowledge, skills, attitudes and satisfaction. Journal of Global Health, 4(1), 010406. doi:10.7189/jogh.04.010406.
Glanz, K., Lewis, F. M., & Rimer, B.K. (1997). Linking theory, research, and practice. In K. Glanz, F. Lewis, & B. Rimer (Eds.). Health behavior and health education. San Francisco, CA: Jossey-Bass Publishers.
Grol, R., & Wensing, M. (2004). What drives change? Barriers to and incentives for achieving evidence-based practice. Medical Journal of Australia, 180, S57–S60.
Hayes, S. C., Bissett, R., Roget, N., Padilla, M., Kohlenberg, B. S., Fisher, G., . . . Niccolls, R. (2004). The impact of acceptance and commitment training and multicultural training on the stigmatizing attitudes and professional burnout of substance abuse counselors. Behavior Therapy, 35, 821–835.
Miller, W. R., Sorensen, J. L., Selzer, J. A., & Brigham, G. S. (2006). Disseminating evidence-based practices in substance abuse treatment: A review with suggestions. Journal of Substance Abuse Treatment, 31, 25–39.
National Association of Social Workers. (2017). Code of ethics. Retrieved from https://www.socialworkers.org/About/Ethics/Code-of-Ethics/Code-of-Ethics-English.
Reeves, P., & Reeves, T. (2008). Design considerations for online learning in health and social work education. Learning in Health and Social Care, 7, 46–58.
Sangrá, A., Vlachopoulos, D., & Cabrera, N. (2012). Building an Inclusive definition of e-learning: An approach to the conceptual framework. The International Review of Research in Open and Distance Learning, 13. Retrieved from http://www.irrodl.org/index.php/irrodl/article/view/1161/2146
Smith, M. J., Whitaker, T., & Weismiller, T. (2006). Social workers in the substance abuse treatment field: A snapshot of service activities. Health and Social Work, 31, 109–115.
Smith-Osborne, A., Bina, R., Harnek Hall, D. M., Mollette, A., Smith-Osborne, A., Yum, J., Sowbel L., & Jani J. (2008). Substance abuse training and perceived knowledge: Predictors of perceived preparedness to work in substance abuse. Journal of Social Work Education, 44, 7–20.
U.S. Department of Health and Human Services. (2016, November). Facing addiction in America: The Surgeon General’s report on alcohol, drugs, and health. Washington, DC: author.
U.S. Department of Health and Human Services. (2017, June). About the epidemic. Retrieved from https://www.hhs.gov/opioids/about-the-epidemic/index.html
van Boekel, L. C., Brouwers, E. P. M., van Weeghel, J., & Garretsen, H. F. L. (2013). Stigma among health professionals towards patients with substance use disorders and its consequences for healthcare delivery: Systematic review. Drug and Alcohol Dependence, 131, 23–35.