National Diabetes Education Program 'Thunder and Lightening' Campaign for Hispanic/Latino Diabetes Control |
|
Disclaimer This example is derived from the National Diabetes Education Program Thunder and Lightning campaign. It was modified and fictionalized in part to conform to the planning process set forth in CDCynergy 3.0. This example originally appeared in CDCynergy Diabetes Edition. |
|
|
|
Step 1.1 Write a problem statement. |
|
Few people with diabetes understand how they can best control their blood sugar, the importance of glucose control, and that complications can be delayed or prevented. The best way to avoid diabetes-related complications is to ensure that blood glucose levels are within normal ranges. The Diabetes Control and Complications Trial (DCCT) showed that intensive blood glucose control in people with type 1 diabetes reduced the risk for developing eye disease, kidney disease, and nerve disease (76%, 50%, and 60% reduced risk, respectively) by lowering hemoglobin A1C from 9 to 7 percent. The United Kingdom Prospective Diabetes Study (UKPDS) showed that intensive blood glucose control in people with type 2 diabetes delayed the onset of eye and kidney disease, reducing the risk by 12 to 33 percent, by lowering hemoglobin A1C levels from about 8 to 7 percent. These studies further showed that any sustained lowering of blood glucose helps, even if the person has a history of poor control. If blood glucose levels are not well managed, complications may arise. It has been documented that diabetes is the leading cause of end-stage renal disease, adult blindness, and non-traumatic lower-extremity amputations. People with diabetes are also two to four times more likely to have a stroke or cardiovascular disease than people without diabetes. Indirect and direct costs associated with diabetes are estimated to be $98.2 billion each year in the United States. Diabetes disproportionately affects minority populations and the elderly and is likely to increase as minority populations grow and the U.S. population becomes older. For more information about diabetes and its prevalence (DB-National_Diabetes_Fact_Sheet-English.pdf). |
|
Step 1.2 Assess the problem's relevance to your program. |
|
Following the release of the DCCT and UKPDS study findings, the Centers for Disease Control and Prevention (CDC) and its partners had responsibility to develop and implement strategies to translate the findings into public health practice. As a result, CDC and the National Institutes of Health (NIH), National Institute of Diabetes and Digestive and Kidney Diseases (NIDDK) became joint sponsors of the National Diabetes Education Program (NDEP). The purposes of NDEP are to improve the treatment and outcomes for people with diabetes, to promote early diagnosis, and ultimately, to prevent the onset of diabetes. To meet the goal of reducing the morbidity and mortality associated with diabetes and its complications, the NDEP has the following objectives:
The NDEP Hispanic Diabetes Control Campaign relates particularly to Objectives 2 and 5. For a more complete description of NDEP goals and objectives (DB_changing_the_way_diabetes_is_treated.pdf). |
|
Step 1.3 Explore who should be on the planning team and how team members will interact. |
|
The NDEP steering committee consists of representatives from different public and private, national, state, and local entities including:
Separate NDEP workgroups were established to develop strategies and campaigns for specific audiences at increased risk. Members of the workgroup dedicated to serving the Hispanic and Latino population include the following:
|
|
Step 1.4 Examine and/or conduct necessary research to describe the problem. |
|
Diabetes surveillance is essential for identifying persons with the disease and for formulating strategies to educate and treat them. The Centers for Disease Control and Prevention, Division of Diabetes Translations Surveillance Section uses data from the U.S. Renal Data System, Behavioral Risk Factor Surveillance System, Vital Statistics, the National Ambulatory Medical Care Survey, the National Hospital Ambulatory Medical Care Survey, National Health Interview Survey, and the National Hospital Discharge Survey for diabetes surveillance. To view surveillance data: DB_Surveillance_1999_Chapt1.pdf, DB_Surveillance_1999_Prevalence.pdf, and DB_Surveillance_1999_Mortality.pdf. The National Diabetes Education Program also hired a contractor to conduct additional formative research regarding the issue of diabetes control in the Hispanic and Latino populations. To view some of the findings of the formative research on the Hispanic/Latino population (DB_English Hispanic_Latino_Fact_Sheet.pdf). |
|
Step 1.5 Determine and describe distinct subgroups affected by the problem. |
|
Public health surveillance of diabetes has shown that minority groups and the elderly are disproportionately represented. It has been reported that Hispanic and Latino Americans have the second highest rate of type 2 diabetes, compared with other populations. Currently, Hispanic and Latino Americans are the second largest minority group in the United States with an estimated population of 27 million people. It is estimated that 1.8 million Hispanic and Latino Americans have diabetes. Risk factors associated with the Hispanic and Latino population for developing diabetes include: family history of diabetes, obesity, smoking, limited access to health care, being over 40 years of age, and a sedentary lifestyle. Within the Hispanic and Latino population, diabetes is the sixth leading cause of death. Among Hispanic and Latino women and the elderly, diabetes is the fourth leading cause of death. Onset of type 2 diabetes has been documented as occurring earlier in life for the Hispanic and Latino population (ages 50 to 59 years versus 60 to 69 years for non-Hispanic populations). Data also indicate that among Puerto Rican Americans and Mexican Americans, the onset of type 2 diabetes occurs even earlier (ages 30 to 50 years). For more information about diabetes and its impact on the Hispanic/Latino population (DB_English_Hispanic_Latino_Fact_Sheet.pdf). National Diabetes Education Program data indicate that Hispanic and Latino Americans are at higher risk of developing and dying from diabetes and are two times as likely to have complications of diabetes, such as heart disease, blindness, amputations, and nerve damage and kidney disease, than non-Hispanics. For additional information about the incidence and prevalence of diabetes and diabetes related complications among the Hispanic/Latino population (DB_Diabetes_in_Hispanic_America.pdf). |
|
Step 1.6 Write a problem statement for each subgroup you plan to consider further. |
|
Three primary subproblems were identified, in relation to diabetes and its complications in Hispanic/Latino adults.
|
|
Step 1.7 Gather information necessary to describe each subproblem defined in new problem statement. |
|
Research was conducted for the National Diabetes Education Program to further understand the incidence, prevalence, and related risk factors for diabetes within the Hispanic and Latino population. Focus groups determined audience knowledge, attitudes and practices in relation to diabetes. To view some of the findings of the formative research on the Hispanic/Latino population (DB_English_Hispanic_Latino_Fact_Sheet.pdf). |
|
Step 1.8 Assess factors and variables that can affect the project's direction. |
|
Strengths: Diabetes has been self-identified by many Hispanic and Latino communities as an important health problem. The Centers for Disease Control and Prevention (CDC) and the National Institutes of Health (NIH), National Institute of Diabetes and Digestive and Kidney Diseases (NIDDK), as well as more than 30 other organizations, are all working together and combining knowledge and experience to promote awareness of diabetes and its risk factors. Much research has been conducted and is available to understand the incidence and prevalence of diabetes and the implications of diabetes within the cultural setting of Hispanic and Latino American communities. Weaknesses: Research reports that Hispanic/Latino adults are often unable to see a doctor because of cost and that culturally or linguistically tailored health care is often not available to Hispanic/Latino individuals. Opportunities: Many health and public health organizations have become conscious of the growing need to have culturally and linguistically appropriate diabetes education made available to the increasing Hispanic and Latino populations. There are many underutilized prevention opportunities to address the burden of diabetes. Threats: A strong cultural belief among many Hispanic and Latino Americans is that "What happens to me is God's will." This belief can often interfere with prevention practices and a person's active participation in their care. |
|
|
|
Step 2.1 List the direct and indirect causes of each subproblem that may require intervention(s). |
|
The Audience Profile
Report developed by a CDC contractor) for the National Diabetes Education
Program, indicates the following factors that contribute to the prevalence
of diabetes and its complications within the Hispanic and Latino population: Subproblem 1 (self-monitoring of blood glucose):
Subproblem 2 (formal diabetes education):
Subproblem 3 (awareness of ability to control diabetes and importance of diabetes control in preventing complications):
To view the report DB_Hispanic_Latino_and_Diabetes-Audience_Profile.pdf. |
|
Step 2.2 Prioritize and select subproblems that need intervention(s). |
|
Subproblems that were identified as possible targets for intervention for this population included:
|
|
Step 2.3 Write goals for each subproblem. |
|
The following overall goals were developed to address the subproblems identified. However, (as described in Step 2.6) only the last goal was a communication goal that would be directly addressed through the "Thunder and Lightning" communication campaign.
|
|
Step 2.4 Examine relevant theories and best practices for potential intervention(s). |
|
The following intervention methods were considered for an overall public health program to address all the goals listed above. Health Communication/Education
Health Policy/Enforcement (cultural/linguistic issues)
Health Policy/Enforcement (access)
Health-Related Community Services
|
|
Step 2.5 Consider SWOT and ethics of intervention options. |
|
Health Communication/Education Strength
Weakness
Opportunity
Threat
Health Policy/Enforcement (cultural/linguistic issues) Strength
Weakness
Opportunity
Threat
Health Policy/Enforcement (access) Strength
Weakness
Opportunity
Threat
Health-Related Community Services Strength
Weakness
Opportunity
Threat
Ethical Considerations
|
|
Step 2.6 For each subproblem, select the intervention(s) you plan to use. |
|
All of the above types of interventions were considered, however health communication interventions were selected as the initial focus to address the last subproblem listed above, related to increasing the knowledge and awareness among Hispanic/Latino adults that they can take control of their diabetes. Health communication strategies work best in changing awareness, knowledge, attitudes, and simple behaviors, especially if they are science-based, theory-driven, grounded in audience research, and implemented sufficiently during the planning, implementation and evaluation processes. |
|
Step 2.7 Explore additional resources and new partners. |
|
Additional partners were added including Hispanic/Latino community leaders, Spanish language media (since a communication campaign had been an identified intervention), and various contractors. Additional resources were secured to fund a number of national minority organizations to assist with implementing interventions and carrying the NDEP messages directly to high risk communities. More CDC staff were also involved at this time. |
|
Step 2.8 Acquire funding and solidify partnerships. |
|
A national conference with all program participants occurred prior to the kickoff of the National Diabetes Education Program general awareness campaign. Here, partners learned how they could each contribute to the development, implementation, and evaluation of the NDEP campaign and about opportunities to participate in the development of campaigns for specific high risk groups such as Hispanic/Latino populations. Directors from the National Institutes of Health and the Centers for Disease Control and Prevention were available during and after the conference to answer any questions and offer support to the partners. Communication was maintained primarily through e-mail, monthly teleconferences, and print. |
|
|
|
Step 3.1 For each subproblem, determine if intervention is dominant or supportive. |
|
Health Communication/Health Education
While other interventions were not to be implemented at this time, it was determined that communication could also be used to support other types of interventions in the future, as described below. Health Policy/Enforcement (cultural/linguistic issues)
Health Policy/Enforcement (access)
Health Related Community Services
|
|
Step 3.2 Determine whether potential audiences contain any subgroups (audience segments). |
|
The following segmentation of the target population was considered:
|
|
Step 3.3 Finalize intended audiences. |
|
Because of an increase in cost for each additional audience segmentation, two distinct priority audiences were selected for the intervention.
* Materials developed for these audiences included those targeted to Hispanic/Latino men and women of varying ages. |
|
Step 3.4 Write communication goals for each audience segment. |
|
The overall communication goal of the campaign was to:
The communication goal was the same for each audience segment. However, the medium in which the message was delivered was different. For example, the message was delivered in both print and audio/visual format (TV and radio). Additionally, all messages (print, radio, TV) were in both Spanish and English. Other related goals were identified to expand on the above goal. These behavioral goals are more directly linked to improved glucose control:
These last three goals were not expected to be accomplished through the Thunder and Lightning campaign alone. Secondary messages would begin to address these issues, and planners determined that these goals could be dealt with in more depth through future interventions, including additional campaigns. Note: The next wave of NDEP campaigns for this audience included messages about nutrition and dietary habits. It included a meal planner and PSAs as well as other elements. For additional information about this campaign http://ndep.nih.gov/materials/puborder/resource.htm. |
|
Step 3.5 Examine and decide on communication-relevant theories and models. |
|
Social Marketing and Social Cognitive theories were used in the development of this intervention. In relation to promoting the concept that a person with diabetes can control their diabetes and to empower them to take control, two constructs from Social Cognitive Theory were especially relevant: self efficacy, a judgment of ones capability to accomplish a certain level of performance, and; outcome expectation, a judgment of the likely consequence such behavior will produce. For more information about how to choose a theory that applies to an intervention Theory_at_a_glance.pdf. |
|
Step 3.6 Undertake formative research. |
|
A CDC contractor obtained primary data by conducting multiple focus groups with the selected target audience to determine the appropriate message settings, communication channels, and materials, the best practices/strategies needed, and to identify those threats and barriers perceived by the target population. To view report DB_Hispanic_Latino_and_Diabetes-Audience_Profile.pdf. Additionally, secondary data was collected from the Centers for Disease Control and Prevention, National Institutes of Health and other agencies with access to valuable information. This information was used to validate the data collected from the focus groups and to better understand the target audience and those intervention strategies that have previously worked. |
|
Step 3.7 Write profiles for each audience segment. |
|
[Both audience segments have the same profile; however, the delivery of the message used multiple media (Spanish/English print versus Spanish/English radio and TV) to accommodate the needs of each audience.] Primary Message: You can control your diabetes. Secondary Message(s): these were only communicated through longer print ad formats
Demographics:
Cultural Norms:
Media Habits and Preferences:
Settings: Messages should, if possible, be conveyed in community-based settings, such as
Activities:
For more information on the Hispanic/Latino audience profile DB_Hispanic_Latino_and_Diabetes-Audience_Profile.pdf. |
|
Step 3.8 Rewrite goals as measurable communication objectives. |
|
The goals below were drafted to expand on the goal above and to help frame future interventions:
|
|
Step 3.9 Write creative briefs. |
|
Target Audience Hispanic/Latinos who speak Spanish as their primary language and those with higher levels of literacy versus those with lower levels of literacy. Objectives
As stated above, it was determined that additional objectives should be addressed through subsequent interventions:
Obstacles
Key Promise If I take control of my diabetes, I will feel happier and healthier. Support Statements/Reasons Why
Tone Culturally sensitive and serious Media Radio, television, posters and print PSAs. Creative Considerations Radio, television and print PSAs should be in both Spanish and English. To view Creative Brief DB_NDEP_creative_brief.pdf. |
|
Step 3.10 Confirm plans with stakeholders. |
|
A meeting with the stakeholders was held to discuss the results of the formative research and all planned communication efforts. The creative brief was shared with all partners, and plans were discussed for the next phases of the campaign. Stakeholders were asked what information they hoped to attain by implementing this campaign and what their desired outcomes were. Their responses led to a discussion of the need for an evaluation of the campaign. After receiving approval for all planned communication efforts, a timeline was drafted for creation of the messages and communication products, implementation of the campaign, and developing and implementing the evaluation strategy. |
|
|
|
Step 4.1 Draft timetable, budget, and plan for developing and testing communication mix. |
|
All partners and stakeholders agreed upon and finalized the timeline and budget for testing the communication messages. Stakeholders were invited to observe the concept and message pre-testing. |
|
Step 4.2 Develop and test creative concepts. |
|
Four creative concepts, which focused on the idea that a Hispanic/Latino person with diabetes can control his or her diabetes through blood glucose control, were developed from the formative research by the contractor and CDC staff, with input from the workgroup. A total of 8 focus groups (one all-male focus group and one all-female focus group in each of 4 major Hispanic markets: Miami, Florida; San Juan, Puerto Rico; New York, New York; and Los Angeles, California) were conducted by a contractor to test these creative concepts. The four concepts were:
Lightning bolt-'Cosas que no se pueden controlar? Muchas! Pero la diabetes no es una de ellas. Entienda su diabetes y comience a controlarla.' (Are there things you can't control? Many, but diabetes is not one of them. Understand your diabetes and begin to control it.)
Antonio-'Antonio tiene mil y un motives para tomar su diabetes muy en serio. Entienda su diabetes. Evite los riesgos.' (Antonio has a thousand and one reasons to take his diabetes seriously. Understand your diabetes. Avoid the risks.)
Family-'No hay mal que por bien no venga. La diabetes me obligo a cuidar mi salud y la de mi familia. Conzcase...Conozca su diabetes.' (Every cloud has a silver lining. Diabetes has caused me to look at my health and the health of my family. Know yourself. Know your diabetes.)
Female-'Conozco y controlo mi diabetes ya no soy carga para nadie. Tome su diabetes en serio para que no se vuelva cosa seria.' (I recognize and control my diabetes. I am no longer a burden for anyone. Take your diabetes seriously so that it doesn't turn into something serious.)
It was concluded that overall, the message for the campaign should be simple and one that lends efficacy to control. |
|
Step 4.3 Develop and pretest messages. |
|
Of the four previous concepts, the "Lightning bolt" concept was further developed into pretest materials. Additional focus groups were conducted to obtain feedback on the visual treatment and on the proposed message in both English and Spanish. Results indicated the message was well received in both languages and that no changes were needed. |
|
Step 4.4 Pretest and select settings. |
|
Focus group data from the formative research and additional market research data indicated the following settings would be most appropriate for this campaign message:
|
|
Step 4.5 Select, integrate, and test channel-specific communication activities. |
|
On the basis of focus group data from the formative research, the most appropriate channels for this campaign were television, radio, and print (pamphlets, brochures, newspaper articles, etc.). Additionally, a 'Community Guide for Partners' and 'Campaign Guide for Partners' were developed to walk local organizations through the steps of disseminating the message to their community members. To view the 'NDEP Community Guide for Partners' DB_diabetes_community_partnership_guide.pdf. To order a copy of the 'NDEP Control Your Diabetes For Life Campaign Guide for Partners' http://ndep.nih.gov/materials/puborder/resource.htm. |
|
Step 4.6 Identify and/or develop, pretest, and select materials. |
|
Final versions of all print, radio, and television PSAs were tested among focus groups. Data indicated that participants felt the materials were clear and easy to understand. Further, the message was perceived to be very motivating. |
|
Step 4.7 Decide on roles and responsibilities of staff and partners. |
|
The communication plan was disseminated among internal staff at the Centers for Disease Control and Prevention and the National Institutes of Health. Roles and responsibilities for staff, contractors, and workgroup members were identified. Additionally, the communication plan was disseminated among external partners at the state, city, and community levels. To aid the implementation of the campaign, a media workshop guide and a National Diabetes Education Program website address were distributed to all partners. |
|
Step 4.8 Produce materials for dissemination. |
|
Final clearance for the campaign materials was obtained and the distribution began. All external partners were given a media kit. This media kit included:
|
|
Step 4.9 Finalize and briefly summarize the communication plan. |
|
Public health surveillance of diabetes has shown that minority groups and the elderly are disproportionately affected. A CDC contractor reported that Hispanic and Latino Americans have the second highest rate of type 2 diabetes compared with other populations. Furthermore, National Diabetes Education Program data indicate that Hispanic and Latino Americans are at higher risk of developing and dying from diabetes and are two times as likely to have complications such as heart disease, blindness, high blood pressure, amputations, nerve damage, and kidney disease than non-Hispanics. Healthy People 2010 indicate that the proportion of Hispanic or Latino adults with diabetes who perform self-monitoring of blood glucose at least once a day is 36 percent compared with 45 percent non-Hispanic white adults with diabetes. The risk of developing complications of diabetes rises when blood glucose is not kept under control. The incidence of diabetes and its complications within the Hispanic/Latino population can be expected to substantially increase unless prevention and control practices can be established within this population. The audience selected for this campaign was Hispanic/Latino persons who speak Spanish as their primary language and those with higher levels of literacy versus those with lower levels of literacy. The communication objective was to promote the concept that a person with diabetes can control his or her diabetes. The final message, titled "Thunder and Lightning," selected for this campaign was designed to be both culturally sensitive and serious. "There are many things in life that can't be controlled. Fortunately Diabetes isn't one of them." The body copy in the print PSA goes on to say: "Millions of us are living with diabetes. For many, it runs in our families. But we don't let that run our lives. Because diabetes can be controlled with the proper diet, exercise, and knowing and managing our blood sugar numbers. The more we learn about how to live with diabetes and avoid the many serious complications, the longer and better we'll live. And there are so many things to live for. Take your diabetes seriously, so it never becomes too serious." The settings for this campaign included community-based organizations such as community recreation centers, malls, churches, homes, billboards, and waiting areas such as bus stops or a doctor's office. The channels selected for this campaign were television, radio, and print (pamphlets, brochures, newspaper articles, etc.). Additional activities conducted by some community organizations included food demonstrations and health fairs with music and dancing. The previously agreed upon communication plan, timeline and budget were disseminated among internal and external partners. Once the final clearance of the campaign materials was obtained, distribution of the materials and a community guide for partners began. All external partners were given a media kit and the guide to help with the implementation of the campaign. |
|
Step 4.10 Share and confirm communication plan with appropriate stakeholders. |
|
A meeting of all stakeholders and key partners was held to share the summary of the communication plan and campaign materials. Buy-in was confirmed with the stakeholders, and campaign kickoff responsibilities were finalized. |
|
|
|
Step 5.1 Identify and engage stakeholders. |
|
The NDEP identified 2 types of stakeholders:
Each of these stakeholders needed to receive information from the NDEP evaluation as it developed and both process and impact findings were included. Level 2 stakeholders and some Level 1 stakeholders were involved in developing and sustaining the NDEP through their support and ongoing activities in local communities; therefore process evaluation findings were of the greatest importance to this group. The majority of Level 1 stakeholders were responsible for policy and funding decisions; therefore, the extent to which NDEP achieved its intended impact objectives was of importance to this group. |
|
Step 5.2 Describe the program. |
|
The primary goal of the campaign was to:
Future interventions would be developed to:
To achieve the primary goal, the following intervention was developed: The campaign materials were developed and pre-tested with the target audience. The data from focus group discussions indicated that the campaign materials were considered to be culturally sensitive and well received in both English and Spanish. Additional focus group data from four leading Hispanic/Latino markets indicated the campaign message, "Thunder and Lightning," was the most appropriate concept for the Hispanic/Latino community. Television, radio and print PSAs , as well as pamphlets and brochures, were developed from the "Thunder and Lightning" concept in English and Spanish and were distributed nationally to NDEP partners for implementation. |
|
Step 5.3 Determine what information stakeholders need and when they need it. |
|
The process evaluation measured what NDEP was doing and monitored its progress in order to identify promising and problematic activities. This information helped to determine if the program was on course for achieving its objectives. The following process evaluation questions were of importance to the stakeholders:
Impact evaluation questions and measures were designed to determine the program's influence on its intended beneficiaries as a result of direct programmatic activities. The impact evaluation focused upon the result of the activities of the NDEP and its partner organizations on the target audiences. Impact evaluation wanted to answer the following questions:
Additional questions of interest would need to be explored in the future, after subsequent interventions were implemented:
For more information about how to determine evaluation measures http://www.cdc.gov/epo/mmwr/preview/mmwrhtml/rr4811a1.htm. |
|
Step 5.4 Write intervention standards that correspond with the different types of evaluation. |
|
Additional intervention standards were tentatively planned to correspond to the previously listed future goals:
|
|
Step 5.5 Determine sources and methods that will be used to gather data. |
|
Both process evaluation (to study the functioning of components of the campaign) and impact evaluation (to determine the extent to which the campaign was accomplishing the stated objectives) were used in NDEP evaluation. The evaluation plan detailed two types of measures, evolving measures and core measures. Evolving measures changed over time to reflect the changing activities of the NDEP and its partners. Because all of the process measures in the evaluation were evolving measures specific to the program, they were gathered through primary data collection methods developed for this purpose. Core measures included impact measures that were assessed at intervals throughout the program. To collect process data from work groups, partners, and other stakeholders, a progress reporting system was developed. Work groups and other NDEP stakeholders were asked to provide data to the NDEP evaluation committee and contractors. The progress reporting system included the following primary components:
Additional process measures included:
Collection of process measurements included media tracking reports; monitoring clipping services; conducting a series of in-depth interviews with key stakeholders; analyzing any available quantitative data on the partner's activities; and reviewing relevant documentation, including memos, letters, reports, press releases, etc. These data were supplemented by direct observation of the program 'in operation', through site visits to partners or stakeholders. Observers collected data on factors such as management strategies, staff skills and activities, application of policies and procedures, and problems and solutions. Successful completion of the activities measured in the process evaluation was expected to have an impact on diabetes control awareness, attitudes, and behaviors of the target audience (Hispanic/Latino persons with diabetes and their families). The impact evaluation measured these effects through a variety of measurements obtained in part through surveys such as Healthstyles and the Behavioral Risk Factor Surveillance System. Some impact measures included:
Additional related impacts to focus on in future interventions were:
For more information about how to determine evaluation measures http://www.cdc.gov/epo/mmwr/preview/mmwrhtml/rr4811a1.htm. |
|
Step 5.6 Develop an evaluation design. |
|
The NDEP primarily used existing secondary data from credible sources such as the Behavioral Risk Factor Surveillance System, the National Vital Statistics System, the National Health and Nutrition Examination Survey and the National Health Interview Survey for evaluation. The NDEP further used primary sources to fill in any 'gaps' the secondary data left out. A classical experimental design was used to collect primary data, with pre/post-test measurements taken from the treatment groups versus a similar-comparison group. In addition, a mixed methods approach that entailed the use of both qualitative and quantitative measures was used. |
|
Step 5.7 Develop a data analysis and reporting plan. |
|
To adequately examine the process and impact measures among the target audience, partners, and stakeholders, a variety of evaluation methods and analysis techniques were employed. Quantitative data from surveys and secondary databases were analyzed using descriptive and inferential techniques. Basic descriptive information about measurements such as totals, means, percentages, and changes in these numbers were reported. Cross-tabulations showing the relationship between two variables of interest were also presented. Inferential techniques, such as analysis of variance, were also used. Repeated measures ANOVAs were used for quantitative data collected at multiple time points. Examination of the qualitative data from interviews and focus groups identified common themes within and across the audience. The relative frequency of these themes among those participating was reported. Further, the themes were compared and contrasted among different audience members. Reports were shared with the workgroup members for discussion. NDEP developed written progress reports and provided annual updates at the partnership network meetings. |
|
Step 5.8 Formalize agreements and develop an internal and external communication plan. |
|
Formal agreements between both internal and external stakeholders were finalized and individual roles and procedures to be used during the evaluation process were clarified. Information will be disseminated to partners and staff through inter-office mail, email, faxes, and conference calls. Additionally, formal reports at the end of each evaluation period were disseminated to Level 1 stakeholders within 3 months of publication and within 6 months of publication for Level 2 stakeholders. |
|
Step 5.9 Develop an evaluation timetable and budget. |
|
The NDEP campaign is a continuous campaign. Each year, the objectives and delivered messages change based on previous years' evaluation data and on current audience research. For this campaign, the kickoff occurred at the beginning of the year 2000. The complete timeline and budget for the evaluation was approved by all stakeholders and disseminated. Contractors and grantees were required to submit evaluation plans and budgets for their proposed NDEP activities. Partners were encouraged to incorporate evaluation components into their activities. NDEP also established a separate evaluation workgroup to assist in overall evaluation efforts. |
|
Step 5.10 Summarize the evaluation implementation plan and share it with staff and stakeholders. |
|
The evaluation implementation plan included process evaluation and impact evaluation for Level 1 and Level 2 stakeholders. The process evaluation findings indicated what strategies were the most successful, and the impact evaluation findings helped to determine what impacts have been made within the diabetes community. A formal report of all evaluation findings was written and published at the end of each evaluation period. |
|
|
|
Step 6.1 Integrate communication and evaluation plans. |
|
A meeting with stakeholders, key partners, and staff was held to reaffirm team members' roles and responsibilities in the integration of the communication and evaluation plans. Persons responsible for communication activities updated all team members on when the communication campaign would begin and how the campaign would be executed. Those responsible for implementation of the evaluation plan brought the team up to date on the collection of baseline measures. Additionally, a series of meetings was arranged throughout the communication and evaluation phases of the campaign to keep all team members informed of the campaign's progress and to acquire feedback. |
|
Step 6.2 Execute communication and evaluation plans. |
|
The communication and evaluation plans were launched simultaneously. The communication campaign was launched with numerous activities including press releases and media spotlights. Evaluation data collection occurred at all phases throughout the campaign. |
|
Step 6.3 Manage the communication and evaluation activities. |
|
During the many phases of the campaign, stakeholders, key partners and staff met and monitored political, social, and health environments surrounding the campaign that could affect its success. Any threats to or weaknesses in the campaign were identified, and proper adjustments were put into action. |
|
Step 6.4 Document feedback and lessons learned. |
|
Stakeholders, key partners and staff were asked to submit reports indicating what each individual felt were the strengths and weaknesses of the communication campaign. Specifically, each team member was asked to evaluate, as objectively as possible, the planning phase and the implementation phase of the communication campaign. Results from these reports were compiled and recommendations were made for future campaigns. It was determined that the next campaign should address nutrition issues for the Hispanic/Latino population with diabetes and the Control Your Diabetes. Its more than food. Its life campaign has been developed. To order materials for that campaign http://ndep.nih.gov/materials/puborder/resource.htm. |
|
Step 6.5 Modify program components based on evaluation feedback. |
|
Feedback from the evaluation of the communication campaign indicated that only minor adjustments needed to be made to the current campaign and that plans to expand the scope of the campaign to include specific messages about nutrition should proceed. |
|
Step 6.6 Disseminate lessons learned and evaluation findings. |
|
A final report of all campaign activities and evaluation results was written. Because not every team member needed a complete comprehensive report, executive summaries were issued. Additionally, the highlights of the campaign were presented at a meeting with all stakeholders, key partners, and staff. |