Program-defined outcomes and user-centered value: Measuring value for “push” and “pull” public engagement programs

One way to think about public engagement that can be useful in identifying impacts and value is identifying the “push” and “pull” aspects of a program or a program element.  A program has “push” characteristics when it aims for specific outcomes defined by the program, typically related to knowledge, behavior, policy, or environment changesIt has “pull” characteristics when its goals are more open-ended and targeted to the provision of a specific service or resource that is then sought out, requested, or “pulled” by the user.   Each of these models is appropriate in different circumstances, many programs include aspects of each, and each is compatible with reciprocal community partnerships, but the appropriate measures of impact and value differ between them.

“Push” activities with program-defined goals include the dissemination of targeted messages or sets of knowledge generally with a specific goal involving a change in the audience’s awareness, attitude, knowledge, ability, or behavior.   In this model, the program uses public engagement/outreach to “get the word out.”  In a university context, the message or knowledge base is often tied to new research or discovery, as exemplified by the classic Extension model and in some “broader impacts” activities identified as a part of NSF grants.  Translational research and op-ed pieces also fit into this category.  Usually there is a targeted public interest outcome, e.g. changes in attitudes on vaccines; change in knowledge about sustainability practices, change in critical thinking abilities, or increase in healthy behaviors.  Associated longer-term target impacts might include reduced incidence of communicable disease, or reduced environmental damage from nutrient runoff or greenhouse gas emissions.

In a “pull” model, by contrast, user-defined value plays a central role.  The program identifies the public interest and develops the program resources, while participants such as residents, community organizations, or businesses actively seek out or initiate access to information, services, or resources in order to meet user-defined goals.  Public engagement activities that fit a “pull” model include developing web-based resources findable with a search engine, providing curricula and courses, and providing a service, facility, or other resource. These engagement activities support “pull” actions by the public such as searching for information on the web, attending a performance, calling a campus-based expert, requesting a service, and using a campus-based facility or resource such as a museum, library, or maker lab.  Target outcomes for these activities might include changes in attitude, awareness, knowledge, ability, or behavior, or they might focus on more open-ended goals such as increased access, increased innovation, increased ability to make an informed decision, or increased connection to community.   Longer-term impacts, such as improved health as a result of a maker-lab supported innovation, may also be part of the program goals.

Many programs have both push and pull elements. Whether a program leans more to the push end or the pull end will likely be informed by the public interest motivating the program.  If it is particularly compelling and urgent that the information be used as broadly as possible, push strategies using effective persuasive, educational, or science communication techniques might be most applicable.  If the public interest aligns most closely with audience-led access and audience-motivated goals, then the most effective delivery might rely more on pull-friendly modes such as on-demand information, services, resources, and facilities.

Recognizing the push and pull features of different programs or for elements within a program helps when creating measurement systems and discussing impacts.  Both “push” and “pull” programs will want to measure the extent to which program goals such as those suggested above were met.  In addition to assessing the specific target outcomes for the program, “pull” programs will want to assess outcomes related to achievement of the users’ goals and the value they received.

ElementsPush programs (program-defined outcomes)Pull programs (user-centered value)
Program purpose/overall goalOften more specific, e.g. increase public knowledge of a social injustice or science factsOften more general, e.g. increase innovation
Targeted resultsSpecific change; e.g. adoption of a recommended practice or specific attitude/beliefVaried; may be open-ended; linked to user’s goals (e.g. user improved knowledge or understanding; stimulated creativity; satisfied an interest)
Examples of community engagementJoint identification of target goals; strategies, and/or activities; partnering to identify and reach targeted groupsJoint identification of target goals; strategies, and/or activities; partnering to identify and reach targeted groups
AND/OR
Community organizations empowered to access and use resources to meet their needs
DeliveryFocus on persuasion or campaign, e.g. based on science communication best practices; can include many types of delivery channels/elementsIncludes on-demand elements, e.g. resource that is accessible all the time (e.g. website) or through ongoing regular hours (e.g. museum or library); can include services and many types of delivery channels/elements
Sample survey questions·    Please rate your level of knowledge on x before and after attending the program (very low/low/moderate/high/very high)
· Please rate the likelihood that you will do x as a result of attending the program
· Which of the following practices have you done as a result of attending the program?
·    Were you able to access the information or resources you needed?
· Please rate your ability to make an informed decision after using the service or resource.
· Please rate the extent to which your goals in using this resource or service were met. (not at all/a little/ somewhat/well/very well)
· How did this resource help you?  
ExamplesOp-ed pieces; dissemination plans; many Extension programs; research translation; news releases; activities promoting specific practices such as health and wellness; sustainable home or business actionsMuseums; performing arts; library; maker labs; services; courses; web-based information; labs/plants with public or industry access; Extension expert assistance