Establish goals

Play 1: Clearly define and communicate your objectives

Federal agencies understand the importance of meaningful public engagement. The type of engagement will differ for each organization, depending on resources, audience and mission. Every agency must set goals that align with its unique definition of success and empowers the public to engage with agencies in order to influence government priorities.

Checklist

• Identify what goal(s) you aim to achieve.
• Develop a plan that includes a timeline, and strategy – ramp up and drawdown.
• Evaluate organizational capacity for engaging and managing public participation.
• Identify internal and external stakeholders and partners.
• Evaluate and change strategy based on feedback and performance data.

Case Studies

• ePolicyWorks is an initiative by the Department of Labor's Office of Disability Employment Policy. The tool is designed around a clear policy-making objective and addresses specific challenges in collaborative policy making.
• The National Archives process for developing regulations uses theRegulations.gov portal during the commenting period to consider public input and make relevant, appropriate revisions. Regulations.gov is a portal that simplifies finding, reviewing, and submitting comments on Rules and Proposed Rules that appear in the Federal Register.

Resources

• The Environmental Protection Agency Public Participation Guide contains numerous resources, including an excellent discussion of situation assessments.
• Synthesis 89, by Scott Giering, discusses The Transit Cooperative Research Program's efforts to coordinate effective public participation strategies for transit, e.g. goal-setting, information exchange, and identifying "the public."
• The Department of Transportation's Federal Highway Administration provides aTransportation Professional Capacity Building site with many resources andcase studies.

Metrics: How Do You Know You Successfully Established Goals?

Metrics will vary according to the platform you use. Some broad examples include:

• Surveys – responses should drive learning and improvement.
• Web – Google Analytics or heat-mapping shows participant engagement.
• Email – click rates, bounce rates, unsubscribes, participation in a call-to-action.
• Social Media – participant engagement analytics, reach, changing behavior, hashtag use.
• Apps (native and mobile) – use statistics, are "transactions" being completed, is there input or feedback from the field (i.e., uploading multiple photos in citizen science app), are people actively using a native app (i.e., logging in multiple times).
• Online Video Chats (i.e., hangouts) – participation rate, views.
• Discussion Threads (i.e., chat rooms) – number of participants, input from the public, up votes / down votes.
• Blog – viewership, comments, action audience takes next.
• Transactional – hard numbers related to online transaction completion.
• Rates of completion for public participation opportunities (e.g., Comment form submissions to federal agencies)
• Rates of API participation and usage, e.g., number of API implementations, queries received via API, successful submissions received via API.

Understand the Playing Field

Play 2: Understand your participants and stakeholder groups

Community and stakeholder understanding is key to organizing a successful participatory effort. Once you determine who you're trying to reach, you can refine your outreach efforts to effectively communicate with participants and stakeholders.

Checklist

• Create personas of your target participants in order to understand their needs.
• Conduct sampling and initial outreach to better define and understand participants.
• Use groupwork methods to draw representations based on observation and not assumption.
• Identify obstacles you must overcome to reach participants, including connectivity and schedules.
• Determine how you will reach diverse groups of people with broad perspectives.
• Identify gatekeepers, including community groups, who can help reach potential participants.
• Customize your engagement strategies for each type of stakeholder.
• Meet your audience where they are, not where you want them to be.

Case Studies

• Peace Corps Application Process Redesign: Peace Corps redesigned their application process to meet the expectations of their target audiences. A shortened application allowed applicants to choose which countries they want to serve in. This receptivity to the audience's needs and experiences resulted in record-breaking application numbers for Peace Corps.
• The National Geospatial Intelligence Agency opened unclassified geospatial intelligence information to the general public through theirEbola Relief Website. They recognized an unprecedented need for a large number of stakeholders to quickly access information that can help NGO's and other workers battle the virus.

Resources

• The CDC: Gateway to Health Communications & Social Marketing Practicewebsite provides resources to build health communication or social marketing programs. It provides tips for analyzing and segmenting an audience, choosing appropriate channels and tools, and evaluating the success of messages or campaigns.

Play 3: Understand and communicate the benefit of participation

Participants must understand how they can contribute to a program and why it is important they participate. Clearly defined and communicated benefits help craft effective messages and inclusive engagement. Understanding what successful participation looks like is critical to knowing whether your program succeeds.

Checklist

• Decide what essential information participants need to know before engagement.
• Build a communications strategy that considers the expectations and motivations of participants and the value of participation.
• Involve senior leadership and ensure they understand the importance of participation.
• Provide inclusive access to orientation materials.
• Define how participants benefit from the process and what they can contribute, including information, permission, and time.
• Inform participants when they should expect to hear follow-up from their engagement.
• Report how participation impacted program efforts.

Case Studies

• National Day of Civic Hacking organizers submit problem statements to explain an issue, explain how the public can address it, and provide resources to help orient people to the problem. They also outline time commitments and skills needed.
• Regulation Room is a pilot project sponsored by the Federal Government and operated by the Cornell e-Rulemaking Initiative. Organizers provide participants resources to better understand the rulemaking process and teach them how to make an effective comment. Participants comment on a report after discussion ends to ensure the program captures ideas before the Government receives a final version.
• Give a Minute is a micro-participation/crowdsourcing event with clear expectations for participants. Moderators collect and share ideas both digitally and in-person.
• NASA Socials are events where NASA's social media followers can learn about NASA's missions, people, and programs. Through these behind-the-scenes experiences, participants are empowered to advocate for the agency and attend future events.

Play 4: Empower participants through public/private partnership

Leveraging corporate and community partners is a powerful way of gaining support and participation for government campaigns and events. Partners and sponsors can help reach new audiences or reinforce messages within existing ones. Public/private partnerships also legitimize both what your agency is trying to accomplish and how that information or program is delivered.

Checklist

• Define and create partnerships.
• Survey agency employees for knowledge of community and stakeholder groups.
• Reach out to nonprofit organizations and community partners who can amplify participation opportunities and connect your agency with additional partners.
• Use a Memorandum of Understanding (MOU), an Interagency Agency Agreement (IAA), or another type of contractual agreement to describe each partner's responsibilities.
• Outline the terms and the specifications of the partnership, including messaging, logo usage, promotional opportunities and boundaries.
• Create an "authorized partners" list to reduce legal issues

Case Studies

• The Heart Truth® campaign, created by the National Heart Lung and Blood Institute (NHLBI), partners with dozens of corporate and community sponsors. The campaign raises awareness about heart disease and its effect on women through both online and in-person fitness events.
• In 2014 LabTV launched its vision of creating a free, scientist-to-student web/video platform aimed at inspiring the next generation of researchers. The National Institutes of Health (NIH) worked with LabTV to produce hundreds of videos of NIH researchers with varying backgrounds and interests, and matched students with scientists who can share information about life as a medical researcher.
• A coalition of private companies, non-profit organizations, and the Department of Homeland Security (DHS) createdSTOP.THINK.CONNECT.™ Partnering with industry leaders, DHS attracts a broader audience and garners significant public participation for online events such as Twitter chats to help spread its message.
• The U.S. Department of Transportation (DOT) Distraction.gov website is dedicated to stamping out distracted driving. DOT enlisted the help of several organizations to spread its message. One partner, the FOX television show "Glee", helped produce public service announcements that feature characters from the show.

Metrics: How do you know you understand the playing field?

Measure:

• Total number of participants.
• Number of participants needed to establish a representative sample size, as well as primary and secondary groups.
• Number of participants from primary and secondary target groups.
• Number of new participants.
• Number of returning participants.
• Percentage of participants satisfied with the "customer experience."
• Rates of conversion, e.g. from contacts > visitors > signups > contributions.
• Volume of sharing, recruitment activity or other promotion.
• Quantify the quality and effectiveness of participation, e.g. relevance of feedback.
• Quantify the value and in-kind services donated by public/private partnerships, e.g. air-time, printing, advertising, or prizes.
• Numerical comparison of current participation with previous efforts and non-partnership activities.

Design Participation

Play 5: Select appropriate design format for public participation

Enable broad participation by offering responsive, accessible, intuitive, mobile-ready tools that can be rendered in multiple formats on a wide range of platforms.

Checklist

• Identify critical and secondary requirements for program success.
• Determine paths for participation and their requirements, such as accessibility, commenting and annotation.
• Design for flexibility using current coding techniques such as HTML5 and CSS.
• Incorporate accessible graphics, color and visual techniques to improve engagement.
• Use analytics and feedback to diversify delivery methods, such as mobile.
• Adopt mobile-friendly capabilities such as responsive web design and link to mobile friendly pages.
• Communicate through channels like SMS, Interactive Voice Responses and Unstructured Supplementary Service Data for participants without wireless connectivity or access to the Internet.
• Continuously improve operations during the life of the project, and to focus on quality, efficiency and effectiveness of processes.

Case Studies

• The White House Engage website allows citizens to submit questions and comments, join online events, and engage with government via social media.
• gov underwent a major redesign to improve participant experience and usability. They used analytics and lessons learned from prior redesigns to determine popular content and participant preferences. Thisblog entry provides an overview of their redesign process.
• The NOAA Release Mako Mobile App allows fishermen to report their releases of Shortfin Mako sharks while in the water. The app uses a device's built-in GPS to fill in exact location coordinates.

Resources

• The Victoria (Australia) Toolkit for Public Participation maintains a great selection of different PP formats.
• The Center for Land Use Education's PDF guide, Crafting an Effective Plan for Public Participation, provides a framework for tailoring a plan that fits local needs and capacities.

Play 6: Design for inclusiveness

Whenever you're running any sort of public engagement effort, make sure the design and setup are inclusive. Consider persons with disabilities who use screen readers, people with limited English proficiency or low literacy skills, and many others who may experience difficulty reading.

Checklist

• Provide accessibility options for persons with disabilities, the aging population and others.
• Evaluate the need for multilingual support, including Spanish language services.
• Use Plain Language communication throughout the life cycle of the program.
• Consider both online and offline support, including a physical version and digital package.
• Analyze if participants can engage without a facilitator.
• Analyze if participants can contribute with 'group input' rather than individual sign-up, and if they can submit input from face-to-face workshops and other sources.
• Provide cultural competency support that values diversity reflected in the participant personas.
• Talk to underrepresented members of communities.
• Recognize common dynamics when cultures interact and develop adaptations to service delivery.
• Design for multiple learning styles.
• Recruit volunteers for visual note taking, or graphic facilitation.
• Provide contact information for official support and accessibility teams.
• Test your content for accessibility at all stages of development.

Case Studies

• Food and Drug Administration: Using Medicines Wisely: Increasing Access to Information for Women with Intellectual Disabilities. FDA's Office of Women's Health teamed up with the Administration for Community Living, the Association of University Centers on Disability, and the Administration on Intellectual and Developmental Disabilities to improve their safe medication materials for women. They conducted a series of focus groups with women in the disability community to learn how word choices, font, layout, and graphics could be improved.
• The Digital Communications Division (DCD) in HHS champions a 508 Program that leads the development and review of HHS Web content, social media, and supporting technologies. They also assembled a 508 Accessibility Lab with a collection of assistive and evaluative technologies.
• Gov is the U.S. federal government website for information on disability programs and services nationwide and is a model for accessibility on the web.

Resources

• Guide to Section 508 Standards. The U.S. Access Board's Section 508 Standards apply to electronic and information technology procured by the federal government, including computer hardware and software, websites, phone systems, and copiers. Issued under Section 508 of the Rehabilitation Act, these require access to such technologies for both members of the public and federal employees.
• Gov contains resources for agencies to evaluate their websites for both usability and accessibility.
• The Improving the Accessibility of Social Media in Government toolkit curates and share best practices to help agencies ensure their social media content is accessible everyone, including h disabilities. Efforts are also being made to work with social media platform and tool developers, citizens and partners to encourage greater accessibility.
• The Department of Homeland Security developed 508 Compliance Test Processes that provide instructions for testing the compliance of applications, Word documents, and PDF documents.

Play 7: Provide multi-tiered paths to participation

Good government is responsive and engaging. To ensure broad participation and involve more citizens in the decision-making process use a variety of channels to communicate. It is important for agencies to be inclusive, and make maximum use of available resources to receive feedback from a wide audience.

Checklist

• Determine the level of participation your program needs to succeed.
• Empower stakeholders as decision makers and share information.
• Determine if you hold the resources to sustain a multi-tiered, two-way engagement strategy.
• Define what input you are looking for and what channels people are most likely to use.
• Notify employees of opportunities for participation and teach them how to respond if approached.

Case Studies

• The Army's Assembled Chemical Weapons Alternatives Programestablished two citizen advisory boards that make project recommendations. The program holds quarterly public meetings with program leadership and maintains an outreach office that's open to the public and receives in-person feedback.
• The U.S. Trade Representative's traveling roadshow that gathersfeedback on trade agreement negotiations from stakeholders: industry, small business, academia, labor unions, environmental groups, and consumer advocacy organizations.
• The Food and Drug Administration solicits multiple levels of participation through its Patient Network site. Participants can comment on proposed regulations and guidance, attend or speak at public meetings, and join an advisory committee to influence decisions like product approvals.

Resources

• The International Association for Public Participation publishes an IAP2 standards document that describes the various levels of public participation and examples of techniques.

Play 8: Provide effective and timely notifications

Throughout the life of an engagement effort, effective communication with participants and stakeholders is key. This play offers tips and advice for letting your audience know about opportunities to participate, and how to keep them engaged and active for the duration of the event.

Checklist

• Develop a comprehensive outreach plan that includes timelines to reach different potential participants.
• Use a variety of channels such as the Federal Register, SMS, postal, email, flyers and social media.
• Provide a toolkit of materials, including an outline of timelines.
• Deliver consistent updates and repeat them across multiple platforms.

Case Studies

• The Union of Concerned Scientists released a scorecard on how agencies allow employees to communicate with the public. USGS received a B+, but updated its public documentation to address suggested improvements within four hours.
• Federal Student Aid's #AskFAFSA campaign sets up monthly office hours for stakeholders to ask questions about student aid via Twitter. This is often faster than asking questions via telephone, and anyone following the hashtag can benefit from the answers. A recap of each session is posted the next day on Storify.

Resources

• DigitalGov Search seeks to provide immediate answers to the public's search questions. In addition to Federal Register documents, it also incorporates results from other specialized government websites and social media accounts.

Play 9: Encourage community building through responsive outreach

Community development and outreach involves communication amongst community stakeholders, partner organizations, research centers, and government agencies. Goals include communicating reliable information about federal programs and policies, including addressing complaints; obtaining public feedback about the impact of government activities; fostering conversations and incorporating feedback into the policymaking process; and strengthening the channels of communication between communities, the government, and public/ private sectors.

Checklist

• Determine shared decision-making responsibilities for success, problem solving, and accountability.

• Develop a dialogue process that:

  • ◦Clearly defines guidelines.
  • ◦Prepares for variations in commitment to participation.
  • ◦Includes time to pause, evaluate input and evolve.
  • ◦Catalyzes the discussion if engagement stalls.

• Moderate and facilitate conversations to encourage connections and discussion of ideas.

• Create regular reports to summarize contributions so people can see what others are saying.

• Publish your programs' points of contact for communities to access.

• Celebrate effectiveness and address critical feedback with participants.

• Continue the conversation and maintain relationships.

• Use the framework you created to develop future uses for the network.

Case Studies

• HHS Office on Women's Health encouraged community organizations around the country to organize events in celebration of National Women's Health Week 2014. They used the platform meetup.com so organizations could add their own events, and participants could search for other events in their area.
• FDA's Office of Women's Health launched the Pink Ribbon Sunday Mammography Awareness Program to educate African American and Hispanic women about early detection of breast cancer through mammography. By partnering with churches and community organizations, they reached over 100,000 women throughout the country.
• US Department of State Bureau of Educational and Cultural Affairs Exchange Program fosters community engagement through the arts. TheAmerican Arts Incubator uses new media and/or mural arts as a means to engage youth, artists, and underserved communities in Asia and address local issues.
• NASA's International Space Apps Challenge is an international collaboration focused on space exploration that takes place over 48 hours in cities around the world. Local communities create and lead events while the project is managed at the Agency level. City leads mobilize participation in their city, choosing challenges and models appropriate to local interests.
• Valley Forge National Historical Park transformed itself through civic input, as reported in "From Isolation to Integration"

Resources

• The National Park Service published two public participation guides, Beyond Outreach: A guide to engaging diverse communities. and Tribal and Public Consultation for Historic Preservation Toolkit
• Two Masters students at Harvard's Kennedy School of Government looked at social media and its role in civic participation in both the U.S. and United Kingdom and published a report, Using Social Media to Enhance Civic Engagement in Federal Agencies.

Play 10: Protect citizen privacy

Public participation is crucial to our democracy, but federal agencies must engage in ways that protect privacy, while maintaining compliance with federal laws such as the Federal Records Act (FRA) and the Privacy Act.

Checklist

• Identify what information your agency stores or maintains, and for how long.
• Publish privacy policies for all engagement channels so the public can see what information you collect, if any, and how you agency may use it.
• Publish terms of use detailing record-keeping practices and reasons why you may remove comments including profanity, sharing personal information and other rules that protect privacy.
• Consult with your agency's Records Officer to determine how to manage digital interactions and public comments as federal records.
• Discuss your privacy policies, terms of service, and other charter documents with your agency's Privacy Officer and/or legal experts.
• Establishing procedures for handling personally identifiable information (PII).
• Provide a medium for people to provide input anonymously.
• Use indirect means like hashtags to join conversations online instead of targeting individual users.

Case Studies

• The FEMA Privacy & Comment Policy comprehensively addresses how social media, web, mobile, and SMS data is used, stored, and managed. The policy is plain-language and concisely explains and justifies the retention and management of different data sets.
• The Peace Corps Privacy Act Notice is a plain language example of a comprehensive, brief, and easy-to-understand privacy policy for the web.

Resources

• CIO Council's Privacy Best Practices for Social Media: This comprehensive guide provides resources and best practices for creating social media policies that are in compliance with the Privacy and Records Keeping Act.
• White House Expands Data Protection: There is no formal guidance on how agencies can use big data. However, this report and recommendation from the White House provides insight on how and when agencies may be able to use big data from the Internet of things, the web, mobile, and more, and whether agencies will be required to extend universal Privacy Act Protections.

Metrics: How do you know you successfully design participation?

• Generate solid baseline data on how much your audience knows, and how well they responded to your message. This will help you determine whether public understanding of your message stayed the same or improved.

• Analyze whether the program becomes integrated with participants or owned by the community.

• Measure:

  • ◦Readership/visits/participation in channels, i.e. programs, sites and blogs.
  • ◦Awareness about the federal agency's goals, mission, purpose, or resources.
  • ◦Public participation (survey responses, event attendance, other contributions i.e., time and ideas).
  • ◦Volume of community organizations participating compared to previous programs.
  • ◦Size of community that your partners reach
  • ◦The role of public engagement in the planning/decision-making process.
  • ◦Variation in percentage between current overall participation and previous efforts
  • ◦Online engagement trends (e.g., comments and shares) over the entire length of the event.
  • ◦Engagement across each channel, independently, to determine the level of participation on each.

• For privacy protection, measure:

  • ◦Percentage of your programs, websites and social media sites contain provisions for protecting PII and maintaining Privacy Act compliance.
  • ◦Frequency that community managers monitor comments for PII.
  • ◦Volume of comments community managers delete for PII on a monthly or bi-monthly basis.
  • ◦Frequency of agency record audits to ensure it is not maintaining broader records than necessary.

• For accessibility, start with guidelines in:

  • ◦W3C Web Accessibility Metrics, a detailed report on useful metrics to assess the accessibility level of websites, including the accessibility level of individual websites, or even large-scale surveys of the accessibility of many websites.
  • ◦Section 508 Checklist, including Pass/Fail criteria for each of the Section 508 standards (e.g., a text equivalent for every non-test element, functional electronic forms).
  • ◦US Access Board guidelines and standards to help meet accessibility and inclusion standards, including Communications and IT.

Facilitate Participation

Play 11: Use Data to Drive Decision Making

Use data to inform decisions about engagement efforts and public participation. Data can help you decide the best format for your engagement effort, the best tools to use, the most receptive audiences and more. Data will help you get the desired results and adapt your engagement effort along the way.

Checklist

• Define your goals and establish key performance indicators.
• Continuously reconsider metrics to make sure they are still the best ones to use.
• Educate employees about open data policies and publicly available datasets.
• Ask data-literate employees to attend and participate in public engagements.
• Include a public feedback track into all decision-making processes–anything from project management to policy decisions–as early and often as possible.
• Assign a member of the team to routinely keep track of public feedback systems, such as, but not limited to: Web Analytics, Emails, FOIA requests, popular datasets on Data.gov and beyond.
• Prioritize next steps based on public feedback in an iterative process and solicit fresh feedback at each incremental step.
• Solicit feedback across platforms, mediums, and from all of the public.

Case Studies

• Govcode is a website that features government open source projects. Open source projects that utilize the 'help wanted' tag proactively solicit contributions from the public on government technology projects.
• Public Participation in Government Web Design: The Data.gov team integrated feedback from virtual, online, face to face testing, and social media platforms to dramatically change the site design in response to customer needs.
• Web Design Changes? Let the Metrics be Your Guide NASA metrics showed that mobile use of NASA.gov initially outpaced desktop use, confirming their decision to change to a mobile-first design.
• Government CX: Finding the Metrics That Matter The Export-Import Bank measures transaction processing times and ease of use to create the best experience for their stakeholders.
• Analyzing Who's Using your Agency's Data with FOIA requests GSA's Open Data Point of Contact reached out to the agency FOIA office to survey FOIA requestors on how they used requested information to get a better sense of the data demands of the public.

Resources

• gov is a rich resource for civic hackers, tech entrepreneurs, data scientists, and developers of all stripes. It includes information about APIs, open source projects, and relevant developer resources across government.
• Enigma is amassing the largest collection of public data produced by governments, universities, companies, and organizations. Concentrating all of this data provides new insights into economies, companies, places and individuals.
• API Release Kit describes the elements agencies should include in federal API releases.

Metrics: How do you know you're successfully facilitating participation?

• Measure the % of decision-making processes–anything from project management to policy decisions–that have a public feedback track.
• Measure how often "next steps" in a decision making process are informed and prioritized in accordance with public feedback.
• Measure how effective public feedback platforms and mediums are at reaching a equitable representation of the public.
• Measure how the timing of public feedback solicitation effects the outcomes of decision-making processes.
• Determine the value and effectiveness of the use of public feedback data, such as, but not limited to: Web Analytics, Emails, FOIA requests, popular datasets on Data.gov, as public sentiment indicators.

Evaluate and Report

Play 12: Transparently report outcomes and performance of participation

Transparency builds and maintains trust and respect with participants. When government takes an active role in allowing public participation in the decision-making process, accountability is key to maintaining trust of participants, by transparently reporting outcomes and performance of participation on a regular basis. Transparency allows for equal assessment with internal agencies, stakeholders and the public.

Checklist

• Establish and communicate where, when and how you will report outcomes and performance.
• Decide what type of open and accessible formats will be most useful for your audience.
• Report outcomes on time while participants are still engaged.
• Explain the importance of transparently reporting outcomes and performance of participation.
• Encourage others to share findings and outcomes from your program.
• Encourage continued community dialogue and awareness of lessons learned and outcomes.

Case Studies

• The National Archives and Records Administration (NARA) used public feedback to develop the agency's third Open Government Plan. It also reported outcomes of NARA's previous Open Government Plans.
• NASA – Disk Detective is a crowdsourcing project whose primary goal is to produce publishable scientific results. It uses citizen science to help astronomers discover embryonic planetary systems, reporting outcomes on time while participants are still engaged.
• Ford's Theatre: Remembering Lincoln on HistoryPin – The HistoryPin platform encourages participants to contribute content, share findings and continue community dialogue. Ford's Theatre is working with a range of partner institutions to collect, digitize and share local responses from the 13 months after former U.S. President Abraham Lincoln's assassination.

Resources

• Managers Guide to Evaluating Civic Participation, by Dr. Tina Nabatchi, discusses the power of process and impact evaluations to assessing citizen participation.
• Effective Engagement: building relationships with community and other stakeholders: The Engagement Planning Workbook, by the Victorian (Australia) Department of Sustainability and Environment, provides tools for evaluating engagement.
• Democracy On-Line: An Evaluation of the National Dialogue on Public Involvement in EPA Decisions, by Thomas C. Beierle, offers insight into a two-week on-line discussion of public participation at the Environmental Protection Agency (EPA).

Metrics: How do you know you're successfully evaluating and reporting outcomes?

• Highlight performance of participation through use of data visualizations and infographics, illustrating transparent outcome.
• Quantify outcomes of performance of participation through Town Hall meetings and/or presentations
• Conduct web research and run social media analytics to analyze ways to report outcomes of participation.
• Post articles directing users to results of past and current initiatives that encourage and require public participation, and reflect outcomes.