Draft Public Participation Playbook Version 2

This document is the unedited draft collection of initial ideas for articulating best practices and performance metrics for public participation in government. It is not intended for distribution as a final product, but instead an in-process collaborative space to contribute ideas and refine approaches before further opening it for wider participation.

To see past comments, please view the first draft of the U.S. Public Participation Playbook. We will continue to incorporate previous comments as the playbook continues to take shape. This will include pairing it down to a more concise size.

To read more about the playbook, please read the initial announcement from the White House OpenGov blog , or the most recent announcement of this second draft from the Digitalgov blog .

Introduction

• Public participation includes all the activities by which people's concerns, needs, interests, and values are incorporated into decisions and actions on public issues. The American people have many ways to participate in their government's decision-making. Whether they are commenting on a new regulation, helping to pick infrastructure projects in their neighborhood, advocating for the environment, participating in open meetings, or engaging in online dialogues, or in other ways, they make their voices heard.

• There are numerous benefits for government agencies that engage directly with the public they serve. This playbook provides examples from federal agencies that have been leaders in public participation. The ways in which each agency pursues public participation differs widely depending on the program, resources, audience, missions and goals. Similarly, the outcomes and impacts of participation will be measured differently with each agency establishing metrics and reporting capabilities that align with their idea of success within their organization.

• As you explore new ways of fostering public participation, we invite to use this playbook. Use the parts which work for you, and if we’ve left something out, let us know so we can include it the next time around.

• All of our agencies have the mandate and drive to engage with our citizens in a meaningful and engaging manner. However, each organization will have different resources, audiences, missions, and internal stakeholders that will affect the definition of "Public Participation" in a digital space. We need to asses our goals, determine our audiences, and establish metrics and reporting capabilities in order to determine our successes. Simply stated, we want to empower citizens to digitally engage in order to influence our government priorities.

1. Clearly define and communicate your objectives

a. Introduction

• Federal agencies understand the importance of meaningful public engagement, but the form that engagement takes will likely be different for each organization, depending on resources, audience and mission. Each agency must determine goals, and establish metrics and reporting capabilities that align with the definition of success for that organization.

b. Checklist (Steps or Questions)

• What does "Public Participation" look like to your agency?

• What are your talking points/messages?

• Who is your team?

• Who else needs to know what "Public Participation" is for your agency?

• Who is your audience? (Do you have micro audiences?)

• What goal are you trying to achieve?

• What platforms do your intended audiences use?

• How will you manage unintended audiences that blur your objectives?

• Has a strategic plan been set? (this includes a timeline, ramp up and draw down strategy, internal reporting and engagement with the public)

• What metrics are you using to evaluate success?

• How are you reporting and sharing information (internally and externally)?

• Are you modifying your play according to how the public is interacting with you?

c. Government Case Studies -- This section needs clarifying statements

• http://www.fcc.gov/comments

• http://open.nasa.gov/blog/2014/06/30/2014-open-gov-awards-citizen-engagement/

• http://forums.e-democracy.org/

• http://fluapp.challengepost.com/submissions/3041-flu-near-you-by-healthmap (this one doesn't influence policy, but does drive government action)

• https://petitions.whitehouse.gov/

• epolicyworks.org

• http://journalistsresource.org/studies/politics/digital-democracy/using-social-media-to-improve-citizen-engagement-with-government-research-brief (see Texas example)

• http://www.govtech.com/e-government/Two-Way-Citizen-Engagement-in-Elk-Grove.html?utm_source=related&utm_medium=direct&utm_campaign=Two-Way-Citizen-Engagement-in-Elk-Grove

• http://www.mpa.unc.edu/sites/www.mpa.unc.edu/files/Amy%20Strecker.pdf (see p 5)

• http://www.slideshare.net/AmericaSpeaks/assessing-public-participation-in-an-open-government-era

• DOT Public Involvement and Reference Tool

• Madison platform and OPEN Act: First crowd-sourced legislation in Congress http://keepthewebopen.com/open

d. Metrics to Clearly Define and Communicate Objectives

Metrics will vary according to the platform you use. Some broad examples include:

• Surveys – responses should drive learning and improvement

• Web – Google Analytics or heat-mapping can show user engagement.

• Email – click rates, bounce rates, unsubscribes, participation in a call-to-action

• Social Media – user engagement analytics, reach, changing user behavior

• Apps (native and mobile) – use statistics, are "transactions" being completed, is there input or feedback from the field (ie uploading multiple photos in citizen science app), are people actively using a native app (i.e., logging in multiple times).

• Online Video Chats (i.e., hangouts) – participation rate, views

• Discussion Threads (i.e., chat rooms) – number of participants, input from the public, up votes / down votes

• Blog – viewership, comments, action audience takes next

• Transactional – hard numbers related to online transaction completion

2. Select appropriate design format for public participation

a. Introduction

• Enable broad participation by offering responsive, accessible, intuitive, mobile- and user-friendly tools that can be rendered in multiple formats on a wide range of platforms.

b. Checklist (Steps or Questions)

• Consider factors of time, money and functionality. What do you absolutely need, what would be nice to have?

• Is there value in mentioning the various steps they may need to take in order to acquire access to that platform (e.g., privacy agreements, terms of service, etc.)? If this is intended to encourage state/local agencies to adopt more digital engagement approaches, it would be helpful to steer them in the right direction.

• Determine hosting platform and key functions needed (e.g., ability to interact/comment/annotate, blogs, social elements, multimedia capabilities, accessibility, SEO)

• Apply current coding techniques in build a flexible design -- HTML5, CSS

• When appropriate, incorporate high-quality graphics, color and visual techniques to increase audience engagement.

• What Mobile Technologies Can be Leveraged to Engage the Audience? If the audience has wireless connectivity and are using smartphones, Web Analytics can be used to help determine a mobile approach . If it's a mobile audience, ensure you use tools that are mobile-friendly (responsive web design, native apps) and link to mobile friendly pages. Audiences without wireless connectivity or data plans should not be ignored. They are communicating in other channels like SMS and can be engaged with these and other telephone systems like Interactive Voice Responses (IVR) and Unstructured Supplementary Service Data (USSD).

c. Government Case Studies

• The White House Engage website allows citizens to submit questions and comments, join online events, and engage with government via social media.

• Streamlined Design and New Features - USDA.gov underwent a major redesign to improve user experience and usability. They used analytics and lessons learned from prior redesigns to determine popular content and user preferences.

• NOAA Release Mako Mobile App was created for fisherman to report their releases of Shortfin Mako sharks while on the water. The app uses a device’s built-in GPS, when available, to fill in exact location coordinates.

• API - HIV Services Locator (AIDS.gov)

• The Victoria (Australia) Tookit for Public Participation has a great selection of different PP formats.

• Determine Participation Objective (i.e., what do you hope to gain from the public's participation? (Awareness, Education, Input, Decision-Making) - refer to diagram on Pg.8

• from EPA's website: [Selecting the right level of Public Participation.] Unsure if this is uptodate.

d. Metrics

• Increased readership/visits to sites/blog/etc.

• Increased awareness about federal agency's goals, mission and purpose

• Measurement of public participation (survey responses, event attendance, other contributions i.e., time, ideas, etc.)

• Measurement of time spent on site, bounce rate, clicking patterns and high-, low traffic areas

3. Understand and communicate the benefit of participation

a. Introduction

• What will motivate people to contribute to a public participation effort? Help participants understand how they can contribute, and what they'll get out of the process. A clearly defined and communicated value exchange will help you craft effective messages that bring participants into the process, and it will help you form your communications strategy. And as we get into metrics later, understanding what is successful participation to your team is key to knowing if you succeeded.

b. Checklist (Steps or Questions)

• Make sure participants know where they are in the process (at the beginning? in the middle?)

• Define what participants need to know before they engage, and ensure that everyone starts off with the same information.

• Understand participant expectations, motivations and objectives. What will your organization give back to the participant for their contributions? What's the reward?

• Define what participants have to give so they can participate (e.g., information, permission, time)?

• Define what the intended outcome will be. What will come out of the participation - how will you use their input?

• Inform participants of how their input will be used, and how their participation will impact the outcome.

• Understand how citizens will benefit from the process (e.g., access, influence, status…)?

• Let people know when they should expect to hear follow-up from their engagement. Then, actually follow up and demonstrate how their input was used.

• Build a communications strategy that includes messaging regarding expectations for participants as well as the value of participation.

• Develop consistent messaging or talking points. Make sure everyone is clear on what they are communicating.

• Thank participants for their contributions.

c. Government Case Studies

• Census Hackforchange Civic hacking, a positive type of community building around data, allows programmers to harness the power of publicly available government statistics to create apps that benefit everyone.

• National Day of Civic Hacking Governments and other organizations submit problem statements that explain an issue, what they hope the public will do to help address the problem, and resources to help orient people to the problem. Event organizers outline time commitments and skills needed to effectively participate.

• Regulation Room Regulation Room is pilot project sponsored by the Federal government and operated by the Cornell e-Rulemaking Initiative (CeRI). Participants are given resources to better understand the rulemaking process, how to make an effective comment, and have the opportunity to browse the issues that are most relevant to them. Discussions among participants are encouraged. After discussions have ended, participants comment on a report of the activity to ensure their views are captured before the final version is sent on to the government.

• Give a Minute : A micro-participation/crowdsourcing event where the expectations of participants are clear: they give one minute to answer a question like, "How can we encourage more biking/walking?" The ideas are collected and shared publicly. Participation can be digital or in-person.

d. Metrics

• Conversion rates, from visitor to signups

• Returning visitor rates

• Sharing & promotion rates (e.g. recruitment activity from participants)

• Quality of participation (e.g., level of discussion (comments on ideas), relevance of feedback)

• Participant satisfaction with the process

• Amount of new participants/users

4. Understand your participants and stakeholder groups

a. Introduction

• Audience understanding is key to running a successful engagement effort. Once you determine who you're trying to reach, you can refine your outreach efforts to effectively communicate with participants and stakeholders.

b. Checklist (Steps or Questions)

• Who are you trying to reach?

  • Identify the types of users you're targeting with your outreach

  • Consider creating personas of your target audience to help you understand and relate to their needs.

  • If you don't have a clear idea of target audience, do some outreach and sampling to better define and understand them

• Listen carefully to the experiences, beliefs and values of potential participants, then use groupwork methods (e.g., clustering) to draw representations based on observation, not assumption.

• Who has tried to reach you?

• Who is contacting you, why are they contacting you, and what are they saying?

• What obstacles do you need to overcome to reach these people (think lack of Internet, busy parents who don't have time, single parents juggling family responsibilities, etc.)

• Ensure you get adequate participation from each subgroup you've identified (without a representative sample, results may be skewed)

• How you recruit these people may vary

• Identify exactly how people will participate (not everyone has Internet access, some may only use mobile devices, social media trends are changing, etc.)

• Select a tool to facilitate participation AFTER you identify project objectives and target audience

• Build into your strategy how you plan to work around obstacles to reach and engage your target audience.

• How will you ensure you’re reaching a diverse group of people with broad perspectives?

• What stakeholders care about the issues presented? Are there other stakeholders who might be interested if presented with the information?

• Defining stakeholders can be difficult. Ask yourself, 'Who has a stake in the project?' An open ended question might tell you who you need to invite, not just who you want to invite. Typically, if a project has a benefactor, there may be other parties who are adversely or tangentially affected

• Are the internal stakeholders looking for something different than your external audience?

• Are there gatekeepers (e.g., community groups, professional societies, industries) who can help you reach your stakeholders?

• What types of engagement strategies might be effective with each type of stakeholder?

• Plan your audience based on the life cycle of your product. Today, your audience may look one way, but what does the data tell you? What will your audience look like over the life of the project? Neighborhoods change over time. Heck, even countries do.

• Meet your audience where they are, not where you want them to be (literally): Go to their community, their platform, their school, etc... and don't expect them to come to your office or location. This also means figuratively, in terms of valuing multiple perspectives, cultural norms and institutions.

c. Government Case Studies

• Department of Transportation http://www.planning.dot.gov/focus_caseStudies.asp

• National Geospatial Intelligence Agency: Ebola Relief Website and Press Release

• Peace Corps Application Process Redesign (increases in applicant numbers and completed applications)

• Resources

• Old Open Gov Wiki: https://opengovdirective.pbworks.com/w/page/4874180/Participation%20Case%20Studies%20and%20Effective%20Practices

• CDC: Gateway to Health Communications & Social Marketing Practice provides resources to build health communication or social marketing programs. It provides tips for analyzing and segmenting an audience, choosing appropriate channels and tools, and evaluating the success of messages or campaigns.

• Dept of Energy: Identify Target Audiences and Behavior Changes

d. Metrics for understanding participants and stakeholder groups

• Total number of users that participated

  • Did you reach the users who are most likely to be the actual end user of the program or product?

  • Number of users in sub-groups identified

  • Compare findings among subgroups. Analyze for similarities and differences

• Identify quantity of users needed to be a representative sample size (total needed broken out by number needed within subgroups)

• Baseline testing could help track impact of project upon completion.

5. Design for inclusiveness

a. Introduction

• Whenever you're running any sort of public engagement effort, make sure the design and setup doesn't exclude anyone from participating. Consider persons with disabilities who use screen readers, people with limited English proficiency or low literacy skills, people who have suffered a stroke and many others who may experience difficulty reading. This play will help ensure that your public participation effort is designed to include as many people as possible.

b. Checklist (Steps or Questions)

• Provide accessibility for persons with disabilities DHS Section 508 Compliance Test Processes

• Evaluate the need for multilingual support Top 10 Best Practices for Multilingual Websites

• Ensure Plain Language DigitalGov Social Media: You Still Need Plain Language

• Provide online and offline support

• Can a physical version or digital package ("meeting-in-a-box") be made available for wider distribution at local, neighborhood, community, or personal family events? Can people engage in the material without a facilitator?

• Provide cultural competency support

• Value diversity

• Talk to members of communities that are typically underrepresented.

• Have the capacity for cultural self-assessment

• Be conscious of the dynamics inherent when cultures interact

• Have institutionalized culture knowledge

• Develop adaptations to service delivery reflecting an understanding of cultural diversity

• Consider multiple learning styles. Some projects are extremely complex and include science or engineering concepts that can be difficult for certain learning styles. Consider recruiting volunteers for visual notetaking , or graphic facilitation online or in meeting, to keep a visual record of the conversation. Employ handouts or models.

• Questions to ask:

• Is your contact information available on your social media account page?

• Is your social media content available through more than one channel?

• Did you provide links or contact information to official social media support and accessibility teams?

• Is your design simple?

• Did you write in plain language?

• Do you periodically test your content for accessibility?

c. Government Case Studies

• Food and Drug Administration: Using Medicines Wisely: Increasing Access to Information for Women with Intellectual Disabilities

• FDA's Office of Women's Health teamed up with the Administration for Community Living, the Association of University Centers on Disability and the Administration on Intellectual and Developmental Disabilities to improve their safe medication materials for women. They conducted a series of focus groups with women in the disability community to learn how word choices, font, layout, and graphics could be improved.

• Improving the Accessibility of Social Media in Government toolkit curates and share best practices to help agencies ensure their social media content is accessible everyone, including users with disabilities. Efforts are also being made to work with social media platform and tool developers, citizens and partners to encourage greater accessibility.

• Ensuring the Accessibility of Web Content . The Digital Communications Division (DCD), part of the office of the Assistant Secretary for Public Affairs (ASPA), leads the development and review of HHS Web content, social media, and supporting technologies.

• The Department of Health and Human Services Accessibility Lab . The Digital Communications Division has assembled a 508 Lab with a collection of assistive and evaluative technologies.

• Guide to Section 5 0 8 Standards . The U.S. Access Board’s Section 508 Standards apply to electronic and information technology procured by the federal government, including computer hardware and software, websites, phone systems, and copiers. They were issued under section 508 of the Rehabilitation Act which requires access for both members of the public and federal employees to such technologies when developed, procured, maintained, or used by federal agencies.

• Usability.Gov: Accessibility . Evaluate your website for both usability and accessibility.

• Disability.Gov : the U.S. federal government website for information on disability programs and services nationwide, and a model for accessibility on the web.

d. Metrics

• W3C Web Accessibility Metrics . Detailed report on useful metrics to assess the accessibility level of websites, including the accessibility level of individual websites, or even large-scale surveys of the accessibility of many websites, including:

  • The number of pictures without an alt attribute.

  • The number of Level A and AA success criteria violations.

  • The number of possible failure points where accessibility issues can potentially happen (such as the number of images in a page).

  • The severity of an accessibility barrier.

  • The time taken to conduct a task.

• Accessibility in Practice: A Process-driven Approach to Accessibility . The best approach to accessible user experience is to integrate accessibility into the design and development process.

• Section 508 Checklist . Pass/Fail criteria for each of the Section 508 standards (e.g., a text equivalent for every non-test element, functional electronic forms).

• The US Access Board produces guidelines and standards to help meet accessibility and inclusion standards, including Communications and IT .

6. Provide effective and timely notifications

a. Introduction

• Effective communication with participants and stakeholders throughout the life of an engagement effort is key. This play offers tips and advice for letting your audience know about opportunities to participate, and how to keep them engaged and active for the duration of the event.

b. Checklist (Steps or Questions)

• Develop a comprehensive outreach plan that includes messages, timelines, branding, and strategies for reaching different audiences, etc.

• Provide your staff members with simple outreach tools (e.g., invitation email, talking points, email signature line runner) so they may invite, encourage, and share messages with their audiences.

• Use a variety of tools and channels (online and in the real world) to invite people into the conversation.

• Encourage partner agencies and organizations to share your messaging with their stakeholder audiences.

• Repeat messages. Ensuring that people are aware of what's happening at all stages of the participatory process is important. If you make one-time announcements about opportunities for citizens to participate there's no guarantee they'll all hear. By repeating your messages, you will significantly increase the chance of more people participating.

• Consistent messages. Make sure all messages are consistent, same dates, deadlines, locations, times, etc.

• Work with community leaders and stakeholders to reach your target audience. Citizens may never think to come to your agency on a given topic, but they can learn about opportunities for participation from community groups, local service providers, religious organizations, etc.

• Provide post-event notifications to participants to thank them for their participation and provide information on how their feedback will be (and eventually is) used.

• Leverage the Federal Register

• Take baseline measurement of past similar events.

• Provide participation notifications that fit each community best (e.g., phone, postal, email, social media).

• Reach out to leaders/influencers in the community and engage them as champions of your message.

c. Government Case Studies

• USGS and Employee Use of Social Media : The Union of Concerned Scientists released a scorecard on how agencies allow employees to communicate with the public. USGS received a B+, but updated its public documentation to address suggested improvements within four hours.

• DigitalGov Search seeks to provide immediate answers to the public’s search questions. In addition to the Federal Register documents, it also incorporates results from other specialized government websites and social media accounts.

• USA.gov encouraged people to share their #ElectionSelfies , showing off "I Voted" stickers. They monitored trending election hashtags on Twitter and reached out in real time to people with questions about voting.

• U.S. Immigration and Customs Enforcement’s Operation Predator App allows anyone to submit anonymous tips about suspected predators to the ICE Tip Line—either wanted fugitives or previously unknown—24 hours a day, seven days a week.

d. Metrics

• Measure overall participation in the event compared to previous efforts

• Measure online engagement trends (e.g., comments and shares) over the entire length of the event. This should show your notifications are improving.

• Measure the positive and negative feedback, post-event, on how participation discussions will be used.

• Measure engagement across each channel, independently, to determine the level of participation on each.

7. Encourage community development and outreach

a. Introduction

• Community development and outreach involves communication amongst community stakeholders, organizations, research centers, government agencies, etc. Goals include communicating reliable information about federal programs and policies, including addressing complaints; obtaining public feedback about the impact of government activities; fostering conversations and incorporating feedback into the policymaking process; and deepening channels of communication between communities, the government and public/ private entities.

b. Checklist (Steps or Questions)

• Moderate and facilitate – encourage connections and discussion of other peoples’ ideas. When discussion stalls, have a plan to catalyze the discussion. At regular intervals, pause to "share out" and let participants see how else they can contribute (this is easy in an in-person meeting, but for online engagements, consider sending weekly emails that summarize the conversation so people can see what other people are saying)

• Identify the appropriate community organizations to which you should reach out.

• Identify the appropriate people in each organization to be your points of contact.

• Ensure that employees who are not designated as points of contact understand the extent to which they may discuss their agency's work with community members outside of work.

• Determine and share the value the community organization will get from helping you. Make sure there is value in it for them, to give them a stake in a successful outcome.

• Share control and decision-making. This can be challenging, but creates shared success, shared problem solving, and shared accountability. A well-designed dialogue process can enable people to discover common concerns and values and agree on a framework to move forward. Create guidelines for how this will work. This is an advanced step for a well-defined process but can be very rewarding.

• Be prepared for a rotating crowd. There might be relevance for people at a certain time that fades and that is fine- getting the right people at the right time.

c. Government Case Studies

• The Commodity Futures Trading Commission hosted a smart investing webinar in partnership with AARP . Both organizations were able to reach out to their respective communities to encourage people to sign up and participate.

• HHS Office on Women's Health encouraged community organizations around the country to organize events in celebration of National Women's Health Week 2014. They used the platform meetup.com so organizations could add their own events and search for other events in their area.

• FDA's Office of Women's Health launched the Pink Ribbon Sunday Mammography Awareness Program to educate African American and Hispanic women about early detection of breast cancer through mammography. By partnering with churches and community organizations, there were able to reach over 100,000 women throughout the country.

• The 2010 National League of Cities (NLC) research report, Making Local Democracy Work: Municipal Officials’ Views About Public Engagement , was part of a project on civic engagement and local democracy. The report found that municipal officials often provide many opportunities for public engagement, but the effectiveness of existing public engagement activities was not always clear.

• US Department of State Bureau of Educational and Cultural Affairs Exchange Program fostering Community Engagement through the Arts. The American Arts Incubator uses new media and/or mural arts as a means to engage youth, artists, and underserved community members in overseas communities in Asia to advance U.S. foreign policy by addressing a local community issue, such as women’s empowerment, HIV-AIDS prevention, social inclusion, conflict resolution, and the environment. (Example not a case study)

d. Metrics: How we know we're encouraging community development and outreach

• Increased number of community organizations support this event compared to previous events

• Size of community that your partners reach

• Does the program become integrated or owned by the community?

8. Empower participants through public/private partnerships

a. Introduction

• Leveraging corporate and community partners can be a powerful way of gaining support and participation for government campaigns and events. Partners and sponsors can help reach new audiences or reinforce messages within existing ones. Public/private partnerships also legitimize both what your agency is trying to accomplish and how that information or program is delivered.

b. Checklist (Steps or Questions)

• Define what a partner is and how partners are approved. Consider using a Memorandum of Understanding (MOU), an Interagency Agency Agreement (IAA), or another type of contractual agreement to provide clear language for what each party is responsible for. Having such an agreement will allow you to define an "authorized partners" list, help to ensure there are no legal issues, and help outline terms for ensuring a mutually beneficial partnership.

• If there isn't an existing agreement, is there some other way to make sure both the agency and the company are getting value out of the partnership?

• Outline the terms of the partnership. Think about messaging, logo usage, promotional opportunities and boundaries.

• Reach out to nonprofits and community partners who could help amplify the message and/or connect with valuable corporate partners.

• Survey agency employees who are not currently involved in agency communications but may have knowledge of communities and stakeholder groups that you are not aware of.

• Ensure that employees who want to interact with the public understand ethics rules and agency policies, and also have opportunities to explore involvement in communications and outreach, where appropriate.

• Recruit volunteers, find ways for people to help/serve.

c. Government Case Studies

• The Heart Truth ®: The Heart Truth® campaign, created by the National Heart Lung and Blood Institute (NHLBI), part of the National Institutes of Health (NIH), has partnered with dozens of corporate and community sponsors since its launch in 2002 to raise awareness about heart disease and its effect on women. Many of these partners have also encouraged public participation. For example, in 2014, corporate sponsor Charles P. Rogers held an online auction of one-of-a-kind custom beds through eBay's Charity Works platform, attracting hundreds of bids and raising nearly $4,000 to support The Heart Truth® campaign.

• LabTV : In 2014, LabTV launched its vision of creating a free, scientist-to-student web/video platform aimed at inspiring the next generation of researchers. LabTV features thousands of videos focused on the scientist, rather than science. Through online tools, LabTV matches young people with working scientists who share similar traits such as "hated math in school," or “grew up with a parent who is a doctor.” Once a student finds a scientist to whom they relate, LabTV connects them, so the student can learn about life as a medical researcher. The National Institutes of Health (NIH) worked with LabTV to produce hundreds of videos of NIH researchers with varying backgrounds and interests.

• STOP.THINK.CONNECT. ™: Launched in 2010, STOP.THINK.CONNECT.™ was created by a coalition of private companies, non-profits and the Department of Homeland Security (DHS), with leadership from the National Cybersecurity Alliance, and the Anti-Phishing Working Group. By partnering with industry leaders, DHS attracted a broader audience and garnered significant public participation for online events like Twitter chats to help spread its message. In October 2014, DHS teamed with dozens of public and private organizations including Microsoft, McAfee, Norton, Intel and Visa to host a series of Twitter chats to highlight National Cybersecurity Awareness Month.

• Distraction.gov : This website, operated by the U.S. Department of Transportation (DOT), is dedicated to stamping out distracted driving. This includes, texting, talking on the phone, eating, drinking or any other activity that takes a driver's focus off the road. DOT has enlisted the help of several organizations to spread this message and motivate the public to join the cause. One partner, the FOX television show "Glee", helped produce public service announcements depicting a character from the show texting while driving, and consequently getting involved in serious car accident. Distraction.gov has a Call-To-Action that specifically encourages people to take (and sign) a pledge stating their commitment to being a safe and distraction-free driver. DOT also uses this website to encourage the use of social media tools for individuals to share online, and offer instructions on how to host an event dedicated to stopping distracted driving.

d. Metrics

• Value and measure in-kind services donated by private partners, such as air-time, printing, advertising, or prizes.

• Use clearly defined web metrics to identify participation levels when utilizing online tools such as social media or websites.

• Compare participation from past events and activities that did not include partnerships to help measure the value of the partnerships.

9. Provide multi-tiered paths to participation

a. Introduction

• Good government is responsive and engaging. To ensure broad participation and involve more citizens in the decision-making process, use a variety of channels to foster participation. It's important for agencies to be inclusive, and use of available resources to receive feedback from a wide variety of sources.

b. Checklist (Steps or Questions)

• Determine the level of participation you're looking for ( IAP2 standards ). Are you empowering stakeholders as decision makers? Involving them? Sharing information?

• Determine that you have the resources -- personnel and otherwise -- to handle a multi-tiered, two-way engagement strategy.

• Define the information you're looking for, whose input you value most, and what communication channels they're most likely to use. Focus your resources there.

• Do employees, beyond those designated as official points of contact, know how to respond if approached by members of the public who are interested in participating in the agency's work?

• Are employees generally aware of opportunities for the public to participate in the agency's work?

• Create clearly defined action items. Leave feedback HERE on THIS ISSUE.

• Acknowledge and respond to feedback.

c. Government Case Studies

• Case Studies in Thunderclap The Department of Health and Human Services (HHS) used Thunderclap to support Open Enrollment Season for Healthcare.gov. The Environmental Protection Agency (EPA) successfully promoted its campaigns for Earth Day, #ActOnClimate ! and Be SunWise on Don’t Fry Day .

• The Army's Assembled Chemical Weapons Alternatives Program has established two citizens advisory boards which are empowered to make project recommendations on the path forward. The program holds quarterly public meetings with program leadership and maintains an outreach office that's open to the public and fully staffed to receive in-person feedback. They also maintain social media accounts, direct email, and a website for maximum interaction (outreach program based on IAP2 guidance).

• FAFSA's #AskFAFSA campaign sets up office hours for stakeholders to ask questions about student aid in multiple languages via Twitter: https://studentaid.ed.gov/node/372

• The U.S. Trade Representative has a traveling roadshow that gathers feedback on trade agreement negotiations. There's an email address to receive information and the office hosts a Twitter account, though I'm unsure if it is set up to receive and pass on feedback.

• FDA has examples of multiple levels of participation, many are listed on its Patient Network site www.patientnetwork.fda.gov/ . Citizens can choose the level of participation that works for them and become more involved as appropriate.

• Comment on proposed regulations and guidance online or by mail.

• Attend public meetings in-person or by webinar.

• Have a more active role in the meeting by registering in advance to speak.

• Join an advisory committee to influence decisions like product approvals.

d. Metrics

• Public meetings held/attendees

• Number of comments received per channel

• Number of avenues available for feedback

• Subjective metric: Value of feedback

10. Maintain responsive, accountable feedback loops

a. Introduction

• If you ask your audience to take time and effort to share thoughts and feedback with you, make sure you let them know that their voice is being heard and how you'll use their feedback. Acknowledge the positive and the constructive openly.

b. Checklist (Steps or Questions)

• Build in time to pause and take stock. Like locks in a canal, these are important moments for you to decide to change direction, stay on the same path, or bring in a new audience

• Celebrate Success! If the feedback is effective, celebrate it with your participation group If corrections are needed, address in a transparent and informative (not critical) manner.

• Begin the conversation and outreach efforts with civic and community engagement efforts that don't necessarily have a direct 'ask' or end in mind. Civic Engagement is a continuous conversation with the public that reinforces public commitment.

• Build relationships with in-person interactions such as hosting 'meet an expert' sessions (in a community or online) to get to know the community, and build an understanding of cultural norms. These open-ended interactions will come in handy when it's time to ask for directed feedback or input.

• These interactions are also important to help you discover things that may inadvertently sideline your efforts. If a community or group is, or was, embroiled in a 'hot topic,' your great idea or important process may not stand a chance.

• Don't let the conversation end. You've just invested a tremendous amount of time and energy into building relationships. Use the participation framework you created to develop future uses for the network. Perhaps the process has highlighted an issue or future product that would also benefit your agency and the public. If you've created a community around a particular topic and participants wish to continue the conversation, assist in keeping the collaboration alive.

c. Government Case Studies

• We the People: The Decision to Excavate the James Dexter Site . The decision by the NPS to reverse its earlier position and excavate the James Dexter site due to engagement with the city's African American community.

• Collaborating for the Future . New Bedford Whaling National Historical Park uses ongoing roundtable dialogues with communities of interest to implement the park's General Management Plan.

• From Isolation to Integration: Valley Forge National Historical Park transforms itself through civic input.

• Evaluating EPAs Public Participation Process in Superfund Sites.

• Beyond Outreach: A guide to engaging diverse communities .

• Tribal and Public Consultation for Historic Preservation Toolkit

• Environmental Protection Agencies Public Participation Guide

• Using Social Media to Enhance Civic Engagement in Federal Agencies

d. Metrics

• Begin with an assessment of where you are. Assuming you've identified your target audience, generate solid baseline data on how much your audience knows, and how well they responded to your message. This will help you determine whether public understanding of your message has stayed the same or improved.

• The National Park Service has created a website to house its public participation process. This web-based approach allows you to tracks hits, collect comments, conduct surveys etc..

• Involve the public in measuring their role in the planning/decision-making process.

• Why are you evaluating?

  • To improve how community engagement is done – is it important to continuously improve our operations during the life of the project, and to focus on quality, efficiency and effectiveness of processes?

  • To gain insight into what makes effective community engagement in different situations - is it important to learn something about the process (can it be applied in another situation) or the community (their needs)?

  • Who wants to know what?

  • What evidence will be collected? How?

  • Victoria (Australia) Page 12 , Tools for Evaluating Public Involvement

  • Resources for the Future: Evaluating Public Involvement

  • Managers Guide to Evaluating Civic Participation

  • Determine schedule and resource allocation

  • Start with the basics. Did the process meet the objectives? Did you stay on schedule, on budget, etc.?

11. Use data to inform decisions

a. Introduction

• Use data to inform decisions about engagement efforts and public participation. Data can help you decide the best format for your engagement effort, the best tools to use, the most receptive audiences and more. Data will help you get the desired results and adapt your engagement effort along the way.

b. Checklist (Steps or Questions)

• Clearly define your goals.

• Establish key performance indicators.

• Have a hypothesis to measure.

• If your hypothesis is correct, what are your next steps? If it's wrong?

• Continuously reconsider these metrics to make sure they are still the best ones to use. Refine if necessary.

• If you determine that a program or technique does not add value, are you willing to revise the process?

• If you host a hackathon or open data event, have you included employees with a thorough understanding of the data that your agency provides?

• Are employees generally aware of the agency's open data policies and publicly available datasets?

• Are data-savvy employees encouraged to attend, participate, and assist with hackathons and open data events?

c. Government Case Studies

• Data.gov is a rich resource for civic hackers, tech entrepreneurs, data scientists, and developers of all stripes. It includes information about APIs, open source projects, and relevant developer resources across government.

• API Release Kit describes the elements agencies should include in federal API releases.

• Govcode features Open source projects by the government.

• Public Participation in Govt Web Design : The Data.gov team integrated feedback from virtual, online, face to face testing, and social media platforms to dramatically change the site design in response to customer needs.

• Web Design Changes? Let the Metrics be Your Guide NASA metrics showed that mobile use of NASA.gov was outpacing desktop use, confirming their decision to change to a mobile-first design.

• Government CX: Finding the Metrics That Matter The Export-Import Bank measures transaction processing times and ease of use to create the best experience for their stakeholders.

• Enigma is amassing the largest collection of public data produced by governments, universities, companies, and organizations. Concentrating all of this data provides new insights into economies, companies, places and individuals.

d. Metrics

(currently in development, but please feel free and encouraged to propose more. We’re listening!)

12. Protect citizen privacy

a. Introduction

• Public participation is crucial to our democracy, but federal agencies must engage in ways that protect privacy, while maintaining compliance with federal laws such as the Federal Records Act (FRA) and the Privacy Act.

b. Checklist (Steps or Questions)

• Document privacy policies for all engagement channels, so the public can see what information you collect (if any), and how it will be used.

• Document terms of use detailing record-keeping practices, reasons why comments might be removed (e.g., using profanity, or sharing personal information on agency-managed social media platforms), and other rules to ensure protection of privacy.

• Consult with your agency's Records Officer to determine how to manage digital interactions and public comments as federal records.

• Invite your agency's Privacy Office and/or legal experts to participate and provide input on your agency's privacy policies, terms of service, or other charter documents. Discuss how to handle personally identifiable information (PII).

• Provide a medium for people to provide input anonymously.

• Include provisions in website and social media terms of use that allow community managers to delete content and comments that contain PII.

• Include the Privacy Policy and terms of use on your website and social media platforms.

• Identify what information will be stored or maintained by your agency, for what purpose, whether your agency will maintain records of conversations, and for how long such records will be maintained.

• Does your agency extend the same data privacy to U.S. citizens, U.S. residents, and to foreign nationals? Per a May 2014 report on Big Data , the White House may encourage agencies to extend universal Privacy Act protection. Consider getting ahead of the curve.

• Use hashtags to join conversations online--don't target individual users.

c. Government Case Studies

The following are great examples of best practices, policies, and terms of service that include provisions for protecting PII and that support Privacy Act compliance.

• CIO Council's Privacy Best Practices for Social Media : This comprehensive guide released in 2013 provides resources and best practices for standing up a Privacy and Records Keeping Act compliant social media policy.

• White House Expands Data Protection : How can your agency use big data? There is no formal guidance yet, but this May 2014 report and recommendation from the White House provides insight on how and when agencies may be able to use big data from the Internet of things, the web, mobile, and more; and whether agencies will be required to extend universal Privacy Act Protections.

• FEMA Privacy & Comment Policy : The FEMA Privacy & Comment Policy comprehensively addresses how social media, web, mobile, and SMS data is used, stored, and managed. The policy is plain-language and concisely explains and justifies the retention and management of different data sets.

• Peace Corps Privacy Act Notice : The Peace Corps' Privacy notice is a great, plain language example of how to formulate a comprehensive, brief, and easy-to-understand privacy policy for the web.

d. Metrics

• Do all of your websites and social media sites contain provisions for protecting PII and maintaining Privacy Act compliance?

• How often do community managers monitor comments for PII?

• How many comments do community managers delete for PII on a monthly or bi-monthly basis?

• Do your screenshots or other records sufficiently protect PII captured by blurring, cropping, or blacking-out names, photos, or other PII?

• How often does your agency "audit" its records to ensure it is not maintaining broader records than necessary to comply with federal records keeping requirements?

• Are you using third party websites that collect data or PII in a way that is inconsistent with the Privacy Act?

13. Transparently report outcomes and performance of participation

a. Introduction

• Transparency builds and maintains trust and respect, both internally and with the public. When government takes an active role in allowing public participation in the decision-making process, accountability is key to maintaining public trust, by transparently reporting outcomes and performance of participation on a regular basis. Transparency allows for equal assessment--internally, with stakeholders and the public.

b. Checklist (Steps or Questions)

• Decide where you will report outcomes and performance. You may need to report it in multiple places, depending on your stakeholders.

• What type of formats will be most useful for your audience? Can the format be open? Is it accessible?

• Report outcomes in a timely manner while participants are still engaged.

• Explain the importance of transparently reporting outcomes and performance of participation at this particular time.

• Disseminate information across multiple platforms to reach more people.

• Encourage others to share information.

• Continue to encourage community dialogue around the topic.

c. Government Case Studies

• Report: Assessing Public Participation in an Open Government Era

• Report: Realizing the Potential of Open Government Data: A Roundtable with the U.S. Department of Commerce

• National Archives and Records Administration - public feedback for the development of the agency's third Open Government Plan as well as NARA reporting outcomes of NARA's previous Open Government Plans.

• USDA - Know Your Farmer Know Your Food compass map shows efforts supported by USDA and other federal partners as well as related information on local and regional food systems.

• NASA - Disk Detective is a crowdsourcing project whose primary goal is to produce publishable scientific results. It uses citizen science to help astronomers discover embryonic planetary systems.

• Ford's Theatre: Remembering Lincoln on HistoryPin (Ford’s Theatre is a public-private partnership between the National Park Service and Ford’s Theatre Society.)

• Transparency International Presentation to Data Transparency Town Hall

• New York City Transparency Working Group

• The People's Roadmap to a Digital NYC

d. Metrics

• Use data visualizations and infographics to illustrate transparent outcomes and highlighting performance of participation.

• Hold Town Hall meetings and/or give presentations reporting outcomes of performance of participation.

• Conduct web research and run social media analytics to analyze ways to report outcomes of participation.

• Post articles directing users to results of past and current initiatives that encourage and require public participation, and reflect outcomes.

• Direct users to articles via social media and other platforms.