Draft Public Participation Playbook

This document is the unedited draft collection of initial ideas for articulating best practices and performance metrics for public participation in government. It is not intended for distribution as a final product, but instead an in-process collaborative space to contribute ideas and refine approaches before further opening it for wider participation.

1. Clearly define and communicate your objectives

a. Introduction

(in development)

b. Checklist (Steps or Questions)

1. What does "Public Participation" look like to your agency?

2. Who is your audience? (do you have micro audiences)

3. What goal are you trying to achieve?

4. What platforms do your intended audiences use?

5. How will you manage unintended audiences that blur your objectives?

6. Has a strategic plan been set? (this includes a timeline, ramp up and draw down strategy, internal reporting and engagement with the public)

7. What metrics are you using to evaluate success?

8. How are you reporting and sharing information (internally and externally)?

9. Are you modifying your play according to how the public is interacting with you?

c. Government Case Studies

1. http://www.fcc.gov/comments

2. http://open.nasa.gov/blog/2014/06/30/2014-open-gov-awards-citizen-engagement/

3. http://forums.e-democracy.org/

4. http://fluapp.challengepost.com/submissions/3041-flu-near-you-by-healthmap (this one doesn't influence policy, but does drive government action)

5. https://petitions.whitehouse.gov/

6. epolicyworks.org

7. http://journalistsresource.org/studies/politics/digital-democracy/using-social-media-to-improve-citizen-engagement-with-government-research-brief (see Texas example)

8. http://www.govtech.com/e-government/Two-Way-Citizen-Engagement-in-Elk-Grove.html?utm_source=related&utm_medium=direct&utm_campaign=Two-Way-Citizen-Engagement-in-Elk-Grove

9. http://www.mpa.unc.edu/sites/www.mpa.unc.edu/files/Amy%20Strecker.pdf (see p 5)

10. http://www.slideshare.net/AmericaSpeaks/assessing-public-participation-in-an-open-government-era

11. DOT Public Involvement and Reference Tool

d. Metrics: How Do You Know Clearly Defined and Communicated Your Objectives

Metrics will vary according to what platform you use. Some broad examples include:

1. Surveys – response drives your learning

2. Web – google analytics or heat mapping can give you input as to user engagement.

3. Email – click rates, bounce rates, unsubscribes, participation in a call-to-action

4. Social media – user engagement analytics, reach, have you changed a user’s behavior, hashtag use

5. App (native and mobile) – use statistics, are "transactions" being completed, is there input or feedback from the field, are people actively using a native app (ie logging in multiple times).

6. Online video-enhanced chats (or hangouts) – participation numbers, views,

7. Thread, or chat room style participation – number of participants, input from the public, up votes / down votes

8. Blog – viewership, comments, action audience takes next

9. Transactional – hard numbers related to online transaction completion

There are more options out there and a mix of metric inputs, measured alongside the defined goals can be reported out in a consistent and easy-to-understand format (graphs, spreadsheets, infographics, documents with tabular reporting, etc.). Metrics reporting on your goal can be relatively straight forward. For example, if your organization wants 50,000 people to visit your digital property and give you input, you can measure that success with raw numbers in simple analytics reporting. If you want 15,000 of those people to then interact with another one of your digital properties, you can measure that as well, but may have to articulate the context in which you know you have either succeeded or need to rethink your participation play.

2. Select appropriate design format for public participation

a. Introduction

(in development)

b. Checklist (Steps or Questions)

1. Determine hosting platform and key features needed (i.e. ability to interact/comment, blogs, social elements, multimedia capabilities, accessibility, SEO)

2. Apply current coding techniques in build a flexible design -- HTML5, CSS

3. Incorporate high-quality graphics, color and visual techniques

4. How Intensely Does the Audience Use Mobile? Can be determined via Web Analytics. If it's a mobile audience ensure you use tools that are mobile-friendly (responsive web design, native apps) and link to mobile friendly pages.

c. Government Case Studies

1. The White House Engage website allows citizens to submit questions and comments, join online events, and engage with government via social media.

2. Streamlined Design and New Features USDA.gov underwent a major redesign to improve user experience and usability. They used analytics and lessons learned from prior website redesigns to determine popular content and user preferences.

3. AIDS.gov Responsive Website Redesign AIDS.gov was among the first full-scale federal websites to use responsive design to deliver better digital services to any device, anytime, anywhere. It allows individuals and providers to enter a ZIP code and find federally-funded HIV testing and care-related services within a selected mile radius.

4. NOAA Release Mako App was created for fisherman to report their releases of Shortfin Mako sharks while on the water. The app uses a device’s built-in GPS, when available, to fill in exact location coordinates.

5. API - HIV Services Locator (AIDS.gov)

6. The Victoria (Australia) Tookit for Public Participation has a great selection of different PP formats.

7. Determine Participation Objective (ie what do you hope to gain from the public's participation? (Awareness, Education, Input, Decision-Making) - refer to diagram on Pg.8

8. from EPA's website: [Selecting the right level of Public Participation.] Unsure if this is uptodate.

d. Metrics

1. increased readership/visits to sites/blog/etc.

2. increased awareness about federal agency's goals, mission, purpose,

3. measurement of public participation (survey responses, event attendance, other contributions ie. time, ideas, etc.)

4. measurement of time spent on site, bounce rate, clicking patterns and high-, low traffic areas

3. Understand and communicate the benefit of participation

a. Introduction

(in development)

b. Checklist (Steps or Questions)

1. Define the expectations from the participant

2. Define participants have to give so they can participate (e.g., information, permission, time)?

3. Define how citizens will benefit from of the process (e.g., access, influence, status…)?

4. Define what participants need to know before they engage? Ensure that all participants have access to the same information at the start of your process.

5. Define the value that the organization will provide back to the participant for their time.

6. Make sure participants know where they are in the process (are we at the beginning of a consultation? in the middle?

7. Inform participants how the input received during participation will be used.

8. Communicate when participants should expect to hear follow-up from their engagement.

9. Build a communications strategy that includes messaging regarding expectations for participants as well as the value of participation.

c. Government Case Studies

1. Census Hackforchange : Civic hacking, a positive type of community building around data, allows programmers to harness the power of publicly available government statistics in order to create apps that benefit everyone.

2. National Day of Civic Hacking : Governments and other organizations submit problem statements that explain the nature of an issue that needs addressing, what they hope the public will do to help address the problem, and resources to help orient people to the problem. Event organizers outline time commitments and skills needed to effectively participate.

3. Regulation Room : Regulation Room is pilot project sponsored by the Federal government and operated by the Cornell e-Rulemaking Initiative (CeRI). Participants are given resources to better understand the rulemaking process, how to make an effective comment, and have the opportunity to browse the issues that are most relevant to them. Discussions among participants are encouraged. After discussions have ended, participants comment on a report of the activity to ensure their views are captured before the final version is sent on to the government.

4. Give a Minute : A micro-participation/crowdsourcing event where the expectations of participants are clear: they give one minute to answer a question like, "How can we encourage more biking/walking?" The ideas are collected and shared publicly. Participants can be engaged digitally or in-person.

**d. Metrics **

1. Conversion rates, from visitor to sign ups

2. Returning visitor rates

3. Sharing & promotion rates (e.g. recruitment activity from participants)

4. Quality of participation (e.g., level of discussion (comments on ideas), relevance of feedback)

5. Participant satisfaction with the process

6. Amount of new participants/users

4. Understand your participants and stakeholder groups

a. Introduction

(in development)

b. Checklist (Steps or Questions)

1. Who are you trying to reach?

2. Identify the types of users you are targeting with your outreach

3. Consider creating personas of your target audience so you can understand and relate to their needs better.

4. If you don't have a clear idea of target audience you can do some outreach and sampling to better define and understand them

5. What might be obstacles you need to overcome to reach these people (think lack of Internet, busy moms who don't have time, single parents juggling family responsibilities, etc.)

6. Getting adequate participation from the various subgroups identified (don't want to skew results by not having a representative sample)

7. How you recruit these people may vary

8. Method in which you are seeking participation (not everyone has internet access, some may only use mobile devices, social media trends are changing, etc.)

9. Tool to facilitate participation should be selected AFTER your project objectives and target audience have been identified.

10. Build into your strategy how you plan to work around those obstacles to reach and engage your target audience.

11. How will you ensure you’re reaching a diverse group of people with broad perspectives?

12. What stakeholders care about the issues presented? Are there other stakeholders who might be interested if presented with the information?

13. Defining stakeholders can be difficult. Sometimes its easy to ask 'Who has a stake in the project?' An open ended question like might tell you who you *need *to invite, not just who you want to invite? Typically if there is a benefactor of a project, there may be a party who is adversely and tangentially affected as well.

14. Are there gatekeepers (i.e. community groups, professional societies, industries) who can help you reach your stakeholders?

15. What types of engagement strategies might be effective with each of these varying types of stakeholders?

16. Plan your audience based on the lifecycle of your product. Today, your audience may look one way, but what does the data tell you? What will your audience look like for the life of the project? Neighborhoods change over time. Heck, even countries do.

17. Meet your audience where they are, not where you want them to be. This can mean literally: go to their community, their platform, their schools etc... and don't expect them to come to your office or location. This also means figuratively in terms of valuing multiple perspectives, cultural norms and institutions.

c. Government Case Studies

1. Department of Transportation http://www.planning.dot.gov/focus_caseStudies.asp

2. National Geospatial Intelligence Agency: Ebola Relief Website and Press Release

3. Peace Corps Application Process Redesign (increases in applicant numbers and completed applications)

4. Open Gov Wiki https://opengovdirective.pbworks.com/w/page/4874180/Participation%20Case%20Studies%20and%20Effective%20Practices

5. CDC: Gateway to Health Communications & Social Marketing Practice provides resources to help build health communication or social marketing programs. It provides tips for analyzing and segmenting an audience, choosing appropriate channels and tools, and evaluating the success of messages or campaigns.

6. Dept of Energy: Identify Target Audiences and Behavior Changes

d. Metrics: How do you know you understand your participants and stakeholder groups

1. Total number of users that participated

2. Number of users in sub-groups identified

3. Compare findings among subgroups. Analyze for similarities and differences

4. Identify quantity of users needed to be a representative sample size (total needed broken out by number needed within subgroups)

5. Consider baseline testing could be helpful to help track impact of project upon completion.

5. Design for inclusiveness

a. Introduction

(in development)

b. Checklist (Steps or Questions)

1. Accessibility for persons with disabilities

2. DHS Section 508 Compliance Test Processes

3. Evaluate the need for multi-lingual support

4. *Ensure Plain-language *

5. DigitalGov Social Media: You Still Need Plain Language

6. Online and offline support

7. Cultural Competency support

8. Valuing diversity

9. Having the capacity for cultural self-assessment

10. Being conscious of the dynamics inherent when cultures interact

11. Having institutionalized culture knowledge

12. Having developed adaptations to service delivery reflecting an understanding of cultural diversity

13. Consider multiple learning styles. Some large projects are extremely complex and include science or engineering concepts that can be difficult certain learning styles. Consider (if you have funding or a volunteer) visual notetaking or graphic facilitation online or in meeting to keep a visual record of the conversation. Employ handouts or models.

14. Questions to always ask:

15. Is your contact information available on your social media account page?

16. Is your social media content available through more than one channel?

17. Did you provide links or contact information to official social media support and accessibility teams?

18. Is your design simple? Did you write in plain language?

19. Do you periodically test your content for accessibility?

c. Government Case Studies

1. Food and Drug Administration: Using Medicines Wisely: Increasing Access to Information for Women with Intellectual Disabilities

2. FDA's Office of Women's Health teamed up with the Administration for Community Living, the Association of University Centers on Disability and the Administration on Intellectual and Developmental Disabilities to improve their safe medication materials for women. They conducted a series of focus groups with women in the disability community to learn how word choices, font, layout, and graphics could be improved.

3. Improving the Accessibility of Social Media in Government toolkit curates and share best practices to help agencies ensure their social media content is accessible everyone, including users with disabilities. Efforts are also being made to work with social media platform and tool developers, citizens and partners to encourage greater accessibility.

4. Ensuring the Accessibility of Web Content . The Digital Communications Division (DCD), part of the office of the Assistant Secretary for Public Affairs (ASPA), leads the development and review of HHS Web content, social media, and supporting technologies.

5. The Department of Health and Human Services Accessibility Lab . The Digital Communications Division has assembled a 508 Lab with a collection of assistive and evaluative technologies for use by OpDivs and StaffDivs.

6. Guide to Section 8 Standards . The U.S. Access Board’s Section 508 Standards apply to electronic and information technology procured by the federal government, including computer hardware and software, websites, phone systems, and copiers. They were issued under section 508 of the Rehabilitation Act which requires access for both members of the public and federal employees to such technologies when developed, procured, maintained, or used by federal agencies.

7. Usability.Gov: Accessibility . Evaluating your website with both usability and accessibility in mind.

d. Metrics

1. W3C Web Accessibility Metrics . Detailed report on useful metrics to assess the accessibility level of websites, including the accessibility level of individual websites, or even large-scale surveys of the accessibility of many websites, including:

2. The number of pictures without an alt attribute.

3. The number of Level A and AA success criteria violations.

4. The number of possible failure points where accessibility issues can potentially happen (such as the number of images in a page).

5. The severity of an accessibility barrier.

6. The time taken to conduct a task.

7. Accessibility in Practice: A Process-driven Approach to Accessibility . The best approach to accessible user experience is to integrate accessibility into the design and development process.

8. Section 508 Checklist . Pass/Fail criteria for each of the Section 508 standards (e.g., a text equivalent for every non-test element, functional electronic forms).

6. Provide effective and timely notifications

a. Introduction

(in development)

b. Checklist (Steps or Questions)

1. Develop a comprehensive outreach plan that includes messages, timelines, and strategies for reaching different audiences, etc.

2. Use a variety of tools and channels (online and in the real world) to invite people into the conversation.

3. Repeat your messages. Ensuring that people are aware of what's happening at all stages of the participatory process is important. If you make one-time announcements about opportunities for citizens to participate there's no guarantee they'll all hear. By repeating your messages, you will significantly increase the chance of more people participating.

4. Work with community leaders and stakeholders to reach your target audience. Citizens may never think to come to your agency on a given topic, but they can learn about opportunities for participation from community groups, local service providers, religious organizations, etc.

5. Provide post-event notifications to participants thanking for their participation and providing information on how their feedback will now be used and incorporated.

6. Leverage the Federal Register

7. Take baseline measurement of past, similar, events.

8. Provide participation notifications that fit each community best (e.g.,: phone, postal, email, social media)

9. Reach out to leaders in the community

c. Government Case Studies

1. USGS and Employee Use of Social Media : The Union of Concerned Scientists released a scorecard on how agencies allow employees to communicate with the public. USGS received a B+, but updated it's public documentation to address suggested improvements within four hours.

2. DigitalGov Search seeks to provide immediate answers to the public’s search questions. In addition to the Federal Register documents, it also incorporates results from other specialized government websites and social media accounts.

3. USA.gov encouraged people to share their #ElectionSelfies , showing off "I Voted" stickers. They monitored trending election hashtags on Twitter and reached out in real time to people questions about voting.

4. U.S. Immigration and Customs Enforcement’s Operation Predator App allows anyone submit anonymous tips to the ICE Tip Line about suspected predators—either wanted fugitives or previously unknown—24 hours a day, seven days a week.

d. Metrics

1. Measure of the overall participation in the event compared to previous efforts

2. Measure online engagement trend (e.g., comments and shares) over entire length of event. This should show your notifications are working better than before.

3. Measure of the positive and negative feedback, post-event, on how participation discussions will be used.

7. Encourage community development and outreach

a. Introduction

(in development)

b. Checklist (Steps or Questions)

1. Moderate and facilitate – encourage connections and discussion of other peoples’ ideas. When discussion stalls, have a plan to catalyze the discussion. At regular intervals, pause to "share out" and let participants see how else they can contribute (this is easy in an in-person meeting, and the analogy in an online thing is the weekly emails that summarize the conversation so people can see what other people are saying)

2. Identify the appropriate community organizations to reach out to.

3. Identify the appropriate point of contact in each organization to be your point of contact.

4. Determine what value the community organization gets out of helping you. Make sure there is value in it for them, so that they feel like they have a stake in a successful outcome of the event.

5. Share control and Decision-Making

6. Shared responsibility and decision-making can be challenging, but it creates shared success, shared problem solving, and shared accountability. A well-designed dialogue process can enable those involved to discover common concerns and values and agree on a framework for moving forward. Create guidelines for how this will work. This is a more advanced step for a well-defined process but can be one of the most rewarding.

c. Government Case Studies

1. The Commodity Futures Trading Commission hosted a smart investing webinar in partnership with AARP . Both organizations are able to reach out to their own communities to encourage people to sign up and participate.

2. HHS Office on Women's Health encouraged community organizations around the country to organize events in celebration of National Women's Health Week 2014. They used the platform meetup.com so organizations could add their own events and search for things in their area.

3. FDA's Office of Women's Health launched the Pink Ribbon Sunday Mammography Awareness Program to educate African American and Hispanic women about early detection of breast cancer through mammography. By partnering with churches and community organizations, there were able to reach over 100,000 women throughout the country.

4. The 2010 National League of Cities (NLC) research report, entitled * Making Local Democracy Work: Municipal Officials’ Views About Public Engagemen*t , as part of a project on civic engagement and local democracy. The report found that municipal officials often provide opportunities for public engagement and through various venues and media. However, a significant point was the response to the question about the effectiveness of existing public engagement activities.

5. US Department of State Bureau of Educational and Cultural Affairs Exchange Program fostering Community Engagement through the Arts. The American Arts Incubator uses new media and/or mural arts as a means for engaging youth, artists, and underserved community members in overseas communities in Asia to advance U.S. foreign policy by addressing a local community issue, such as women’s empowerment, HIV-AIDS prevention, social inclusion, conflict resolution, and the environment. ( Example not a case study )

d. Metrics: How we know we're encouraging community development and outreach

1. Increased number of community organizations support this event compared to previous events

2. Size of community that your partners reach

3. Does the program become integrated or owned by the community?

8. Empower participants through public/private partnerships

a. Introduction

(in development)

b. Checklist (Steps or Questions)

1. Define what a partner is and how partners are approved. Is there an MOU or IAA in place or some sort of contractual agreement with the agency?

2. If there is not one, is there some other way in place to make sure both the agency and the company are getting value out of the partnership?

3. Outline the terms of the partnership. Think about messaging, logo usage, promotional opportunities and boundaries.

4. Reach out to nonprofits and community partners who could help amplify the message and/or connect with valuable corporate partners.

5. Also support and outreach to Senior Citizen/ elderly communities

6. Some of this may overlap with what we we have above about working with gatekeepers. May want to consider moving gatekeeper info here.

7. Volunteering! how can they help? serve?

8. Overlaps with inclusiveness too. What vehicles?

c. Government Case Studies

1. The Heart Truth : The Heart Truth campaign, created by the National Heart Lung and Blood Institute (NHLBI), part of the National Institutes of Health (NIH), has partnered with dozens of corporate and community sponsors since its launch in 2002 in an effort to raise awareness about heart disease and its effect on women. And many of these partners have carried out its mission by encouraging public participation. For example, in 2014, corporate sponsor Charles P. Rogers held an online auction of one-of-a-kind custom beds through eBay's Charity Works platform which attracted hundreds of bids and raised nearly $4,000. All of the proceeds benefited the Foundation for the National Institutes of Health in support of The Heart Truth campaign.

2. LabTV : In 2014, LabTV launched its vision of creating a free, scientist-to-student web/video platform aimed at inspiring the next generation of researchers. LabTv features thousands of videos focused on the scien tist , rather than science. Through online tools offered on its website, LabTv matches young people with working scientists who share similar traits such as "hated math in school," or “grew up with a parent who is a doctor.” Once a student finds a medical scientist that they can relate to, LabTV allows participants to connect with the scientist so the student can learn about life as a medical researcher, and how they can follow in their footsteps. In support of this effort, the National Institutes of Health (NIH) worked with LabTV to produce hundreds of videos of NIH researchers with varying backgrounds and interests.

3. STop.Think.Connect : Launched in 2010, Stop.Think.Connect is a program created by a coalition of private companies, non-profits and the Department of Homeland Security (DHS), with leadership provided by the National Cybersecurity Alliance, and the Anti-Phishing Working Group. By partnering with industry leaders, DHS is able to attract a broader audience and garner significant public participation for online events like Twitter chats to help spread its message. In October of 2014, DHS teamed with dozens of public and private organizations including Microsoft, McAfee, Norton, Intel and Visa to host a series of Twitter chats to highlight National Cybersecurity Awareness Month.

4. Distraction.gov : Distraction.gov is a website operated by the U.S. Department of Transportation (DOT), dedicated to stamping out distracted driving. This includes, texting, talking on the telephone, eating, drinking or any other activity that takes a driver's focus off the roadway. To this end, DOT has enlisted the help of several organizations to spread its message and to motivate the public to join them in its cause. One partner, the FOX television show "Glee", helped produce public service announcements depicting a character from the show texting while driving, and consequently getting involved in serious car accident. Distraction.gov has a Call-To-Action on its website, specifically encouraging the public to take (and sign) a pledge stating their commitment to being a safe and distraction-free driver. DOT also uses this website to encourage the use of social media tools for individuals to share online, and offer instructions on how to host an event dedicated to stopping distracted driving.

5. Federal advisory committees

6. Challenge.gov

7. CitizenCorps

d. Metrics

1. Value and measure in-kind services donated by private partners, such as air-time, printing, advertising, or prizes.

2. Use clearly defined web metrics to identify participation levels when utilizing online tools such as social media or websites,

3. Compare participation from past events and activities that did not include partnerships to help measure the value of the partnerships.
   

9. Provide multi-tiered paths to participation

a. Introduction

(in development)

b. Checklist (Steps or Questions)

1. Determine the level of participation you're looking for ( IAP2 standards ). Are you empowering stakeholders as decision makers? Involving them? Sharing information? Are you just trying to interact with them?

2. Determine that you have the resources -- personnel and otherwise -- to handle a multi-tiered, two-way engagement strategy.

3. Define the information you're looking for, whose input you value most, and what communication channels they're most likely to use. Focus your resources there.

4. Create clearly defined action items. Leave feedback HERE on THIS ISSUE.

5. Acknowledge and respond to feedback*.*

c. Government Case Studies

1. Case Studies in Thunderclap The Department of Health and Human Services (HHS) used Thunderclap to support Open Enrollment Season for Healthcare.gov. The Environmental Protection Agency (EPA) successfully promoted its campaigns for Earth Day, #ActOnClimate ! and Be SunWise on Don’t Fry Day .

2. The Army's Assembled Chemical Weapons Alternatives Program has established two citizens advisory boards which are empowered to make recommendations on the path forward for the project. The program holds quarterly public meetings with program leadership and maintains an outreach office that is open to the public and fully staffed to receive in-person feedback. Program also maintains social media accounts, direct email, and website - runs the gamut of options for interaction. (also bases its outreach program on IAP2 guidance)

3. FAFSA's #AskFAFSA campaign sets up office hours for stakeholders to ask questions about student aid in multiple languages via Twitter: https://studentaid.ed.gov/node/372

4. The U.S. Trade Representative has a traveling roadshow that gathers feedback on trade agreement negotiations. There's an email address to receive information and the office hosts a Twitter account, though I'm unsure if it is set up to receive and pass on feedback.

5. FDA has examples of multiple levels of participation, many are listed on its Patient Network site www.patientnetwork.fda.gov/ . Citizens can choose the level of participation that works for them and become more involved as appropriate.

6. Comment on proposed regulations and guidance online or by mail.

7. Attend public meetings in-person or by webinar.

8. You can have a more active role in the meeting by registering to speak in advance.

9. Become a member of an advisory committee to influence decisions like product approvals.

d. Metrics

1. Public meetings held/attendees

2. Number of comments received per channel

3. Number of avenues available for feedback

4. Subjective metric: Value of feedback, credibility of source...

5. (i gotta look back at some old comms plans here...)

10. Maintain responsive, accountable feedback loops

a. Introduction

(in development)

b. Checklist (Steps or Questions)

1. Build in time to pause and take stock. Like locks in a canal, these are important moments where you may decide to change direction, stay on the same path, or bring in a new audience.

2. Celebrate Success! If the feedback is positive celebrate it. with your participation group If corrections are needed, address in a transparent and informative (not critical) manner.

3. Begin the conversation and outreach efforts with civic and community engagement efforts that do no necessarily have a direct 'ask' or end in mind. Civic Engagement is a* continuous* conversation with the public that reinforces public commitment.

4. In Person, this may include hosting 'meet an expert' sessions in community or online that focus on getting to know each other and understanding cultural norms. Building relationships through these open-ended interactions will come in handy when its time to ask for directed feedback or input.

5. This is also important in terms of surveying the field for past or present occurrences or projects that may inadvertently sideline your process. If a community or group is or was embroiled in a 'hot topic' or painful scar, your great idea or important process may not stand a chance.

6. Don't let the conversation end. You have just invested a tremendous amount of time and energy into building relationships. Use the framework you created for participation to develop future uses for the network. Perhaps the process has highlighted an issue or future product that would also benefit your agency and the public.

c. Government Case Studies

1. We the People: The Decision to Excavate the James Dexter Site . The decision by the NPS to reverse its earlier position and excavate the James Dexter site due to engagement with the city's African American community.

2. Collaborating for the Future . New Bedford Whaling National Historical Park uses ongoing roundtable dialogues with communities of interest to implement the park's General Management Plan.

3. From Isolation to Integration: Valley Forge National Historical Park transforms itself through civic input.

4. Evaluating EPAs Public Participation Process in Superfund Sites.

5. Beyond Outreach: A guide to engaging diverse communities .

6. Tribal and Public Consultation for Historic Preservation Toolkit

7. Environmental Protection Agencies Public Participation Guide

8. Using Social Media to Enhance Civic Engagement in Federal Agencies

d. Metrics

1. Begin with an Assessment of where you are. Assuming you have identified your target audience, generate a solid baseline of data on how much your audience knows and how well they respond to your message. Then you can move toward asking how public understanding of your message has stayed the same or improved.

2. The National Park Service has created a website to house its public participation process. This web-based approach allows you to tracks hits, collect comments, conduct surveys etc..

3. Involve the public in measuring their role in the planning/decision-making process.

4. Why are you evaluating?

5. To improve how community engagement is done – will it be important to continuously improve our operations during the life of the project, and to be seen to have a focus on quality, efficiency and effectiveness of processes?

6. To gain insights into what is effective community engagement in different situations - will it be important to learn something about the process (can it be applied in another situation) or the community (their needs)?

7. Who wants to know what?

8. What evidence will be collected? How?

9. Victoria (Austrialia) Page 12 , Tools for Evaluating Public Involvement

10. Resources for the Future: Evaluating Public Involvement

11. Managers Guide to Evaluating Civic Participation

12. Determine schedule and resource allocation

13. Start with the basics. Did the process meet the objectives? Stay on schedule, on -budget etc...?

11. Use data to drive decisions

a. Introduction

(in development)

b. Checklist (Steps or Questions)

1. Clearly define your goals.

2. Establish key performance indicators.

3. Have a hypothesis to measure.

4. If your hypothesis is correct, what are your next steps? If it's wrong?

5. If something does not add value for the public, are you willing to eliminate it? Do you have the resources to do an overhaul?

c. Government Case Studies

1. [Data.gov](http://www.data.gov/developers) is a rich resource for civic hackers, tech entrepreneurs, data  scientists, and developers of all stripes. It includes information about APIs, open source projects, and relevant developer resources across government. 
   

2. API Release Kit describes the elements agencies should include in federal API releases.

3. Govcode is a website to feature Open source projects by the government.

4. Public Participation in Govt Web Design : The data.gov team integrated feedback from virtual, online, face to face testing, and social media platforms to dramatically change the design in response to the needs of the users of open data.

5. Web Design Changes? Let the Metrics be Your Guide NASA noticed for the first time, metrics indicated that NASA.gov’s mobile users outpaced their desktop users. This confirmed the decision to change to a mobile design.

6. [Government CX: Finding the Metrics That Matter](http://www.digitalgov.gov/2014/01/13/government-cx-finding-the-metrics-that-matter/) The Export-Import Bank measures transaction processing times and ease to create the best experience for their stakeholders.
   

7. Enigma is amassing the largest collection of public data produced by governments, universities, companies, and organizations. Concentrating all of this data provides new insights into economies, companies, places and individuals.

d. Metrics

12. Protect citizen privacy

a. Introduction

(in development)

b. Checklist (Steps or Questions)

1. Invite your agency's privacy office and/or legal experts to participate and provide input on your agency's privacy policies, terms of service, or other charter documents.

2. Include provisions in website and social media terms of use that allow community managers to delete content and comments that contain PII.

3. Include the Privacy Policy on your website and social media platforms.

4. Identify what information will be stored or maintained by your agency, for what purpose the information will be stored, and indicate whether your agency will maintain records of conversations and for how long such records will be maintained.

5. Does your agency extend the same data privacy to U.S. citizens, U.S. residents, and to foreign nationals? Per a May 2014 report on Big Data , the White House may encourage agencies to extend universal Privacy Act protection. Consider getting ahead of the curve.

6. Use hashtags to join conversations online--don't target individual users.

c. Government Case Studies

The following are great examples of best practices, policies, and terms of service that include provisions for protecting PII and that support Privacy Act compliance.

1. CIO Council's Privacy Best Practices for Social Media : This comprehensive guide released in 2013 provides resources and best practices for standing up a Privacy and Records Keeping Act compliant social media policy.

2. White House Expands Data Protection : How can your agency use big data? There is no formal guidance yet, but this May 2014 report and recommendation from the White House provides insight on how and when agencies may be able to use big data from the Internet of things, the web, mobile, and more; and whether agencies will be required to extend universal Privacy Act Protections.

3. FEMA Privacy & Comment Policy : The FEMA Privacy & Comment Policy comprehensively addresses how social media, web, mobile, and SMS data is used, stored, and managed. The policy is plain-language and concisely explains and justifies the retention and management of different data sets.

4. Peace Corps Privacy Act Notice : The Peace Corps' Privacy notice is a great, plain language example of how to formulate a comprehensive, brief, and easy-to-understand privacy policy for the web.

d. Metrics

1. Do all of your websites and social media sites contain provisions for protecting PII and maintaining Privacy Act compliance?

2. How often do community managers monitor comments for PII?

3. How many comments do community managers delete for PII on a monthly or bi-monthly basis?

4. Do your screenshots or other records sufficiently protect any PII captured by blurring, cropping, or blacking-out names, photos, or other PII?

5. How often does your agency "audit" its records to ensure it is not maintaining broader records than necessary to comply with federal records keeping requirements?

6. Are you using third party websites that collect data or PII in a way that is inconsistent with the Privacy Act?

13. Transparently report outcomes and performance of participation

a. Introduction

(in development)

b. Checklist (Steps or Questions)

1. Decide where you will report outcomes and performance. You may need to report it in multiple places, depending on your stakeholders.

2. What type of formats will be most useful for your audience?

3. Report outcomes in a timely manner while participants are still engaged.

4. Define to the user/viewer/stakeholder why it is important to report outcomes transparently and performance of participation at this particular time.

5. Disseminate information across multiple platforms to reach as many audiences as necessary.

c. Government Case Studies

1. Report: Assessing Public Participation in an Open Government Era

2. Report: Realizing the Potential of Open Government Data: A Roundtable with the U.S. Department of Commerce

3. National Archives and Records Administration - public feedback for the development of the agency's third Open Government Plan as well as NARA reporting outcomes of NARA's previous Open Government Plans

4. USDA - Know Your Farmer Know Your Food compass map shows efforts supported by USDA and other federal partners as well as related information on local and regional food systems.

5. NASA - Disk Detective is a crowdsourcing project whose primary goal is to produce publishable scientific results. It uses citizen science to help astronomers discover embryonic planetary systems.

6. Ford's Theatre: Remembering Lincoln on HistoryPin (Ford’s Theatre is a public-private partnership between the National Park Service and Ford’s Theatre Society.)

7. Transparency International Presentation to Data Transparency Town Hall

8. New York City Transparency Working Group

9. The People's Roadmap to a Digital NYC

d. Metrics

1. Do you have data visualizations and infographics illustrating transparent outcomes and highlighting performance of participation?

2. Do you hold Town Hall meetings and/or give presentations reporting outcomes of performance of participation?

3. Do you conduct web research and run social media analytics for analysis and ways you can report outcomes of participation?

4. Do your blog posts direct users and highlight the results of past and current initiatives that encourage and require public participation?

5. Do you publish articles reflecting outcomes and direct users to articles via social media and other platforms?