Improve RetroPGF Review Process: Mission Request Draft

Overview

Here is a draft mission request to improve the RetroPGF review process. We’d also be happy to share the notion page where this is written and give you access to edit or leave comments directly on the page if you’d like. You can see this article for a solution that we envision to execute this mission. Thanks for reading, we hope you enjoy and find it helpful!

Mission Request

Delegate Mission Request Summary:

Enhance the review process for RetroPGF by creating more efficient processes for impact evaluation, reducing burden on badgeholders, and increasing discoverability for applicants as more participation grows in future rounds. Teams creating proposals for this mission should provide solutions that make RetroPGF more effective at rewarding impact, while also making the experience more enjoyable for everyone in the Optimism Collective.

S5 Intent 5: Improve governance accessibility (Intent 4) or Grow the Superchain (Intent 2)

Proposing Delegate: Delegate name/pseudonym, linked to delegate profile

Proposal Tier 7: Please specify the Tier required for Mission Applications under this Request (ie. applications must be submitted by teams at the Fledging Tier)

Baseline grant amount: This amount should reflect the minimum amount required to execute the Mission. Additional rewards may be received via RetroPGF, based on impact/outperformance. We recommend a 50/50 split between Mission/RetroPGF rewards in Season 5 to incentivize quality execution. Over time, the proportion of baseline grant to retroactive rewards should shift towards RetroPGF until everything is funded by RetroPGF.

Should this Foundation Mission be fulfilled by one or multiple applicants: Select from: “One,” “Up to X” or “Multiple”

Multiple

Submit by: To be set by Grants Council

Selection by: To be set by Grants Council

Start date: If applicable

Completion date: If applicable. Please note Missions must be completed within 12 months (i.e. marked as done).

Specification

How will this Delegate Mission Request help accomplish the above Intent?

This Delegate Mission Request aims to improve the governance accessibility of Optimism by incentivizing solutions that make the review process more efficient and enjoyable for everyone involved. RetroPGF has grown significantly in each round. Round 1 had 76 projects, Round 2 had 195 projects, and Round 3 had 644 projects eligible for voting. With about 145 badgeholders volunteering to distribute life-changing amounts of funding amongst so many projects, there was a lot of pressure for both badgeholders and applicants. As a result, a large amount of the feedback from both badgeholders and applicants indicated that the experience was stressful.

The most consistent feedback from badgeholders during the evaluation process was around the overwhelming quantity of projects to review. The most consistent feedback from applicants was a high threshold of votes needed to earn funding. While round 3 introduces some changes that help with the review process and how applications are presented, these changes don’t address the issue at the fundamental level. Unfortunately, this is a situation where RetroPGF is a victim of its own success: the more successful it becomes at public goods funding, the more applications it will attract and the more burden it will have to put on the badgeholders.

This Delegate Mission Request will help improve the governance accessibility in the following ways:

• Increase the scalability and resiliency of RetroPGF
• Improve the experience and understandability of RetroPGF
• Enable RetroPGF to recur on a more frequent cadence by creating a smoother experience for all participants
• Enable the Collective to more effectively fund public goods creators who are creating a positive impact

All of these benefits will help grow the Superchain by fostering development on the OP Mainnet and help improve governance accessibility for everyone interested in the Optimism Collective.

What is required to execute this Delegate Mission Request?

There are various ways that the Mission Request can be executed. Options include:

• Create better systems for impact measurement, metrics, and evaluation. These systems should be easily accessible, understandable, and accurately convey the impact of public goods creators by tapping into the wisdom of the crowd.

• Develop a user interface for RetroPGF that shows impact metrics, prioritizes projects that deserve more funding, and enables allow citizens to sort projects according to helpful measurements.

• Develop tooling that enables badgeholders to easily create lists based upon impact measurement, metrics, and evaluation. It would also be helpful to develop tooling that enables more detailed ranking and measurement of lists.

• Provide resources and/or structured events where RetroPGF applicants can promote their work and badgeholders can easily learn about public goods projects.

• Develop a credibly neutral qualification system where projects must be nominated by badgeholders in order to be eligible for funding in RetroPGF

You can see an example of how this mission might be accomplished by reading this post.

How should the Token House measure progress towards this Mission?

Measuring progress towards the mission of enhancing the review process for Optimism's RetroPGF requires a multifaceted approach, given the complexity and diversity of the goals involved. Here are some suggested metrics and methods to effectively track progress:

1. Survey Feedback from Participants: Conduct regular surveys with badgeholders and applicants to gather feedback on their experience. Improvement in satisfaction scores over time would indicate progress.
2. Application and Review Efficiency Metrics: Track the average time taken to review each project and the time applicants spend completing the application. A reduction in these times would indicate more efficient processes.
3. Impact Metrics for Funded Projects: Develop and track key performance indicators (KPIs) for projects funded through RetroPGF. These could include the project's reach, user engagement, or specific outcomes aligned with the project's goals. Improvements in these metrics would indicate a successful funding impact.
4. Volume of Applications and Fund Distribution: Monitor the number of applications received per round and the diversity in the distribution of funds. An increase in applications alongside a more equitable distribution of funds could signal a more accessible and appealing process.
5. Badgeholder Participation and Workload Metrics: Measure the number of badgeholders actively participating in each round and track their workload (e.g., number of projects reviewed per badgeholder). An increase in participation with a manageable workload would suggest a more scalable and badgeholder-friendly system.
6. Qualitative Analysis of Project Outcomes: Beyond quantitative metrics, regularly review case studies or success stories of funded projects. This qualitative analysis can provide deeper insights into the program's effectiveness in funding impactful projects.
7. System Usability Metrics: If new tools or interfaces are developed, track usability metrics such as user error rates, task completion times, and user satisfaction scores.
8. Applicant Success Rate and Repeat Applications: Monitor the success rate of applicants and the rate of repeat applications. An increase in successful first-time applicants and repeat applications can indicate a more encouraging and clear process.
9. Public Perception and Community Engagement: Assess public perception through social media sentiment analysis, community engagement in forums, and participation in public discussions about RetroPGF.
10. Innovation and Diversity in Funded Projects: Track the variety and innovation in projects receiving funding. A broader range of innovative projects being funded would demonstrate success in fostering a diverse ecosystem.

By employing a combination of these quantitative and qualitative methods, Optimism can gain a comprehensive understanding of the progress being made towards the mission and make informed adjustments as needed.

1. Survey Feedback from Participants: Conduct surveys with badgeholders and applicants to gather feedback on their experience. Improvement in satisfaction scores over time would indicate progress. The surveys could track the average time taken to review each project and the time applicants spend completing the application. A reduction in these times would indicate more efficient processes.

2. Impact Metrics for Funded Projects: Develop and track key performance indicators (KPIs) for projects funded through RetroPGF. These could include the project's reach, user engagement, or specific outcomes aligned with the project's goals. Improvements in these metrics would indicate a successful funding impact.

• Ad-hoc calls with winner applicants which present their implementation progress using presentations, presenting key metrics and source of latest tech developments so Token House delegates can accurately track execution

• Applicants can provide trackable milestones containing expected completion dates for each milestone

• Token House delegates can join events and/or look through the provided resources where RetroPGF applicants can promote their work and badgeholders can easily learn about public goods projects

• Token House delegates can check and evaluate the credibly neutral qualification system provided by applicants, where projects must be nominated by badgeholders in order to be eligible for funding in RetroPGF

How should badgeholders measure impact upon completion of this Mission?

• Create a feedback form targeting a group of individuals who have previously applied to RetroPGF and/or are familiar with the process in previous rounds. The purpose will be to collect their feedback in both, quantitative and qualitative manner

• Create a feedback form targeted at badgeholders and evaluate their feedback on whether the mission request enhancements had contributed positively to their overall evaluation and voting processes

• Present github repos showing developed tooling, including ones that enable more detailed ranking and measurement of lists

• Have you engaged a Grant-as-a-service provider for this Mission Request? If so, please disclose the details of this arrangement, to the extent possible:

No.

• Has anyone other than the Proposing Delegate contributed to this Mission Request? If so, who, and what parts of this application did they contribute to?

Application Instructions:

To be defined by the Grants Council in accordance with their internal operating procedures. Mission Applications must collect email addresses and twitter handles. Applications should specify whether they are applying for a growth or builders grants or a combination of the two (in which case, the proportion of rewards should be clear.) Suggested Template here 3.