Related resources

These resources explore the topics of monitoring and evaluation - in relation to government funding of community and voluntary organsaitions.

Monitoring

Auditor-General's Inquiry into Certain Aspects of Te Wananga o Aotearoa
This report by the Auditor-General discusses poor decision-making practices involving significant expenditure, inadequate management of conflicts of interest and unacceptable practices concerning management expenses – practices damaging to the credibility of a public entity.

Auditor-General's report on NZ Trade and Enterprise Administration of Grants Programmes
This report by the Auditor-General is a performance audit of New Zealand Trade and Enterprise (NZTE) grants programmes following the merger, in July 2003, of Trade New Zealand and Industry New Zealand. It recommended NZTE apply sound administrative principles and standards to all aspects of its grants programmes.

Dangerous Accountabilities: Remaking Voluntary Organisations in Someone Else's Image
Increased accountability is often assumed to only be desirable for non-profit and voluntary organisations. This papers outlines some of the risks and dangers from externally-imposed, 'classic', legalistic accountability arrangements found in contracting. A typology of 'funder capture' risks is presented. Rather than advocating a vacuum in responsibility, the paper examines the Social Audit tool (developed by the New Economics Foundation) as an accountability measure. Garth Nowland-Foreman’s paper presented to 29th Annual Conference of the Association for Research on Nonprofit Organisations and Voluntary Action, New Orleans, Nov 2000.

A Genuine Progress Indicator for the Auckland Region Summary Report – published by the Auckland Regional Council, 2009
Like Gross Domestic Product (GDP), the Genuine Progress Indicator (GPI) is a measure of economic progress. The difference is that GPI also takes into account social and environmental costs as well as the benefits of growth.

Ministry for Culture and Heritage’s Monitoring Process
The Ministry for Culture and Heritage is committed to having clear expectations and maintaining good relationships with the national cultural organisations it funds. This section provides further details on their monitoring process.

Ministry of Social Development’s Performance Measurement Framework

New Zealand Trade and Enterprise: Administration of the Visiting Investor Programme This report by the Auditor-General discusses the management of the visiting investor programme for companies and individuals considering New Zealand as a location for establishing operations. It recommends establishing robust policies and processes for the administration of the programme to minimise the risk of inappropriate expenditure, ensure adequate record keeping, and to improve transparency.

Report on the 2007/08 Audits of the Local Government Sector
The Department of Internal Affairs report on the 2007/2008 Local Authority Census data notes that many local authorities failed to explicitly link any discussion of the effects of activities to community well-being. Some links could be inferred, but this depended on the quality of a local authority’s performance management framework, and in many cases these links were not clearly made.

Statistics NZ Table Builder
Table Builder is a web tool that enables you to build your own tables of data from datasets such as census, business statistics, household economic statistics, recorded crime statistics plus many more. A help section is available on the Table Builder homepage and it includes a series of video clips with step-by-step instructions.

Treasury Guidelines for Contracting with Non-Government Organisations for Services Sought by the Crown: Monitoring Section 4.1
These Treasury guidelines encourage the use of better contracting practices by all departments and Crown entities involved in negotiating arrangements with NGOs for the provision of services.

Treasury Guidelines: Relationship Management Section 4.2
These Treasury guidelines encourage the use of better contracting practices by all departments and Crown entities involved in negotiating arrangements with NGOs for the provision of services.

Reviewing the funding process

The Accountability of Voluntary Organisations: Implications for Government Funders
The mechanisms of accountability of government-voluntary sector contracting are problematic for both government agencies and voluntary organisations. If they are to be revised, new mechanisms need to be appropriate for both parties. While the public accountability system has been relatively well described and analysed, the accountability systems of voluntary organisations have not. This research aimed to explore accountability from the perspective of voluntary sector managers and board members asking to whom, for what and why they thought themselves accountable. Four organisational case studies were undertaken involving 34 in-depth interviews with managers and board members.

Accounting for Something: Voluntary Organisations, Accountability and the Implications for Government Funders
A paper based on Jo Cribb's doctoral research, which aimed to explore accountability from a voluntary-sector perspective and the implications for government funders.NZ Non-Government, published in the Social Policy Journal of New Zealand: Issue 26, Nov 2005.

Making Good Use of Resources
Colleen Pilgrim's article, published in the Chartered Accountants Journal, explores the changing roles and responsibilities of public entities. Over the past 20 years, public sector reform has led to a broadening of the base of service delivery, both in New Zealand and overseas. NGOs are now frequently involved in delivering public services, influencing policy design and implementation, or performing a facilitative role to ensure that government objectives are met. Second, activities that were once carried out by public entities are increasingly carried out by NGOs for public entities. Colleen Pilgrim, Office of the Auditor-General - NZ government, October 2006.

Management of Overseas Aid Programmes: NZ Agency for International Development
This Office of the Auditor-General Performance Audit report outlines concerns about NZAID needing clear and accessible processes and procedures for putting in place and monitoring funding arrangements for delivering aid programmes. The issues of setting performance expectations and monitoring performance are covered for programmes in challenging parts of world.NZ government, 2008.

Office of the Auditor-General - Inquiry into Public Funding of Organisations Associated with Donna Awatere Huata MP
This report examines how one funding agency did or did not take steps – through the medium of the funding contracts – to check and verify that the funds made available had been spent properly and for the purposes for which they had been given. In January 2003, the leader of the ACT New Zealand parliamentary party, asked the Auditor-General to inquire into certain allegations of financial impropriety involving one of his party’s list members, Donna Awatere Huata MP. The allegations involved money owned by the Pipi Foundation Trust, a private trust established by Mrs Awatere Huata in 1999 to deliver a children’s reading programme, which Mrs Awatere Huata had developed in the 1970s. (2003)

Philanthropy NZ’s Giving NZ 2006 report
This report provides measurement of New Zealanders' philanthropic funding for the 2005/2006 year, and what these funds supported.

Survey of Government Funding of Non-profit Organisations gives a broad view of central government’s funding of non-profit organisations
In June 2006, a questionnaire was sent to government departments seeking information on the size of government funding flows to non-profits. The results provide a broad overview of central government’s funding of non-profit organisations. In 2005/06, the largest amounts of departmental payments are made to non-profits in the social services, education and health activity sectors and contract payments for services dominate over grant funding. The survey provided limited information about payments made by Crown entities. There are two distinct streams of funding: payments of $1.25 billion to NPOs from departments and payments of at least $141 million to NPOs from Crown entities(Note: Previous estimates of Crown entity payments to NPOs suggest this figure is likely to be significantly under-reported. In 2007 it was estimated that payments in the order of $2 billion were provided to non-profits for health and disability services from district health boards and other Crown entities in Vote: Health. This estimate could not be substantiated from the survey.)This was the first time that a comprehensive survey has tested information about government payments to this sector. Although exploratory, the survey provided useful benchmark information about the Government’s interactions with the non-profit sector.

Treasury Guidelines for Contracting with Non-Government Organisations for Services Sought by the Crown
Chapter 5 of the Treasury guidelines deals with review and evaluation of contracts. While there is some overlap with monitoring, review and evaluation merit separate treatment given that this extends further than an individual contract.

What the Ombudsmen can Investigate?
A page on the Office of the Ombudsmen's website describing what the Ombudsmen do and do not investigate. Under the Ombudsmen Act, an Ombudsman is able to investigate complaints about the administrative acts, decisions, recommendations and omissions of central and local government agencies. New Zealand government.

Evaluating project progress

Evaluation: Good Practice Participate
If you need to research or evaluate a project, first find out what information already exists. Information to help you can be found on the Good Practice Participate website.

Lead Funder Prototype Pilot
The pilot project was an initiative to review funding arrangements and to seek ways of reducing the compliance costs for community organisations.

Measuring What Matters - To Prove (and Improve) Effectiveness
The website of the Office for the Community and Voluntary Sector includes a number of links to help community organisations and funders focus on what counts when resources are tight.

Nga Puna Wai o Hokianga
Urban Water Decision-Making Project: Learning from the Stories of Nga Puna Wai o Hokianga (2005) reports on a collaborative evaluation undertaken by Institute of Environmental Science and Research (ESR) with Hauora Hokianga and Whirinaki Maori Committee. It found several contributing factors played a part in the pilot being a success, such as the enabling political climate, consultation and liaison, community ownership, project management and the employment of kanohi ki te kanohi as a negotiating and relationship building tool.

Not-for-profit Performance Generates More than Just Figures
This four-hour NZ Institute of Chartered Accountants (NZICA) course helps organisations to identify, describe, measure and report non-financial performance indicators based on the objectives and outcomes they aim to achieve for users, stakeholders, clients and founders.

Simple Evaluation Planning for Non-Evaluators workshops
People working in many different roles have to develop evaluation plans. These workshops teach a process to develop an evaluation based around a visual results model (also called an intervention logic or outcomes model).

Social Policy Evaluation and Research (SPEaR) Committee
SPEaR is a cross-agency group overseeing government investment in social policy research and evaluation.

Trying Hard is Not Good Enough - Results Based Accountability Framework
"Is anyone better off?" is the question at the core of a simple but effective way of measuring results – one increasingly being used by government and non-government agencies.

Overseas guidance and resources

Accountability Glossary
Harvey's 2004 Analytic Quality Glossary. Provides some interesting definitional issues around the term "accountability" (2004)

Alliance Magazine
Alliance Magazine has free online articles on impact evaluation and information required to make social investment decisions. USA

Better Purchasing Guides: Queensland Government
The 'Better Purchasing Guides' support the State Procurement Policy by providing information about current best practice procurement in the Queensland Government. Titles include: ‘Ethics, Probity and Accountability in Procurement’, ‘Managing and Monitoring Suppliers' Performance’, ‘Relational Contracting Guide’, and ‘Value for Money’. The full series of 28 stand-alone procurement guides is available as downloadable booklets via this link. Australia, 2009.

Beyond the Checkbook: A Financial Management Guide for Leaders of Small Youth-Serving
From the Finance Project, which helps leaders make smart investment decisions, develop sound financing strategies, and build solid partnerships that benefit children, families, and communities. USA.

Building a Common Outcome Framework to Measure Non-Profit Performance
The USA’s Urban Institute and its project partner, The Center for What Works, collaborated from June 2004 through May 2006 to identify a set of common outcomes and outcome indicators or “common framework” in the measurement of performance for non-profits. The aim was to avoid non-profits with multiple projects and multiple funders having to deal with different requirements for tracking outcomes for similar programs.The idea was that if agreement on a common core set of outcome indicators can be reached, then outcome reporting can be efficient and focused. (These aims are similar to New Zealand’s own Integrated Contracts initiative and the work done to introduce integrated contracts.)The report from the US provides suggested core indicators for 14 categories of non-profit organisations and then expands the notion of common core indicators to a much wider variety of programmes by suggesting a common framework of outcome indicators for all non-profit programmes.

Delivering on the Promise of Non-Profits
This article suggests that to be truly effective, non-profits need to rigorously confront a few essential, interdependent questions: Which results will we hold ourselves accountable for?, How will we achieve them?, What will results really cost, and How can we fund them? Harvard Business Review, USA, December 2008.

Does Government Funding Suppress Nonprofits’ Political Activity?
Autonomy from the state has been considered a core feature of civil society, and understanding the consequences of perceived threats to that autonomy has been a central theme in social and political theory. This article examines a specific question: What is the effect of government funding on non-profit organisations' political activity?Research identifies some mechanisms by which government funding might reduce non-profit political activity and others that might enhance such activity. By investigating this relationship with two data sets (a national sample of religious congregations and a longitudinal sample of non-profit organisations in Minneapolis–St. Paul), the researchers find some consistent and compelling results: the relationship between government funding and non-profit political activity is either positive or nil; government funding does not suppress non-profit political activity. Published by the American Sociological Association in the American Sociological Review, Volume 69, Number 2, 1 April 2004 , pp. 292-316. The full article can be purchased online

Does Public Accountability Work? An Assessment Tool
In this article, the authors develop an instrument for systematically assessing public accountability arrangements, drawing on three different normative perspectives. In the democratic perspective, accountability arrangements should effectively link government actions to the ‘ democratic chain of delegation ’ . In the constitutional perspective, it is essential that accountability arrangements prevent or uncover abuses of public authority. In the learning perspective, accountability is a tool to make governments effective in delivering on their promises. M Bovens, T Schillemans and P T'Hart. 2007.

Framework for Collecting Performance Measurement Data – an example
This sample performance-measurement framework draws on work with not-for-profit organisations in the United States involved in job search assistance and job training. The framework is used to identify key organisational goals. Some quantitative data is collected for each goal to show what has been achieved. Data is collected over several years to show trends. Putting all the data on one form allows stakeholders to quickly see what progress is being made for each goal. Community organisations can use this type of format for their own goals and performance measures. USA

Getting More for Less: Efficiency in the Public Sector – Demos
This report argues that the route to public sector efficiency is to focus on effectiveness. Effective services are personalised - driven by people's needs. They take aim at the cause of problems rather than the consequences, and they are delivered collaboratively. Services driven by these principles result in better outcomes for citizens, a better quality of service, and happier staff. They also save money because getting things right the first time always works out cheaper. United Kingdom.

House of Commons Debates: Third Sector Funding
Oral Answers to Questions to Ed Milliband, the former UK Government's Minister for the Third Sector. Interesting figures on total government funding of the third sector. There are doubts about these figures and they may be an underestimate. International Government, United Kingdom, 18 April 2007.

Intelligent Monitoring – an element of Financial Relationships with Third Sector Organisations
Practical guidance, published by the UK’s National Audit Office in 2009, designed to save charities and other voluntary and community organisations time and money by reducing the paperwork they are required to collect when providing public sector-commissioned services. The guidance is in response to a 2007 report that suggested UK charities were being required to jump through an inordinate number of bureaucratic hoops to access local and central government funding, with funders often demanding excessive amounts of inappropriate information. United Kingdom, 2009.

Managing Contract Performance - A Transaction Costs Approach
The complex task of managing contracts is often exacerbated by high transaction costs inherent in negotiating, implementing, and monitoring contract relationships with vendors. Through analyses of data from a 1997 study of the ways municipal and county governments respond to transaction cost factors inherent in contract service delivery is examined. The results of the analyses demonstrate that when governments contract for services in contexts that risk contract failure, they engage in a variety of monitoring techniques to improve their ability to monitor and correct vendor performance.T Brown and M Potowski, the Association for Public Policy Analysis and Management. 2002.

Measuring Impact - Who Counts?
An interview with the President of the Hewlett Foundation, Paul Brest stating that funders need to get better at assessing the impact of the money they spend. He notes that it’s clearly impossible to have a separate, specifically funded evaluation for every grant. He asks, "When are evaluations needed, and how can funders be persuaded to pay for them? And how good are foundations at sharing information?" A revealing and informative interview which examines the issue of evaluation and who should pay. International Non-Government, From Alliance magazine (Vol 12, No. 4) for Philanthropy and Social Investment Worldwide. USA, Dec 2007.

New Guidance to Help Cut Red Tape
Charities that receive public funding have to account to government funders for how they have spent this money and should show the impact they have achieved with it. The cost of producing this information, however, must be proportionate to the risks and benefits involved. Cutting unnecessary red tape can free up time and money better spent focusing on the key services charities and others provide. The term for achieving this balance and avoiding poor practice is ‘intelligent monitoring’. New intelligent monitoring guidance, published by the UK’s National Audit Office along with the Office of the Third Sector (OTS) in the Cabinet Office and HM Treasury, will help the UKgovernment cut paperwork, while still enabling it to monitor the £12 billion it gives to charities and other voluntary and community organisations each year. New principles for the monitoring of funding for the third sector have also been unveiled. United Kingdom

Non-profits for Hire: The Welfare State in the Age of Contracting
This report provides strong arguments for reform of government contracting procedures when funding non-profits. As Director of the Nancy Bell Evans Center on Nonprofits and Philanthropy, Professor Steven Rathgeb-Smith argues strongly for government to invest in the equity capital and operational needs of non-profits. Steven is Professor of Public Affairs in the Evans School of Public Affairs at the University of Washington. USA (1993)

The Price of Complacency
Voluntary organisations risk being sidelined by the private sector unless they can show the public that their money is well spent. An article in the UK Guardian by Martin Narey, chief executive of Barnardo's. The article is adapted from a speech made to the Association of Chief Executives of Voluntary Organisations annual conference in late November 2007 (Guardian 5 December 2007)

Principles of Proportionate Monitoring and Reporting
Issued by the UK’s Office of the Third Sector in the Cabinet Office. The principles and guidance aim to lessen the unnecessary burden of monitoring on charities, social enterprises and voluntary organisations and help them and departments gain better value from it. The principles commit Government departments to understanding the cost of reporting for third sector organisations and to working closely with them when establishing monitoring requirements. The principles were produced as a response to a 2008 report by New Philanthropy Capital called Turning the Tables: Putting Charities in Control of Reporting. United Kingdom, 2009.

Public Service & the Third Sector: Rhetoric & Reality – House of Commons
This first ever Select Committee report focused on the third sector was released in July, and specifically looks at the policy of commissioning public service delivery from organisations in the third sector. The UK Public Administration Select Committee wanted to assess the impact on government, the third sector and, most importantly, 'the effect on service users and the public at large'. United Kingdom.

Sustaining Grassroots Community-Based Programs
Time-limited grant funding and categorical funding streams leave grassroots organisations in a constant scramble to find funds and pay rent, utilities, salaries, and programme expenses. Relying on grant awards and contracts alone will not ensure a future for such organisations: Once those funds are gone, critical services for clients may cease to exist. Again and again, organisations that rely on funds from grants and contracts come back to the same question: “How do I find the resources to sustain and support my programme services over time?” A Toolkit for Community & Faith-based Service Providers – Substance Abuse & Mental Health Services Administration Center for Substance Abuse Treatment, US Department of Health & Human Services USA.

Toward Greater Effectiveness in Community Change: Challenges and Responses for Philanthropy.
This paper identifies challenges within three core dimensions of philanthropic activity: how foundations think about community change, how they do their community-change work, and how they learn from their efforts. By Prudence Brown, Robert J. Chaskin, Ralph Hamilton, Harold Richman - Chapin Hall, USA.