Part of the answer here is that not every organization – particularly small community-based or non-governmental ones – will necessarily have extensive resources to conduct a formal evaluation.  They may have to be content with less formal evaluations, which can still be extremely helpful in providing direction for a program or intervention.  An informal evaluation will involve some data gathering and analysis. This data collection and sensemaking is critical to an initiative and its future success, and has a number of advantages.

1.  The data can show whether there was any significant change in the dependent variable(s) you hoped to influence. Collecting and analyzing data helps you see whether your intervention brought about the desired results.  The term “significance” has a specific meaning when you’re discussing statistics.  The level of significance of a statistical result is the level of confidence you can have in the answer you get.  Generally, researchers don’t consider a result significant unless it shows at least a 95% certainty that it’s correct (called the .05 level of significance, since there’s a 5% chance that it’s wrong).  The level of significance is built into the statistical formulas: once you get a mathematical result, a table (or the software you’re using) will tell you the level of significance.   Thus, if data analysis finds that the independent variable (the intervention) influenced the dependent variable at the .05 level of significance, it means there’s a 95% probability or likelihood that your program or intervention had the desired effect.  The .05 level is generally considered a reasonable result, and the .01 level (99% probability) is considered about as close to certainty as you are likely to get. A 95% level of certainty doesn’t mean that the program works on 95% of participants, or that it will work 95% of the time.  It means that there’s only a 5% possibility that it isn’t actually what’s influencing the dependent variable(s) and causing the changes that it seems to be associated with.

2.  They can uncover factors that may be associated with changes in the dependent variable(s). Data analyses may help discover unexpected influences; for instance, that the effort was twice as large for those participants who also were a part of a support group. This can be used to identify key aspects of implementation.

3. They can show connections between or among various factors that may have an effect on the results of your evaluation.  Some types of statistical procedures look for connections (“correlations” is the research term) among variables.  Certain dependent variables may change when others do.  These changes may be similar – i.e., both variables increase or decrease (e.g., as children’s proficiency at reading increases, the amount of reading they do also increases).  Or the opposite may be observed – i.e. the two variables change in opposite directions (as the amount of exercise they engage in increases, peoples’ weight decreases).  Correlations don’t mean that one variable causes another, or that they both have the same cause, but they can provide valuable information about associations to expect in an evaluation.

4.  They can help shed light on the reasons that your work was effective or, perhaps, less effective than you’d hoped.  By combining quantitative and qualitative analysis, you can often determine not only what worked or didn’t, but why.  The effect of cultural issues, how well methods are used, the appropriateness of your approach for the population – these as well as other factors that influence success can be highlighted by careful data collection and analysis.  This knowledge gives you a basis for adapting and changing what you do to make it more likely you’ll achieve the desired outcomes in the future.

5.  They can provide you with credible evidence to show stakeholders that your program is successful, or that you’ve uncovered, and are addressing limitations. Stakeholders, such as funders and community boards, want to know their investments are well spent. Showing evidence of intermediate outcomes (e.g. new programs and policies) and longer-term outcomes (e.g., improvements in education or health indicators) is becoming increasingly important to receiving – and retaining – funding.

6.  Their use shows that you’re serious about evaluation and about improving your work.  Being a good trustee or steward of community investment includes regular review of data regarding progress and improvement.

7.  They can show the field what you’re learning, and thus pave the way for others to implement successful methods and approaches.  In that way, you’ll be helping to improve community efforts and, ultimately, quality of life for people who benefit.