A complete and adequate description of the components of a program is essential to assess its implementation.
– The strategies,
– Ways of communication and
– Technologies for the implementation of the program and the specification of the beneficiaries and where the implementation takes place.
A proper and accurate identification of the components of the program will assess what aspects of the program were implemented as planned, and what factors of possible influence in the implementation differences.
The correct specification of the components to assess concerns as the scope of the program (intended beneficiaries) was observed. Moreover, conjecture about the possible links between the results of the implementation and results of the program itself (in terms of production, intermediate results, impacts, etc.) …
At the same time the specification (or detail) of the contents of the program is a requirement the process for evaluation.
The own initiative to plan and carry out the assessment process contributes to a specification most appropriate and realistic content of the program. This is an important condition to ensure that the program is more effective (because the internal consistency of the program has gone through a preliminary screening) and, secondly, that the evaluation of results and impact is more effective since the program's performance will be compared with targets and more consistent and realistic expectations.
To allow the assessment that the process can improve the design and specification of a public program, some techniques may be used.
1 – Formative Assessment: based on data collected from pilot projects and beneficiaries over the conduct of a particular intervention and giving information on the feasibility of certain activities and instruments and to what extent they are appropriate to the design plan and beneficiaries provided;
2 – Verification of the "evaluability" systematic set of procedures for the proper development of the theory behind a public program, detail and clarify theplanned uses for the data in the evaluation process, before the start of an assessment in full scale.
His most important steps include (Scheirer, 1994: 49-50):
a) Involve key policy makers, managers and staff through a series of meetings to clarify their expectations for the program and the assessment itself;
b) Using a model called matrix logic diagram, detailing the expected causal relationships between three aspects of the program: resources allocated to the program, implementation of specific activities planned the program and expected results;
c) Refinement of the theory behind the program by an interactive process, using visits to project sites and information available, to examine the reality of operations in the field and the extent to which proposed theory is plausible;
d) Clarify the uses planned for the information obtained from the evaluation, through discussions with policy makers and managers of the program, including changes in the program;
e) Use of theory to help in the specification of the program. theories application of relevant to the substantive issue of which comes the program, and the use of data to elucidate the underlying processes.
This type of evaluation process is important not only to specify the content of the program but also to link program activities with measures (indicators) of income to be used in subsequent impact assessments.
The term theory here refers to the inter-related principles that explain and to assume the behavior of a person, group or organization.
Chen (1990) distinguishes two types of theories:
– The normative, which defines what a program should be and
– The causal, which describes empirically the causal relationships between proposed solutions (including contextual factors) and outcome.
The central problem in this case is to investigate the effectiveness of the program and to achieve this purpose, it uses the mechanisms to establish causal relationships between actions of a program and the final result.
The purpose of such assessment can be defined as to identify the net effects of a social intervention. Like the evaluation of goals, this approach is held after the end of the program or the same steps.
Assessment processes – This type of evaluation research in a systematic manner the development of social programs for the purpose of measuring the coverage of the social program, establish the degree to which it is reaching the target, and, especially, monitor their internal processes. The objective is to detect possible defects in the development of procedures to identify barriers and obstacles to its implementation and to generate important data for your reprogramming, through the record of events and activities.
Thus, the appropriate use of information produced during the development of the program allows changes in its content during the execution. Unlike, therefore, the previous approaches, this method of assessment is carried out simultaneously with the development of the program, also called formative assessment. Its implementation requires, however, we can design flows and processes of the program.
Moreover, presupposes the existence of adequate management information system, which served as the basis for the work of managers and evaluators when appropriate.
An application of the methodology of evaluation of social programs:
A comprehensive assessment system using methodologies that provide for the evaluation of results and evaluation processes. Further, the settings and forms of operation used in the proposed model.
Assessment of results:
Here, results are defined as consisting of immediate results, results (impacts) and the medium-term results (impacts) for the long term.
For the evaluation it is suggested the use of impact indicators for measuring the results of long-term, related to the objectives of the program and output indicators to measure the immediate results and medium term. The output indicators measure the effects of the program: from the target population as a whole and among u
sers of the program. In the first case, should be raised two types of output indicators, with research in the field or the help of databases and / or existing entries:
– Degree of global coverage:
Measures the rate of coverage of the target population for the program. Both the deficit and the surplus of people benefiting are the reasons for changes in the route. The first demonstrates the need for expansion, and second, that there is waste of resources (non-eligible as target population are benefiting);
– Degree of coverage varies from program:
Measures the participation of different subgroups of the target population proposal. This rate can portray the discrimination (or bias) in the selection of clients of the program depending on region, age, sex etc .. On the second point, ie the evaluation of results to users of the program, can be used to measure indicators of benefits, which take into account the specific objectives of each program or project.
Rob Vos (1993) gives some examples of indicators most commonly used among users of the program and target population:
1 – for programs of nutrition – malnutrition rates by age, mortality and morbidity;
2 – for programs of education – illiteracy rates, the repetition, of evasion; coefficients of schooling and degrees of education;
3 – to programs of health – mortality rates in general, child mortality, maternal mortality, and birth, of fertility and life expectancy at birth;
4 – for housing programs – quantitative deficit for housing, quality of construction of housing and availability of basic services. The indicators show the input means or the resources available to achieve the objectives. Scarce resources and inadequate (in financial terms, of labor, equipment etc.). Almost always tend to undermine the expected results.
Vos (1993) mentions some examples of most common indicators of input such as:
a) – to nutrition programs – availability of food per person;
b) – to education programs – relationship pupil / teacher, student / school, number of series offered by the school and availability of teaching materials for students;
c) – for health programs – number of doctors per capita, for health posts per capita; of beds per inhabitant and vaccines available per capita.
But the indicators of access to identify the determinants that make effective use of available resources in programs to achieve the goals envisaged. The most common are:
a) – to programs of health – the number of medical consultations per adult equivalent; distance to nearest health service, disposable income per family (as useful to facilitate the purchase of medicines, for example) and cultural factors;
b) – for programs of education – away from the school, curriculum adequacy and income available for family (to enable, for example, the purchase of school supplies).
Furthermore, the use of questionnaires allows for the satisfaction of the customer, being a good indicator of quality but not the only nor the most complete. In this sense it is still possible to establish composite indicators through the construction of indices formed by a set of attributes defined from the characteristics of the service.
Evaluation Process The evaluation process can be defined as a way to identify the real content of a public program, where he is being held as planned, is reaching the audience for which it was intended and whether the benefits are being distributed in the planned intensity ( Scheirer, 1994: 40).