Chapter 2.
Conceptual Framework

Chapter 3.
Participatory and Consultative Process: Deciding What Issues to Study

Chapter 4.
Methodological Guidelines

Chapter 5.
Verification of Results and Further Consultations

SAPRI FIELD INVESTIGATION  METHODOLOGICAL FRAMEWORK October 1997

1) This exercise at its foundation is a participatory one involving civil society, the World Bank, and governments in a broad consultative process as well as in research to review structural adjustment policies. It seeks, in particular, to facilitate the full participation of those that have traditionally been excluded from decision-making in the assessment, deliberations, and research on structural adjustment programs in the participating countries.

2) The research component of the SAPRI exercise will be geared toward targeted learning on those issues about which there is an identified need for further understanding as identified through the opening National Forum and deliberations of the National Steering Committees.

3) The study will take a gender aware political economy approach, meaning that a key goal of the research is to understand political, social, and institutional structures and processes that shape policy decisions and impacts.

4) As such, we are interested in both understanding the complex relationship among policies, intermediating economic, social and political institutions (e.g. markets, regulatory and legal frameworks, government etc.) and outcomes at the individual, family, and community and sectoral level and, to the extent possible, establishing direct attributions of causality. To reach such a full understanding we must look at the full range of groups and sectors both those who have been favorably and unfavorably affected by policies, with emphasis placed on the economically and politically marginalized groups.

5) The review will recognize the equal validity of quantitative and qualitative methods of research and, whenever possible, use more than one approach to validate information.

6) In light of the participatory and consultative nature of the exercise, and the range of countries and actors involved, there will be flexibility regarding processes employed, methodological approaches utilized, and time frame needed for completion of the study.

7) The results of this review will be the basis for serious review of adjustment measures and development of mechanisms for including all parties in policy design in future policy reform operations.

Chapter 2. Conceptual Framework

Political economy approach. In the entire SAPRI review process, both in the consultative activities and in the field research, a political economy approach is required. The political economy approach acknowledges and analyzes the realities of concentration of wealth and power, differential access to markets and resources, the impact of discriminatory policies and practices, and the complexity of factors shaping human action.

Linking macro to micro effects. This implies a focus on the intermediating institutions, markets and authorities that link the macro level to the micro. Matching the evidence of impact at the macro level with that at the micro level will help facilitate the identification of those institutional mechanisms and processes, as well as other intervening structures that shape policies and their outcomes at each level, across sectors, and amongst groups.

Units of analysis. Incorporating various levels of analysis into the research design, and respecting the inter-relatedness of the political and the economic, requires that the researchers analyze different units of analysis--the whole economy, different sectors, markets, and institutions, groups within sectors, on down to households and individuals. Regardless of unit or level of anlaysis, all data should be disaggregated by gender.

Range of Policies

The country teams will identify specific adjustment policies (not just Bank-financed adjustment measures) and examine the processes, markets, and institutions that have intervened in the application of the programs in order to fully understand the impact of the policies. Typical adjustment policies may be categorized as follows:

1. The role of the state in the economy

 privatization
 public expenditures
 deregulation, etc.

2. Trade and investment policy

 domestic and foreign investment related policy
 import liberalization
 export promotion
 competitiveness programs, etc.

3. Macro-economic stabilization

 monetary policy
 fiscal policy
 balance of payments
 exchange rates, etc.

Categories and Indicators of Impact

There are various categories of impact that may be considered in the review. These impacts may include final policy outcomes, as well as effects on intermediate processes through which the outcomes are determined. The indicators that may be used to measure the impacts will vary widely, from conventional indicators (such as per capita household expenditures, income levels, wages, food prices, employment, etc.) to less conventional indicators (such as levels of social capital, quality of life, environmental and resource quality, etc.).

Welfare, however broadly measured, will be the ultimate indicator of interest and can be thought to encompass many aspects of the following categories of areas in which policies may have an impact:

1) Basic rights (education, health, food security, employment);

2) Insecurity/vulnerability;

3) Poverty/inequality/income;

4) Productive capacity;

5) Market structures (labor, land, etc.);

6) Social/institutional capital; and

7) Environment/natural capital;

8) Gender/reproductive capacity.

Indicators will vary or overlap among these categories, and in all cases, there should be a focus on the differential impact based on gender. The research teams should also be sensitive to the differential impact on various ethnic groups and regions. See Appendices A and B for examples of questions relating these categories of impacts to different levels and sectors of the economy, and Appendix C for a discussion of research tools and data. (The Appendices A and B have no ambition to cover all issues or impacts that may be encountered during the exercise; they are also not strait-jackets that must be observed by all researchers regardless of local conditions. They are simply given to provide some guidance as to the possible ways to approach the topic).

Chapter 3. Participatory and Consultative Process: Deciding What Issues to Study

Participatory nature of the process

The participatory nature of the process (see Principle 1) is reflected in the inclusion of all segments of society via the National Forum and the processes leading up to it in the discussion of effects of economic policies, and in the peoples ability to articulate and formulate the issues which need to be studied. This Chapter describes the participatory and consultative nature of the process.

Preparatory work

Prior to the convention of the opening National Forum (see section below) which is a defining moment in the SAPRI process preparatory work needs to be done at the global and national level. At the global level, preparatory work includes formation of Global Steering Committee and Technical Teams responsible for country selection, definition of methodology, and organization of the Global Launch. The Global Steering Committee is also responsible to make sure that all relevant information pertaining to adjustment policies be properly disseminated and accessible to all participants in the SAPRI process. Preparatory work at the global level also includes the Global Launch of the SAPRI which should ensure both the publicity for the exercise and that all elements needed for the work are firmly put in place.

At the national level, numerous Civil society organizations (CSOs) are organizing: they are constituting their own Steering Committees (which may be time-consuming due to the extremely large number of NGO/civil society organizations that are participating), debating the issues they would like to see studied within the framework of the SAPRI, and establishing contacts with governments and the

World Bank. At the level of national governments, ministries and individuals who will be government representative on the National Steering Committees are being designated. The same process is going on in the World Bank.

Following the Global Launch, the National Forums will be convened.

The format for the country fora will depend on the particulars of each country situation. For example, a preliminary forum or a series of mini-fora may be convened by civil society organizations to give voice to civil society actors on all key argument issues before several (approximately three to four) issues are selected for the opening National Forum.

The purpose of the opening National Forum is threefold. First, it should begin the public dialogue about adjustment, its multiple impacts, and joint analysis and problem identification among citizens groups, government and Bank officials. Second, the information, perspectives, and analysis presented at the forum by civil society and that which emerges from the ensuing discussion with government and Bank officials will be reflected in the final results of the country exercise. The third purpose is to select issues for research.

It has been tentatively agreed that in each opening forum the joint (NGO/CS, government and Bank) information team (defined in the Information disclosure agreement) will provide a basic history of adjustment loans, supporting policies and programs, and the macro-economic and sectoral objectives of those policies. Though a series of panels, civil society will present their analysis of the outcome of the identified adjustment measures on their respective constituencies. Bank and government officials will also be participants in these panels and will offer their analysis.

Through this process areas of agreement and disagreement among the participants and areas where there are significant information gaps will be identified, and consequently the research questions that should be decided upon. Emphasis should be placed on research questions that have important implications for policy and/or address basic assumptions about adjustment programs; issues identified by groups that have borne a disproportionate burden of the costs associated with adjustment measures and/or those that have failed to respond to economic stimuli as expected; and issues for which there are gaps in the research for which targeted learning will have significant payoffs. The joint steering committee and the Technical team should remain open to the emergence of pressing issues/policy concerns in this meeting that may alter their preliminary issue selection.

After the forum, the national technical team will formulate the exact research hypotheses, define the methodology to be followed (as per discussion in Chapter 4 below; see also examples in Appendices A and B), and draft the terms of reference for the researchers who will do the study. In order to minimize a possible conflict of interest, only a half of the members of the Technical Team may serve as researchers. The research hypotheses and the terms of reference will have to be ratified by the National Steering Committee. The research questions, hypotheses and terms of reference will have to be approved by the National Steering Committee, using the criteria embodied in the principles.

The researchers which will conduct the studies will be selected by the two parties: NGO/Civil society, and the Bank/government. Each party will propose the list of suitable of researchers. (The researchers can come from the Technical Team too, under the proviso from mentioned in the previous paragraph.) The final list of researchers (composed of an equal number of researchers proposed by the two parties) will be decided upon by the technical committee by consensus. The Technical Team will, all other elements being the same, give preference to local researchers who speak the language and reside in the countries which participate in SAPRI. While the duration of field research will vary depending on the country and complexity of the topic, it is important that some guidelines about when the first draft of the study must be submitted to the Technical Team for review be observed. It is suggested that the first draft of each study be completed and submitted to the Technical Team 6 to 12 months after the final selection of the researcher.

Chapter 4. Methodological Guidelines

The richness of this study will derive from its holistic approach applied to the analysis of the impact of structural adjustment. Such an approach requires the use of a combination of research methods that are best suited for the targeted learning this exercise will undertake. Techniques to be used and who the informants/participants are will depend on a range of factors such as the sectors identified, the policy instruments being reviewed, and distinctive characteristics of each country. The country teams, with support from global technical teams need to sort these issues out.

Nonetheless, all the teams must be mindful of the following guidelines:

The purpose of this study is to analyze the outcomes generated by the introduction of macro-economic interventions with a particular focus on how policies interact in systems characterized by concentrations of power and wealth; differential access to markets, information, and productive resources by both individuals and groups; and discrimination based on factors such as ethnicity or gender (see Principle 3). In other words, economic restructuring has differential inter-sectoral, intra- sectoral, inter-household, and intra-household impacts, and those impacts in turn create new sets of economic and power relationships which influence the outcomes of the next round of policy interventions. This study should link more explicitly macro-, meso-, and micro changes with a focus on outcomes and behaviors at the community level, as this is an area where systematic research, data collection, and incorporation of results has been deficient in policy development.

For example, under this approach, an analysis of fiscal reform would not simply look at the design of policy or its elements, such as tax reform, but would also analyze the government's willingness and ability to implement such reform, different groups ability to engage in high levels of tax evasion, compensatory efforts by the state to generate tax revenues from those over whom it has some control, the immediate and longer term impact of those efforts on the well-being of those individuals, communities, and/or sectors, and the conditions this has created for the viability of future reform efforts.

Furthermore, in keeping with the conceptual framework, researchers need to employ methods that will evaluate impacts not only in terms of economic outcomes (e.g. impact on incomes and production) but also will weigh environmental impacts, the impacts on social and institutional capital, distribution of income, and general welfare and quality of life issues, with a particular focus on the differential impacts on women.

Establishing causal relationships

The research is meant to establish the relationship between adjustment measures and outcomes with the view of identifying future interventions (i.e. policy alternatives) that maximize well-being in both economic terms and in terms identified by those groups that are the focus of this exercise -- those that have traditionally been marginalized from decision-making and who have not benefitted substantially from adjustment programs or have been harmed.

Hypothesis Generation

The starting point for the analysis should be a hypothesis or set of hypotheses which clearly identifies what the expected outcomes of a policy or packages of adjustment policies were, including their impact on alleviating poverty and establishing the conditions for sustainable development (looking at both the economic and political viability of these policies). A second order of hypotheses will identify what intervening factors either contributed to or hindered expected outcomes.

Stated in general terms, we are concerned in answering the following questions:

1. Were objectives as outlined in World Bank and government documents achieved? Are claims of success/failure correct?

2. For whom were these policies beneficial, neutral, or negative and what characterized these individuals/groups/communities?

3. Did expected negative impacts correct themselves (e.g. did people who lost jobs/income find new employment/income)?

4. What have been the secondary (or other unanticipated) effects -- economic, social, environmental, etc. (e.g. on productive capacity, health, social infrastructure, gender division of labor, use of non- renewable resources)?

5. What were the economic, social and political factors (both national and global) that shaped the selection of policies to begin with and how did these intervening factors -- for example, the nature of gender roles within the household or high concentrations of wealth pre-adjustment -- contribute to the success or failure of policies overall and/or for specific groups and sectors?

6. In the final analysis, were these objectives the correct ones and were outcomes desirable? Did they or do they have the potential for alleviating poverty and generating broad-based prosperity?

To establish causality time series data will need to be used to establish baseline (pre-adjustment) information and well as identify change over time after adjustment measures have been implemented. Comparative analysis will also need to be carried out among those that have been able to respond to the economic incentives provided by adjustment, and those that have not (see Principle 4). In this sense, sampling will be inclusive. At different levels of analysis, the sampling should also be representative in that individuals/groups/communities should reflect the range of characteristics that may effect policy outcomes (e.g. female/males; indigenous/non-indigenous; dominant party affiliation/minority party affiliation; land reform community/non-reform community; etc.)

In keeping with the participatory and consultative nature of SAPRI (Principle 1), the research should utilize techniques that incorporate respondents analysis and assessment of the changes they have experienced under adjustment and what they would like to see in the future. While it is not expected that the research can undertake exhaustive public opinion surveys, it is important to solicit information that will help policy makers understand what people value as this will help policy-makers predict how they will respond to incentives and to policy proposals. This is particularly important in democratic systems.

Combining quantitative and qualitative data; traditional and participatory research methods

Given the goals of this research, it is clear that a combination of quantitative and qualitative methods need to be employed (see Principle 5). Traditional, quantitative methods are powerful research tools, however, researchers have increasingly agreed that employing a combination of methods greatly enhances the quality of research.

First, data may not exist about many of the questions that are of interest to the researchers, and needs to be constructed in cooperation with participating communities. Second, the meaning of data generated by quantitative analysis may not be clear and other means may be needed to complete the analysis (thus the popularity of combining survey research with focus group discussions). Thirdly, there are some things that are simply not easily measurable, yet may be decisive in explaining outcomes, such as the behavior of key institutions and even individuals.

The National research teams, once the key adjustment-related issues are discussed at the opening National Forum, are faced, along with the Technical team and National Steering Committee, with the challenge of identifying the information gaps that need to be filled and determining the combination of methods that will most effectively explain current outcomes and establish the types of information base needed for future policy interventions that enhance the well-being of workers, small producers, the poor and their communities. It is likely, for example, that gender-disaggregated data will not be widely available on the types of issues of interest to the researchers. This may require additional effort on the part of the national research teams to collect information about and from women, which should be planned for during the research design phase.

Chapter 5.  Verification of Results and Further Consultations

In keeping with the consultative nature of the research exercise and its contribution to the broader debates on economic policy, the National Steering Committee is expected to call a second forum, once an initial draft of the research is completed, for presentation, discussion, and verification by the public. The results of the field research and the information, perspectives and materials presented at the opening National Forum will provide the basis for the findings of the national review process, the development of recommendations for policy changes, and for the integration of civil society in future economic policy making.

(back to top of page)

Return to Operational Guidelines

Return to SAPRIN home