Abstract
In the first part of this chapter, the hypotheses will be tested by making comparisons between the explanatory power of the five specifications of the overarching implementation model. Do implementation agencies make use of their room for maneuver and the presence of political dissension to deviate from political decisions? the explanatory power of these model specifications will be analyzed on the basis of the data collected on the implementation of the three local authority policy programs: social renewal in Weststellingwerf; the restructuring of social-cultural work in Groningen; and neighborhood oriented work in Arnhem. These policy programs contain a total of 134 policy performances. It will be investigated whether, as expected, the mixed conflict model offers the best explanation for the practice of implementation.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
Notes
Because the predictions are based on as many known variables as possible, the actual outcome of the political decision, as identified independently of the model variables, was used in the model analyses. the size of the toleration interval was also calculated on the basis of the actual political decision. Another possibility is to take the decision as predicted by a model of decision making. This is explored in the last section of this chapter.
The mathematical formulas belonging to different models of policy implementation were implemented in an SPSS command. It is then possible to conduct statistical analyses on the outcomes of the simulations. the program can be obtained on request from the author.
The actual policy performances and policy deviations were measured at a later time point when the implementation was being conducted. In addition, these measurements were operationalized independently of the model variables.
These two measures were discussed in section 3.7.
All policy decisions were scaled between the zero and one. the values ‘0’ and ‘1’ were assigned to the two most extreme policy alternatives (cf. Bueno de Mesquita and Stokman, 1994: 217). This method of scaling was discussed extensively in chapter four.
Such cases occur when a consensus between the decision makers and the implementation agency is reached at an early stage in the decision making. This is true of twelve cases in Weststellingwerf (40 per cent). There were only six cases (9.6 per cent) in Groningen in which the implementation agency’s policy position was the same as the political decision. This is true of seven cases (16.6 per cent) in Arnhem.
In the graphical presentation of the model error, the cases in which the political decision was the same as the agency’s policy position are included. This does not affect the form of the curves (the differences between the models). Given the hypotheses, this is what we are most interested in. the inclusion of these cases does have some effect on the height of the curves.
If a negative sign occurs in the least number of cases in the example, the difference in the model error is generally positive, and the model error of model C is smaller than the model error of a in most cases.
The squared difference between the predicted policy performance Ă´di and the actual policy performance odi on the policy scale is smaller for the mixed model than for the political decision model.
It was also tested whether the three models indeed generate significantly different model errors. This was tested with the help of Friedman’s test. This test is a generalization of Wilcoxon’s test to more than two variables, in which an analysis of variance is applied. Friedman’s test indicates the presence of significant differences between the model errors in the policy program of restructuring in Groningen. Neither in Weststellingwerf nor in Arnhem can it be demonstrated that the political decision, the implementers’ preference or the mixed model differ from each other in terms of model error. This points to a problem regarding the statistical ‘power’ of the test in Weststellingwerf and Arnhem, caused by the relatively low number of cases. the implication is that care should be taken not to reject hypotheses without due consideration.
The null hypothesis, H0: (MSEA — MSEC) = 0, can be tested against another alternative hypothesis, H2: (MSEA — MSEC) ≠0, in Groningen. No direction is specified when testing this hypothesis. the two tailed Wilcoxon’s test reveals that the error of the political decision model is significantly (p >.01) smaller than that of the mixed model.
Because the political decision model predicts no deviation whatsoever, this model will, by definition, underestimate any policy deviation which actually occurred. Likewise, by definition the political decision model will never overestimate the amount of policy deviation.
The result for Arnhem is somewhat surprising because the smallest model error occurs by an α of 0.25.
The information in the table also shows that the implementers’ preference model underestimated the actual policy deviations in a limited number of cases. In practice, these implementation agencies appear to have deviated to a greater extent than we would expect on the basis of their policy positions, as indicated by the key informant. This can be explained either by an incorrect estimation of the policy position or by a change in the policy position over time.
The method of calculation is presented in appendix A. the actual political decisions were referred to when making these calculations.
In the local authorities of Weststellingwerf and Arnhem the error of the political conflict model is significantly (p <.01) greater than the error of the mixed model. Therefore, there is no point to conducting this analysis on the data from Weststellingwerf or Arnhem.
In Groningen, the model errors of the political decision model and the political conflict model are significantly smaller (p <.01) than the errors of the mixed model and the mixed conflict model. Therefore, there is little point conducting this analysis on the data on implementation in Groningen.
The null hypothesis can also be tested against a second alternative hypothesis: the hypothesis that there is a difference between the model errors, H2: (MSEC-MSED) ≠0. the null hypothesis can be rejected on the basis of the data on Arnhem with respect to the version of the mixed model with the assumption that α = 0.25. the best fitting version of the mixed model becomes significantly worse on the basis of the data from Arnhem if the effect of political conflict is included!
In addition, the pattern of the model error of each of the five model specifications was studied for each of the issues. With the exception of budgetary issues, this pattern is identical to the total picture for each local authority policy program. the model errors associated with budgetary issues have the same pattern as that found in Groningen. This can be explained by the fact that the implementation of financial decisions is often subject to stricter procedures.
In this study, the variables in the policy formation model were mostly measured shortly after the policy formation in the local authority council took place. the variables from the policy formation model were measured prior to the policy formation in the local council only with regard to the issue of ‘establishing the neighborhood platforms’ in Groningen. Once again, it is emphasized that the variables in this study were established before the implementation.
The number of influence rounds has to be established on the basis of theoretical considerations. In this study, the number of influence rounds was set to two. the first round concerns the formulation of an administrative proposal for the governing body of the mayor and office holders, followed by decision making within the governing body. the second round concerns the decision making in the local council on the proposal from the governing body of the mayor and office holders.
This criterion is used by many researchers (cf. Laumann and Knoke, 1987; Stokman and Van den Bos, 1992). According to this criterion a prediction is either correct or incorrect. the outcome of the model prediction can in principle take the value of any of the points on the policy scale. With regard to policy decisions with a number of ordered policy alternatives (discrete decisions), the policy alternative closest to the predicted value is taken as the predicted outcome. If this predicted policy alternative is the same as the policy alternative decided upon, then the prediction is correct. With regard to continuous decisions, such as is the case with budgetary decisions, a 12 per cent interval is constructed around the policy alternative decided upon: this is the interval within which a prediction is assumed to be correct (Bueno de Mesquita and Stokman, 1994:221). the criterion used by Berveling (1994: 268, n. 17) is that the prediction should not be more than one standard deviation from the actual decision.
In all the previous analyses of model fit, the actual political decision is used. It is empirically possible (although theoretically unlikely) that those models which fit the practice of implementation best on the basis of the actual political decision, cease to do so when we use the predicted political decision. to test this hypothesis, it was analyzed whether the best fitting models using the predicted political decision predicted worse than the other models (which were also based on the predicted decision). Analyses with the help of a Wilcoxon’s Matched Pairs Signed Ranks test reveal that this null hypothesis can be rejected, with levels of significance varying between p <.05 and p <.01. This leads to the conclusion that the relative predictive power of the best fitting models does not depend on whether the actual or the predicted political decision is used as the basis of the prediction of the policy performances.
This appears to be a consequence of the precision of the policy scales used in the analyses in Arnhem. In comparison with the scales in the other policy programs, the scales in the Arnhem program contain many different policy alternatives. This means that a small change in the assumption regarding normative control leads to the prediction of an entirely different policy alternative. If the policy alternatives lie closer to each other than 12 per cent of the interval, errors will lead to incorrect predictions more easily than if this is not the case. In future research, the analyses could be carried out separately for discrete and continuous policy decisions.
The information in table 6.6 shows that the percentage of correct predictions hardly changes if the predicted political decision, rather than the actual political decision, is taken as the basis for predicting the policy performances.
Theoretically we would expect that when the reputation sensitivity has a smaller value, the difference between the integrated model and the ‘regular’ implementation model disappears. This can be explained easily. the smaller the value of α, the less weight the implementation agencies attribute to the political decisions in their calculations. As a result, the errors in the prediction of the political decision affect the predictions of the implementation to a much lesser extent when α has a low value.
Author information
Authors and Affiliations
Rights and permissions
Copyright information
© 2000 Springer Science+Business Media Dordrecht
About this chapter
Cite this chapter
Torenvlied, R. (2000). Towards an Integrated Prediction Model. In: Political Decisions and Agency Performance. Library of Public Policy and Public Administration, vol 2. Springer, Dordrecht. https://doi.org/10.1007/978-94-011-4285-4_6
Download citation
DOI: https://doi.org/10.1007/978-94-011-4285-4_6
Publisher Name: Springer, Dordrecht
Print ISBN: 978-94-010-5857-5
Online ISBN: 978-94-011-4285-4
eBook Packages: Springer Book Archive