COMPREHENSIVE EXAMINATION

Submitted by
Louis Hertz Albert
Graduate School of Public and International Affairs
University of Pittsburgh
June 9, 1986


BACK(go and read!)

Public Policy and Research Analysis, PPRA, is a systematic, methodological approach to human problem solving. It deals with human problems within the framework of political decision-makers and must clamor for their attention as much as any other method of intervention, as well as meet certain guidelines expected of social science research (Lindblom). This process is complex and to decompose it in an endeavor to portray its functions may in fact limit one's view of the process.

Viewing PPRA as a series of separate and distinct processes is somewhat like trying to describe a Macrame knot by untying the threads, lying them in a straight row and talking about the now-decomposed knot as if the individual strings represented the knot as a whole. During the PPRA process one or more of the five areas -- 1) Problem Structuring and Concept Formation, 2) Policy Research Methods, 3) Decision-Analytic Methods, 4) Organizational Theory & Policy-Making Processes, and 5) Knowledge Use and Public Policy -- may be in the foreground. But, like any true multi-tasking project, the other areas are being processed in the background, while the whole PPRA process operates within an organizational framework.

In this exercise I propose to try and follow the threads of the PPRA policy knot without untying it, without becoming lost in the pattern or being tangential to the purpose of the project. Problem structuring is not a unique and individual part of the policy process: it rises to the foreground and co-resides with "uncertainty" and "ambiguity" as well as "knowledge utilization" to a level of dominance or diminution, depending upon the position of the task in the process, or, as in the example above, the thread in the knot. Further complicating this cognitive framework is Kincaid's notion of the iterative nature of the knowledge process and Hogwood & Peter's policy succession and incremental change. What we are left with is a broad fabric of not just one knot but a design of knots woven into a pattern with different procedures taking precedent, without a real beginning nor end.

If one assents to Simon's definition of "a natural science [as being] a body of knowledge about some class of things-- objects or phenomena-- in the world: about the characteristics and properties that they have; about how they behave and interact with each other," then natural science would be viewed as a descriptive process. However, the social sciences not only fill this descriptive role, but perform a prescriptive one as well. This is especially true of the policy sciences or policy analysis from its ex ante. point of reference. This bit of knowledge should weigh heavily on the moral implications of a policy analyst's actions. "Problems and solutions, it seems come in pairs: changing one changes the other." (Wildavsky) Yet the notion involved in policy making is to confront the issue at hand instead of being "overwhelmed by the inconceivable." (Dery)

There are many ways of approaching the problem of describing this process and many processes that can be described. If in fact the activity is iterative, the choice of where to begin the descriptive narrative can become arbitrary. However, instead of reaching out and grabbing onto the carrousel at random, I have chosen to begin with the notion of problem structuring and its role in public policy making. As with Dunn's six steps, problem structuring is the first step, or, in the words of Levy & Bardach, one is to "seize the problem." In virtually all problem solving processes the first step espoused is to "seize" or "structure" the problem. Unfortunately, this area has been quickly passed over partially due to its difficulty. As Rein and White state, problem structuring is "an integral part of the work of the scientist, perhaps the most crucial part, but it has traditionally been the part least well codified in the canons of methodology and 'normal science.' There is in fact, no orderly or prescribed way of doing it."

Yet if one reads Lasswell, Lindblom, Dror, Dye, Meltsner, Rein and Schon, Wildavsky, MacRae and Wilde, there is an institutionalization of policy problem definition and structuring with this emphasis as the major point in public policy analysis. Yet only a subset of authors point to what one should expect at the end of the definition process (Meltsner; Rein and Schon; Wildavsky; Dunn). Kelly identifies part of the problem as different personal constructs leading to different understandings of the "same reality": (assuming that reality is the same if it is perceived differently.) What is at work here is people trying to solve/define problems set in a "human" framework by their own assumptions and value judgments of what constitutes reality and without concern about how their own perceptions are formed (Vickers). "Different people ... are likely to perceive their ... environment differently and to advocate and support different strategies for coping" (Weilenmann).

Problems imply a lack of a ready solution (Duckner; Newel & Simon; Davis). Nor are they puzzles,(Kuhn) as puzzles imply answers. The problem definition process has been divided into areas of "ill-structured" problems and, in my terms, not ill-structured problems (Dunn; Mason & Mitroff; Mitroff & Sagasti). Problems are seen as "messy" (Ackoff), "squishy," (Strauch) and a complexity of problems rather than a single problem. If a problem is well-structured, is it a problem? I think not, at least in PPRA terms.

Introducing the Type III error, that of defining the wrong problem, (Mitroff & Mason; Dunn) is confirmed by Quade who states "it is more important to choose the right objective than it is to make the right choice between alternatives. The wrong objective means that the wrong problem is being solved." In Ackoff's hotel elevator anecdote, the issue at hand involved a human-related solution and a technically-related answer. The end result was human engineering, situating the mirrors in order to occupy the people waiting for the elevators. In this example there were competing problem definitions that faced both the problem definition and the solution alternatives quite differently (Kilmann & Mitroff). If one accepts the view that goals are set a priori, then Kilmann and Mitroff's definition of a problem as the discrepancies between "what is" and "what should be" may be sufficient. While Merton adds that undesirable states alone are insufficient, to define a problem, he adds that there must be a disparity between the states for a problem to exist. Are goals a priori or a posteriori? If goals are independent of analysis, then there is no need for the analysis. If the mirrors had not solved the problem, Kilmann and Mitroff believe that an evaluation would point to the wrong problem definition. Yet, as Dery states, by "dealing with definitions of different problems, rather than definitions of the same problem, program evaluation" would only point out the problem of the "particular program employed with the problem definition."

If an undesirable state is equivalent to a social problem, then it would be a simple transformation to define the problem by describing it. Yet, knowing the problem state's definition may not give us enough information to intervene. Becker holds that the "crucial points of intervention" are often outside the control of causality. He, along with Lindblom and Meltsner, maintain that causes are often neither items that can be controlled or "worth manipulating." But if analysis is a process, then the existence of any discrepancies are not sufficient for a problem. Simon says that these discrepancies need to be joined by a process that removes the discrepancies. Others hold that for a problem to exist, in policy terms, there must be some form of solution (Lemert; Rose). This notion of problem definition does not address the notion of benefits. There is the notion that the problem definition should address "a positive net benefit" between the "what is" and the "what should be." And that it may be possible that some problems are not solvable, due to the cost involved. (Dery)

It may seem to be somewhat difficult to accept this judgment of tying problem definition to problem solving. But I am discussing this in terms of policy analysis, not in terms of strictly economic, political or sociological disciplines. Problems are composed of analytical constructs developed internally and are not "objective entities in their own right" (Wildavsky; Rein and Schon). Restating this notion, Simon holds that problems are mazes with different paths and different rewards at different ends and that social science should show and "point out constraints ... which a proposed policy would have to satisfy in order to be feasible" (Majone).

As I stated earlier, the problem definition or structuring aspect of the policy analysis process is not an isolated event. In order for this process to proceed, the defining process must join with the total policy process. Ackoff claims that a problem is "solved" when its value outcome is maximized. When a policy maker "satisfies," as opposed to maximizes, then he claims the problem has been resolved. One dissolves a problem when the values are no longer meaningful within the framework of alternatives. Wildavsky describes this process as a "route" that the process travels. Going back to my original metaphor of the macrame knot, it is the twisting of the threads together that brings the problem to definition and ultimate solution.

Putting this in terms of the Challenger Space Shuttle disaster, there are two general and somewhat separate problems to be defined and dealt with. The first is the technical cause of the disaster. What caused the shuttle to explode? The second issue is much more policy oriented as to man's role in space flights. It appears that the former, technical issue will be "solved." The solution will in fact be maximized and there will be a general consensus and agreement about the actual causation. It is the latter issue, that of man's role in space, that will not be solved. The issue may be "resolved" or, in Ackoff's terms, the issue may be "dissolved" and the process started anew. For while the technical issue should bring consensus, the policy issue can only hope to bring a majority to bear.

These two different types of problem definition processes and policy issues interact on the consequences of causation differently. In one breath Ackoff states "our natural inclination is to try to find the cause of the deficiency that gives rise to the problem and to remove or suppress [or remedy] it." This works for the technical problem or approach. However, for the humanistic problem of man's role in space, Ackoff's position is that it "is [the wrong] formulation of a problem that leads to the suppression of symptoms rather than the removal of the cause of a deficiency that creates the problem." In the case at hand, solving the problem of the "O" rings will make man safe in space, by removing or suppressing the cause of the problem, but that cause does not address the issue of man in space. This latter issue can only be thought of in terms of the interpersonal comparisons of opportunities and denials within the political process (Wildavsky). Further problem structuring must not only identify conflicting values, but must delineate those values that are important (Brown).

The methodology of this process includes many procedures. Dunn differentiates between the second-order of policy analysis, problem structuring, and the first-order, problem solving. Invoking the principle of appropriateness of method to the level of problem structuring, Dunn states that "most methods of policy analysis are inappropriate for those second-order problems which have been characterized as messy, squishy, or ill-structured." He confirms the notion that policy structuring is paramount to policy analysis. Continuing on, he states that "problems at one level (first-order problems) are members of the class of problems at the next higher level (second-order problems)." By my previous definition the first-order problem is not a "problem" in terms of problem structuring and definition, since it already has been structured and defined, that is until the policy process "routes" the "problem" through to the dissolving stage and back into the realm of problem structuring and defining. At this point it is anti-climactic to state "the aim of policy design is to create and to apply problem structuring methods to second-order problems, transforming them into first-order problems that may be solved by applying standard methods of policy analysis" (Dunn).

Dunn divides the methods of the second-order into two major schools, a dichotomy between "Quasi-algorithmic" and "Non-algorithmic" methods. Not necessarily implying algebraic or computerized formulas, the "Quasi-algorithmic" mainly refers to the reproducibilty of results by differing researchers. He considers these methods "a class of approximations or truth-estimates which ...employ precise, unambiguous, and readily applicable heuristic prescriptions. These methods ... are oriented towards discovery rather than confirmation or proof..." As examples of quasi-algorithmic methods he provides us with "Hammond's social judgment theory, Saaty's analytical hierarchy process, and Mitroff & Mason's strategic assumption surfacing and testing." However, he maintains that these methods lack any device for quantifying the discovery process. Dunn holds that personal construct theory and its applications hold the answer to this problematic issue. Based on the three components of, 1) Divergent Stakeholder Sampling; 2) Elicitation of Policy Constructs; and, 3) Collective Boundary Estimation, he proposes to be able define policy problems.

Part of my critique of this method is based on inductive reasoning, "knowledge" (the subject of what constitutes knowledge is not questioned here) and past learning frames of reference. Also, the critique is aimed at the "human" problem area and that distinction between "human" and "technical problems." The opinion that the divergent stakeholder sampling will include "the total universe" is doubtful on at least two accounts. I do not accept that salient stakeholders or authority figures will "know" or be able to identify all stakeholders, either at an individual or group level, that until at least one, if not more, iterations of the policy process take place, many important stakeholders may not become evident, even if one were able to identify all salient stakeholders. Can the procedure work on a large "human" problem? Referring back to the notion of a priori goal setting, is it safe to assume that even if one were able to sample a significant portion of the universe of salient stakeholders, that the reality of the problem would reside within their multiple frames of reference?

The major quandary with the procedure at hand, though, is in step two, elicitation of policy constructs. Psychology and the discipline of Industrial Organization both tell us that one cannot equate measures of attitudes to direct implementation of actions (Cooper). Also, thoughts or intellect are not the same as emotions, which are, in turn, different from behavior. Testing for one does not necessarily provide an indicator of the other (Fishbein & Azjen). This is supported by Argyris with his "'Theory-in-use'... that people actually use" as opposed to "'espoused theories' ... that they wrote or talked about." Tests such as the Minnesota Multiphasic Personality Inventory (MMPI), a personalty test with 556 true/false questions, has an internal validity process which includes "a large body of standardized data, including evidence of an association with delinquency" (Herrnstein). However, abnormal scores on the test do not indicate deviant behavioral patterns, just that deviants "as a group also earn high scores on the scale." This raises three questions: Are the constructs and elements deciphered valid? Do they actually represent the beliefs of the participants? Do they represent what they are purported to? (For a similar problem of cognitive accuracy and individual recall in a much simpler environment see the series of articles by Bernard, Killworh and Sailer.) The problematic involved at this point is whether the constructs and elements that are generated by the procedure in question are accurately translatable into the problem definition process and if the definition itself lies within the cognitive framework in general and those surveyed in particular. This method appears to have a usefulness in the problem solving of the technical area, but is limited in the application of "human" problems. It logically follows that if the first two elements are flawed, the results of the third will also be flawed. Referring to Morris Kline's allegorical homesteader, as used by Dunn, the homesteader must maintain his current crop area, and when he clears new wilderness, it should be in area of fertile, tillable ground.

I must confess that after my critique of Dunn's proposal I have no positive new procedures to offer. If problem definition and structuring is messy and squishy, then the only "sure" way of finding the definition is to stick one's hands into the squishy mess and get them dirty. Dunn's text has a multitude of methods to help with the actual processes that need not be developed here. Often in class reference was made to looking at the policy process as an "all knowing" being; unfortunately, as social scientists we have not yet reached this position.

The thesis underlying this work is the interconnectedness of the various threads of the policy analysis process. In fact, after writing just the first third of this paper, I am willing to go down one more level of analysis in the metaphor and call the threads multi-stranded and multi-colored, as well. At this point it may be beneficial to switch to a different one of these all-inclusive threads and examine the results and "utilization of research-based and experiential knowledge on the policy process."

Since we are policy analysts, we are concerned with the usefulness of our work. While we may endeavor to move back the frontiers of our individual fields of social science, the notion of application of our work to society exists. This is not to deny the applications of other social scientists, yet policy science is an applied social science. How we create, diffuse and utilize knowledge is an important issue in that our primary function is to provide knowledge in a usable and timely fashion. The previous part of this essay dealt with the notion of the ontology of the problem, how we as policy scientists structure and define the nature of the problem, i.e., reality. This part of the essay will attempt to deal with the notions of epistemology, "the study or theory of the origin, nature, methods, and limits of knowledge" (Coles Concise English Dictionary). Knowledge use studies are an enlightening of the "concepts and theoretical perspectives" that are an inherent part of policy process (Weiss).

If problem structuring and defining has been labeled messy and squishy, knowledge use has been labeled "soggy" (C. Weiss). The three broad areas of knowledge use, creation, diffusion, and utilization, have been artificially divided, and work in these areas has traditionally tended to be bounded within each sub-field (Ganz; Rich). Efforts have been made to remedy this separate approach to knowledge studies (Dunn). One of the underlying themes of this essay is that when the policy process is decomposed there is the risk of losing the underlying "holistic" approach. Knowledge utilization is one of the desired goals of policy analysis and its tenets should stay with one during the policy process. "... the under-utilization of social indicators by policy-makers cannot be satisfactorily understood apart from the whole system of problems [of] which it is a part" (Dunn). Dunn also holds that, at least at this level of analysis, "it is impossible to decompose ... the problems of knowledge use," while Ackoff adds that knowledge problems are not "natural," but rather, "products of thought acting in environments."

Trying to define "what is knowledge" can be confounding. Machlup states that "... the subjective meaning of the know[ledge] (is) to the knower ..." We all "make use of knowledge from within ... [our] own cognitive structure and ... [our] own social system would be ...[an] accurate and ... useful" statement (Kincaid). One can categorize knowledge as mundane knowledge, scientific knowledge, humanistic knowledge, social-science knowledge, artistic knowledge, and knowledge without words (Machlup). If one views an "organism," i.e., an individual or organization, as being in a constantly changing pattern in a "matrix of conditional probabilities" of many possible behaviors, "information" is the tool that allows the organism to orient itself within the matrix (MacKay). This still does not give a policy definition of knowledge. Borrowing from Simon's Sciences of the Artificial, one can make the argument that information is natural, i.e., that it exists in nature, and that knowledge is the artificial result of human endeavor on information. Knowledge is artificial, developed within a cognitive framework and subject to certain rules and criteria in order for it to qualify as knowledge, as opposed to faith or intuition, (not to imply any "evil" to either) in the realm of social science.

Policy science knowledge tends to deal with causes. "Knowledge of what is likely to happen if something is deliberately varied has great practical value for social policy" (Cook and Campbell). Unfortunately, one's perception of these cause may take many routes, i.e., Hume's positivists; essentialists; Mill's method of concomitant variation; Popper's falsification; Collingwood's activity theory of causation; and the position of Cook and Campbell of an evolutionary critical-realist. This last perspective selectively adopts features from many of the previously mentioned philosophies of science and adds to the synthesis the notion of evolution. It accepts that knowledge is a result of the cognitive environment of the individual or organizations and that these perceptions are "subjective,"

"but at the same time it stress that many causal perceptions constitute assertions about the nature of the world which go beyond the immediate experience of perceivers and so have objective contents which can be right or wrong (albeit not always testable.) The perspective is realist because it assumes that causal relationships exist outside of the human mind, and it is critical-realist because it assumes that these valid causal relationships cannot be perceived with total accuracy by our imperfect sensory and intellective capacities. And the perspective is evolutionary because it assumes a special survival value to knowing about causes and, in particular, about manipulable causes." Cook and Campbell

The notion of the evolutionary element is extremely important, in that it allows us as a society to make the discrimination between the haphazard relationship "and the truly causal relationship, between forecasting and diagnosing manipulable causes which will enable us to change the world" (Cook and Campbell). This latter "manipulation" may be the purpose of "human society" to allow us, not only to adapt to conditions, but rather to adapt conditions for a positive net benefit. It is definitely the underlying goal behind policy analysis.

Dunn divides knowledge into four categories: 1) definitive; 2) designative, 3) evaluative and 4) advocative. Regardless of the category "knowledge use is a cognitive relation among two or more purposively behaving actors" (Dunn). Combining these categories with the cognitive relationship between actors, one develops a set of "motives" revolving around the intentions of "producers and users" (Dunn). Knowledge users, i.e., policy makers, have been shown to use three basic techniques to deal with the incoming amount of knowledge, i.e., policy analysis. These techniques are 1) relevance to the context; 2) a reliability or "truth" test; and 3) a "utility" test (Weiss and Bucuvalas). Using these processes shows how knowledge is filtered through and subjected to the framework of the "organism" that receives the knowledge.

Two notions arise at this point, the first being Lindblom and Cohen's differentiation between ordinary or experiential knowledge and scientific knowledge. The preceding paragraphs should have served as a definition of scientific knowledge. Ordinary knowledge, or "soft" knowledge, is the "gray market" of knowledge that one arrives at through life's experiences and what can be called common-sense or intuition. If we think of knowledge types in the same terms as problem solving techniques, then scientific knowledge can be thought of in the terms of being "quasi-algorithmic." Scientific knowledge has a high level of validity and should be able to be reproduced by different researchers, at least those working within the same paradigm. Within this dichotomy of knowledge views, Caplan holds that it is the former, experiential knowledge, that has the most input into policy matters. He further suggests that use of this type of knowledge may have a better fit to the ill-structured policy problems. Greg finds that scientific knowledge has a more significant bias: it is frequently "reductionistic" in style and not appropriate for ill-structured policy problems -- obviously a problem in the problem structuring process of the analysis.

Knowledge use can be viewed as in the "two-communities" metaphor (Caplan; Rich; Dunn), two-communities, co-residing, the policy analysts in one and the policy makers in the other. These "epistemic communities" view knowledge from within their own cognitive frameworks and social constructs. They apply the above "truth and utility" tests within their own reality and within their "particular contexts" (Holzner and Marx). In terms of knowledge utilization Dunn found that 1) The quality of its "reliability" was directly related to its application; 2) The quality of its "validity was directly related to its application; 3) Knowledge was applied more in profit motivated organizations than in public organizations; 4) Knowledge created by "formal" parts of the organization will be used to a higher degree than that produced by those "unaffiliated"; 5) "The greater the overall influence of policy- makers, social scientists, and other stakeholders in each phase of the policy-making process, the greater the knowledge utilization"; and 6) "The more the social scientists use a diffusion style that encourage feedback on the results of their research, the greater the knowledge utilization."

Knowledge has a continuum of utilization, from instrumental use (Caplan, et al; Knor; Dunn) to conceptual use (Weiss and Bucuvalas; Patton et al; Dunn). Instrumental use implies an observable cognitive relationship, including the transmission and reception of knowledge (Machlup) and its specific use by decision makers in the policy process (Patton; Caplan, et al.; Rich; Weiss). At the other end of the continuum is conceptual use. Conceptual use is subjective in its cognitive relationships and is difficult to measure and track the use and/or value of the knowledge received (Machlup), including the perspectives of the all the parties involved (Patton; Caplan, et al.; Rich; Weiss). This latter form, conceptual use, is generally accepted as being the most utilized of the two forms (Caplan; Rich; Patton). While we as policy analysts may prefer to think of our work as having a direct impact on policy decisions, it can be seen as "enlightening" policy makers to the notions of social science and those ideas we wish to present (Weiss). Yet, when all the research is consumed and digested the resultant cognitive map of the policy maker may have to be changed in such a way that, regardless of the direction of the action, the product of the knowledge process has been utilized.

Tangential to this continuum is the notion of the political framework that policy analyst operates in. The conceptions discussed above are applicable only in an atmosphere of free intellectual exchange. In reality a policy analyst may find his boundaries severely limited and the range of policy alternatives limited to those that are deemed acceptable by the decision makers. This need not apply only to non-democratic national regimes, but can be applicable to organizations as well. In 1982, for instance, I did a study of police research units in Cleveland, Akron, Canton and Summit County, Ohio. For the most part these units functioned at what Larson called the "symbolic" level. Their research was primarily aimed at the codification and legitimizing of already-made decisions.

There appears to be an "[in-]coherent conceptualization of the communication process upon which almost all knowledge utilization and generation depends..." (Kincaid). Knowledge generation-utilization theories can be viewed in two schools with the broad headings of "linear approach" and "atomism" which, according to Kincaid, account for the way most knowledge is to be transmitted and disseminated. The linear model is characterized by a "one-way motion of communication ... where the spread of a new idea moves in a linear fashion from its source of invention or creation to its ultimate users or adopters," (Kincaid) this is an oversimplification of the need. Communication, according to Rogers, is to be "the transfer of ideas from a source with a viewpoint of modifying the behavior of receivers." Recent writers have begun to place increased importance on the effort to discern the needs of users and to provide for this in the communication by knowledge producers. Also, the notion that users should be both the "major focus and starting point" as opposed to the "product orientation" of knowledge has emerged. Part of this movement encourages the notion that "knowledge generation-utilization be thought of as an [inter-] active exchange or transaction process" (Kincaid).

Atomism is the view that, unlike the linear flow of knowledge, the differing parts, such as knowledge generation, utilization, and dissemination, are, in fact, separate, distinct elements of the process that can be delineated, defined and delegated as separate responsibilities of the process to either the user or the researcher (Larsen). The argument here is one of the possibility or practicality of separating one concept or idea from the whole system of ideas and concepts that are interrelated in their applied contexts. This is a logical extension of the notion that if knowledge use cannot be divided in order to survey the field, it cannot be decomposed in its operations.

Again one faces the dilemma of the dichotomy between technical and human problems. In dealing with a technical diffusion of knowledge, as opposed to human problem where pieces of the research can be exclude at will, it is much more difficult to accept "only part of the technology itself (half of a seed or a tractor is useless), technology transfer reinforces an atomistic conception of the knowledge generation-utilization process" (Kincaid). It is much easier to accept partial notions of human problems into one's conceptual, subjective framework. Kincaid states that "concepts cannot be fruitfully isolated and singled out from the network of related concepts with which they are applied in particular contexts."

Zaltman has widened the notion of conceptual knowledge with the conception of "re-invention" of knowledge, to adapt it to the current singular contextual situation. It involves a refitting of the model to include all current applicable variables, not driven by the producers, but by the users (Glazer) while removing those variables that are no longer pertinent. Therefore, accordingly, the knowledge generation-utilization technique can neither be a linear process with a well-defined current and purpose "nor should knowledge be ... an indivisible entity , separated from other aspects of knowledge generation-utilization, ... incapable of growth."

Kincaid further states that "knowledge in its collective sense has grown beyond the grasp of any single individual." He adopts Popper's view of collapsing reality into three categories: 1) Physical objects and events; 2) Conscious experiences; and 3) The "logical contents" of books, libraries, etc. Knowledge is the integration of these three elements with their interaction in the process of "human communications." Then understanding may be considered the application of these principles and their ongoing relationships. Kincaid adds to my previous definition of knowledge as being man's artificial interpretation of information by requiring knowledge to have an element of communication. Yet he develops an argument that language, concepts and signs, the elements of communications, are conceptual and imprecise and that these elements have "develop[ed] over time in relationship with [our] other conceptual knowledge."

Accepting the above brings us to a position of calling for "a cybernetic approach in which understanding is developed over several cycles of application and exchange with others in a social process of communication" (Kincaid), a cyclic application of moving to a better understanding and shared consensus of the information available with each individual cycle bringing greater degrees of understanding. It is this "shared understanding which develops among members of a collectivity" (Kincaid) that creates, diffuses and utilizes knowledge. A researcher can no longer simply transmit his work. He must be sure it reaches its intended receiver and that the communications are multiplex in nature. The researcher and the decision maker need to be part of the same network. This "will tend over time to conceptualize matters more similarly than those who have not shared in the exchange of the same information" (Kincaid).

This process of network knowledge creation and diffusion is confirmed by Dunn's finding (reported on page 15, #5 and #6). Dunn also states that the "appropriate conceptual framework" of both individuals and collectivities must be developed, that individuals do not make policy decisions, the aggregation does. "Knowledge is rather a social construction which is transacted by multiple stakeholders with different frames of reference"(Dunn). It is for this purpose that studying knowledge networks has value.

In most networks there is a very complex pattern of relationships developed that are "not confined to observable exchanges of goods, services, or information but extended to beliefs values, assumptions and [political loyalties] as well" (Dunn). In the development of a network data set, one develops a set of matrices based on the question of "who does _______ with whom?" These matrices are then examined by various quantitative and qualitative research methods. The underlying research perspectives try to analyze one or more of the following three issues:

1) Identifying cliques within the total system and determining how these structural subgroupings affect communication behavior in the system.
 

2) Identifying certain specialized communications roles such as liaisons, bridges, and isolates.
 

3) Measuring various indices of communication structure (e.g., degree of connectedness) for individuals, dyads, personal networks, cliques, or entire systems. Kincaid

What Kincaid is seeking is to place the weight of the relationship "on the difference between two or more groups who might communicate with one another. The model does not call for a users' orientation or a producers' orientation in the knowledge utilization process, but rather a comparison between their relative positions and the degree to which convergence or divergence is occurring over time as a result of the process and patterned networks of their communication." I perceive this as a challenge to move away from the spirit of the "two communities" by removing such barriers as the overt complexity and obtuseness in the language used by researchers in their reports to end users and Rothman's notion of research and development centers engaged strictly in "social-engineering." Policy researchers (policy analysts) must network themselves into a collaboration with policy and decision makers in order for their knowledge generation program to be validated by its dissemination, utilization and ultimate acceptance. These networks will help to reach the primary purpose of communication: mutual understanding and a removal of the uncertainty inherent in the information-exchange process. This will result from the increased reliance on the different nodes of the network and their convergence toward the common focus of public policy.

At this point in the essay the metaphoric macrame knot has begun to take shape. Unfortunately, the two strands are insufficient to give it a definable pattern that can be appreciated as a "whole." The third strand must be woven in order for the pattern to appear. If the ontology of the problem is problem structuring and the epistemology of the problem is knowledge utilization, all within a policy frame of reference, then it seems to follow that the problem is the relationship between the two, the relationship between policy analysis and the formation of public policy. Yet, the previous two discussions have had this topic as their underlying theme, and have covered much of the material that would be introduced if this essay were a separate unconnected work, for in fact the whole discourse is central to the issue of policy analysis and its relationship to public policy.

"Policy analysis is an applied social science discipline which uses reason and evidence to clarify, appraise, and advocate solutions for public problems" (Dunn). It is applied because it is oriented to the solution of practical problems and its concern is based in facts, values, and actions (MacRae). It brings multi-perspectives to the policy process and is political in its interrelated phases (Lasswell; Jones). It can approach problems either from a philosophical position exploring the alternative claims of different paradigms (Dror) or in a decision/analytic mode defined by application of specific methods and techniques as advocated by the Kennedy school. Most important, policy analysis has an ethical element (MacRae), since it deals with alternatives to the human condition, and it is administrative, in that it deals with the anticipation of implementation (Williams).

Dunn has identified four methods of inquiry, different perspectives based on their reference to time frames and procedures: 1) retrospective -- political science, which looks at the past in order to predict the future; 2) prospective -- economics, which attempts to look at events ex ante in order to predict their outcome; 3) evaluative -- an attempt to determine a value through a comparison of stated goals to results; and 4) advocative -- a prescriptive, normative modeling approach. Policy analysis borrows the techniques for these methods from the individual disciplines, but does not apply them in the same manner or style that the individual disciplines would use. Policy analysis has a broader generic focus and level of analysis then the specific domain of the disciplines. In order for policy analysis to work it must combine the notions of many disciplines (Lasswell). Policy analysis is above all prescriptive and looks to play the part of an interventionist. In a social science problem, policy analysis is not as concerned with the "causes" of the social problem as it is with the points where policy makers can intervene and the capabilities available for that intervention. With these two components in mind it is easy to see how policy analysis can be considered a "political activity." The danger of policy analysis is in using the above four methods outside the analyst's framework and within the conventional social science disciplines. In fact Spector and Kitsuse "specifically urge" the conventional disciplines to abandon most questions of "public concern."

A significant difference between conventional discipline and policy science is the definition of what constitutes a "social problem." (Note the difference between defining a "social problem" and the previous discussion of problem structuring and definition.) Dery offers two criteria, parallelling the previous discussion: 1) Social problems need to be defined in ways that will not make them unsolvable; 2) the new desired state must offer an "improvement" over the existing condition. While these are similar notions to the "what is" and "what should be" discussion, the level of analysis is different in that here the paradigms between the policy sciences and the rest of the social sciences are divergent. A sociologist does not view his research as having to be policy relevant and actionable. In this essay I have used the words "research" and "analysis" almost interchangeably. Yet there is a subtle difference. I perceive research, in the single disciplines, as taking a descriptive approach, while policy analysis is prescriptive.

One last word of caution is developed around the area of "policy evaluation." Instead of the Popperian notion of falsification, evaluators tend to be "justificationists at heart" (Majone). Majone holds that "like good behaviorists, evaluators think of themselves as objective experimental scientists supplying the policy-maker with hard facts, the 'potential falsifiers' of the programme. In reality, they formulate goals, assign them relative weights, identify actors, define system boundaries and choose yardsticks... Evaluation does not assume a fully-articulated policy or programme; it creates it." Program evaluation asks "Does it work?" or better yet, "What should it do?" Policy analysis replaces this question with "Why does it do what it does?" The issue is not what the program does, but what its consequences are (Cronbach, et al.).

In the policy analysis process the perception of the "ill-structured problem" is joined by "complexity" (Rittel and Webber). In the world at-large, every problem is inevitably tied to every other problem (Mason and Mitroff; Churchman). "Every time a policy maker attempts to solve a particular policy problem he or she must consider its potential relationship with all other problems" (Mason and Mitroff). Dery claims that it is a policy analyst's "choice" whether he formulates a problem as simple or complex. The approach that the analysts take is dependent on the "objectives," the solution, and the means available "to solve the problem." He continues, "Whatever goals we intend to achieve via solution are our goals, our choice of ends. And if our objectives are multiple, interconnected, and conflicting, they are still our goals."

The policy analyst who deals in both ill-structured and complex problems may find himself in trouble. As policy analysts, we operate in a world of "macro" problems with "micro" tools. As stated earlier, the areas that we work in are described as being "messy, squishy and soggy." Yet, the

analytical methods and quantitative analyses that are available to policy analysts are inadequate for handling even the simplest amounts of complexity and lack of structure. From a technical viewpoint progress is being made: our stock of methods and quantitative procedures has been constantly expanding. Policy analytical problems are those that have intervention points. Without these points the problem is not actionable, and if the problem is not actionable, then, by previous definition, it is not within the realm of policy problems. But from the viewpoint of social science's call for mutidisciplinary work, matters are not improving, and the prediction for the future is not bright (Gregg, et al.). Not all agree with the incorporating of problems with "the perception" of practical applications or that defining and dealing with problems in a conceptual framework that formulate alternatives in a realistic "bureaucratic control" has led to defining problems supportive of those already in control (Habermas).

Since Lerner and Lasswell's call in the early 1950's for a new policy discipline, progress has been slow and spotty. After the spurt of growth in the 1960's and 1970's, policy sciences seem to be in a position of trying to regenerate themselves. The term "policy science" has been adopted by elements of political science, which seem to have not accepted the notion of ex ante prescriptive analysis, yet still call themselves policy scientists.

If political science can "co-opt" our terms, their conceptions have relevancy for us. One of the major debates within the field of policy analysis is the conception of incrementalism and "policy succession" as presented by Hogwood and Peters. It seems that a difficulty in distinguishing between the considerations of incrementalism in science and incrementalism in policy applications has developed. In terms of technical science, improvements are made in increments. There is no other way. For example why did the Wright Brothers design the "Wright Flyer" instead of the B1 bomber or the Space Shuttle? The same analogies can be made about virtually any development in science and technology. The model "T" is not as sophisticated a car as Dr. Weilenmann's old Studebaker, which is not the touring car that my 1979 Mazda RX7 is, and so on.

Incremental change in automobile technology has not been linear. In the mid 1970's the paradigm switched and the emphases changed from full-size cars to smaller cars with more pollution controls. A combination of necessity, fuel shortages and policy in pollution control drove the paradigm shift. Yet in the mid 1980's the marketplace has moved the paradigm back to the more traditional idea of the automobile in our society. It is not the incremental nature of the technology that drives the acceptance of policies or paradigm shifts, but rather the incremental acceptance of these policies and shifts that causes the problems. While the marketplace or technology breakthroughs can generate shifts in their paradigms, it is extremely difficult to make these shifts in the social sciences. These shifts by their nature are one of personal constructs and philosophies, individual traits that are not easily changed. This problem presents the major task to policy analysts in relation to policy making.

In policy terms, by our definitions, analysis can only work in incremental terms. The tools and methods that are available do not provide the policy scientist with the capabilities to do mega-analysis. Even in the fields of the hard sciences, breakthroughs are incremental in nature. This does not mean that incremental change cannot be significant or paramount to the field, but that it is built on previous knowledge and research developments. The incremental or piecemeal implementations of the generated policies are the problem.

Hogwood and Peters describe incrementalism as being the method of government action "short of wars, revolutions and crises." However, they differentiate between level of incrementalism, in that a budget change is not the same as increasing an agency's area of responsibility. Policy innovation and policy termination do not fall under the umbrella of incremental change, but these rarely occur. More often policy actions fall into either the category of policy succession of policy maintenance. Either one of these normal operations involves incrementalism. Incremental change can, though, lead to problems. For instance, it does not work under all conditions, and once a bureaucracy is established, it is difficult to make major changes in direction or program.

Four major models of incrementalism have been identified: 1) Disjointed incrementalism (Braybrooke & Lindblom; Wildavsky) -- described as reactive and remedial; 2) Logical incrementalism (Quinn) -- proactive and prospective; 3) Mixed Scanning (Etzioni) -- adaptive and purposeful; and Normative optimum (Dror) -- determistic. Several areas have been identified as being inappropriate for incrementalism: 1) policies that are not partitionable, that cannot be "disaggregated" into various parts. Like the space shuttle "O" ring problem, implementing only part of the fix will not be sufficient: the whole fix must be implemented is a necessity (Schulman); 2) when there are significant or major changes in the external environment that necessitate radical visible distinct modifications in current policy (Jones); 3) "win-lose" situations where anything other than total resolution will only put off problems in time; 4) where the lack of a "self-correcting mechanism" will lead to contradictory, self-defeating actions.

Not every decision made needs to be justified on a "zero-based" decision rule. Certain concepts are readily accepted by large elements of the population. Such American institutions as bicameral legislatures, for example, are not questioned in every policy discussion. Policy succession and maintenance do not require an examination of the basic underlying philosophy of the policy every time it is modified. However, analytic input into "What is the policy doing?" and "Where is the policy going?" are legitimate concerns of the policy maker. If the answer to these questions is unsatisfactory, the basic premise of the policy can be surfaced either to be modified or terminated.

Paraphrasing Wallace, policy arguments can be made in four ways: 1) Authoritarian -- "those who are socially defined as qualified [experts]"; 2) Mystical -- information arrived at from "a natural or supernatural occupant of a particular position ... 'a state of grace.'"; 3) Logico-rational -- "statements purporting to be true ... [relying] on the formal rules of logic"; and 4) Scientific -- which "combines a primary reliance on the observational effects of the statements in question, with secondary reliance on the procedures (methods) used to generate them" (Wallace). Policy making uses all four methods: elder statesmen, politicians and others announce their beliefs and positions; stands are taken on such issues as the "right to life" based on religious principles and doctrines; every Sunday, people such as William Buckley pronounce their logico-rational arguments on the public affairs news programs; and, lastly, the policy analysts bring all these together within their own cognitive framework as they perform the scientific role. It is important to note that policy analysts are often salient stakeholders themselves. For example, when I do consulting work outside the Summit County area, where I live, the only concern that I have about the acceptance of my recommended alternatives is one of professional integrity. Yet when I deal with the same issues within Summit County my perspective changes, and I am at the same time both the professional analyst and a concerned stakeholder with interest in the outcome. If Dunn is correct (referring to the classifications on pp. 14-15), I will have more impact as a change agent by being a formal part of the organization, a county employee, but reduced impact by being perceived to have less status then the "out-of-town-expert."

Wallace's policy wheel (see Dunn also) shows policy as a circular iterating process. If the diagram can be placed in a three-dimensional space, it would seem to occupy a spiral similar in conception to an Einsteinian view of the universe, without beginning nor end. This is the reality that faces conceptual policy analysts. Policy analysis consists of conceptual workings that include "perception, analysis and choice," while political decision making incorporates "a social process of implementing policies formulated by means of organizational structure, systems of measurement and allocation, and systems for reward and punishment" (Bauer).

"Social life does form a totality, and we must see it as a totality if we are to choose social policies wisely. The problem with social sciences is not simply that it breaks the whole into parts -- all good science does that. The problem is that it ignores the totality and thus breaks it down in ways that can not be reintegrated" (Gregg et al).

As Dunn so aptly puts it, it is the "process of knowledge utilization, and not the conduct of the policy analysis per se, [that] links policy analysis to the policy making process." Given that the product of the analysis meets the criteria of useful scientific knowledge and has alternatives that are acceptable to the policy process, then it is the position and centrality in the network that can determine the utilization of the resultant analyses.
 

AFTERWORD

In writing this exercise it became rapidly apparent that a winnowing process had to be employed in order to limit the work to its desired length. This is especially true of knowledge use. The study of epistemology is an immense undertaking. As I prepared my essay, I became aware that the more I sought to include, the more there was to include, and the less of the field could be included within the scope of a single essay, given the working constraints. I felt like an individual battling a field of Kudzu: the more I worked, the faster the field grew.

Further, the lack of discussion of general systems theory is due to the operational level of the essay. I consider myself a "systems' thinker." The essay was written from the resultant systematic frame of reference, yet while I conceived the work in systems terms, systems itself was at a level of analysis different than that which I addressed. Along the same line of reasoning, if the paper (to follow Dunn's framework) were to have been written at the first-order of analysis, then systems thinking would have been a valid tool to consider. At the second-order of analysis, the level at which I have attempted to write, systems is not the subject, but the paradigm within which one operates.

BIBLIOGRAPHY
 

Ackoff, R. L. Redsiging the Future: A Systems Approach to Societal Problems. New York: John Wiley and Sons, 1974.
 

________. "The Art and Science of Mess Management," Interfaces 11: no.1, (Feb 1981) 20-26.
 

________. "An Interactive View of Rationality." Journal of Opperations Research Society, 34: (1983) 719-722.
 

Allison, G. T. "Conceptual Models and the Cuban Missile Crisis." The American Political Science Review, 63: (1969), 698-717.
 

Amy, D. "Why Policy Analysis and Ethics are Incompatible?" Journal of Policy

Analysis and Management, 3: (1984) 573-591.
 

Argyris, Chris. Reasoning, Learning, and Action. Washington: Jossey-Bass, 1982.
 

________. "Action Science and Intervention." The Journal of Applied Behavioral Science. 19: no. 2 (1983) 115-140.
 

________, Schoen, D. A. Organizational Learning: A Theory of Action Perspective. Reading, Mass: Addison-Wesley, 1978.
 

Ashby, W. R. An Introduction to Cybernetics. New York: John Wiley & Sons, 1963.
 

Bardach, Eugene. "Problems of Problem Definition in Policy Analysis." In Research in Public Policy Analysis and Management, edited by John P. Cresine, vol. 1, pp. 161-71. Greenwich, Conn.: Ai Jai, 1981.
 

Bauer, Raymond A., "The Study of Policy Formation: An Introduction." In The Study of Policy Formation. edited by Raymond A. Bauer and Kenneth J. Gergen.
 

Bernard, H. R. and Killworth, P. D., "Informant accuracy in Social Network Data II." Human Communication Research. 4: (1977) 3-18.
 

________, Killworth, P. D. and Sailer L., "Informant Accuracy in Social Network Data IV: A Comparison of Clique-level structure in Behavioral Cognitive Network Data." Social Networks 2: (1980) 191-218.
 

Beer, S. Brain of the Firm. New York: John Wiley & Sons. 1981.
 

Braybrooke, D. and Lindblom, C. E. A Strategy of Decisions. New York: The Free Press, 1963.
 

Camillus, J. C., "Reconciling Logical Incrementalism and Synoptic Formalism- Am integrated Approach to Designing Strategic Planning Processes," Strategic Management Journal. Vol 3 1982, 277-283.

Campbell, Donald. T. "Pattern Matching as an Essential in Distal Knowing," Chapter 2 in The Psychology of Egon Brunswik. edited by K. R. Hammond, New York: Holt, Rinehart and Winston, Inc., 1966.
 

________. "Evolutionary Epistemolgy," edited by P. A. Schipp, The Philosophy of Karl Poper. La Salle, Ill.: Open Court, 1974, 413-463, Vol. 14-I.
 

________. "Qualitative Knowing in Action Research." Kurt Lewin Award Address, Society for the Psychological Assocation, New Orleans, Sept. 1974.
 

________. "Reforms as Experiments" edited by E. L. Struening and M. Guttentag, Handbook of Evaluation Research. Vol. 1, Beverly Hills, Ca.: Sage, 1975.
 

________, Stanley, J. C. Experimental and Quasi-Experimental Designs for Research. Chicago: Rand McNally, 1966.
 

________, and Kimmel, Allan J. Guiding Preventative Intervention Research Centers for Research. Report to the Center for Prevention Research, Division of Prevention and Special Mental Health Programs, National Institute of Mental Health. Rockville, Md.: 1985.
 

Caplan, N. "Social Research and National Policy: Who gets used, by whom, for what purpose, and with what effects?" International Social Science Journal. 28: (1976) 187-194.
 

________. "A Minimal Set of Conditions Necessary for the Utilization of Social Science Knowledge in Policy Formulation at the National Level: A Report to respondents. Amm Arbor: University of Michigan, 1975.
 

________. "The Two Communities Theory." American Behavioral Scientst. 22 1979, 459-470.
 

Churchman, C. W. The Design of Inquiring Systems. New York: Basic Books, 1971.
 

________. The Systems Approach and its Enemies. New York: Basic Books, 1979.
 

Cohen, M. D., J. G. March and Olsen, J. P. "A Garbage Can Model of Organizational Choice." Administrative Science Quarterly, 17, no. 1 (March 1972) 1-25.
 

Coleman, James S. Policy Research in the Social Sciences. Morristown, NJ.: General Learning Press, 1972.
 

Cronbach, Lee J., Ambron, Sueann R., Dornbush, Sanford M., Hess, Robert D., Hornik, Robert C., Phillips, D. C., Walker, Decker F., and Weiner, Stephen S. Toward Reform of Program Evaluation. San Francisco: Jossey- Bass, 1980.
 
 

Cook, Thomas D. and Campbell, Donald T. Quasi-Experimentation: Design and Analysis Issues for Field Settings. Chicago: Rand McNally, 1979.
 

Cooper, Elizabeth A., Personal conversations, Asst. Prof. Management, University of Rhode Island.
 

Czepiel, John A. "Patterns of Interorganizational Communications and the Diffusion of a Major Technological Innovation in a Competitive Industrial Community." Academy of Management Journal. 18: No. 1, (March 1975) 6-23.
 

Dahl, Robert A. A Preface to Democratic Theory. Chicago: The University of Chicago Press, 1966.
 

________. Modern Political Anaysis. 3rd. Edition, Englwood Cliffs, NJ.: Prentice Hall, 1975.
 

Dery, David. Problem Definition in Policy Analysis. Lawrence, KS: University of Kansas Press, 1984.
 

Deutsch, Karl. The Nerves of Government: Models of Political Communication and Control. New York: Free Press, 1966.
 

Dror, Yehezkel. Design for Policy Sciences. New York: American Elsevier, 1971.
 

________. "Some Features of a Meta-Model for policy Studies" In Problems of Theory in Policy Analysis. Edited by Phillip Gregg. Lexington, Ma: D. C. Heath and Company, 1976.
 

________. "A critical Survey of Normative Models of Public Policy Making." In Public Policy Reexamined. New Brunswick, NJ: Transaction, 1983.
 

Dryzek, John S. "Don't Toss Coins in Garbage Cans: A Prologue to Policy Design." Journal Of Public Policy, (1983)
 

Dunn, William N. "The Two-Communities Metaphor and Models of Knowledge Use," Knowledge: Creation, Diffusion, Utilization. I No. 4, (June 1980)

515-536.
 

________. Public Policy Analysis Analysis: An Introduction. Englewood Cliffs, NJ.: Prentice-Hall, 1981a.
 

________, Bangs, Ralph and Rahmanian, Hassan. "Studing Knowledge Networks." Working Paper KU-05. University of Pittsburgh, Program for the Study of Knowledge Use, 1981b.
 

________. "Usable Knowledge: A Meta-Theory of Policy Research in the Social Sciences." Conference on Non Traditional Approaches to Policy Research, School of Business Administration, University of Southern California, 1981c.
 

________."Reform as Arguments," Knowledge: Creation, Diffusion, Utilization. 3 (March 1982a) 293-326.
 

________, Holzner, Burkart and Zaltman, Gerald. "Knowledge Utilization." In International Encyclopedia of Education: Research and Studies. Edited by Hans Weiler,November 1982b.
 

________, Kearns, Keven P., Cahill, Anthony G. "Application of the Policy Grid to Work-Related Frames of Reference." Working paper KU203, University of Pittsburgh, Program for the Study of Knowledge Use, 1983.
 

________, Cahill, Anthony G., Dukes, Mary J., and Ginsberg, Ari. "The Policy Grid: A Cognitive Methodology for Assessing Policy Dynamics." A paper, University of Pittsburgh, Program for the Study of Knowledge Use, 1984.
 

________. "Evaluating the Effects of Policy Analysis: Toward a Theory of Applications." Prepared for Stuart S. Nagel (ed.), Research Annals in Public Policy Analysis and Management. Greenwich, CT: JAI Press, 1985a.
 

________. "Methods of the Second Type: On Coping with the Wilderness of Public Policy Analysis." Paper prepared for presentation at the Annual Research Conference, Association for Public Policy and Management, Washington, DC. Oct. 1985b.
 

Dye, Thomas R. Understanding Public Policy. 2nd ed. Englewood Cliffs, NJ: Prentice-Hall, 1975.
 

Etzioni, Amitai. "Mixed Scanning: a Third Approach to Decision-Making," Public Administration Review, 27 (1967) 385-392.
 

Fishbein, Martin and Azjen, Icek. Belief, Attitude, Intention and Behavior: An Introduction to Theory and Research. Reading, MA: Addison-Wesley Pub. Co., 1975.
 

Ganz, Carole. "Linkages Between Knowledge Creation, Diffusion, and Utilization." In The Knowledge Cycle. Edited by Robert F. Rich, Beverly Hills, CA: Sage Publications, 1981.
 

Glaser, Edward M., Abelson, Harold H., Garrison, Kathalee, N. Putting Knowledge to Use. San Francisco, CA: Jossey-Bass Inc., 1983.
 

Greg, G., Preston, T., Geist, A. and Caplan, N. " The Caravan Rolls on: Forty Years of Social Problem Research." Knowlege: Creation, Diffusion, Utilization. 1, no.1, (1979) 31-61.
 

Habermas, Jurgen, Theory and Pratice. Boston: Beacon, 1973.
 
 
 
 

Hammond, K. R., and Adelman, L. "Science, Values and Human Judgement." In Judgment and Decision in Public Policy Formation. Edited by K. R. Hammond, Boulder: Westview Press, 1978.
 

Herchman, A. O. and Lindblom, C. E., "Economic Development, Research and Development, Policy Making: Some Converging Views," Behavioral Science. (April 1962) pp. 211-22.
 

Holzner, Burkhart and Marx, John. Knowledge Application: The Knowledge System in Society. Boston: Allyn and Bacon, Inc., 1979.
 

Hogwood, Brian W., and Peters, B. Guy, Policy Dynamics. New York: St. Martins Press, 1983.
 

Jones, Charles O. An Introduction to the Study of Public Policy. 2nd Ed. North Scituate, MA: Duxbury Press, 1977.
 

Kahneman, D., Slovic, P., and Tversky, A. Judgment under Uncertainty. New York: Cambridge University Press, 1982.
 

Knorr, K. D., " Policymakers' Use of Science Knowledge; Symbolic or Instrumental?" in Using Social Research in Public Policy Making. Edited by Carol Weiss, Lexington, MA: Lexington Books, 1977.
 

Lasswell, Harold D. "The Policy Orientation." in The Policy Sciencies: Recent Developments in Scope and Method." Edited by Daniel Lerner and Harold D. Lasswell, Stanford, CA: Stanford University Press, 1951.
 

________. A Pre-view of Policy Sciences. New York: Elsevier, 1971.
 

Lindblom, Charles. The Intelligence of Democracy. New York: MacMillan, 1968.
 

________. Incrementalism and Environmentalism." In Managing the Environment. E. P. A., pp. 83-84. Washington, DC: Government Printing Office, 1973.
 

________, Cohen, David K. Usable Knowledge: Social Science and Social Problem Solving. New Haven, CN: Yale University Press, 1979.
 

________. The Policy-Making Process. 2nd Ed. Englewood Cliffs, NJ: Prentice- Hall, Inc., 1980.
 

Mason, Richard O., and Mitroff, Ian I., Challenging Strategic Planning Assumptions: Theory, Cases and Techniques. New York: Wiley, 1981.
 

Machlup, F. Knowledge and Knowledge Production. Princeton: Princeton University Press, 1980.
 

Mackay, Donald M., "The Informational Analysis of Questions and Commands." in Modern Systems Research for the Behavioral Scientist. Edited by Walter Buckley, Chicago: Aldine Publishing Co., 1968.
 
 

MacRae, Duncan, Jr. Policy Analysis for Public Decision. North Scituate, MA: Duxbury Press, 1979.
 

Merton, Robert K. "Social Problems and Sociological Theory." In Contemporary Social Problems.Edited by Robert K. Merton and Robert A. Nisbet, New York: Harcourt, Brace & World, 1961.
 

Miller, J. G. Living Systems. New York: McGraw-Hill, 1978.
 

Mitroff, Ian I. and Mason, Robert O. Creating a Dialectical Social Science: Concepts, Metods, and Models. Dordect, Holland: D. Reidel Publishing Company, 1981.
 

________, Bonoma, Thomas V. "Psychological Assumptions, Experimentation and Real World Problems." Evaluation Quarterly, 2 no. 4 (Fall 1976) pp. 639-662.
 

Morgan, Garth (ed.), Beyond Method: Strategies for Social Research. Beverly Hills, Ca: Sage Publications, 1983.
 

Nagel, Stuart S., Policy Evaluation: Making Optimum Decisions. New York: Praeger Publishers, 1982.
 

Osborn, A. Your Creative Power. New York: Charles Schribner, 1948.
 

Patton, Michael Q., Grimes, P. S., Guthrie, K. M., Breman, N. J., French, B. D. and Blyth, D. A. "In Search of Impact: An Analysis of the Utilization of Federal Health Evaulation Reseach." In Using Social

Research in Public Policy Making. Edited by Carol Weiss, Lexington, MA:D.C. Heath and Company, 1977.
 

________. Utilization-Focused Evaluation. Beverly Hills, CA: Sage Publications, 1978.
 

Popper, Karl R. The Logic of Scientific Discovery. 2nd ed. New York: Harper & Row, 1965
 

_________. The Open Society and Its Enemies. Vol. 2 New York: Harper & row, 1967.
 

Quade, Edward S. Analysis for Public Decisions. New York: Elsevier, 1975.
 

Quinn, J. B. Strategies for Change: Logical Incrementalism. Chicago: Richard D. Irwin, Inc., 1980
 

Rein, Martin. "Methodology for the Study of the Interplay between Social Science and Social Policy." International Social Science Journal 32 , no. 2 (1980): 361-368.
 

________, and Schon Donald H. "Problem Setting in Policy Research." In Using Social Research in Public Policy-Making. Edited by Carol H. Weiss, Lexington, MA: Lexington Books, 1977.

Rich, Robert F. "Use of Social Science Information by Federal Bureaucrats: Knowledge for Action versus Knowledge for Understanding." In Using Social Researcher in Public Policy Making. Edited by Carol H. Weiss, Lexington, MA: Lexington Books, 1977.
 

Rittel, Horst W. J., and Webber, Melvin M. "Dilemmas in a General Theory of Planning." Policy Sciences 4, no. 2 (june 1973)
 

Rogers, E. and Kincaid, D. Communications Networks: Toward a New Paradigm for Research. New York: Free Press, 1981.
 

Rothman, J. Social R & D: Research and Development in the Human Services. Englewood Cliffs, NJ: Prentice-Hall, 1980.
 

Simon, Herbert A. Administrative Behavior: A study of Decision Making Processes in Administrative Organizations. 3rd ed. New York: Free Press, 1976.
 

________. Sciences of the Artificial. 2nd ed. Cambridge: MIT Press, 1981.
 

________. Reason in Human Affairs. Stanford: Stanford University Press, 1981.
 

Spector, Malcolm, and Kitsuse, John I., Constructing Social Problems. Menlo Park, CA: Cummings, 1977.
 

Toulmin, S., Rieke, R, and Janik, A. An Introduction to Reasoning. New York: MacMillan Publishing Co. Inc. 1979.
 

Wallace, Walter. The Logic of Science in Sociology. Chicago: Aldine . Athrerton, Inc., 1971.
 

Weilenmann, Alexander "Cybernetics." Encyclopedia of Computer Science and Technology. Vol, 7. 53-69
 

________. Evaluation Research and Social Change. Paris: UNESCO, 1980.
 

Weiss, Carol H. "Introduction." In Using Social Reasearch in Public Policy Making. Edited by Carol H. Weiss, Lexington, Ma: D.C. Heath and Company, 1977.
 

Weiss, Janet A., And Weiss Carol H. "Social Scientists and Decision Makers look at the Usefulness of Mental Health Research." American Psychologist. 36 no. 8 (Aug. 1981): 837-847.
 

________, and Bucuvalas, M. "Truth Tests and Utility Tests." The American Socialogical Review, 45:(180) 302-312.
 

Wildavsky, Aaron, Speaking Truth to Power: The Art and Craft of Policy Making. Boston: Little, Brown and Company, 1979.
 

________. The Politics of the Budgetary Process. 4th ed. Boston: Little, Brown and Company, 1984.

________. "Forword" in Problem Definition in Policy Analysis. by David Dery, University Press of Kansas, 1984.
 

Wilson, James Q. "'Policy Intellectuals' and Public Policy." Public Interest, No. 64 (Summer 1981): 31-46.
 

________, Ed. Crime and Public Policy. San Francisco: Institute for Contemporary Studies, 1983.
 

Zaltman, Gerald. "Knowledge Utilization as Planned Social Change." Knowledge: Creation, Diffusion, Utilization. 1, no.1 (Sept. 1979): 82-105.
 

________. "Constructing Knowledge Use." Working Paper KU-03. University of Pittsburgh, Program for the Study of Knowledge Use, 1982.
 
 

thought provoking quotes
WA8PHD
Machu Picchu
"FLUENT" APHASIA
Stoke and Diabetes
Aphasia, Stoke and Diabetes
Academic and Ph.D. Studies
Flying
Criminal Justice Work
Weather
pho-tog-ra-phy
 Personal and more
 
The Institute for Idiotic Pursuits
(Home)
 Site Map