Open Source Intelligence An Intelligence Lifeline

Stevyn Gibson

Stevyn Gibson is currently undertaking PhD research into

‘OSINT and the National intelligence Machinery” at the Department of

Defence Management and Security Anatysis, Royal Military CoEege of Science, Cranfield University. This

article was” awarded First Prize in the RUSJ Trench Gascoigne Prize Essay

Competition 2003

RUSI JOURNAL FEBRUARY 2004

Lord Hutton’s comprehensive verdict in favour of the government and against the BBC (read media generaLLy) was delivered on 28 January 2004.A leading Law Lord, Hutton gave his ‘ruling’ in a judicial fashion: he found in favour of one of the two sides implicated in the Inquiry into Dr David KelLy’s death. ALthough critics have challenged the one-sidedness of his report, its findings were based on an exhaustive review of evidence led by an eminent jurist with a reputation for integrity and independence. That said, a central question that has dominated the national debate since Dr KeLly’s death – whether pre-war intelligence informed government decision-making or was formed to support decision-making – was not addressed in the Inquiry. Its omission fuelled caLLs for a new independent inquiry, which was duly announced by Prime Minister Blair on 3 February. Expected to report by summer 2004, this new inquiry will investigate the accuracy, validity and reLiabiLity of the pre-war intelligence product, particuLarLy concerning Iraq’s WlVlD threat but perhaps also the former regime’s alleged [inks to AL-Qa’ida.

Until then, the public wilL be none the wiser about the challenges of the intelligence process and no dearer on the traditionally stated and acceptable role of intelligence: to inform decision-making- independently, impartially and with integrity. Hopefully, this article, written before the Hutton Inquiry was published and unchanged since, may go some way to clarify that.

Introduction The September 2003 Hutton Inquiry represents a low point in the standing and raison d’etre of the UK national security intelligence function. The low point both

reflects and obscures the key question from which this Inquiry has emanated: has intelligence, with respect to WMD and links to At-Qa’ida, been pushed or pulled in order to derive a casus belli? More broadly p u t – is intelligence used to inform decision and policy-making or is intelligence formed to support their pre- determination? The perception in the public mind ranges from confusion to boredom, while the worrying conclusion being drawn by many eminent scholars of inteLLigence seems to be coalescing around the latter view, that intelligence is being selected and harvested to prop up pre-determined policy. ~ If this is the case, then inteLLigence, both product and process, wiLL be tainted as a result.

However, stilt greater forces are at w o r k – reflections of contemporary society that are even more capable of overwhelming the intelligence function than Hutton. This article attempts to throw inteLLigence a lifeline by examining the emerging role of open source inteLLigence (OSINT), drawing together the contextual influences that are bringing about its potentiaLLy starring rote and identifying the contribution it can make to defence and security in return.

Forces of change Emerging from the Hutton Inquiry are equally encouraging signs that demonstrate a desire for openness in civil society generally and in the intelligence community in particular. The very public nature of the inquiry, the unprecedented scrutiny of key inteLLigence and security service officials, and the dissection of the intelligence process at its highest levels, all demonstrate an acquiescence, if not willingness and determination, to bring intelligence out of the closet. ~ Nevertheless, not wishing to diminish t h e

D ow

nl oa

de d

by [

A m

er ic

an P

ub li

c U

ni ve

rs it

y S

ys te

m ]

at 1

8: 19

1 3

M ar

ch 2

01 3

The intelligence function can reform and adapt to all three shapers o f global, postmodern, risk society or react to maintain the status quo and become irrelevant in the process

benefits for national corporate governance that Hutton is having, the verdict at the Royal Courts of Justice will not be sufficient to preserve intelligence, informing or formed, from new forces at • work in contemporary society.

Three such forces seem pre-eminent. First, the framework in which much of civil society is conducted, not just in developed nations but also globally, has changed irreversibly to the context of ‘postmodernism’. Second, taken-for- granted concepts of the nation-state, democracy, trust and freedom are under threat from the activity of g[obalization in its free-market, consumer-oriented form, which nation-state governments seem unwilling or unable to protect themselves against, s Third, the ubiquitous • phenomenon of risk is the new altar at which all decision and policy-makers must now worship. The intelligence function can reform and adapt to all three shapers of global, postmodern, risk society or react to maintain the status quo and become irrelevant in the process. • Rathmell acknowledges this dilemma when he discusses the need for a ‘postmodem intelligence’ responsive to these forces of change. 4

P o s t m o d e r n i s m • Postmodernism is largely a developed- nation phenomenon but with global consequence. It is the culmination in the evolution of Western societies from hunter-gatherer through settled • agriculturalist, the Enlightenment, industrial revolution and the information- technology communication age to the contemporary world in which we live. s Its • characteristics include:

• The end of industrialization and the era of information processing

into knowledge as the single most significant portion of the service sector;

The end of mass production and the recognition of the individual together w i t h the consequent creation of niche markets;

• The globalization of commerce and economics;

• The ease of travel and collapse of borders;

• The explosion of information and information overload; and

The emergence of the citizen w i t h rights, aspirations, education and influence. 6

Postmodernist themes are dominated by growing realizations that:

There is no grand formula for life rather a continual process of dealing with combined complexity, uncertainty and ambiguity;

ObJective evidential science is no longer enough. Rather, social and cultural constructs of how the world ‘is’, also play their part. 7

Sciences, disciplines and philosophies are blurring, sharing and learning from each other;

The spread of knowledge is transforming hierarchies and centralized bureaucracies into networked individual centres of excellence; and

• A growing recognition that everything is connected to everything else.

During the course of the last fifteen years – with the collapse of the Soviet Union, the removal of the Cold War’s bi- polarizing world influence, the more recent 9111 terrorist attack (as much an attack on the perceived ends of globalization as on America), the combined ‘paradise’ of liberal democracy and moral consciousness that Europe luxuriates in, the pre-emptive power displays t h a t the US dips in and out of as it chooses to move between world policeman and isolationist state e – the world has transitioned from the modem to the postmodern. 9 The ramifications for many, certainly in most of the developed world, is that life has become more complex, fast, interdependent and uncertain than it has ever been. Equally, postmodernism is creating a world dominated by risk to such a degree that its management (particularly where the risk is negative) has become almost mandatory, for individuals, organizations and societies alike, to undertake. I° Postmodernism travels under many pseudonymous, from Ulrich Beck’s ‘risk society’ to S[ovic’s ‘post normal science’ to Fukuyama’s ‘posthuman future’. They all particularly note the change in the conduct and order of civil society effected by science’s spin- off – technology. 11

G l o b a t i z a t i o n – d e m o c r a c y , t r u s t a n d f r e e d o m The end of the Cold War dragged us out of a torpid, linear and polarized historical cul-de-sac and propelled us back into history’s more customary but turbulent flow. Yet postmodernism is not only characterized by the collapse of

17

D ow

nl oa

de d

by [

A m

er ic

an P

ub li

c U

ni ve

rs it

y S

ys te

m ]

at 1

8: 19

1 3

M ar

ch 2

01 3

The issue f o r OSINTis no longer its validity or usefulness but rather how could it be developed, institutionalized and rolled out as a discipline common to government intelligence analysts and commercial knowledge workers alike

communism, the release of tribalism and the emergence of catastrophic terrorism but also by globa[ization’s impact upon the ‘nation-state’ and its attendant mainstays of democracy, trust and freedom. 12

The very g[obalization that brings much in the way of progress is also the rallying point against which ‘co[[ective- Jihad’ is waged, not only by Islamist religious extremists but also by a range of the dissatisfied, disenchanted and disenfranchised. These range from ‘anti- g[obatizers’ to ‘countryside-alliancers’, across multi-national, multi-religious and multi-class groupings, curiously united against centralizing governments and, as Barber describes them, ‘McWor[d’ type corporations, as they see not progress but only threats to their way of life. ‘s Therefore, it is no longer just the nation- state that requires an intelligence function, which until recent times has been exclusively delivered by the public sector, but also the corporate world, and for that both are turning to OSINT particularly when the public sector appears reticent or unable to assist. ~4

Confucius said: ‘without trust we cannot stand. ‘Is In any given society, democracy survives because trust in all its institutional manifestations is firmly rooted in the people and culture of that society. From the loyal opposition to the apathetic but ‘content’ voter, democracy is embraced. Democracies break down because that trust is not deep-rooted enough – geographically, socially, culturally and temporally – to survive ‘difficult’ times such as occurred in Germany’s post-First World War Great Depression or the USSR satellite states of the late 1980s. ‘6 Trust occurs across a rainbow of relationships from inter- persona[ to international. Key contributors

RUSI JOURNAL FEBRUARY 2004

to the establishment of trust at any level, whether between you and your neighbour or between nation and nation, are openness, co-operation, communication and the persona[ nature of its giving and receiving. The intelligence function within a democratic society enJoys a two-way relationship with the public it serves. The public trusts it and it creates trust in the collective mind of the public. The currency of exchange is information, in the broadest sense of the word. If that currency is restricted then trust diminishes with it.

Risk – complexity, uncertainty and ambiguity Risk is measured by the product of its constituents: likelihood (p) and impact (I). If only risk were that simple! From SARS to Sadaam we live in a complex world. However, complexity is now too simple a description of today’s world. The promise of unlimited progress, offered by human intervention through science and technology at the dawn of the industrial revolution, now brings fear and detriment in equal measure at the end of it. Renn elucidates a catch-all taxonomy that characterizes postmodern risk as predominantly complex, uncertain or ambiguous. 17

Complex risk can be managed to a considerable degree by the application of science and technotogy, then driving them (such as those presented by nuclear power in the early 1970s) towards acceptable levels. Complexity is not the problem. Uncertainty and ambiguity, present to varying degrees in each and every risk alongside complexity, bring additional challenges to risk management that can contradict and negate the work of scientists. Ambiguity and uncertainty are a growing feature of postmodem

society. Ambiguity is science’s political equivalent of ‘debate’.There is little scientific argument about the data, the methodology or the observation, but considerable disagreement about what all this measurement means.The polarized scientific debate over GMOs is proving a classic example. ~° Nanotechnology is already on the horizon! Uncertainty is manifest where scientific regulation struggles to play catch-up with the very scientific development that it is supposed to be regulating. Uncertainty is not just ambiguity’s inability to quantify or qualify impact but additionally the incapacity to scientifically measure likelihood. Terrorist risk displays both – immeasurable likelihood and unimaginable impact.

Thus risk has a ‘dual nature’, characterized by risk theoreticians as a combination of its objective reality and its social construct. Scientists can manage the objective nature but its social construct demands repeated reassurance from people who can deliver valid messages – risk managers, whether they are regulators, government or private bodies – that they are not only endeavouring to assess and treat risks but also identify them, communicating and disseminating what they find. A critical step in this process is the creation of knowledge to inform social construct. Knowledge that can be communicated and disseminated freely is a small but valuable part of the response to the changing force of complex, ambiguous and uncertain risk.

Finally for risk – it is managed not solved. Risk management is a process not an obJective. Managed risks leave behind residual risks, which are re-examined or accepted until such time as science or other methods catch up and deal with them. In the intervening period society

D ow

nl oa

de d

by [

A m

er ic

an P

ub li

c U

ni ve

rs it

y S

ys te

m ]

at 1

8: 19

1 3

M ar

ch 2

01 3

should be educated to accept and live with those risks by becoming habituated to them rather than irrationally expecting them to be removed instantly. TM

Open Source Intelligence ( O S I N T ) OSINT is the analytical exploitation of information that is legally available and in the public domain. 2° That is to say it is neither acquired clandestinely through espionage or illegal means nor ‘cLosed’ to the public by government or commercial sensitivity. Such information has always been available but the last two decades have given it a recognition and usage commensurate with many changing aspects of contemporary society as both a product of it and a tool to deal with it.

OSINT need not necessarily be obtained openly in that the acquirer Leaves a calling card. It can be discretely acquired. Information, obtained clandestinely or openly, whose disclosure creates vuLnerabi[ities for sources, methods or intentions, must of course become ‘closed’ by classification or commercial sensitivity procedures. However, classification without justification, preventing communication and dissemination rather negates the potential of OSINT. Regrettably, ‘need to know’ has become a debate complicated more by issues of organizational culture and personal vested interest than operational security. The mounting dilemmas of global, postmodern, risk society and the recognition of the value of OSINT, of themselves, are creating pressures to change this. However, a reactionary intelligence community wishing to preserve all that is ‘traditional’ will only compound and reinforce these dilemmas.

What sources are there? The Intemet,

and before that newspaper ‘cut-and- paste’, are no longer the stereotypes of OSINT. Indeed the Internet is not of itself a source but merely the means by which sources are accessed. Open sources can broadly be categorized into: traditional media broadcast such as that captured by the BBC Monitoring Service or Foreign Broadcast Monitoring Service (FBIS); commercial ‘on-line premium’ such as Factiva, Lexis-Nexis or Dialog for global media coverage; specialist technical/tactical coverage such as Jane’s, Oxford Analytica or the Economist Intelligence Unit; ‘grey Literature’- information which is obtained from expert channels including academia and private information brokers; overt human observers – the most valuable means of ascertaining ‘ground truth’ such as International Alert and Amnesty International; commercial imagery – there are some eleven private (commercia D high-resolution (near Im) remote sensing satellites available to credit card holders; 21 and mapping specialists such as Eastview Cartographic, suppliers to the US DoD for Afghanistan, Iraq and most recently Iran(!). 22 It is worth noting that in all of these categories a significant and critical issue implicit to each of them, and one that remains to be addressed by OSlNT as we[[ as intelligence generally, is the issue of language. We ignore at our peril Steele’s estimate that twenty-nine languages are considered minimum entry for a complete intelligence picture. 23

Intelligence or knowledge, regardless of the origin of its precursor-information (open or clandestine), must be timely, accurate, relevant and verifiable. 24 It must answer a question and it must engender proactive actionable decision-making even if that decision is not to act. One of the criticisms of OSINT is that it is not

easily verifiable or evaluated. This perception is particularly true of information derived freely via the Internet. It is a less expressed criticism of information derived from premium content sites, academic peer-reviewed grey-Literature or ground truth experience. Like all sources of information, trust, the passage of time, and analyst expertise become the defining arbiters of value. Being in the public domain is not to be confused with being available to the public. There are barriers to entry, notably, money and effort. The exchange of information for money or endeavour, or both, still remains a potent validation of the worth of that information in a free market economy. The assertion that the value of intelligence represented by degree of classification is the defining mark is at best misguided and worst psychotic. Closed information displays a degree of sensitivity of the source, the method by which it was obtained or the intention for which it is being used not the value it affords the creation of knowledge, decision-making and action. The open source convention is to consider and review the following checklist for each and every open source: 2s

• Authority- does the source command respect from its peers or customers?

Accuracy- is the source corroborated and benchmarked against other validated all-source material?

Objectivi~ – does the source advocate or balance views? To whom does it link? Who or what does it represent?

Currency- is it dateltimelp[ace/ author-tagged for currency?

19

D ow

nl oa

de d

by [

A m

er ic

an P

ub li

c U

ni ve

rs it

y S

ys te

m ]

at 1

8: 19

1 3

M ar

ch 2

01 3

R U S I

OSINT offers a li eUne to intelligence by allowing it the freedom to communicate and disseminate risk issues, thereby informing perception and creating trust. In its turn, intelligence, informing rather than formed, can offer a lifeline to the beleaguered- democracy, trust and freedom

Coverage – is it relevant (i.e., adds to understanding) or is it just interesting or circular reporting?

OSINT is accepted practice in the private sector where it merges with knowledge management and competitive intelligence. It is becoming more sophisticated with specifically developed techniques, tools, evaluation procedures and expert training. It would seem sensible to conclude that if OSINT is such a significant and growing input to private sector decision-making then public sector defence and security (intelligence included) should sit up and take note. The issue for OSINT is no longer its validity or usefulness but rather how could it be developed, institutionalized and rolled out as a discipline common to government intelligence analysts and commercial knowledge workers alike.

OSINT’s contribution OSINT is both a product of and tool for dealing with all three forces driving contemporary change. Open source information is a front-end ingredient for the process of analysis by which intelligence or knowledge is created in support of decision and policy-making, whether it is in defence, security or any function of society. But in an age characterized by instantaneous, distributed, publicly available, open source information, uninformed decision-making arising from an inability to understand, harness and exploit the potential of this new breed of information becomes a significant security weakness. Information gaps create communication credibility challenges, which lead to mistrust and a destructive cycle of stigma, increased

RUSI JOURNAL FEBRUARY 2004

mistrust and further credibility challenges for all policy makers. 26

Why is OSINT so good? This presupposes that it is good relative to something else and that ‘something else’ is traditionally held to be intelligence obtained through espionage. The more perceptive organizations that require knowledge to function are beginning to appreciate that the two are not in competition but mutually supportive. It has been estimated by many senior representatives of the intelligence community, that approximately 80 per cent of knowledge, upon which decisions are made and action is taken in the public sector, derives from OSINT. 27 The original source of this figure may very well have been Allen Du[[es (former Director CIA), when in 1947 he made the following comment as part of his testimony to the Senate Committee on Armed Services, 25 April 1947:

A proper analysis o[the intelligence obtainable by these overt, normal and aboveboard means would supply us with over 80 percent, I should estimate, o [ the information required[or the guidance o [ our national policy. 28

His testimony was only nine pages long and hastily written; but in it, as Markowitz has noted, he began the process of the demystification of the art of intelligence. 29 Anecdota[ly, this figure may be nearer 90 per cent and, for some all-source intelligence agencies, is the preferred ‘knowledge’ of choice3 ° But 90 per cent of what? Is it 90 per cent of a final intelligence report or 90 per cent of action outcomes, i.e. an arrest or a threat interdiction?

Whatever it is a percentage of, it remains a subjective judgment but its perceived efficacy by practitioners and more importantly satisfied customers is likely only to increase31

At the [eve[ of intelligence qua process and product, and as an ‘INT’ in its own right alongside the clandestine ‘INTs’ (Humint, Sigint, Elint etc.), the main benefits of OSINT include the following:

It is fast, flexible, dynamic and cheap; 3z

It is communicable, sharable, trust creating and partner-forming, particularly for multi-national organizations such as NATO and the UN engaged in peacekeeping operations, where nationally- supplied intelligence has a restricted flow and therefore limited value; ~3

It identifies and mitigates risk at strategic, operational, tactical and technical levels -‘horizon scanning’ to sophisticated targeting;

It spans ‘quick and dirty’ evaluation to in-depth analysis; 34

It contextualizes the intelligence requirement both historically and currently, providing the matrix in which the clandestine ‘INTs’ can set their nuggets of closed information, as well as the foundation upon which they can be more effectively and efficiently directed;

It contributes to the all-source collection process of itself and by ‘freeing-up’ other ‘INTs’ for their own more concentrated espionage;

D ow

nl oa

de d

by [

A m

er ic

an P

ub li

c U

ni ve

rs it

y S

ys te

m ]

at 1

8: 19

1 3

M ar

ch 2

01 3

Defence a n d i I n t e r n a t i o n a [ ~ u r ~ i i

It provides ‘cover’ and risk communication possibilities for the other ‘INTs’; and

It provides ‘horizon-scanning’ to focus the other ‘INTs’. ‘If it is 85 per cent accurate, on time and I can share it, this is a lot more useful to me than a compendium of TS [Top Secret] Codeword materials t h a t are too late, too much and requires a safe and three security officers to escort it around the battlefield. ‘~5

OSINT can usefully contribute to the wider management of risk by enhancing the informing of perception, where little or none exists, through utilizing risk communication theory and generating virtuous circles of trust and confidence rather than mistrust and stigma. Perception is moulded by a variety of factors including: how the information is framed for communication; ~ the bias or culture of the sender and receiver; 37 the amplification of the signal or groups of signals t h a t form the message; s8 the ‘availability heuristic’ predisposing us to remember the most recent and/or most prominent signals; 39 and the theory of ‘affect’ which recognizes, amongst others, intuition, emotion and judgment in the formation of perceptionY The more people know about the risks they face – provided t h a t the informing has been balanced, honest, open and having preferably emanated from a trusted figure – the more likely they will be to cope, habituate and ultimately change behaviour? 1

At the level of national and international policy and decision- making, OSINT w i l l have its biggest role to play in generating resilience and competitive advantage. This would be

achieved by habituating citizen- decision-makers to risks, reducing their fear and impotence and returning decision-making and its corollary, action, to those individual decision- makers. Where the management of complex, uncertain and ambiguous risk is concerned – from prions to ‘dirty- b o m b s ‘ – OSINT can be used to inform, educate and habituate the perceptions of those risks before, during and after they have occurred. 42 Equally, if and when these risks do occur, as we have been promised they will, then a concerted risk management c o n t i n u i t y and recovery effort can be enhanced by the dissemination of useful information to the public through the media? 3 Before, during and after – the appropriate communication and dissemination of risk issues can in turn contribute to the preservation of democracy, trust and freedom or help reinstate it where it is lacking.

Conclusion OSlNT is not a new breed of intelligence per se. It is a common enough technique used by intelligence organizations in all sectors and by many generations of intelligence professionals. However, it is new to elements of the national intelligence and national security machineries in so much as it has been formally recognized and accorded its place by the relatively recent creation of open source intelligence cells and the appointment of open source intelligence specialists. Whether there is a strategy for its newly-elevated role in these machineries is of even greater interest. Is it w o r t h y of single-source, collection agency status or should it be integral to all source analysis?

At a functional level, it is already setting the context in which the other clandestine ‘INTs’ can operate and be focused as well as contributing to strategic, operational, tactical and technical intelligence analysis in its own right. However, in a far deeper way, it is also responsive to the new forces shaping our contemporary society by virtue of being forged out of them and empathic w i t h them. Where policy and decision-makers for resilience, security and defence must manage the complexity, uncertainty and ambiguity of the global, postmodern, risk society, OSINT offers a lifeline to intelligence by allowing it the freedom to communicate and disseminate risk issues, thereby informing perception and creating trust. In its turn, intelligence, informing rather than formed, can offer a lifeline to the beleaguered – democracy, trust and freedom. These should be the measures of the standing of intelligence, not Hutton. •

This concern was expressed by speaker after This concern was expressed by speaker after speaker at the ‘2002/2003 Intelligence Seminar Series’ at St Anthony’s College Oxford.

Hutton Inquiry, http :llwww. the-hutton- inquiry, org. uk/ as a t September 2003.

Tim Jackson and Laurie Michaelis, Policies for Sustainable Consumption: A Report to the Sustainable Development Commission (September 2003).

Andrew Rathmell, ‘Towards postmodem intelligence; Intelligence and National Security, (VoL 17, No. 3, 2002), pp. 87-104.

John Gray, Straw Dogs: Thoughts on Humans and Other Animals (London: Granta Books, ZOO2).

Samuel P. Huntington, The Clash of Civilizations and the Re-making of World Order (London: Simon ~ Schuster, 1996).

D ow

nl oa

de d

by [

A m

er ic

an P

ub li

c U

ni ve

rs it

y S

ys te

m ]

at 1

8: 19

1 3

M ar

ch 2

01 3

“(666L) “ssaJd ~!s.laAluli p~oJxo ‘(‘pg) “ueu#e9 “>1 pue Z~auuag “d “q~leaH

3!lqnd pue uo!:le3tunu~u~o9 ~lSlY Ul “, ssago~d uoispaq e se uo!:le3!unwwo9 )IS!y, ‘pleuo(73IV

auuv pue ~alo9 p!Ae(7 ‘:l:lauuag #a:la d

” L 6Z-E Z Z ” dd ‘ ( ZOOZ “Z “ON Z Z “lOA) “~e7 pue f3tlOd ‘Dl;flOd q~leaH JO lewnof ‘, aSualleV9

~,sa:leaJ 9 aq.~ ~ua~a.td fain ~aJJ:~ q:lleaH u~al-Suo7 pue :Da4pul aql :tu~poJ~al ~eapn N

~o le3!5oto!g ‘le3luJaq9 o,~ 5ulpuodsay, ‘fla~SaA4 UOWlE “fqdmiv sa~ue~ d ‘swe/(H q:lauua>1

“5~’- L # “dd ‘(Z86 L ” L L • IOA) fSOlOaplJo leumoffpa~enO,zpJezeH

lel~ua~od o.~ ~asuodsay lepos aq~ U!eldx 3 )lfly Jo uo!:lda:z~ad 3!lqn d sao(7, ‘.mzeiv “V

pue : L OOZ “‘~P do ‘~alqa~ “1 “e~oy ..3 “uuay “0 pue.~aSaef “D :E66L “uoi~epuno:l aSes llassny

“~lflY uo 5Ul~oday “fua~pu:7 “d pue ~aSuiE “E :EZ#L-2Z~”L “dd (Z “loA) :la~ue? “,s~o:De_#

5uHa55pZ pue pea~ds :uouawouaqd sse141 ~” –~!u~apld3 qe~efua~ aq.L , ‘lzze.to~aA “D pue

“qoeAay “IV ‘./a,Y.3V “….3 ?Seuoq “V ‘z~Je~E “V “1 ‘uflso9 “9 ‘5.~aquas~!aA4 “9 ‘qsO.ql “IV “uepoiv “g :60Z-8oz’dd (#66L ’60E “ON) leu.mof le3!paiv

qSl-fl~g ,,elpal, V aq.~ pue Ifqou.~aq9, “u!~a~g l ace

“L OOZ “~P do “le.~a.~aSaef “g pue p!ql

“O00Z “~P do “3!~OlE

” L OOZ ‘uofl3V leuopey pue f:lu!eTda3uli “)lily “~alqaA4 seu~ottl

‘esoy auaSn:? ‘uuay ui,~Td 0 pue ~aSaef ope9

‘~66 l “‘~P do “scuepv

“O00Z “~P do “~!AOlE

~OL d ‘( LOOZ) ppoA~ uado ue u I f3a~3aS pue sa!dE :a3uaSllla;Ul

uo ‘alaa~E ~aqoy u! L 66 L “pepclSe8 oJu! 5ul,’~ pea I atl~ JO,~apuew~o9 fAeN Ell

“ZOOZ’,,tapeay a_~uaS!lla.~u ! a3JnoE uado,

“(~OOZ “ssa~d leuo!;eula:lUl SEO) a~n~nd aq:l JoJ s~da3uo9 5Ul6~aw3 :a3uaS!lla~Ul 5u!daa~la3ead (‘pa)

‘alaa~ E :paqoy pue a#eld Sa!A4 ‘Suof a(7 ueg

“(LOOZ “sSa~d leuofleu~a~Ul SSO) ppoA,1 uedo ue u! f~a~aS

pue sa!d E :a~uaS!lla~ul uo ‘a/aa:lS ~aqoy

• ~o~3a~ 3!lqnd aq~ Ul ~4aJ e ,~S!l o:l afl3x_~ z 2 swo:~m.3

IVH >119 “VI9 WIO Ell “(70IV q3~no ‘(70141 Vs!PaA4E “(70IV >tl3 ‘li3 ‘7odoyn3 “01VN

‘E#

“Z#

“L#

“0#

“6E

“8E

“9E

“EE

“#E

“Z~

“LE

“~OOZ “~P “do ‘uosq!9 “O’Z

• (566 L ) ‘au~e~5oJd a3.moE uado f~lun~cuo9 Ell ‘ueld

3!6a~e,~ a3.moE uado ‘z~!A4O>lJeiv qdasof

“SZE dd ‘(#66L) “~alln(7 UallV Jo a,,(!7 aq.L :,(dE ue{,ual~ua9 ‘aso.~9 .~a;ad u! pa~uno3aJ se “SZ-SZ5 dd

“Z#6L “uolssaE :lsL “~uatuCl~llqe.;s:7 asuaJa(7 leUOl:leN aq:l uo sSuueaH

‘sa~!AlaE paw1V uo aa~:l!wu~o9 a~euaE

“#66L A”.m.~e.toqe7 leUOl.;eN sOWelV so 7 “uo!,~ez!ueS~o uope~aJ!lo~d

-~a~uno9 “!SUile3 E elne d J(7 “~9S dd “(# “ON “S L “lOA) a3uaS!lla;Ul-~a,tuno3

pue a3ua6!lla~ul Jo leumof leuo!:lewa,~u I ui >PiUlnH “s Jm.p:V ~osseJo~d ‘ L 9 L dd

‘(Z “ON “EL “lOA) “flla:l.~en~) uoi~ewJoJul 3uawu, laAoD U/pa~l~ (E/£O Jo Jo:l:)a~!(7

~acu~oJ) ~Po~13 p~eA,1 “(Z66 L ~aqwaAON Z L ) ~aU~lj. uo:16u!qse ~ (Vlq ~O~a~!(7

,~auJ.~oJ) UOSllA4 u~eE uaO 37 :a~e~l~sa Slq~ JoJ sa3.mos alq!pa.D snoJawnu a~e aJaql

“6Z

• (686 L “sSaJd uJn Uald :~Ho,~ ~aN) suo!~ez!ue5~o :luauJwaAoS-uoN pue ~uau~u~agOD Jo f?.iflq!suodsay pue aloy

aql :uo!~e~!unw~uo9 ~flY aAI:DaJJ3 “3!AOlS Ined pue ue~upues ~a~ad “OllaAO9 ;ua3u!A

“SZ

“(ZOOE) 01VN “~au~a.;ul aq~ Jo uo!~e:floldx 3 a3uaSilla.~u I pue ‘( L OOZ) 01VN ‘)looqpueH a3uaSllla~Ul a3~no~; uado

‘ZZ

“s:lsa55ns al:l!:l S~l ~o sA4Olle fe~sa flq:l ueq:l a3ed~ a~ou~ q3nu~ sa~!nba~ uopel;Ua~aJJ!p fuv • uop3ui~ipJo )l:)el aq:l q:l!~ aa~6e/~lpessa3au

~ou plno~4 sleuoissaJoJd a3uaS!lla:lu I • Su!q~ awes eq:l uea~u o,~/(lqeaSueq3~a~u!

aSpal~ow I pue a3uaS!lla~u! sam fessa flql

“gz

“SZ

“#Z

“( LOOZ) ‘am;n:~ aq~ ~oJ s~da3uo.o 5u!5~aw9 :a3ua6!lla~.Ul 5uidaa,ya3ead ‘alaa.L~ .;Jaqoy “EZ

EOOZ ~aqu~a;daE :le se dse’~ deu~ ~!vde~5odo,vxb/ue4/jsea–aEZpp!wlSOE z

“qfl3npo~d/~'(Tl/OOSa.~e9/ L EL “(TlUOlSaY/EE “qlea~WdEWbx/~uo3 “3!qde~5o~e3″44,w~//: d~4q’zz

(ZOOZ) “O.LVN “/apea~l a3uaS!fla~Ul a:)mo E uado, u! ,plJo,4A aqT, aSueq9

11! ~ £~a6ewl a~llla~eE lep~a~uu~o9 MOLL :ales .~oJ s7.a~aE, “!UHOld uuv pue epezuebqaa eft.le A ” L Z

“8-9 ;aSed ” (EOOZ “5 “ON “~ “lOA) Jo,woiv a3ua!ltsayz2 /O!~n~as

puelawOH s, auef/IEFlY ‘,SuPtelN-UO!Sl3aq Jo poolqaJ, q aql :~NIEO, “uosqto u~a:lE “OZ

‘~OOZ .,%.~IVN~IB3::I 1VN~NO[ ISNH

ZO6L -ZO6L “dd (ZOOZ ‘6f, E6″ON ’09E “IO4) ‘;a3ue-i

aql ‘ >pe.~e ~s!Jona:l pue 3!lqnd aq.I. Z 3!ued ~o a3ua!l!say , ‘/(la~SaA~ uouJ!E pue a!pom(7 ill.8 “6L

peolu,~op#tu:l{t’9 -67n/60-~n/5~o’a.;eqap3!lqndwSww~4//:d~: N

“(EOOE laq~a~ da E #~) “(EOOZ) 7~oday aa~!usuJo9 5u!Jaa~S – fSolouq~a~o!g

le:luau~uo.qAU9 pue lem.~ln~!lSv ; a,~eqaq ~!lqnd aq~ Jo sSu!pu!:l aql ZuofleN IVD, “SL

“#60l-ZZOL dd “(ZOOZ ‘9 “ON “ZZ “lOA) s!sfleuV ~ls!Y ‘,sa!Sa:le.L~E pa~e8 -a~mo3flO pue “pa~eg-uopne3a.~d ‘paseg-~lfl~l

:~uau~aSeuelN pue uoi:lenleA 9 )lS!y o; q3eo~ddv ~aN V, ‘ayu!l>1 seapuv pue uuay u!,~O “Z L

“OZ L -88 “dd'(666 t “sSa~d ~l~aA!Uli aSpi~qu~e9 :aSp~iq~eg) ~sn~J. pue

f:)e.~3ocuao ‘(‘pa) ua~.~eA,1 )peiv :ui,f3e~o~ap pue 5ulaq-lla,~ “,~mpl, “,ueqal6ul pleuoy “9~

“( ~68 L “~a~d uopuaJeD :p.~oJxo ) ueakV aq~ Jo au!.t~3o(7 aql pue “Su!u.~ea 7

~ea, tO aql “~3aleuv uepnJuo~ “a55a7 sau, tef “S L

‘uo!~e~ado, o~ Jo SlaAa 1 alqe:lua~ue 1 o~aq:llq ~e 4edsap s/O:lsnpu! uofleiAe aq.~ 5u!~4OllOJ slo.;~as o~, aCl.~ uaa~/~aq

uope3iunwwo3 Jo sueaw la:l:laq aJoldxa o.~ am:luaA :o.~3as a:leApdD!lqnd ;u!of e “wo~tun

:13afOJd Jo EOOZ u! dn 5up:las aq~ a3uaH “~, L

“P!ql “EL

(s66L ‘pHo~4suell :uopuoT)f3et3oLuaq o~ aSualleq9

s, u~si~o~al :ppo,4A~iv “SA peq!f UaqJeg UlLUefuag “Z L

(zooz “~oog al!JO~ d :uopuo-i) uo!:lnloAay f5olouq_3a.~o!8

aq.?,Jo sa::)uanbasuo9 :apn.~nj ueLunq~so d mO “ewefn)ln_~ si3uel:t pue ‘(O00Z “ue3sq~Je-2

:uopuoT) >IS!y Jo uo!:lda3Jad aql ‘3!AOlE lned ” L L

“( L OOZ “ssa~d ,9!lOd :a5p!~quleg) f:lal3OE )lS!y ppoA4 “)Pag q3tJln “oL

“(EOOZ ‘S>lOOg Jl:lUeflV :uopuoT) JapJo ppoA,1 MaN aq.; u! adom_~ pue

e~!Jawv uaMod pue as!paled “uefe>1 .;Jaqoy

w~s’#9EEEL Else3!JawelppoA41!W Z />ln’o~’3qq’sMaUll:d74 q

“(EOOZ ~aqwa~daE EL) ‘flqudassv leJaua9 NIl aq~ o~ q~aads flq u! ueuuv tJo>1 leJauaD £Je~al~as NIl fq o3 panaJaJ £11e~oNnbauR

“(566L “aSpaFlnoy :uopuoT) )IS!y “suJepv uqof

D ow

nl oa

de d

by [

A m

er ic

an P

ub li

c U

ni ve

rs it

y S

ys te

m ]

at 1

8: 19

1 3

M ar

ch 2

01 3