Meaning-Making Forum 1

Meaning-Making Forum 1 (Week 4)

Meaning-Making Forums 1-4 are this course’s unique final project. Be fully engaged in Phase One! After reviewing the readings, presentations, lecture notes, articles, and web-engagements, and previous assignments, artificially move your predetermined careseeker (i.e., Crossroads’ Careseekers: Bruce, Joshua, Brody, Justin, or Melissa) through Phase One.

NOTE: These research-based forums require that you draw upon ALL of the course readings and learning activities to date, in order to substantively develop each phase in our Solution-based, Short-term, Pastoral Counseling (SbStPC) process.  Noticeably support each core assertion.

  • Ministry or Agency-based Context. Introduce classmates to your actual or anticipated role in a ministry or agency-based context and your predetermined careseeker.
  • Guiding Purpose Statement. Concisely point out how a Guiding Purpose Statement will help you be and become more like Christ in every relational context, especially this pastoral counseling scenario.
  • Rapport and Relational Alignment. Briefly discuss how to build rapport and shift your relational style in order to best align with the careseeker’s style (i.e., use DISC language) and current behavioral position (i.e., attending, blaming, or willing).
  • Phase One Distinctive Features. Narrate movement of careseeker through Phase One’s distinctive features (i.e., purpose, goal, chief aim, role/responsibility, use of guiding assumptions) and apply pertinent insights and techniques from ALL the readings, previous assignments, and the Bible.
  • Phase One Marker. Describe a marker that indicates you have been invited into the careseeker’s story.
  • Food for Thought: After reviewing the readings and SbStPC Handout’s “The Art of Triage and Referral” websites, point out the essential elements in pastoral care triage and referral?

TIPS:

  • Carefully Follow Meaning-Making Forum Guidelines & Tips!
  • Make sure to use headings (6) so that the most inattentive reader may easily follow your thoughts.
  • Use the annotated outline approach. Bullets should have concise, complete, well-developed sentences or paragraphs.
  • Foster a “noble-minded” climate for investigating claims via well-supported core assertions (i.e., consider the validation pattern of the Bereans; Acts 17:11).  Noticeably support assertions to facilitate reader’s further investigation and to avoid the appearance of plagiarism.
  • Since you have the required materials (e.g., Solution-Focused Pastoral Counseling), abridge related citations (Kollar, p. 47) and do not list the required source in a References’ section.
  • Secondary sources must follow current APA guidelines for citations and References.
  • Make every effort to prove that you care about the subject matter by proofreading to eliminate grammar and spelling distractions.

A substantive thread (at least 450 words) is due by 11:59 p.m. (ET) of the assigned week/module in Course Schedule (Friday). One substantive reply (at least 150 words) to a classmate (via QUOTE function) is due by 11:59 p.m. (ET) of the assigned week/module in Course Schedule (Sunday).

PACO 500

Meaning-Making Forum Rubric (Based on 125 Point Total)

Criteria Levels of Achievement    
Criteria Advanced 92-100 (A- to A):

Satisfies criteria w/ excellence

Proficient 84-91 (B- to B+) :

Satisfies Criteria

Developing (C- to C+):

Satisfies most criteria

Below Expectations (F to D+):

Does not satisfy criteria

Not Present Points

Earned

Content 70% (87.5 pts.)          
Thread

 

65-70 pts.

· All key components of the Meaning-Making Forum prompt are answered in the thread.

· The thread has a clear, logical flow. All major points are stated clearly.

· All major points are supported by required evidence-based sources/readings to date and good examples or thoughtful analysis.

 

59-64 pts.

· All key components of the Meaning-Making Forum prompt are answered in the thread.

· The thread has a logical flow. Most major points are stated.

· Most major points are supported by required evidence-based sources/readings to date and examples or analysis.

53-58 pts.

· The Meaning-Making Forum prompt is addressed.

· The thread lacks flow and content. Major points are unclear or confusing.

· Major points include minimal examples or analysis.

1-52 pts.

· The Meaning-Making Forum prompt is addressed minimally or not at all.

· The thread lacks content. Major points are unclear, confusing or not discussed at all.

· Major points are not supported by examples or analysis.

 

0 points  
Reply

 

16.5-17.5 pts.

· One Reply with Quote directly addresses a related thread.

· The reply is a significant contribution supported by at least 1 required evidence-based source, thoughtful analysis of subject matter and thread.

15.5 pts.

· One Reply with Quote directly addresses a related thread.

· The reply is a contribution that reflects evidence-based thoughtful analysis of subject matter and thread.

13.5-14.5 pts.

· One Reply with Quote addresses a related thread.

· The reply lacks flow and content. Reply is unclear or confusing.

1-12.5 pts.

· One Reply with Quote marginally addresses a related thread.

· The reply lacks relevancy or clarity.

0 points  

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

Structure 30% (37.5 pts.)          
Organization / Style/Sources

 

23-25 pts.

· The thread is presented with appropriate headings in bold, annotated outline with concise sentences, and organizational clarity.

· Thread’s minimum word count of 450 words is met or exceeded.

· The reply contains a salutation and meets or exceeds 150 word count.

· Required sources/readings to date are noticeably present with appropriate APA or Turabian citations/references with format errors.

21-22 pts.

· The thread is presented with most headings in bold, annotated outline with sentences, but slightly lacking organizational clarity.

· Thread’s minimum word count of 450 words is met or exceeded.

· The reply contains a salutation and meets or exceeds 150 word count.

· Required sources/readings to date are noticeably present with appropriate APA or Turabian citations/references with minimal format errors.

19-20 pts.

· The thread is presented with partial headings, without annotated outline and/or clear sentences, and/or lacks organizational clarity.

· Thread’s minimum word count of 450 words is met or exceeded.

· The reply does not contain a salutation and/or meet 150 word count.

· Most required sources/readings to date are present yet reflect several APA or Turabian citations/references errors.

1-18 pts.

· The thread is presented without headings and/or clear sentences, and lacks organizational clarity.

· Thread’s minimum word count of 450 words is not met or exceeded.

· The reply does not contain a salutation and meet 150 word count.

· Three or more required sources to date are not present; Sources present lack appropriate APA or Turabian citations/references.

0 points  
 

Grammar/

Spelling

 

11.5-12.5 pts.

· Spelling, grammar are correct. Sentences are complete, clear, and concise.

· Paragraphs contain appropriately varied sentence structures.

 

9.5-10.5 pts.

· Sentences are reasonably complete, clear, and concise. Minor issues with proofreading/editing are noted.

· Paragraphs contain appropriately varied sentence structures.

 

8.5 pts.

· Sentences are less complete, clear, and concise. More pervasive / significant issues with proofreading / editing are noted.

· Paragraphs contain appropriately varied sentence structures.

 

1-7.5 pts.

· Writing is not at the graduate level. It was clear that the work had not been edited or proofread. Multiple issues are noted.

· Run-on paragraphs are observed. Sentence structure is not varied.

 

0 points  
Total / 125

 

Page 2 of 2

Ethical Biases

A minimum of  3 scholarly peered reviewed article  must be sited using APA format 500 words for each topic 81 and 82

 

Topic 81

Biases are something we all have, and it is important to be aware of what biases you have in order to monitor them carefully. Biases can be personal or professional in nature and confronting biases can help to overcome them. Discuss some areas on which you hold biases (divorce, domestic violence, death penalty, spanking, etc.)

What are your personal/professional biases? What harm can result from not being aware of them? What standards are violated if they are not acknowledged and addressed?

Topic 82

Dr. Jones has a client who is of a different culture and faith. He is not comfortable dealing with this patient due to past negative feelings from childhood. What are his ethical and legal obligations? Why are these legal and ethical obligations in place?

65

3 ETHICAL GUIDELINES FOR USING

SPIRITUALLY ORIENTED INTERVENTIONS

WILLIAM L. HATHAWAY

The empirical literature pertaining to clinical practice with religious and spiritual issues is still at a relatively early stage, but in recent years a substantial amount of attention has been paid to ethical issues in this domain (Gonsiorek, Richards, Pargament, & McMinn, 2009; Hathaway & Ripley, 2009; Plante, 2007, 2009; Richards & Bergin, 2005; Sperry & Shafranske, 2005). This liter- ature has focused on a wide range of ethical concerns, such as protecting against harmful bias, practicing within one’s boundaries of competence, and exploring role considerations in working with religious issues.

In this chapter, I begin by bringing attention to how psychologists’ rela- tive lack of religious commitment has the potential for creating and introduc- ing biases into treatment. A brief introduction provides readers with common conceptualizations of spiritually oriented interventions in the recent psycho- logical literature. This is followed by an examination and application of rele- vant ethical codes to the use of spiritually oriented interventions. Spiritually oriented interventions are then discussed from an accountable practice perspec- tive. Training recommendations are also provided to help facilitate the ethical application of such interventions. Brief clinical examples and questions are also offered to help readers delve deeper into thinking about the ethical issues that

To the psychologist the religious propensities of man must be at least as interesting as any other of the facts pertaining to his mental constitution.

—William James (1997)

12350-04_CH03-rev.qxd 1/11/11 11:44 AM Page 65

http://dx.doi.org/10.1037/12313-003 Spiritually Oriented Interventions for Counseling and Psychotherapy, by J. D. Aten, M. R. McMinn, and E. L. Worthington, Jr. Copyright © 2011 American Psychological Association. All rights reserved.

Co py

ri gh

t Am

er ic

an P sy

ch ol og ic al A ss oc ia ti on . No t fo r fu

rt he

r di

st ri

bu ti

on .

 

 

must be considered before using spiritually oriented interventions in clinical practice.

POTENTIAL FOR PROBLEMATIC BIASES

It has been frequently noted that professional psychologists appear to be atypically irreligious compared with the general North American popu- lation. Plante (2009), for instance, cited Gallup polls indicating that 95% of Americans believe in God and 40% of attend religious services on a weekly basis. Despite a widespread prevalence of religiousness in the general popula- tion, researchers (Bergin & Jensen, 1990; Delaney, Miller, & Bisono, 2007; Hathaway, Scott, & Garver, 2004; Shafranske, 2000) have noted that, relative to the general population, psychologists (a) have double the rate of claiming no religion, (b) are more likely by a factor of three to report religion being unim- portant in their life, (c) have a five-fold higher rate of denying belief in God, and (d) report lower likelihoods of attending religious services, being a member of a congregation, or engaging in prayer.

The risk is that this lower level of conventional religiousness among psy- chologists may result in biasing blind spots that lead them to erroneously dis- regard significant religious issues in clinical practice. Unfortunately, there is evidence that just this sort of neglect is occurring. Russell and Yarhouse (2006) found that over two thirds of a sample of training directors at American Psychological Association (APA) internships never foresaw offering training in religious and spiritual issues at their sites. Brawer, Handal, Fabricatore, Roberts, and Wajda-Johnston (2002) surveyed training directors of APA- accredited doctoral training programs and found that only 17% reported sys- tematic coverage of religion and spirituality in their programs. There is little evidence that such findings cause much concern outside of the niche of psychologists who specialize in the clinical psychology of religion. Imagine if such lassitude in the profession were the case for any of the other named diversity domains highlighted for particular attention in the APA (2010) Ethics Code (hereafter referred to as the Code).

Yet the situation may be even more problematic than just a climate of indifference. There is evidence that psychologists may be more likely than the general population to be hostile and prejudicial to conventional religion. Delaney, Miller, and Bisono (2007) noted that “it appears to be a relatively fre- quent experience among psychologists to have lost belief in God and disaffili- ated from institutional religion” (p. 542). They found this experience to be nearly 7 times more frequent in their sample of psychologists than in the gen- eral population. In a study of whether antireligious discrimination may be occurring in admissions to doctoral programs in clinical psychology, Gartner

66 WILLIAM L. HATHAWAY

12350-04_CH03-rev.qxd 1/11/11 11:44 AM Page 66

Co py

ri gh

t Am

er ic

an P sy

ch ol og ic al A ss oc ia ti on . No t fo r fu

rt he

r di

st ri

bu ti

on .

 

 

(1986) found that a sample of faculty at doctoral programs accredited by the APA were less likely to grant admission or to have positive feelings about appli- cants whose admissions protocols contained a conventional religious identifi- cation than about those whose protocols were otherwise identical except for the absence of such religious identification.

It seems unlikely that a negative or less receptive atmosphere among psy- chologists toward conventional religion would not translate into problematic clinical practice patterns toward this client population or niche. In a random national sample of clinical psychologists, Hathaway et al. (2004) found that most psychologists do not routinely assess for clinically relevant spiritual or reli- gious issues in practice. They also noted that a sizeable portion of their sample did not feel that religion is more than a slightly important adaptive domain for such focus.

Although there is no systematic research on the prevalence of apparent antireligious biases and/or overt discrimination toward conventionally religious clients by psychologists, numerous anecdotes have been recounted by clini- cal psychologists (Cummings, O’Donohue, & Cummings, 2009). A doctoral psychology intern at a respected internship informed me about being instructed by his supervisor to diagnose a client with a delusional disorder because the client expressed belief in intelligent design as opposed to evolution. The client reportedly did not display any other indications of thought disorder, psychotic process, or life impairment related to her beliefs. The intern expressed concern about giving this diagnosis, but the supervisor insisted and explained that the intelligent design belief itself was sufficient to warrant the diagnosis.

Let us assume that naturalism is true and all of the varieties of beliefs self- identified as intelligent design are false. This would hardly justify a mental health professional diagnosing a believer in intelligent design with a delusional disorder. It has become common in the polemics surrounding the new atheism to declare either theistic or atheistic belief to be a delusion (Dawkins, 2006; Hart, 2009). In terms of pure logic, either atheism or theism is true, but not both, so one of the two groups believes something that is false. Yet having a false belief is not the same thing as having a delusional belief, in a technical psy- chological sense. Delusional beliefs involve a disordered thought process and not just acceptance of beliefs that turn out to be factually incorrect (Clarke, 2001). Giving such a diagnosis in the absence of a genuine psychotic process runs a significant risk of iatrogenic consequences for the client, such as bearing the stigma of receiving an unwarranted diagnosis of a serious psychiatric con- dition, potentially having career and life options adversely affected, or being the recipient of unnecessary treatments. Thus, this practice may constitute a violation of the cardinal ethical concern of doing no harm.

Although negative or undervaluing biases toward conventional religion appear to be a common risk among psychologists, problems can also arise from

ETHICAL GUIDELINES 67

12350-04_CH03-rev.qxd 1/11/11 11:44 AM Page 67

Co py

ri gh

t Am

er ic

an P sy

ch ol og ic al A ss oc ia ti on . No t fo r fu

rt he

r di

st ri

bu ti

on .

 

 

proreligious biases. The American Psychiatric Association (1989) adopted guidelines warning against a psychotherapist imposing his or her religious val- ues or beliefs on clients. The ethical principle of nonmalfeasance (i.e., doing no harm) implies that psychologists should not attempt experimental proce- dures in lieu of standard proven psychotherapies without clear warrant and informed consent. I have encountered some psychologists who are personally religious abandoning standard approaches to common clinical problems for which well-supported treatments exist in favor of stand-alone explicitly reli- gious interventions. The stand-alone approaches eschew any other form of assistance apart from the religious or spiritual practice. Typically, these reli- gious caregivers have justified the stand-alone spiritual approaches in terms of their own religious beliefs about what is right for the person. Sometimes these spiritual-only-approach psychotherapists are licensed mental health professionals and other times they are not.

Some of their care recipients report benefits from such stand-alone spiritual-only approaches, but others do not. Their clients are not typically given any scientific data about likely responses to the approach (e.g., success rates, rates of nonresponders, adverse risks), although testimonials of success are frequently shared with the clients. Some persons in our community sought assistance from nonreligious caregivers after dropping out from these stand- alone spiritual-only treatments. The stand-alone dropouts indicated that they were not typically informed by the spiritual-only-approach provider about standard treatments for their concerns or about the experimental nature of the approach. In cases in which this care was being provided by a nonmental health professional, this is perhaps not surprising, but some of these cases involved licensed mental health professionals.

The stand-alone dropouts typically reported that their presenting issues had not improved. In fact, they sometimes now had added guilt and shame over not getting better from the stand-alone spiritual approach. When the lack of positive treatment response is attributed to God being unable or unwilling to help, it may deepen recipients’ faith conflicts and emotional pain.

Now, I am not suggesting that such anecdotes prove the stand-alone spir- itual treatments to be ineffective or noxious. Every treatment, even ones with good empirical support, has nonresponders and dropouts. Furthermore, there is a growing body of evidence that spiritually focused and accommodative approaches that combine spiritual interventions with standard psychotherapeu- tic techniques and relational skills are benign and helpful to clients (Tan & Johnson, 2005; Worthington & Sandage, 2002). My concern has more to do with ethical issues raised by the way the licensed caregivers engaging in the stand-alone spiritual approaches practiced (Gonsiorek et al., 2009).

These stories call attention to the need for psychologists to adequately consider the range of relevant ethical principles, standards, and other consid-

68 WILLIAM L. HATHAWAY

12350-04_CH03-rev.qxd 1/11/11 11:44 AM Page 68

Co py

ri gh

t Am

er ic

an P sy

ch ol og ic al A ss oc ia ti on . No t fo r fu

rt he

r di

st ri

bu ti

on .

 

 

erations that should guide our practice with regard to religious and spiritual issues (Knapp & VandeCreek, 2006). Hathaway and Ripley (2009) pointed out that such guidance can be found by reflecting on relevant ethical codes, pol- icy statements, practice guidelines, legal precedents, exemplar guidance, and evidence-based practice considerations. Let us now reflect on their relevance for the explicit use of spiritually oriented interventions by psychologists.

SPIRITUALLY ORIENTED INTERVENTIONS

A growing literature on spiritually oriented interventions provides detailed descriptions of how to conduct such interventions competently and ethically (Plante, 2009; Richards & Bergin, 2005; Schlosser & Safran, 2009). There is no standard language used to identify this group of interventions. Plante (2009) described them as spiritual practices or tools. Richards and Bergin (2005) referred to them as either theistic or spiritual interventions (p. 281). Schlosser and Safran (2009) called them spiritual interventions and techniques (p. 199). There is considerable overlap among the spiritually oriented interven- tions enumerated by these authors (see Table 3.1). Although some of these would likely be readily thought of as spiritual by most individuals (e.g., the use of prayer), others may be less obvious examples to some of a specifically “spiri- tual” intervention (e.g., meaning making or relaxation).

Among psychotherapists who seek to incorporate an explicitly spiritual aspect to treatment, Schlosser and Safran (2009) also distinguished between two general approaches: “spiritually accommodative approaches typically com- bined a manualized treatment with practices and beliefs from a particular world religion, whereas spiritually oriented approaches are typically less standardized and more inclusive” (p. 200). It should be noted that none of the psychologists whose work is cited in Table 3.1 is proposing a stand-alone use of spiritual and religious interventions or techniques regardless of whether they are used in a spiritually accommodative or spiritually oriented manner.

RELEVANT ETHICAL GUIDANCE

The Code provides psychologists with a list of ethical aspirational princi- ples and enforceable standards that are either explicitly or implicitly relevant to the use of spiritually oriented interventions in clinical practice. An example of an explicitly relevant principle from the Code is Principle E, titled “Respect for People’s Rights and Dignity.” The principle states the following:

Psychologists respect the dignity and worth of all people and the rights of individuals to privacy, confidentiality, and self-determination.

ETHICAL GUIDELINES 69

12350-04_CH03-rev.qxd 1/11/11 11:44 AM Page 69

Co py

ri gh

t Am

er ic

an P sy

ch ol og ic al A ss oc ia ti on . No t fo r fu

rt he

r di

st ri

bu ti

on .

 

 

Psychologists are aware that special safeguards may be necessary to pro- tect the rights and welfare of persons or communities whose vulnerabil- ities impair autonomous decision making. Psychologists are aware of and respect cultural, individual, and role differences, including those based on . . . religion . . . and consider these factors when working with mem- bers of such groups. Psychologists try to eliminate the effect on their work of biases based on those factors, and they do not knowingly participate in or condone activities of others based upon such prejudices. (APA, 2010, p. 1063)

The earlier discussion of the role of bias in psychological practice with religious issues directly and explicitly intersects with Principle E. Psychologists should be diligent and intentional in preventing relevant biases from affect- ing religious or spiritual issues in treatment. The simplistic characterization of

70 WILLIAM L. HATHAWAY

TABLE 3.1 Three Lists of Spiritually Oriented Interventions

Theistic/spiritual Spiritual interventions interventions and techniques Spiritual practices or tools

Richards & Bergin (2005) Schlosser & Safran (2009) Plante (2009)

Therapist prayer Teaching spiritual

concepts Reference to Scripture Spiritual self-disclosure Spiritual confrontation Spiritual assessment Religious relaxation

or imagery Therapist and client prayer Blessing by therapist Encouragement for

forgiveness Use of religious community Client prayer Encouragement of client

confession Referral for blessing Religious journal writing Spiritual meditation Religious bibliotherapy Scripture memorization Dream interpretation

Prayer (therapist or client guided)

Teach spiritual concepts Forgiveness Reference sacred writings Meditation Spiritual self-disclosure Encourage altruism and

service Spiritual confrontation Spiritual assessment Spiritual history Spiritual relaxation and

imagery Clarify spiritual values Use Spiritual community

and spiritual programs Spiritual journaling Experiential focusing

methods Encourage solitude and

silence Use spiritual language

and metaphors Explore spiritual elements

of ereams Spiritual genogram

Prayer Meditation Meaning, purpose, and

calling in life Bibliotherapy Attending community

services and rituals Volunteerism and charity Ethical values and

behavior Forgiveness, gratitude,

and kindness Social justice Learning from spiritual

models Acceptance of self and

others (even with faults)

Being part of something larger than oneself

Appreciating the sacred- ness of life

12350-04_CH03-rev.qxd 1/11/11 11:44 AM Page 70

Co py

ri gh

t Am

er ic

an P sy

ch ol og ic al A ss oc ia ti on . No t fo r fu

rt he

r di

st ri

bu ti

on .

 

 

a client’s belief in intelligent design as a delusional disorder illustrates the inap- propriate operation of such a bias. However, the desire of a proreligious psy- chologist to promote a spiritual activity in psychotherapy when the spiritually oriented intervention is not chosen in deference to an informed client’s own beliefs, values, and preferences would also be example of such bias.

The Code’s second standard on competence has both explicit and implicit relevance to the use of spiritually oriented interventions. Standard 2.01a states, “Psychologists provide services, teach, and conduct research with populations and in areas only within the boundaries of their competence, based on their education, training, supervised experience, consultation, study, or professional experience” (APA, 2010, p. 1063). Thus, a psychologist who is not trained to appropriately use spiritually oriented interventions would be wise in avoiding their use until he or she takes steps to ensure competency and avoidance of client harm. Until such training is received, it would be appropriate to make a referral for a client who requests explicit use of spiritually oriented inter- ventions or who presents with prominent religious and spiritual issues.

Yet this does not mean that psychologists should be content simply to avoid this domain indefinitely. Standard 2.01b further states the following:

Where scientific or professional knowledge in the discipline of psychol- ogy establishes that an understanding of factors associated with . . . religion . . . is essential for effective implementation of their services or research, psychologists have or obtain the training, experience, consulta- tion, or supervision necessary to ensure the competence of their services, or they make appropriate referrals. (APA, 2010, pp. 1063–64)

The relatively ubiquitous nature of religion and spirituality renders it pro- pitious for general practitioners to obtain at least a basic competence in this domain. Standard 2.01c states that “psychologists planning to provide services, teach, or conduct research involving populations, areas, techniques, or tech- nologies new to them undertake relevant education, training, supervised experience, consultation, or study” (APA, 2010, p. 1064). Given that formal training in this domain is the exception for most psychologists as part of their prelicensure preparation for practice, seeking out continuing education, con- sultation, and a supervisor to obtain a religious and spiritual practice compe- tency would be a commendable priority for many psychologists. For those psychologists who do have a proficiency in working with religious and spiritual issues, continued consultation and professional development is advised, partic- ularly when encountering clients whose spirituality diverges from one’s prior preparation.

APA’s Division 36 (Psychology of Religion) appointed an ad hoc com- mittee that has formulated preliminary practice guidelines for clinical work with religious and spiritual issues (Hathaway, 2005). The guidelines were developed by identifying common shared recommendations offered by over

ETHICAL GUIDELINES 71

12350-04_CH03-rev.qxd 1/11/11 11:44 AM Page 71

Co py

ri gh

t Am

er ic

an P sy

ch ol og ic al A ss oc ia ti on . No t fo r fu

rt he

r di

st ri

bu ti

on .

 

 

20 exemplar professionals in the clinical psychology of religion. Guidelines related to assessment, intervention, and relevant multicultural competency issues in the domain were formulated. The subset of preliminary guidelines specifically addressing the use of religious and spiritual interventions can be found in Exhibit 3.1. The full set of preliminary guidelines can be found in Hathaway and Ripley (2009).

As part of the ad hoc committee’s ongoing work, I presented a set of five principles that undergird the preliminary practice guidelines (Hathaway, 2009). The Division 36 preliminary guideline principles are presented in Exhibit 3.2 and include awareness, respect, routine assessment focus, clinically congruent roles, and competence. These themes converge with those noted as relevant for clinicians seeking to appropriately incorporate spirituality and religion into practice by Plante (2004, 2007, 2009) and others (Gonsiorek et al., 2009). Plante has summarized the relevance of five ethical principles derived from the Code for guiding psychologists in the use of spiritually oriented interventions in practice under the acronym RRICC (i.e., Respect, Responsibility, Integrity, Competence, and Concern).

ETHICAL USE IS APPROPRIATE CLINICAL USE

Many of the spiritual and religious practices listed in Table 3.1 are fre- quently used in nonclinical contexts and for other purposes. For instance, the use of directed prayer or scripture reading could be used as an evangelistic tool designed to cultivate or instill faith. If these practices were used in an evan- gelistic context, such use would be congruent with the explicit purpose of the context. However, if they were being used for this purpose in the context of professional psychological practice, it would likely violate numerous ethical principles and standards.

Principle B of the Code states the following:

Psychologists establish relationships of trust with those with whom they work. They are aware of their professional and scientific responsibilities to society and to the specific communities in which they work. Psychologists uphold professional standards of conduct, clarify their professional roles and obligations. (APA, 2010, p. 1062)

A professional psychologist operates within a publically and legally granted fiduciary space (Reaves, 1996). As licensed professionals, psychologists agree to practice congruent with applicable scope of practice, standards, legal precedent and other structures arising from relevant regulatory codes.

To appropriately use spiritually oriented interventions, licensed psychol- ogists must do so congruent with their clinical role and regulating standards.

72 WILLIAM L. HATHAWAY

12350-04_CH03-rev.qxd 1/11/11 11:44 AM Page 72

Co py

ri gh

t Am

er ic

an P sy

ch ol og ic al A ss oc ia ti on . No t fo r fu

rt he

r di

st ri

bu ti

on .

 

 

ETHICAL GUIDELINES 73

EXHIBIT 3.1 Division 36 Preliminary Religious and Spiritual Intervention Guidelines

1. Psychologists obtain appropriate informed consent from clients before incorpo- rating religious/spiritual techniques and/or addressing religious/spiritual treat- ment goals in counseling.

2. Psychologists accurately represent to clients the nature, purposes, and known level of effectiveness for any religious/spiritual techniques or approaches they may propose using in treatment.

3. Psychologists do not use religious/spiritual treatment approaches/techniques of unknown effectiveness in lieu of other approaches/techniques with demon- strated effectiveness in treating specific disorders or clinical problems.

4. Psychologists attempt to accommodate a client’s spiritual/religious tradition in congruent and helpful ways when working with clients for whom spirituality/ religion is personally and clinically salient.

5. Religious/spiritual accommodations of standard treatment approaches/protocols are done in a manner that (a) does not compromise the effectiveness of the standard approach or produce iatrogenic effects, (b) is respectful of the client’s religious/spiritual background, (c) proceeds only with the informed consent of the client, and (d) can be competently carried out by the therapist.

6. Psychologists are mindful of contraindications for the use of spiritually/ religiously oriented treatment approaches: (a) Generally, psychologists are discouraged from using explicit religious/spiritual treatment approaches with clients presenting with psychotic disorders, substantial personality pathology, or bizarre and idiosyncratic expressions of religion/spirituality. (b) Psychologists should discontinue such approaches if iatrogenic effects become evident.

7. When psychologists use religious/spiritual techniques in treatment, such as prayer or devotional meditation, they (a) clearly explain the proposed technique to the client and obtain informed consent, (b) do so in a competent manner that is respectful of the intended religious/spiritual function of the technique in the client’s faith tradition, and (c) adopt such techniques only if they are believed to facilitate a treatment goal.

8. Psychologists appreciate the substantial role faith communities may play in the lives of their clients and consider appropriate ways to harness the resources of these communities to improve clients’ well-being.

9. Psychologists avoid conflictual dual relationships that might arise in religious/ spiritually oriented treatment or in adjunctive collaborations with faith communities.

10. Psychologists set explicitly religious/spiritual treatment goals only if (a) they are functionally relevant to the clinical concern, (b) can be competently addressed within the treatment, (c) can be appropriately pursued within the particular context and setting in which treatment is occurring, and (d) are consented to by the client.

11. Psychologists commit to a collaborative and respectful demeanor when addressing aspects of a client’s religion/spirituality the psychologist deems maladaptive or unhealthy. The preferred clinical goal in such cases is to pro- mote more adaptive forms of the client’s own faith rather than to undermine that faith.

12350-04_CH03-rev.qxd 1/11/11 11:44 AM Page 73

Co py

ri gh

t Am

er ic

an P sy

ch ol og ic al A ss oc ia ti on . No t fo r fu

rt he

r di

st ri

bu ti

on .

 

 

This can be illustrated by considering one such spiritually oriented intervention. Moriarty and others have been investigating interventions to alter God image (Moriarty & Hoffman, 2007), defined as “the complex, subjective emotional experience of God” (p. 2). A variety of strategies have been deployed in an effort to alter God image, including bibliotherapy, appropriate use of cognitive psy- chotherapy strategies, group psychotherapy, and integrative–psychodynamic approaches. Certain types of God image are more associated with depressed states and others with less depressed states. Let us suppose that God image psy- chotherapies are successful at fostering a shift toward God images that are less associated with depression. When would such an intervention be appropriate? Several factors that are highlighted in Exhibit 3.1 would impinge on determin- ing whether such an intervention is ethical. Did the client provide informed consent from the intervention? Did the information provided to the client prior to this consent “accurately represent to the client(s) the nature, purposes, and known level of effectiveness” (see Exhibit 3.1, item 2) for the God image inter- vention? Was the God image intervention used adjunctively and not in lieu of other interventions that have higher levels of demonstrated effectiveness for treating the clinical concern (unless the God image intervention has been demonstrated to be equally effective as a stand-alone treatment through ade- quate research)? Is the God image intervention being used in a manner that is respectful of the client’s religious and spiritual tradition? If the God image inter- vention is being used adjunctively with another established treatment, is it being done in a manner that does not compromise the effectiveness of the stan- dard treatment? Can the psychologist using the God image intervention do so in a competent manner? A negative answer to any of these considerations would contraindicate that use of the God image intervention.

The fourth Division 36 preliminary practice guideline principle states, “When engaged in spiritually oriented practice activities, psychologists should do so congruently with their clinical roles” (see Exhibit 3.2, item 4). So in

74 WILLIAM L. HATHAWAY

EXHIBIT 3.2 Division 36 Preliminary Practice Guideline Principles

1. Awareness: Psychologists aspire to cultivate deliberate and nuanced awareness of relevant religious and spiritual issues in practice.

2. Respect: Psychologists seek to maintain a respectful demeanor towards the religious and spiritual domain in clinical practice.

3. Routine assessment focus: Psychologists strive to routinely and intentionally assess for relevant religious and spiritual considerations in practice.

4. Clinically congruent roles: When engaged in spiritually oriented practice activities, psychologists should do so congruently with their clinical roles.

5. Competence: Psychologists seek to maintain ongoing competence in their spiritually oriented practice activities.

12350-04_CH03-rev.qxd 1/11/11 11:44 AM Page 74

Co py

ri gh

t Am

er ic

an P sy

ch ol og ic al A ss oc ia ti on . No t fo r fu

rt he

r di

st ri

bu ti

on .

 

 

addition to the clinical manner of deploying the God image intervention, it would be important to reflect on whether its use is congruent with the psychol- ogist’s clinical role.

Psychologists address many issues facing their clients that overlap with those addressed by religious caregivers (Pargament, 2007, 2009). Some have attempted to distinguish the approach taken by psychologists and religious caregivers in terms of differing epistemologies that operate in each sphere. I served on an APA Presidential Working Group (2008) that drafted a policy statement on “resolution on religious, religion-based, and/or religion-derived prejudice.” The resolution offers guidance about issues and bias and prejudice associated with religion and about what should be the proper response of psy- chology to these issues. Some members of the working group initially proposed language asserting that psychology proceeds from a qualitatively different epis- temology than religion and consequently that religious considerations were irrelevant to the clinical or scientific decisions made by psychologists in the course of their professional and disciplinary activities. But we soon realized that the issue is not that simple. Pargament (2009) noted the following:

Religion and science are not totally separable. Although we can draw con- trasts between contemporary psychology and religious traditions, these contrasts can be overdone. . . . Values, subjectivity, and judgment are an intrinsic part of science. . . . On the other hand; religion does not reject critical reflection or evaluation of the external world. (p. 391)

The final version of the APA (2008) resolution admitted some distinctive epistemic tendencies but no watertight separation between psychological and religious knowledge claims.

Professional role differences offer a more promising distinction between religious and psychological approaches to spiritual practices than does a focus on epistemic differences. The same spiritual practice may function differentially as a form of religious care or psychological care depending on the contextual, role-, and goal-related factors under which it is deployed. Religious caregivers tend to encourage religious and spiritual practices for explicitly and intrinsically religious goals (Hathaway, 2009). Such caregivers often operate with a religio- legal authority that is alien to the psychologist’s professional identity. It is com- monly the case that the traditional role of a religious professional allows him or her to speak prescriptively to care recipients about what should be the proper form of spiritual life.

In contrast, a psychologist who attempts to operate from such a position of religious authority would be engaging in role-incongruent value imposition on the client. For instance, a client may have religious convictions about the sanctity of marriage that leads him or her to persist in an unhappy marriage with a personality disordered spouse even after clinical options for improving the

ETHICAL GUIDELINES 75

12350-04_CH03-rev.qxd 1/11/11 11:44 AM Page 75

Co py

ri gh

t Am

er ic

an P sy

ch ol og ic al A ss oc ia ti on . No t fo r fu

rt he

r di

st ri

bu ti

on .

 

 

situation have been exhausted. A psychologist might be tempted to make an authoritative religious pronouncement such as “God would not want you to continue to suffer when there is no real hope for change.” I have unfortunately encountered psychologists making just these sorts of statements to their clients. Such a claim is outside of the psychologist’s proper role and competence, as noted in APA (2008) policy: “Psychologists are encouraged to recognize that it is outside the role and expertise of psychologists as psychologists to adjudi- cate religious or spiritual tenets” (p. 433).

The APA (2008) policy clarifies, however, that “psychologists can appro- priately speak to the psychological implications of religious/spiritual beliefs or practices when relevant psychological findings about those implications exist” (p. 433). So it may be appropriate for the psychologist to point out to the client that the sorts of difficulties that arise within such a relationship are likely to per- sist. To be fully compliant with the principles in the Ethics Code, the psychol- ogist would need to convey this observation in a manner that is fully respectful to the client and his or her religious beliefs.

Let us return to the issue of the propriety of a God image intervention. The foregoing discussion about professional roles suggests that it would be inappropriate to attempt such an intervention with the explicit goal of help- ing the client change their God image in a direction that the psychologist deems theologically superior simply because this is the religious judgment of the psychologist or a tenet from the psychologist’s personal religious tradition. The Division 36 preliminary practice guidelines on interventions indicate that psychologists should set such explicitly religious treatment goals only if they are “functionally relevant to the clinical concern” and “are consented to by the client” (see Exhibit 3.1, item 10). If altering a God image can be done in a man- ner that comports well with the earlier guidelines and is being done to improve clinical functioning with client informed consent, then such an explicit spiri- tually oriented intervention would be prima facie appropriate (Pargament, 2009). Does feeling this way rather than that about God allow the client to find relief from his or her depression?

Still, what if the client is primarily concerned about the impact of his or her depression on religious or spiritual functioning? The client may come into treatment asking for help in feeling differently about God, reengaging their faith community, or performing religious activities that reportedly have been adversely impacted by his or her psychological difficulties. When such altered spiritual functioning is the result of a clinical disorder, a religious caregiver may be less able to offer effective assistance. Assuming the psychologist has the rel- evant clinical competencies, self-awareness, and relational skills needed to engage such goals, when might it be appropriate for the psychologist to take on a case with such explicitly spiritual or religious treatment goals rather than refer it out to a religious caregiver?

76 WILLIAM L. HATHAWAY

12350-04_CH03-rev.qxd 1/11/11 11:44 AM Page 76

Co py

ri gh

t Am

er ic

an P sy

ch ol og ic al A ss oc ia ti on . No t fo r fu

rt he

r di

st ri

bu ti

on .

 

 

I have elsewhere argued that religious and spiritual functioning can be a significant adaptive domain that can be adversely impacted by psychological disorders (Hathaway, 2003). If the religious and spiritual domain is salient to the client, the domain is adversely affected by a psychological difficulty and the client desires treatment of the psychological difficulty to alleviate the clinically significant religious impairment, then having an explicit goal of altering the client’s religious experience or functioning may be within the bounds of the professional psychologist role. This sort of clinical focus may require a higher level of niche competence than simply using spiritually oriented interventions to assist with more standard treatment goals (Hathaway, 2008). Close collabo- ration with religious professionals may also be wise in such cases.

ACCOUNTABLE PRACTICE

The fiduciary space and oversight afforded by the regulatory bodies that govern our practice provide an important mental frame within which psychol- ogists should conceptualize the ethical use of spiritually oriented interventions. During graduate training, doctoral psychology students receive extensive men- toring and close supervision to inculcate practice virtues, skills, and competen- cies. Once a psychologist becomes licensed for independent practice, most of his or her practice activities will be free of this direct external scrutiny. Yet psy- chologists are always subject to potential review by state licensing boards, cre- dentialing bodies, courts, or other controlling authorities (Reaves, 1996). Even when such review does not in fact occur, well-trained psychologists would habitually practice with a communal consciousness that is always mindful of the potential audience that is attending to one’s performance. Thus, a psychologist should always be prepared to offer cogent and appropriate rationales for any approach, treatment decision, action (or nonaction) in one’s practice:

Any mental health professional should practice in a professionally self- aware manner, with an internalized community of wise peers. When a pro- fessional treats a client, the client should be treated not merely with his or her psychotherapist’s innate skill and the body of objective scientific knowledge but also from the implicit communal wisdom and calibration that emerges from myriad dialogues about analogous cases in supervision sessions, published cases, and professional peer interactions. When such internalized voices are not sufficient, ethical professionals invite still other professionals into the conversation through ongoing consultation. (Hathaway, 2009, p. 111)

A prudent goal of maintaining ethical awareness in practice might be to avoid malpractice liability from incompetent practice or to prevent commis- sion of an ethical violation that could result in loss of licensure and livelihood.

ETHICAL GUIDELINES 77

12350-04_CH03-rev.qxd 1/11/11 11:44 AM Page 77

Co py

ri gh

t Am

er ic

an P sy

ch ol og ic al A ss oc ia ti on . No t fo r fu

rt he

r di

st ri

bu ti

on .

 

 

The more overriding concern for such awareness should be to maximize our benefit to our clients and minimize the chance of their harm.

Given that spiritually oriented practice competencies may not be widespread in professional psychology, it might be tempting to continue to avoid this area as risk management strategy. But such a strategy would likely only lead to continued neglect of this important domain that may be of cen- tral salience for clients in dereliction of our broader ethical aspirations as profession (Hathaway et al., 2004).

CONCLUSION

There are now many resources, such as this volume, that provide training at various competency levels in spiritually oriented treatments. At least since the 1990s the APA has offered continuing education workshops by leading clinical psychologists of religion in this practice area. Such workshops are also frequently offered at state psychological associations and in other venues. The APA has several psychotherapy training videos focusing on spiritual issues. Becoming self-aware of the needs for practice competency in this domain is the first step toward ethical practice with spiritually oriented interventions. The next step is to make appropriate referrals based on client presentation and pref- erence to others who have achieved the competency until one acquires the skill or can practice under sufficient consultation to ensure competent practice. There is an entire division of the APA (Division 36) dedicated to the psychol- ogy of religion that has many niche practitioners in this domain among its members.

Fisher (2003) noted the following:

The Ethics Code is not a formula for solving . . . ethical challenges. The Ethics Code provides psychologists with a set of aspirations and broad gen- eral rules of conduct that must be interpreted and applied as a function of the unique scientific and professional roles and relationships in which they are embedded. (p. 237)

For many of our clients, this relational embedding will include religious and spiritual forms of life. In regard to appropriate professional roles in this practice niche, a thought experiment may provide such guidance: Imagine giving an account of practice habits with religious and spiritual issues and in using spiri- tually oriented interventions before a licensing board or some other body. Can the practitioner think through a clear rationale for how and why a specific spir- itually oriented intervention was used with a particular client? Were all of the relevant ethical considerations well managed? What was the outcome? The ability to offer such a cogent rationale may be a helpful clue about one’s readi-

78 WILLIAM L. HATHAWAY

12350-04_CH03-rev.qxd 1/11/11 11:44 AM Page 78

Co py

ri gh

t Am

er ic

an P sy

ch ol og ic al A ss oc ia ti on . No t fo r fu

rt he

r di

st ri

bu ti

on .

 

 

ness to pursue this practice niche as well as a guide for whether such an inter- vention would be ethical.

REFERENCES

American Psychiatric Association. (1989). Guidelines regarding possible conflict between psychiatrists’ religious commitments and psychiatric practice. Washington, DC: Author.

American Psychological Association. (2008). Resolution on religious, religion-based, and/or religion-derived prejudice. American Psychologist, 63, 431–434.

American Psychological Association. (2010). Ethical principles of psychologists and code of conduct (2002, Amended June 1, 2010). Retrieved from http://www.apa. org/ethics/code/index.aspx

APA Presidential Working Group. (2008). Resolution on religious, religion-based, and/or religion-derived prejudice. American Psychologist, 63, 431–434.

Bergin, A. E., & Jensen, J. P. (1990). Religiosity and psychotherapists: A national sur- vey. Psychotherapy: Theory, Research, & Practice, 27, 3–7. doi:10.1037/0033-3204. 27.1.3

Brawer, P. A., Handal, P. J., Fabricatore, A. N., Roberts, R., & Wajda-Johnston, V. A. (2002). Training and education in religion/spirituality within APA-accredited clinical psychology programs. Professional Psychology, Research and Practice, 33, 203–206. doi:10.1037/0735-7028.33.2.203

Clarke, I. (Ed.). (2001). Psychosis and spirituality: Exploring the new frontier. Philadelphia, PA: Whurr.

Cummings, N., O’Donohue, W., & Cummings, J. (Eds.). (2009). Psychology’s war on religion. Phoenix, AZ: Zeig, Tucker & Theisen.

Dawkins, R. (2006). The God delusion. Boston, MA: Houghton Mifflin.

Delaney, H. D., Miller, W. R., & Bisono, A. M. (2007). Religiosity and spirituality among psychologists: A Survey of clinician members of the American Psycho- logical Association. Professional Psychology, Research and Practice, 38, 538–546. doi:10.1037/0735-7028.38.5.538

Fisher, C. B. (2003). Decoding the ethics code: A Practical guide for psychologists. Thousand Oaks, CA: Sage.

Gartner, J. D. (1986). Antireligious prejudice in admissions to doctoral programs in clinical psychology. Professional Psychology, Research and Practice, 17, 473–475. doi:10.1037/0735-7028.17.5.473

Gonsiorek, J. C., Richards, P. S., Pargament, K. I., & McMinn, M. R. (2009). Ethical challenges and opportunities at the edge: Incorporating spirituality and religion into psychotherapy. Professional Psychology, Research and Practice, 40, 385–395. doi:10.1037/a0016488

Hart, D. B. (2009). Atheist delusions: The Christian revolution and its fashionable enemies. New Haven, CT: Yale University.

ETHICAL GUIDELINES 79

12350-04_CH03-rev.qxd 1/11/11 11:44 AM Page 79

Co py

ri gh

t Am

er ic

an P sy

ch ol og ic al A ss oc ia ti on . No t fo r fu

rt he

r di

st ri

bu ti

on .

 

 

Hathaway, W. L. (2003). Clinically significant religious impairment. Mental Health, Religion & Culture, 6, 113–129.

Hathaway, W. L. (2005, August). Proposed practice guidelines for religious/spiritual issues. Paper presented at the meeting of the American Psychological Association, Washington, DC.

Hathaway, W. L. (2008). Clinical practice with religious/spiritual issues: Niche, profi- ciency or specialty. Journal of Psychology and Theology, 36, 16–25.

Hathaway, W. L. (2009, August). Proposed practice guidelines for clinical work with religious and spiritual issues. Paper presented at the meeting of the American Psychological Association, Toronto, ON.

Hathaway, W. L. (2009). Clinical use of explicit religious approaches: Christian role integration issues. Journal of Psychology and Christianity, 28, 105–112.

Hathaway, W. L., & Ripley, J. S. (2009). Ethical concerns around spirituality and reli- gion in clinical practice. In J. D. Aten & M. M. Leach (Eds.), Spirituality and the therapeutic process: A comprehensive resource from intake to termination (pp. 25–52). Washington, DC: American Psychological Association. doi:10.1037/11853-002

Hathaway, W. L., Scott, S. Y., & Garver, S. A. (2004). Assessing religious/spiritual functioning: A neglected domain in clinical practice? Professional Psychology, Research and Practice, 35, 97–104. doi:10.1037/0735-7028.35.1.97

James, W. (1997). The varieties of religious experience. New York, NY: Simon & Schuster. (Original work published 1902)

Knapp, S. J., & VandeCreek, L. D. (2006). Practical ethics for psychologists: A positive approach. Washington, DC: American Psychological Association. doi:10.1037/ 11331-000

Moriarty, G. L., & Hoffman, L. (Eds.). (2007). God image handbook: For spiritual coun- seling & psychotherapy: Research, theory, and practice. Binghampton, NY: Haworth.

Pargament, K. I. (2007). Spiritually integrated psychotherapy: Understanding and address- ing the sacred. New York, NY: Guilford Press.

Pargament, K. I. (2009). The psychospiritual character of psychotherapy and the ethical complexities that follow. Professional Psychology, Research and Practice, 40, 391–393.

Plante, T. G. (2004). Do the right thing: Living ethically in an unethical world. Oakland, CA: New Harbinger.

Plante, T. G. (2007). Integrating spirituality and psychotherapy: Ethical issues and principles to consider. Journal of Clinical Psychology, 63, 891–902. doi:10.1002/ jclp.20383

Plante, T. G. (2009). Spiritual practices in psychotherapy: Thirteen tools for enhanc- ing psychological health. Washington, DC: American Psychological Association. doi:10.1037/11872-000

Reaves, R. P. (1996). Enforcement of codes of conduct by regulatory boards and pro- fessional associations. In L. J. Bass, S. T. DeMers, J. R. P. Ogloff, C. Peterson, J. L.

80 WILLIAM L. HATHAWAY

12350-04_CH03-rev.qxd 1/11/11 11:44 AM Page 80

Co py

ri gh

t Am

er ic

an P sy

ch ol og ic al A ss oc ia ti on . No t fo r fu

rt he

r di

st ri

bu ti

on .

 

 

Pettifor, R. P. Reaves . . . R. M. Tipton (Eds.), Professional conduct and discipline in psychology (pp. 101–108). Washington, DC: American Psychological Association.

Richards, P. S., & Bergin, A. E. (2005). A spiritual strategy for counseling and psychother- apy (2nd ed.). Washington, DC: American Psychological Association. doi:10. 1037/11214-000

Russell, S. R., & Yarhouse, M. A. (2006). Religion/spirituality within APA-accredited psychology predoctoral internships. Professional Psychology, Research and Practice, 37, 430–436. doi:10.1037/0735-7028.37.4.430

Schlosser, L. Z., & Safran, D. A. (2009). Implementing treatments that incorporate client spirituality. In J. D. Aten & M. M. Leach (Eds.), Spirituality and the thera- peutic process: A comprehensive resource from intake to termination (pp. 193–216). Washington, DC: American Psychological Association. doi:10.1037/11853-009

Shafranske, E. P. (2000). Religious involvement and professional practices of psychia- trists and other mental health professionals. Psychiatric Annals, 30, 525–532.

Sperry, L., & Shafranske, E. P. (Eds.). (2005). Spiritually oriented psychotherapy. Washington, DC: American Psychological Association.

Tan, S. Y., & Johnson, W. B. (2005). Spiritually oriented cognitive behavioral therapy. In L. Sperry & E. Shafranske (Eds.), Spiritually oriented psychotherapy (pp. 77–103). Washington, DC: American Psychological Association. doi:10.1037/10886-004

Worthington, E. L., Jr., & Sandage, S. J. (2002). Religion and spirituality. In J. C. Norcross (Ed.), Psychotherapy relationships that work: Therapist contributions and responsiveness to patients (pp. 383–399). New York, NY: Oxford University Press.

ETHICAL GUIDELINES 81

12350-04_CH03-rev.qxd 1/11/11 11:44 AM Page 81

Co py

ri gh

t Am

er ic

an P sy

ch ol og ic al A ss oc ia ti on . No t fo r fu

rt he

r di

st ri

bu ti

on .

Psychopharmacology-A16 Final Exam

FINAL EXAM

PSY 87700:

Psychopharmacology

 

Highlight in bold the correct answer to each question. Save and upload the entire exam.

 

1) Which of the following medical conditions is not typically associated with depression?

a. AIDS

b. Diabetes

cGout

d. Cushing’s Disease

 

2) A symptom which is found in clinical depression but not in grief is:

a. Intense sadness

b. Sleep disturbance

c. Crying

d. Loss of self-esteem

 

3) Uncomplicated bereavement is best reduced by

a. Antidepressant medication

b. Psychoanalytic psychotherapy

c. Mourning and the passage of time

d. Antianxiety medication

 

4) Some symptoms are common to all depressions, whereas others indicate a biochemical dysfunction which may be responsive to antidepressants. Which one of the following symptoms is not uniquely associated with biochemical depression?

a. Appetite Disturbance

b. Sleep disturbance

c. Reduced Libido

d. Suicidal Ideas

 

5) A patient on an antidepressant complains of dry mouth, blurred vision, constipation, and trouble urinating. These symptoms are most likely:

a. Symptoms of depression

b. Anticholinergic side effects

c. Hypochondriacal complaints

d. Due to inactivity

 

6) When under treatment with an MAO inhibitor, one should avoid:

a. Red meat

b. Pea Soup

c. Aged cheese

d. Jello

 

7) A patient is taking Prozac for depression and is not responding. His physician asks you if you think augmentation with Parnate would be a good idea. You should tell the physician:

a. This is a sensible treatment approach

b. No — this combination could be fatal

c. The combination is harmless, but probably won’t reduce depression

d. Combining these two medications will probably increase depression

 

8) A depressed patient has a history of sexual dysfunction, and fears that anti-depressants will make his problem worse. You would prescribe:

a. buproprion

b. desipramine

c. imipramine

d. doxepin

 

9) Which one of the following is not likely to be a cause of depression?

a. Antihypertensives

b. Antibiotics

c. Antiparkinson drugs

d. Birth control pills

 

10) A depressed patient has been taking adequate doses of an SSRI for ten days and complains to you that she has felt no improvement. You would advise this patient:

a. That she needs to stay on the medication longer before she feels Improvement

b. To ask her physician to try a new medication

c. To discontinue her medication

d. To take St. John’s Wort along with her medication

 

11) Medications which should be initially considered for Generalized Anxiety Disorder include:

a. benzodiazepines and SSRIs

b. buspirone and lithium

c. buspirone, Effexor, and SSRIs

d. Valium, Librium, and Xanax

 

12) Classes of medications typically used for treating anxiety disorders include:

a. SSRIs

b. Benzodiazepines

c. MAO inhibitors

d. All of the above

 

13) Stress induced insomnia is typically treated with short-term:

a. SSRIs

b. Benzodiazepines

c. MAO inhibitors

d. Neuroleptics

 

14) Beta blockers control which anxiety-related symptoms?

a. Sense of dread

b. Anxiety-evoking cognitions

c. Rapid heartbeat

d. Concentration difficulties

 

15) The two phases of treating panic disorder are:

a. Reducing panic intensity with medication or relaxation; exposure to feared situations

b. Benzodiazepine treatment; SSRI treatment

c. Immobilization; Reactivation

d. Antidepressant medication; antianxiety medication

 

16) Medication used for social phobia is usually:

a. MAOIs or beta blockers

b. Benzodiazepines

c. Haldol

d. Amphetamines in low doses

 

 

17) Which of the following is probably not a biochemical cause of anxiety?

a. High levels of caffeine consumption

b. Heavy use of aspirin

c. Alcohol withdrawal

d. Steroid use

 

18) A patient has a history of frequently engaging in risky and self-defeating behaviors which produce anxiety. The treatment of choice would be:

a. Long-term benzodiazepines

b. Xanax

c. Psychotherapy

d. PRN benzodiazepines

 

19) Serotonin levels are increased by:

a. SSRIs

b. Benzodiazepines

c. Antipsychotic drugs

d. Desipramine

 

20) Medical causes of anxiety symptoms include:

a. Hyperthyroidism

b. Adrenal tumor

c. Hypoglycemia

d. Any of the above

 

21) A schizophrenic patient taking antipsychotic medication is extremely restless, unable to sit still. This probably is:

a. An attention deficit disorder

b. A medication side effect

c. A reaction to delusional thoughts

d. A primary symptom of psychosis

 

22) A patient has been on antipsychotic medication for the past ten years. He shows odd lip, tongue, and extremity movement. This is probably:

a. Tardive dyskinesia

b. A response to hallucinations

c. A compulsive ritual

d. Seizure activity

 

23) A patient on antipsychotic medication presents with her head twisted to one side. The likely treatment would be:

a. Hypnosis and relaxation

b. An IM anticholinergic agent

c. Antispasmodic drugs

d. Discontinuing all medication

 

24) If a sedating effect is desired when treating a psychotic patient with antipsychotic drugs, one would most likely use:

a. A high potency antipsychotic

b. A low potency antipsychotic

c. Augmentation with a benzodiazepine

d. Q.i.d. dosing

 

25) An advantage of newer antipsychotic drugs such as clozapine over traditional antipsychotics is:

a. Reduced risk of agranulocytosis

b. Better control of positive psychotic symptoms

c. Better control of negative psychotic symptoms

d. Antidepressant effects

 

 

 

26) Which of the following antipsychotic medications is least likely to cause weight gain?

a. Abilify

b. Haldol

c. Risperdal

d. Stelazine

 

27) Dementias can be differentiated from schizophrenia in the following way:

a. In schizophrenia, orientation and short-term memory is relatively intact

b. In dementias, orientation and short-term memory is relatively intact

c. Early morning awakening is more likely to be seen in schizophrenic patients

d. Late evening confusion is more likely to be seen in schizophrenic patients

 

28) A dose of 1000 mg. of Thorazine has the equivalent antipsychotic effect as what dose of Zyprexa?

a. 3 mg.

b. 20 mg.

c. 500 mg

d. 1000 mg

 

29) The first schizophrenic symptoms to respond to antipsychotic medication are:

a. Hallucinations

b. Delusions

c. Agitation

d. Poor reality testing

 

 

30) In treatment of a psychotic depression:

a. Both an antipsychotic and antidepressant might be used

b. The prescriber would always choose between an antipsychotic and antidepressant; these two types of medication would not be combined

c. The treatment would always be only an antidepressant

d. The treatment would always be only an antipsychotic

 

31) In the treatment of Bipolar Disorder, a combination of antipsychotic medication and lithium would most likely be used when the patient:

a. presents in a depressed phase

b. is suicidal

c. presents in a manic episode

d. is Bipolar ll

 

32) In the treatment of Bipolar Disorder, a combination of bupropion and lithium would most likely be used when the patient:

a. presents in a depressed phase

b. presents in a manic episode

c. is Bipolar I

d. these two medications should never be combined

 

33) A patient on lithium presents with lethargy, nausea, slurred speech, and complains that she hears a ringing in her ears. You would suspect:

a. That she is experiencing a psychotic depression

b. Lithium toxicity

c. Alcohol abuse

d. That she has not been taking her lithium

 

34) Blood tests are frequently done with bipolar patients before initiating lithium treatment. The reason for this is:

a. Determine if a lithium deficiency is present

b. To determine kidney function is adequate

c. Differentiate bipolar illness from major depression

d. Differentiate Bipolar I from Bipolar II

 

35) Bipolar I disorder, compared to Bipolar II has:

a. Manic episodes which are more pronounced

b. Manic episodes which are less pronounced

c. No depressive episodes

d. Has depressive and manic episodes which are more pronounced

 

36) Rapid cycling bipolar patients are often treated with:

a. Anticonvulsants

b. MAOI medication

c. Benzodiazepines

d. Stimulants

 

37) A man is brought in by his wife, who notes that he has not slept in two days. He presents with rapid, pressured speech, racing thoughts, grandiose plans, and euphoria. The least likely diagnosis is:

a. Methamphetamine intoxication

b. Manic phase of bipolar illness

c. Encephalitis

d. Anxiety disorder

 

38) A typical therapeutic blood level of lithium is:

a. 0.5 mEq/L

b. 1.2 mEq/L

c. 3.0 mEq/L

d. 600 mEq/L

 

39) When symptoms of depression and mania occur simultaneously, this is called:

a. Dysphoric mania

b. Delirium

c. Ataxia

d. Cognitive dissonance

 

40) The medications other than lithium which are used as mood stabilizers are in which class of drugs?

a. Antidepressant

b. Antiviral

c. Anticonvulsant

d. Neuroleptic

 

41) Medications used to treat obsessive-compulsive disorders are:

a. Antidepressants which increase serotonin availability

b. Antidepressants which increase norepinephrine availability

c. Anxiolytics

d. Neuroleptics

 

42) The classic medical treatment for ADD is:

a. Stimulants

b. ECT

c. Sedatives

d. Barbiturates

 

43) Pharmacokinetics refers to:

a. Absorption, Distribution, Biotransformation, Excretion

b. Regulation of neurotransmitters

c. Manufacture and purification of medications

d. The physical properties of a medication (pill, capsule, liquid, etc.)

 

44) The first pass effect can be circumvented by administering a drug:

a. Orally

b. t.i.d.

c. IV

d. At bedtime

 

45) A prodrug is a

a. Drug which activates another drug

b. A relatively inactive compound which is metabolized into a more active form

c. Vitamin

d. Certain drugs taken by professional athletes to improve performance

 

46) Steady state refers to

a. Stabilization of psychiatric symptoms

b. A form of a drug which can be stored for long periods without a loss of potency

c. The point at which administration and elimination of a drug are in equilibrium

d. A drug isotope which is not radioactive

 

 

 

47) With regular dosing of a drug with a half-life of 12 hours, steady state would be reached in about:

a. 12 hrs.

b. 24 hrs.

c. 2 1/2 days

d. 30 days

 

48) Drug X produces a therapeutic response in 50% of patients at a dose of 50 mg. It is lethal in 50% of patients at a dose of 400 mg. The Therapeutic Index for this drug is:

a. 0.125

b. 4

c 8

d 16

 

49) The insulation covering nerve fibers is the

a. Myelin sheath

b. Neuronal channel

c. Calcium channel

d. Lipid layer

 

50) In a neuron at rest

a. The electrical potential of the immediate interior and exterior of the neuronal membrane is equal

b. The electrical potential of the immediate interior of the neuronal membrane is negative relative to the exterior

c. The electrical potential of the immediate interior of the neuronal membrane is positive relative to the exterior

d. There is no electrical potential

 

 

51) The body’s main inhibitory neurotransmitter/neuromodulator is:

 

a. glutamate

b. L-dopa

c. GABA

d. tryptophan

 

52) Which of the following are found in the soma?

 

a. Terminal button and axon.

b. The dendrites and the terminal buttons

c. The axon and the dendrites

d. The nucleus and the mitochondria.

 

53) Which of these is the correct progression of events?

 

a. Action potential, exocytosis, re-uptake.

b. Exocytosis, action potential, re-uptake.

c. Re-uptake, action potential, exocytosis.

d. None of these.

 

54) The physical space between neurons is called:

 

a. the intracellular space

b. the extracellular space

c. the cytoplasm

d. the synapse

 

 

 

 

 

 

55) GABA is the body’s main inhibitory neurotransmitter substance, and ___________ is the excitatory substance.

 

a. tryptophan

b. calcium phosphate

c. glutamate

d. none of these

 

56) A non-medical psychotherapist says that she has focused her training exclusively on psychotherapeutic intervention. She says that she has intentionally not learned about psychopharmacology so that she can avoid liability for medication issues. You would respond to her by suggesting:

 

a. That her stance is reasonable

b. That this is not an effective strategy for avoiding liability

c. That her stance is only reasonable if she discloses to her clients that she has no knowledge of psychopharmacology

d. That she is on safe ground if she says “talk to your doctor” whenever a medication issue is raised or noticed

 

57) Excessive amounts of ________ may be implicated in psychosis.

 

a. serotonin

b. norepinephrine

c. GABA

d. dopamine

 

58) When the client focuses on medication issues throughout the therapeutic session, to the exclusion of focus on inner thoughts and feelings, this may be seen as a form of therapeutic:

 

a. Resistance

b. Insight

c. Hysteria

d. Sociopathy

 

59) Dopamine, norepinephrine, and serotonin are all types of:

 

a. Neurotransmitters

b. Medications

c. Neurons

d. Hormones

 

60) The terminal bouton in a neuron is a:

 

a. Long fiber

b. Post-synaptic structure

c. Pre-synaptic structure

d. Neurochemical

 

61) Patients with an obsessive-compulsive personality style might be expected to respond to a prescription for psychotropic medication in the following way:

 

a. Insist on detailed information about the medication and its side effects

b. Discuss even minor side effects in great detail

c. Bring up issues around loss-of-control due to medication effects or side effects

d. Any of the above

 

62) Tiny containers for neurotransmitters are found in the terminal bouton and protect the neurotransmitters from enzymes. These containers are called:

 

a. Corpuscles

b. Micro-boutons

c. Axons

d. Vesicles

 

63) Second Messengers are:

 

a. Extracellular signaling substances

b. Intracellular signaling molecules that trigger physiological changes

c. Medications that activate neural transmission

d. A way to transmit a prescription to a pharmacy

 

 

64) Heart rate, breathing and blood pressure are regulated by the:

 

a. Limbic system

b. Cortex

c. Basal ganglia

d. Brain stem

 

65) The part of the brain associated with procedural learning, such as riding a bicycle, is the:

 

a. Limbic system

b. Cerebellum

c. Amygdala

d. Brain stem

 

66) Digestion, decreased heart rate, and muscle relaxation is associated with the:

 

a. Sympathetic nervous system

b. Somatic nervous system

c. Spinal cord

d. Parasympathetic nervous system

 

67) The study of how a patient’s body absorbs, distributes, modifies and excretes a drug is known as:

 

a. Pharmacodynamics

b. Physiological processing

c. Metabolism

d. Pharmacokinetics

 

68) Which of the following diseases/disorders has been known to cause or worsen depression?

 

a. Alzheimer’s

b. Congestive heart failure

c. Fibromyalgia

d. All of the above

 

69) A person’s body using enzymes to chemically convert medication to other substances is an example of:

 

a. Excretion

b. Absorption

c. Reduction

d. Metabolism

 

70) The kidney is an organ of:

 

a. Metabolism

b. Absorption

c. Digestion

d. Excretion

 

71) Which of the following describes a drug interaction?

 

a. One drug increases the effect of another

b. One drug cancels the effect of another

c. Two drugs, taken at therapeutic doses, become lethal when combined

d. Any of the above

 

72) A medication that binds to a serotonin receptor and imitates the effect of serotonin is a(n):

 

a. Agnostic

b. Antagonist

c. Pathomimetic drug

d. Agonist

 

73) You are working with a patient with OCD. Which of the following medications is not considered an anti-obsessive?

 

a. Prozac

b. Anafranil

c. Wellbutrin

d. Luvox

 

74) “Serotonin syndrome” includes all of the following symptoms except:

 

a. slow heart rate

b. fever

c. muscle rigidity

d. hypertension

 

75) A typical daily dose of Prozac or Paxil is:

 

a. 2-4 mg.

b. 20-40 mg.

c. 75-100 mg.

d. 200-300 mg.

 

76) Which of the following benzodiazepines is often used for its mood-stabilizing properties?

 

a. Xanax

b. Ativan

c. Klonopin

d. All of the above

 

77) A patient presents with high levels of anxiety of recent origin. Your first diagnostic step would be to:

 

a. Develop a thorough understanding of his personality structure

b. Distinguish between various anxiety diagnoses: panic disorder, phobia, GAD, etc.

c. Rule out non-psychiatric medical causes

d. Interview significant family members to develop a full picture of the client

 

78) Typical two-drug combinations for Bipolar Illness include:

 

a. An SSRI plus an MAOI

b. An SSRI plus a neuroleptic

c. Lithium plus an anticonvulsant

d. All of the above

 

79) Which of the following is approved primarily for Generalized Anxiety Disorder?

 

a. Buspar

b. Ativan

c. Xanax

d. None of these

 

80) In treating Social Anxiety Disorder, the best results are achieved with:

 

a. TCAs

b. Atypical Antipsychotics

c. SSRIs or MAOIs

d. None of these

 

81) For spot treating specific social phobias like public speaking, one may consider:

 

a. Buspar

b. Klonopin

c. Prozac

d. Inderal

 

82) All of the following EXCEPT ______ are typically used to treat PTSD symptoms.

 

a. SSRIs

b. Effexor

c. Mood stabilizers

d. Stimulants

 

83) A patient comes to you, reporting that he started seeing “ghosts” a few days ago. He realizes the experience is abnormal, he has never had odd experiences like this before, and has not had any recent stresses that he can think of. You would:

 

a. Work on improving his insight

b. Suspect schizophrenia and refer him for antipsychotic medication

c. Suggest he ignore the ghosts and get on with his life

d. Suspect a non-psychiatric medical problem and refer him for medical evaluation

 

84) All of the following are atypical antipsychotics except:

 

a. Zyprexa

b. Fanapt

c. Abilify

d. Navane

 

85) Which of the following is a vegetative depressive symptom?

 

a. Feeling sad

b. Low self-esteem

c. Obsessional thinking

d. Weight loss and anorexia

 

 

 

86) How can you distinguish Major Depression with Psychotic Features from Schizoaffective Disorder?

 

a. The distinction is based upon clinical judgment

b. With Schizoaffective Disorder, psychosis occurs only during a depressive episode

c. Schizoaffective Disorder always includes manic episodes

d. With Major Depression, psychosis occurs only during a depressive episode

 

87) Irregularities of which neurotransmitter(s) has been implicated in ADHD?

 

a. Serotonin

b. Norepinephrine

c. Dopamine

d. both b and c

 

88) Which of the following is not a stimulant per se, but rather an antidepressant with stimulant qualities?

 

a. Cylert

b. Ritalin

c. Stratera

d. Adderal

 

89) Monoamines are:

 

a. Enzymes that break down neurotransmitters

b. The neurotransmitters dopamine, noradrenaline and serotonin

c. A class of amino acids

d. A byproduct of polyamines

 

90) To be diagnosed with Bipolar I disorder, a client needs to have had:

 

a. Two or more manic or mixed episodes

b. At least one manic or mixed episode and at least one depressive episode

c. Two or more manic episodes and at least one depressive episode

d. At least one manic or mixed episode

 

91) The anticholinergic side effects are more common with which type of antidepressants?

 

a. tricyclics

b. SSRIs

c. MAOIs

d. none of these

 

92) When ruling out non-psychiatric medical causes for mania, one would be most likely to review which lab findings?

 

a. Lipids

b. CBC

c. Thyroid function tests

d. PSA

 

93) Bipolar illness often starts with specific stressful events, but later manic episodes are more likely to occur spontaneously. This is best predicted by:

 

a. Kindling theory

b. Cognitive-behavioral theory

c. Operant conditioning

d. Psychoanalytic theory

 

94) Which of the following drugs is thought to be the “gold standard” for treating Bipolar Illness?

 

a. Lamictal

b. Topamax

c. Lithium

d. Tegretol

 

95) When prescribing benzodiazepines, one of the most important concerns is:

 

a. Addictive potential

b. Increased seizure potential

c. Weight gain

d. Triggering a manic episode

 

96) S-adenosyl-methionine or SAM-e is available without a prescription. Research has shown it to be:

 

a. A stimulant, similar to caffeine

b. An effective antidepressant

c. No more effective than placebo

d. A useful dieting aide

 

97) Continuous writhing movements of the extremities and involuntary movements of the mouth and tongue are symptoms of:

 

a. Acute dystonia

b. Psychosis

c. Tardive dyskinesia

d. Seizure disorder

 

98) The two broad areas of dysfunction found in ADHD are:

 

a. Inattention and social deficits

b. Hyperactivity and inattention

c. Hyperactivity and social deficits

d. Inattention and antisocial behavior

 

99) Lithium has a narrow therapeutic index. This means that:

 

a. The therapeutic dose is close to the toxic dose

b. Large doses are required for effectiveness

c. Similar doses are required for patients with different characteristics

d. The ineffective dose is close to the effective dose

 

100) The serum concentration level of Drug Z is 100 mcg/ml at noon and 25 mcg/ml at 4 pm. The drug’s half-life is:

 

a. One hour

b. Two hours

c. Four hours

d. Eight hours

Social work evaluation: Enhancing what we do.

Assignment: Drafting a Process Evaluation

The steps for process evaluation outlined by Bliss and Emshoff (2002) may seem very similar to those for conducting other types of evaluation that you have learned about in this course; in fact, it is the purpose and timing of a process evaluation that most distinguish it from other types of evaluation. A process evaluation is conducted during the implementation of the program to evaluate whether the program has been implemented as intended and how the delivery of a program can be improved. A process evaluation can also be useful in supporting an outcome evaluation by helping to determine the reason behind program outcomes.

There are several reasons for conducting process evaluation throughout the implementation of a program. Chief among them is to compare the program that is being delivered to the original program plan, in order to identify gaps and make improvements. Therefore, documentation from the planning stage may prove useful when planning a process evaluation.

For this Assignment, you either build on the work that you completed in Weeks 6, 7, and 8 related to a support group for caregivers, or on your knowledge about a program with which you are familiar. Review the resource “Workbook for Designing a Process Evaluation”.

Submit a 4-page plan for a process evaluation. Include the following minimal information:

· A description of the key program elements

· A description of the strategies that the program uses to produce change

· A description of the needs of the target population

· An explanation of why a process evaluation is important for the program

· A plan for building relationships with the staff and management

· Broad questions to be answered by the process evaluation

· Specific questions to be answered by the process evaluation

· A plan for gathering and analyzing the information

References (use 3 or more)

Dudley, J. R. (2014). Social work evaluation: Enhancing what we do.(2nd ed.) Chicago, IL: Lyceum Books.

Chapter 8, “Improving How Programs and Practice Work” (pp. 167–207)

Document:Bliss, M. J., & Emshoff, J. G. (2002). Workbook for designing a process evaluation.Retrieved from http://beta.roadsafetyevaluation.com/evaluationguides/info/workbook-for-designing-a-process-evaluation.pdf (PDF)

Georgia Department of Human Resources, Division of Public Health.

Workbook

for Designing a Process Evaluation

 

Produced for the

Georgia Department of Human Resources

Division of Public Health

By

Melanie J. Bliss, M.A. James G. Emshoff, Ph.D.

Department of Psychology Georgia State University

 

July 2002

 

 

Evaluation Expert Session July 16, 2002 Page 1

 

What is process evaluation?

Process evaluation uses empirical data to assess the delivery of programs. In contrast to outcome evaluation, which assess the impact of the program, process evaluation verifies what the program is and whether it is being implemented as designed. Thus, process evaluation asks “what,” and outcome evaluation asks, “so what?”

When conducting a process evaluation, keep in mind these three questions:

1. What is the program intended to be? 2. What is delivered, in reality? 3. Where are the gaps between program design and delivery?

This workbook will serve as a guide for designing your own process evaluation for a program of your choosing. There are many steps involved in the implementation of a process evaluation, and this workbook will attempt to direct you through some of the main stages. It will be helpful to think of a delivery service program that you can use as your example as you complete these activities. Why is process evaluation important? 1. To determine the extent to which the program is being

implemented according to plan 2. To assess and document the degree of fidelity and variability in

program implementation, expected or unexpected, planned or unplanned

3. To compare multiple sites with respect to fidelity 4. To provide validity for the relationship between the intervention

and the outcomes 5. To provide information on what components of the intervention

are responsible for outcomes 6. To understand the relationship between program context (i.e.,

setting characteristics) and program processes (i.e., levels of implementation).

7. To provide managers feedback on the quality of implementation 8. To refine delivery components 9. To provide program accountability to sponsors, the public, clients,

and funders 10. To improve the quality of the program, as the act of evaluating is

an intervention.

 

 

Evaluation Expert Session July 16, 2002 Page 2

Stages of Process Evaluation Page Number

1. Form Collaborative Relationships 3 2. Determine Program Components 4 3. Develop Logic Model* 4. Determine Evaluation Questions 6 5. Determine Methodology 11 6. Consider a Management Information System 25 7. Implement Data Collection and Analysis 28 8. Write Report**

Also included in this workbook:

a. Logic Model Template 30 b. Pitfalls to avoid 30 c. References 31

 

Evaluation can be an exciting, challenging, and fun experience

Enjoy!

 

* Previously covered in Evaluation Planning Workshops. ** Will not be covered in this expert session. Please refer to the Evaluation Framework

and Evaluation Module of FHB Best Practice Manual for more details.

 

 

 

 

 

Evaluation Expert Session July 16, 2002 Page 3

Forming collaborative relationships

A strong, collaborative relationship with program delivery staff and management will likely result in the following:

Feedback regarding evaluation design and implementation Ease in conducting the evaluation due to increased cooperation Participation in interviews, panel discussion, meetings, etc. Increased utilization of findings

Seek to establish a mutually respectful relationship characterized by trust, commitment, and flexibility.

Key points in establishing a collaborative relationship:

Start early. Introduce yourself and the evaluation team to as many delivery staff and management personnel as early as possible.

Emphasize that THEY are the experts, and you will be utilizing their knowledge and

information to inform your evaluation development and implementation.

Be respectful of their time both in-person and on the telephone. Set up meeting places that are geographically accessible to all parties involved in the evaluation process.

Remain aware that, even if they have requested the evaluation, it may often appear as

an intrusion upon their daily activities. Attempt to be as unobtrusive as possible and request their feedback regarding appropriate times for on-site data collection.

Involve key policy makers, managers, and staff in a series of meetings throughout the

evaluation process. The evaluation should be driven by the questions that are of greatest interest to the stakeholders. Set agendas for meetings and provide an overview of the goals of the meeting before beginning. Obtain their feedback and provide them with updates regarding the evaluation process. You may wish to obtained structured feedback. Sample feedback forms are throughout the workbook.

Provide feedback regarding evaluation findings to the key policy makers, managers,

and staff when and as appropriate. Use visual aids and handouts. Tabulate and summarize information. Make it as interesting as possible.

Consider establishing a resource or expert “panel” or advisory board that is an official

group of people willing to be contacted when you need feedback or have questions.

 

 

 

 

 

Evaluation Expert Session July 16, 2002 Page 4

Determining Program Components

Program components are identified by answering the questions who, what, when, where, and how as they pertain to your program.

Who: the program clients/recipients and staff What: activities, behaviors, materials When: frequency and length of the contact or intervention Where: the community context and physical setting How: strategies for operating the program or intervention

BRIEF EXAMPLE: Who: elementary school students What: fire safety intervention When: 2 times per year Where: in students’ classroom How: group administered intervention, small group practice

1. Instruct students what to do in case of fire (stop, drop and roll). 2. Educate students on calling 911 and have them practice on play telephones. 3. Educate students on how to pull a fire alarm, how to test a home fire alarm and how to

change batteries in a home fire alarm. Have students practice each of these activities. 4. Provide students with written information and have them take it home to share with their

parents. Request parental signature to indicate compliance and target a 75% return rate. Points to keep in mind when determining program components Specify activities as behaviors that can be observed

If you have a logic model, use the “activities” column as a starting point

Ensure that each component is separate and distinguishable from others

Include all activities and materials intended for use in the intervention

Identify the aspects of the intervention that may need to be adapted, and those that should

always be delivered as designed. Consult with program staff, mission statements, and program materials as needed.

 

 

 

 

 

Evaluation Expert Session July 16, 2002 Page 5

Your Program Components

After you have identified your program components, create a logic model that graphically portrays the link between program components and outcomes expected from these components.

Now, write out a succinct list of the components of your program. WHO: WHAT: WHEN: WHERE: HOW:

 

 

 

 

 

Evaluation Expert Session July 16, 2002 Page 6

What is a Logic Model

A logical series of statements that link the problems your program is attempting to address (conditions), how it will address them (activities), and what are the expected results (immediate and intermediate outcomes, long-term goals).

Benefits of the logic model include:

helps develop clarity about a project or program, helps to develop consensus among people, helps to identify gaps or redundancies in a plan, helps to identify core hypothesis, helps to succinctly communicate what your project or program is about.

When do you use a logic model Use… – During any work to clarify what is being done, why, and with what intended results – During project or program planning to make sure that the project or program is logical and complete – During evaluation planning to focus the evaluation – During project or program implementation as a template for comparing to the actual program and as a filter to determine whether proposed changes fit or not. This information was extracted from the Logic Models: A Multi-Purpose Tool materials developed by Wellsys Corporation for the Evaluation Planning Workshop Training. Please see the Evaluation Planning Workshop materials for more information. Appendix A has a sample template of the tabular format.

 

 

 

 

 

Evaluation Expert Session July 16, 2002 Page 7

Determining Evaluation Questions

As you design your process evaluation, consider what questions you would like to answer. It is only after your questions are specified that you can begin to develop your methodology. Considering the importance and purpose of each question is critical.

BROADLY…. What questions do you hope to answer? You may wish to turn the program components that you have just identified into questions assessing: Was the component completed as indicated? What were the strengths in implementation? What were the barriers or challenges in implementation? What were the apparent strengths and weaknesses of each step of the intervention? Did the recipient understand the intervention? Were resources available to sustain project activities? What were staff perceptions? What were community perceptions? What was the nature of the interaction between staff and clients?

These are examples. Check off what is applicable to you, and use the space below to write additional broad, overarching questions that you wish to answer.

 

 

 

 

 

Evaluation Expert Session July 16, 2002 Page 8

SPECIFICALLY … Now, make a list of all the specific questions you wish to answer, and organize your questions categorically. Your list of questions will likely be much longer than your list of program components. This step of developing your evaluation will inform your methodologies and instrument choice. Remember that you must collect information on what the program is intended to be and what it is in reality, so you may need to ask some questions in 2 formats. For example:

How many people are intended to complete this intervention per week?” How many actually go through the intervention during an average week?”

Consider what specific questions you have. The questions below are only examples! Some may not be appropriate for your evaluation, and you will most likely need to add additional questions. Check off the questions that are applicable to you, and add your own questions in the space provided. WHO (regarding client): Who is the target audience, client, or recipient? How many people have participated? How many people have dropped out? How many people have declined participation? What are the demographic characteristics of clients?

Race Ethnicity National Origin Age Gender Sexual Orientation Religion Marital Status Employment Income Sources Education Socio-Economic Status

What factors do the clients have in common? What risk factors do clients have? Who is eligible for participation? How are people referred to the program? How are the screened? How satisfied are the clients?

YOUR QUESTIONS:

 

 

 

 

 

Evaluation Expert Session July 16, 2002 Page 9

WHO (Regarding staff): Who delivers the services? How are they hired? How supportive are staff and management of each other? What qualifications do staff have? How are staff trained? How congruent are staff and recipients with one another? What are staff demographics? (see client demographic list for specifics.)

YOUR QUESTIONS: WHAT: What happens during the intervention? What is being delivered? What are the methods of delivery for each service (e.g., one-on-one, group session, didactic instruction,

etc.) What are the standard operating procedures? What technologies are in use? What types of communication techniques are implemented? What type of organization delivers the program? How many years has the organization existed? How many years has the program been operating? What type of reputation does the agency have in the community? What about the program? What are the methods of service delivery? How is the intervention structured? How is confidentiality maintained?

YOUR QUESTIONS: WHEN: When is the intervention conducted? How frequently is the intervention conducted? At what intervals? At what time of day, week, month, year? What is the length and/or duration of each service?

 

 

 

 

 

 

Evaluation Expert Session July 16, 2002 Page 10

YOUR QUESTIONS: WHERE: Where does the intervention occur? What type of facility is used? What is the age and condition of the facility? In what part of town is the facility? Is it accessible to the target audience? Does public transportation access

the facility? Is parking available? Is child care provided on site?

YOUR QUESTIONS: WHY: Why are these activities or strategies implemented and why not others? Why has the intervention varied in ability to maintain interest? Why are clients not participating? Why is the intervention conducted at a certain time or at a certain frequency?

YOUR QUESTIONS:

 

 

 

 

 

Evaluation Expert Session July 16, 2002 Page 11

Validating Your Evaluation Questions

Even though all of your questions may be interesting, it is important to narrow your list to questions that will be particularly helpful to the evaluation and that can be answered given your specific resources, staff, and time.

Go through each of your questions and consider it with respect to the questions below, which may be helpful in streamlining your final list of questions. Revise your worksheet/list of questions until you can answer “yes” to all of these questions. If you cannot answer “yes” to your question, consider omitting the question from your evaluation.

Validation

Yes

No

Will I use the data that will stem from these questions?

 

 

Do I know why each question is important and /or valuable?

 

 

Is someone interested in each of these questions?

 

 

Have I ensured that no questions are omitted that may be important to someone else?

 

 

Is the wording of each question sufficiently clear and unambiguous?

 

 

Do I have a hypothesis about what the “correct” answer will be for each question?

 

 

Is each question specific without inappropriately limiting the scope of the evaluation or probing for a specific response?

 

 

Do they constitute a sufficient set of questions to achieve the purpose(s) of the evaluation?

 

 

Is it feasible to answer the question, given what I know about the resources for evaluation?

 

 

Is each question worth the expense of answering it?

 

 

Derived from “A Design Manual” Checklist, page 51.

 

 

 

 

 

Evaluation Expert Session July 16, 2002 Page 12

Determining Methodology Process evaluation is characterized by collection of data primarily through two formats: 1) Quantitative, archival, recorded data that may be managed by an computerized

tracking or management system, and 2) Qualitative data that may be obtained through a variety of formats, such as

surveys or focus groups.

When considering what methods to use, it is critical to have a thorough understanding and knowledge of the questions you want answered. Your questions will inform your choice of methods. After this section on types of methodologies, you will complete an exercise in which you consider what method of data collection is most appropriate for each question.

Do you have a thorough understanding of your questions?

Furthermore, it is essential to consider what data the organization you are evaluating already has. Data may exist in the form of an existing computerized management information system, records, or a tracking system of some other sort. Using this data may provide the best reflection of what is “going on,” and it will also save you time, money, and energy because you will not have to devise your own data collection method! However, keep in mind that you may have to adapt this data to meet your own needs – you may need to add or replace fields, records, or variables.

What data does your organization already have? Will you need to adapt it?

If the organization does not already have existing data, consider devising a method for the organizational staff to collect their own data. This process will ultimately be helpful for them so that they can continue to self-evaluate, track their activities, and assess progress and change. It will be helpful for the evaluation process because, again, it will save you time, money, and energy that you can better devote towards other aspects of the evaluation. Management information systems will be described more fully in a later section of this workbook.

Do you have the capacity and resources to devise such a system? (You may need to refer to a later section of this workbook before answering.)

 

 

 

 

 

Evaluation Expert Session July 16, 2002 Page 13

Who should collect the data?

 

Given all of this, what thoughts do you have on who should collect data for your evaluation? Program staff, evaluation staff, or some combination?

Program Staff: May collect data from activities such as attendance, demographics, participation, characteristics of participants, dispositions, etc; may conduct intake interviews, note changes regarding service delivery, and monitor program implementation.

Advantages: Cost-efficient, accessible, resourceful, available, time-efficient,

and increased understanding of the program. Disadvantages: May exhibit bias and/or social desirability, may use data for critical

judgment, may compromise the validity of the program; may put staff in uncomfortable or inappropriate position; also, if staff collect data, may have an increased burden and responsibility placed upon them outside of their usual or typical job responsibilities. If you utilize staff for data collection, provide frequent reminders as well as messages of gratitude.

 

Evaluation staff: May collect qualitative information regarding implementation, general characteristics of program participants, and other information that may otherwise be subject to bias or distortion.

Advantages: Data collected in manner consistent with overall goals and timeline

of evaluation; prevents bias and inappropriate use of information; promotes overall fidelity and validity of data.

Disadvantages: May be costly and take extensive time; may require additional

training on part of evaluator; presence of evaluator in organization may be intrusive, inconvenient, or burdensome.

 

 

 

 

 

 

Evaluation Expert Session July 16, 2002 Page 14

When should data be collected?

Conducting the evaluation according to your timeline can be challenging. Consider how much time you have for data collection, and make decisions regarding what to collect and how much based on your timeline. In many cases, outcome evaluation is not considered appropriate until the program has stabilized. However, when conducting a process evaluation, it can be important to start the evaluation at the beginning so that a story may be told regarding how the program was developed, information may be provided on refinements, and program growth and progress may be noted. If you have the luxury of collecting data from the start of the intervention to the end of the intervention, space out data collection as appropriate. If you are evaluating an ongoing intervention that is fairly quick (e.g., an 8-week educational group), you may choose to evaluate one or more “cycles.” How much time do you have to conduct your evaluation? How much time do you have for data collection (as opposed to designing the evaluation, training, organizing and analyzing results, and writing the report?) Is the program you are evaluating time specific? How long does the program or intervention last? At what stages do you think you will most likely collect data?

Soon after a program has begun Descriptive information on program characteristics that will not change; information requiring baseline information During the intervention Ongoing process information such as recruitment, program implementation After the intervention Demographics, attendance ratings, satisfaction ratings

 

 

 

 

 

Evaluation Expert Session July 16, 2002 Page 15

Before you consider methods

A list of various methods follows this section. Before choosing what methods are most appropriate for your evaluation, review the following questions. (Some may already be answered in another section of this workbook.)

What questions do I want answered? (see previous section)

Does the organization already have existing data, and if so, what kind?

Does the organization have staff to collect data?

What data can the organization staff collect?

Must I maintain anonymity (participant is not identified at all) or confidentiality

(participant is identified but responses remain private)? This consideration pertains to existing archival data as well as original data collection.

How much time do I have to conduct the evaluation?

How much money do I have in my budget?

How many evaluation staff do I have to manage the data collection activities?

Can I (and/or members of my evaluation staff) travel on site?

What time of day is best for collecting data? For example, if you plan to conduct

focus groups or interviews, remember that your population may work during the day and need evening times.

 

 

 

 

 

Evaluation Expert Session July 16, 2002 Page 16

Types of methods

A number of different methods exist that can be used to collect process information. Consider each of the following, and check those that you think would be helpful in addressing the specific questions in your evaluation. When “see sample” is indicated, refer to the pages that follow this table.

 

√ Method Description

 

Activity, participation, or client tracking log

Brief record completed on site at frequent intervals by participant or deliverer. May use form developed by evaluator if none previously exists. Examples: sign in log, daily records of food consumption, medication management.

Case Studies Collection of in-depth information regarding small number of intervention recipients; use multiple methods of data collection.

Ethnographic analysis

Obtain in-depth information regarding the experience of the recipient by partaking in the intervention, attending meetings, and talking with delivery staff and recipients.

Expert judgment Convene a panel of experts or conduct individual interviews to obtain their understanding of and reaction to program delivery.

Focus groups Small group discussion among program delivery staff or recipients. Focus on their thoughts and opinions regarding their experiences with the intervention.

Meeting minutes (see sample)

Qualitative information regarding agendas, tasks assigned, and coordination and implementation of the intervention as recorded on a consistent basis.

Observation (see sample)

Observe actual delivery in vivo or on video, record findings using check sheet or make qualitative observations.

 

Open-ended interviews – telephone or in person

Evaluator asks open questions (i.e., who, what, when, where, why, how) to delivery staff or recipients. Use interview protocol without preset response options.

Questionnaire Written survey with structured questions. May administer in individual, group, or mail format. May be anonymous or confidential.

Record review

Obtain indicators from intervention records such patient files, time sheets, telephone logs, registration forms, student charts, sales records, or records specific to the service delivery.

 

Structured interviews – telephone or in person

Interviewer asks direct questions using interview protocol with preset response options.

 

 

 

 

Evaluation Expert Session

July 16, 2002 Page 17

Sample activity log

This is a common process evaluation methodology because it systematically records exactly what is happening during implementation. You may wish to devise a log such as the one below and alter it to meet your specific needs. Consider computerizing such a log for efficiency. Your program may already have existing logs that you can utilize and adapt for your evaluation purposes.

Site:

Recorder:

Code

Service

Date

Location

# People

# Hours

Notes

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

Evaluation Expert Session July 16, 2002

Page 18

Meeting Minutes

Taking notes at meetings may provide extensive and invaluable process information that can later be organized and structured into a comprehensive report. Minutes may be taken by program staff or by the evaluator if necessary. You may find it helpful to use a structured form, such as the one below that is derived from Evaluating Collaboratives, University of Wisconsin-Cooperative Extension, 1998.

Meeting Place: __________________ Start time: ____________ Date: _____________________________ End time: ____________ Attendance (names): Agenda topic: _________________________________________________ Discussion: _____________________________________________________ Decision Related Tasks Who responsible Deadline 1. 2. 3. Agenda topic: _________________________________________________ Discussion: _____________________________________________________ Decision Related Tasks Who responsible Deadline 1. 2. 3. Sample observation log

 

 

 

 

 

Evaluation Expert Session July 16, 2002

Page 19

Observation may occur in various methods, but one of the most common is hand-recording specific details during a small time period. The following is several rows from an observation log utilized during an evaluation examining school classrooms.

CLASSROOM OBSERVATIONS (School Environment Scale) Classroom 1: Grade level _________________ (Goal: 30 minutes of observation)

Time began observation: _________Time ended observation:_________ Subjects were taught during observation period: ___________________

PHYSICAL ENVIRONMENT

Question

Answer 1. Number of students

 

2. Number of adults in room: a. Teachers b. Para-pros c. Parents

Total: a. b. c.

3. Desks/Tables a. Number of Desks b. Number of Tables for students’ use c. Any other furniture/include number (Arrangement of desks/tables/other furniture)

a. b. c.

4. Number of computers, type

 

5. How are computers being used?

 

6. What is the general classroom setup? (are there walls, windows, mirrors, carpet, rugs, cabinets, curtains, etc.)

 

7. Other technology (overhead projector, power point, VCR, etc.)

 

8. Are books and other materials accessible for students?

 

9. Is there adequate space for whole-class instruction?

 

12. What type of lighting is used?

 

13. Are there animals or fish in the room?

 

14. Is there background music playing?

 

15. Rate the classroom condition Poor Average Excellent

 

16. Are rules/discipline procedures posted? If so, where?

 

17. Is the classroom Noisy or Quiet? Very Quiet Very Noisy

 

Choosing or designing measurement instruments Consider using a resource panel, advisory panel, or focus group to offer feedback

 

 

 

 

 

Evaluation Expert Session July 16, 2002

Page 20

regarding your instrument. This group may be composed of any of the people listed below. You may also wish to consult with one or more of these individuals throughout the development of your overall methodology.

Who should be involved in the design of your instrument(s) and/or provide feedback? Program service delivery staff / volunteers Project director Recipients of the program Board of directors Community leader Collaborating organizations Experts on the program or service being evaluated Evaluation experts _________________________ _________________________ _________________________

Conduct a pilot study and administer the instrument to a group of recipients, and then

obtain feedback regarding their experience. This is a critical component of the development of your instruments, as it will help ensure clarity of questions, and reduce the degree of discomfort or burden that questions or processes (e.g., intakes or computerized data entry) elicit.

How can you ensure that you pilot your methods? When will you do it, and whom will you use as participants in the study? Ensure that written materials are at an appropriate reading level for the population.

Ensure that verbal information is at an appropriate terminology level for the population. A third or sixth-grade reading level is often utilized.

Remember that you are probably collecting data that is program-specific. This may

increase the difficulty in finding instruments previously constructed to use for questionnaires, etc. However, instruments used for conducting process evaluations of other programs may provide you with ideas for how to structure your own instruments.

 

 

 

 

 

Evaluation Expert Session July 16, 2002

Page 21

Linking program components and methods (an example) Now that you have identified your program components, broad questions, specific questions, and possible measures, it is time to link them together. Let’s start with your program components. Here is an example of 3 program components of an intervention.

Program Components and Essential Elements: There are six program components to M2M. There are essential elements in each component that must be present for the program to achieve its intended results and outcomes, and for the program to be identified as a program of the American Cancer Society.

Possible Process Measures

1) Man to Man Self-Help and/or Support Groups The essential elements within this component are:

• Offer information and support to all men with prostate cancer at all points along the cancer care continuum

• Directly, or through collaboration and referral, offer community access to prostate cancer self-help and/or support groups

• Provide recruitment and on-going training and monitoring for M2M leaders and volunteers

• Monitor, track and report program activities

 

 

• Descriptions of attempts to schedule and advertise group meetings

• Documented efforts to establish the program • Documented local needs assessments • # of meetings held per independent group • Documented meetings held • # of people who attended different topics and speakers • Perceptions of need of survey participants for

additional groups and current satisfaction levels • # of new and # of continuing group members • Documented sign-up sheets for group meetings • Documented attempts to contact program dropouts • # of referrals to other PC groups documented • # of times corresponding with other PC groups • # of training sessions for new leaders • # of continuing education sessions for experienced

leaders • # and types of other on-going support activities for

volunteer leaders • # of volunteers trained as group facilitators • Perceptions of trained volunteers for readiness to

function as group facilitators

 

 

 

 

 

Evaluation Expert Session July 16, 2002

Page 22

2) One-to-One Contacts The essential elements within this component are:

• Offer one-to-one contact to provide information and support to all men with prostate cancer, including those in the diagnostic process

• Provide recruitment and on-going training and monitoring for M2M leaders and volunteers

• Monitor, track and report program activities

 

• # of contact pairings

• Frequency and duration of contact pairings

• Types of information shared during contact pairings

• # of volunteers trained

• Perception of readiness by trained volunteers

• Documented attempts for recruiting volunteers

• Documented on-going training activities for volunteers

• Documented support activities

 

3) Community Education and Awareness

The essential elements within this component are:

 

• Conduct public awareness activities to inform the public about prostate cancer and M2M

• Monitor, track and report program activities

 

• # of screenings provided by various health care

providers/agencies over assessment period • Documented ACS staff and volunteer efforts to

publicize the availability and importance of PC and screenings, including health fairs, public service announcements, billboard advertising, etc.

• # of addresses to which newsletters are mailed • Documented efforts to increase newsletter mailing list

 

 

 

 

 

 

Page 23

Linking YOUR program components, questions, and methods

Consider each of your program components and questions that you have devised in an earlier section of this workbook, and the methods that you checked off on the “types of methods” table. Now ask yourself, how will I use the information I have obtained from this question? And, what method is most appropriate for obtaining this information?

 

Program Component

Specific questions that go with this

component

How will I use this

information?

Best method?

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

Page 24

Program Component

Specific questions that go with this

component

How will I use this

information?

Best method?

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

Evaluation Expert Session July 16, 2002

Page 25

Data Collection Plan Now let’s put your data collection activities on one sheet – what you’re collecting, how you’re doing it, when, your sample, and who will collect it. Identifying your methods that you have just picked, instruments, and data collection techniques in a structured manner will facilitate this process.

Method

Type of data (questions, briefly indicated)

Instrument used

When implemented

Sample

Who collects

E.g.: Patient interviews in health dept clinics

Qualitative – what services they are using, length of visit, why came in, how long wait, some quantitative satisfaction ratings

Interview created by evaluation team and piloted with patients

Oct-Dec; days and hrs randomly selected

10 interviews in each clinic

Trained interviewers

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

Page 26

Evaluation Expert Session July 16, 2002

 

Consider a Management Information System

Process data is frequently collected through a management information system (MIS) that is designed to record characteristics of participants, participation of participants, and characteristics of activities and services provided. An MIS is a computerized record system that enables service providers and evaluators to accumulate and display data quickly and efficiently in various ways.

Will your evaluation be enhanced by periodic data presentations in tables or other structured formats? For example, should the evaluation utilize a monthly print-out of services utilized or to monitor and process recipient tracking (such as date, time, and length of service)?

YES

NO

Does the agency create monthly (or other periodic) print outs reflecting services rendered or clients served?

YES

NO

Will the evaluation be conducted in a more efficient manner if program delivery staff enter data on a consistent basis?

YES

NO

Does the agency already have hard copies of files or records that would be better utilized if computerized?

YES

NO

Does the agency already have an MIS or a similar computerized database?

YES

NO

If the answers to any of these questions are YES, consider using an MIS for your evaluation. If an MIS does not already exist, you may desire to design a database in which you can

enter information from records obtained by the agency. This process decreases missing data and is generally efficient.

If you do create a database that can be used on an ongoing basis by the agency, you may

consider offering it to them for future use.

 

 

 

 

 

Page 27

Evaluation Expert Session July 16, 2002

 

Information to be included in your MIS Examples include: Client demographics Client contacts Client services Referrals offered Client outcomes Program activities Staff notes

Jot down the important data you would like to be included in your MIS. Managing your MIS What software do you wish to utilize to manage your data? What type of data do you have? How much information will you need to enter? How will you ultimately analyze the data? You may wish to create a database directly in the program you will eventually use, such as SPSS? Will you be utilizing lap tops?

 

 

 

 

 

Page 28

Evaluation Expert Session July 16, 2002

 

If so, will you be taking them onsite and directly entering your data into them?

How will you download or transfer the information, if applicable?

What will the impact be on your audience if you have a laptop? Tips on using an MIS If service delivery personnel will be collecting and/or entering information into the MIS

for the evaluator’s use, it is generally a good idea to provide frequent reminders of the importance of entering the appropriate information in a timely, consistent, and regular manner.

For example, if an MIS is dependent upon patient data collected by public health officers

daily activities, the officers should be entering data on at least a daily basis. Otherwise, important data is lost and the database will only reflect what was salient enough to be remembered and entered at the end of the week.

Don’t forget that this may be burdensome and/or inconvenient for the program staff.

Provide them with frequent thank you’s. Remember that your database is only as good as you make it. It must be organized and

arranged so that it is most helpful in answering your questions. If you are collecting from existing records, at what level is he data currently available?

For example, is it state, county, or city information? How is it defined? Consider whether adaptations need to be made or additions need to be included for your evaluation.

Back up your data frequently and in at least one additional format (e.g., zip, disk, server).

Consider file security. Will you be saving data on a network server? You may need to

consider password protection.

 

 

 

 

 

Page 29

Evaluation Expert Session July 16, 2002

 

Allocate time for data entry and checking.

Allow additional time to contemplate the meaning of the data before writing the report.

 

 

 

 

 

Page 30

Evaluation Expert Session July 16, 2002

 

Implement Data Collection and Analysis

Data collection cannot be fully reviewed in this workbook, but this page offers a few tips regarding the process.

General reminders: THANK everyone who helps you, directs you, or participates in anyway.

Obtain clear directions and give yourself plenty of time, especially if you are traveling

long distance (e.g., several hours away). Bring all of your own materials – do not expect the program to provide you with writing

utensils, paper, a clipboard, etc. Address each person that you meet with respect and attempt to make your meeting as

conducive with their schedule as possible. Most process evaluation will be in the form of routine record keeping (e.g., MIS). However, you may wish to interview clients and staff. If so: Ensure that you have sufficient time to train evaluation staff, data collectors, and/or

organization staff who will be collecting data. After they have been trained in the data collection materials and procedure, require that they practice the technique, whether it is an interview or entering a sample record in an MIS.

If planning to use a tape recorder during interviews or focus groups, request permission

from participants before beginning. You may need to turn the tape recorder off on occasion if it will facilitate increased comfort by participants.

If planning to use laptop computers, attempt to make consistent eye contact and spend

time establishing rapport before beginning. Some participants may be uncomfortable with technology and you may need to provide education regarding the process of data collection and how the information will be utilized.

If planning to hand write responses, warn the participant that you may move slowly and

 

 

 

 

 

Page 31

Evaluation Expert Session July 16, 2002

 

may need to ask them to repeat themselves. However, prepare for this process by developing shorthand specific to the evaluation. A sample shorthand page follows.

 

 

 

 

 

Page 32

Evaluation Expert Session July 16, 2002

 

Annual Evaluation Reports The ultimate aim of all the Branch’s evaluation efforts is to increase the intelligent use of information in Branch decision-making in order to improve health outcomes. Because we understand that many evaluation efforts fail because the data are never collected and that even more fail because the data are collected but never used in decision-making, we have struggled to find a way to institutionalize the use of evaluation results in Branch decision-making. These reports will serve multiple purposes:

The need to complete the report will increase the likelihood that evaluation is done and data are collected.

The need to review reports from lower levels in order to complete one’s own report hopefully will cause managers at all levels to consciously consider, at least once a year, the effectiveness of their activities and how evaluation results suggest that effectiveness can be improved.

The summaries of evaluation findings in the reports should simplify preparation of other reports to funders including the General Assembly.

Each evaluation report forms the basis of the evaluation report at the next level. The contents and length of the report should be determined by what is mot helpful to the manager who is receiving the report. Rather than simply reporting every possible piece of data, these reports should present summary data, summarize important conclusions, and suggest recommendations based on the evaluation findings. A program-level annual evaluation report should be ten pages or less. Many my be less than five pages. Population team and Branch-level annual evaluation reports may be longer than ten pages, depending on how many findings are being reported. However, reports that go beyond ten pages should also contain a shorter Executive Summary, to insure that those with the power to make decisions actually read the findings. Especially, the initial reports may reflect formative work and consist primarily of updates on the progress of evaluation planning and implementation. This is fine and to be expected. However, within a year or two the reports should begin to include process data, and later actual outcome findings. This information was extracted from the FHB Evaluation Framework developed by Monica Herk and Rebekah Hudgins.

 

 

 

 

 

Page 33

Evaluation Expert Session July 16, 2002

 

Suggested shorthand – a sample The list below was derived for a process evaluation regarding charter schools. Note the use of general shorthand as well as shorthand derived specifically for the evaluation.

 

 

 

CS Charter School

mst

Most

Sch School b/c Because Tch Teacher, teach st Something P Principal b Be VP Vice Principal c See Admin Administration, administrators r Are DOE Dept of Education w/ When BOE Board of Education @ At Comm Community ~ About Stud Students, pupils = Is, equals, equivalent Kids Students, children, teenagers ≠ Does not equal, is not the same K Kindergarten Sone Someone Cl Class # Number CR Classroom $ Money, finances, financial, funding,

expenses, etc. W White + Add, added, in addition B Black < Less than AA African American > Greater/more than SES Socio-economic status ??? What does this mean? Get more

info on, I’m confused… Lib Library, librarian DWA Don’t worry about (e.g. if you wrote

something unnecessary) Caf Cafeteria Ψ Psychology, psychologist Ch Charter ∴ Therefore Conv Conversion (school) ∆ Change, is changing S-up Start up school mm Movement App Application, applied ↑ Increases, up, promotes ITBS Iowa Test of Basic Skills ↓ Decreases, down, inhibits LA Language arts X Times (e.g. many x we laugh) SS Social Studies ÷ Divided (we ÷ up the classrooms) QCC Quality Core Curriculum C With Pol Policy, politics Home, house Curr Curriculum ♥ Love, adore (e.g. the kids ♥ this) LP Lesson plans Church, religious activity Disc Discipline O No, doesn’t, not Girls, women, female 1/2 Half (e.g. we took 1/2) Boys, men, male 2 To

 

 

 

 

 

Page 34

Evaluation Expert Session July 16, 2002

 

F

Father, dad

c/out

without

P Parent

2B

To be

M Mom, mother

e.g.

For example

i.e. That is

If the person trails off, you missed information

Appendix A

Logic Model Worksheet

Population Team/Program Name __________________________ Date _______________________ If the following CONDITIONS AND ASSUMPTIONS exist…

And if the following ACTIVITIES are implemented to address these conditions and assumptions

Then these SHORT-TERM OUTCOMES may be achieved…

And these LONG-TERM OUTCOMES may be acheived…

And these LONG- TERM GOALS can be reached….

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

Page 35

Evaluation Expert Session July 16, 2002

 

 

Appendix B Pitfalls To Avoid Avoid heightening expectations of delivery staff, program recipients, policy makers, or

community members. Ensure that feedback will be provided as appropriate, but may or may not be utilized.

Avoid any implication that you are evaluating the impact or outcome. Stress that you are

evaluating “what is happening,” not how well any one person is performing or what the outcomes of the intervention are.

Make sure that the right information gets to the right people – it is most likely to be utilized

in a constructive and effective manner if you ensure that your final report does not end up on someone’s desk who has little motivation or interest in utilizing your findings.

Ensure that data collection and entry is managed on a consistent basis – avoid developing an

evaluation design and than having the contract lapse because staff did not enter the data.

 

 

 

 

 

 

Page 36

Evaluation Expert Session July 16, 2002

 

Appendix C References

References used for completion of this workbook and/or that you may find helpful for additional information.

Centers for Disease Control and Prevention. 1995. Evaluating Community Efforts to Prevent Cardiovascular Diseases. Atlanta, GA. Centers for Disease Control and Prevention. 2001. Introduction to Program Evaluation for Comprehensive Tobacco Control Programs. Atlanta, GA. Freeman, H. E., Rossi, P. H., Sandefur, G. D. 1993. Workbook for evaluation: A systematic approach. Sage Publications: Newbury Park, CA. Georgia Policy Council for Children and Families; The Family Connection; Metis Associates, Inc. 1997. Pathways for assessing change: Strategies for community partners. Grembowski, D. 2001. The practice of health program evaluation. Sage Publications: Thousand Oaks. Hawkins, J. D., Nederhood, B. 1987. Handbook for Evaluating Drug and Alcohol Prevention Programs. U.S. Department of Health and Human Services; Public Health Service; Alcohol, Drug Abuse, and Mental Health Administration: Washington, D. C. Muraskin, L. D. 1993. Understanding evaluation: The way to better prevention programs. Westat, Inc. National Community AIDS Partnership 1993. Evaluating HIV/AIDS Prevention Programs in Community-based Organizations. Washington, D.C. NIMH Overview of Needs Assessment. Chapter 3: Selecting the needs assessment approach. Patton, M. Q. 1982. Practical Evaluation. Sage Publications, Inc.: Beverly Hills, CA.

 

 

 

 

 

 

Page 37

Evaluation Expert Session July 16, 2002

 

Posavac, E. J., Carey, R. G. 1980. Program Evaluation: Methods and Case Studies. Prentice-Hall, Inc.: Englewood Cliffs, N.J. Rossi, P. H., Freeman, H. E., Lipsey, M. W. 1999. Evaluation: A Systematic Approach. (6th edition). Sage Publications, Inc.: Thousand Oaks, CA. Scheirer, M. A. 1994. Designing and using process evaluation. In: J. S. Wholey, H. P. Hatry, & K. E. Newcomer (eds) Handbook of practical program evaluation. Jossey-Bass Publishers: San Francisco. Taylor-Powell, E., Rossing, B., Geran, J. 1998. Evaluating Collaboratives: Reaching the potential. Program Development and Evaluation: Madison, WI. U.S. Department of Health and Human Services; Administration for Children and Families; Office of Community Services. 1994. Evaluation Guidebook: Demonstration partnership program projects. W.K. Kellogg Foundation. 1998. W. K. Kellogg Foundation Evaluation Handbook. Websites: www.cdc.gov/eval/resources www.eval.org (has online text books) www.wmich.edu/evalctr (has online checklists) www.preventiondss.org

When conducting literature reviews or searching for additional information, consider using alternative names for “process evaluation,” including: formative evaluation program fidelity implementation assessment implementation evaluation program monitoring