Reporting a Process Evaluation

Discussion 1: Reporting a Process Evaluation

Just as in needs assessments, interviews and focus groups are common tools for obtaining information about the processes involved in the implementation of programs. Process evaluation should include specifics about purpose, questions which the evaluation will address, and methods that social workers will use to conduct evaluations.

Review the many examples of process evaluation results described in Chapter 8 of Dudley, J. R. (2014). Social work evaluation: Enhancing what we do. (2nd ed.) Chicago, IL: Lyceum Books, or in the optional resources. Select an example of a process evaluation that produced valuable information. Compare the description of those results with the Social Work Research Qualitative Groups case study located in this week’s resources.

· Post a description of the process evaluation that you chose and explain why you selected this example. 

· Describe the stage of program implementation in which the evaluation occurred, the informants, the questions asked, and the results.

· Based upon your comparison of the case study and the program evaluation report that you chose, improve upon the information presented in the case study by identifying gaps in information.

· Fill in these gaps as if you were the facilitator of the focus group. Clearly identify the purpose of the process evaluation and the questions asked.

References (use 3 or more)

Dudley, J. R. (2014). Social work evaluation: Enhancing what we do. (2nd ed.) Chicago, IL: Lyceum Books.

Chapter 8, “Improving How Programs and Practice Work” (pp. 167–207)

Plummer, S.-B., Makris, S., & Brocksen S. (Eds.). (2014b). Social work case studies: Concentration year. Baltimore, MD: Laureate International Universities Publishing. [Vital Source e-reader].

Read the following section:

“Social Work Research: Qualitative Groups” (pp. 68–69)

Document: Bliss, M. J., & Emshoff, J. G. (2002). Workbook for designing a process evaluation. Retrieved from http://beta.roadsafetyevaluation.com/evaluationguides/info/workbook-for-designing-a-process-evaluation.pdf (PDF)

Georgia Department of Human Resources, Division of Public Health.

Example of Process Evaluation

Boyce, C., & Neale, P. (2006). Conducting in-depth interviews: A guide for designing and conducting in-depth interviews for evaluation input. Pathfinder International Tool Series: Monitoring and Evaluation – 2. Retrieved from http://www.cpc.unc.edu/measure/training/materials/data-quality-portuguese/m_e_tool_series_indepth_interviews.pdf

Social Work Research: Qualitative Groups

A focus group was conducted to explore the application of a cross-system collaboration and its effect on service delivery outcomes among social service agencies in a large urban county on the West Coast. The focus group consisted of 10 social workers and was led by a facilitator from the local office of a major community support organization (the organization). Participants in the focus group had diverse experiences working with children, youth, adults, older adults, and families. They represented agencies that addressed child welfare, family services, and community mental health issues. The group included five males and five females from diverse ethnicities.

The focus group was conducted in a conference room at the organization’s headquarters. The organization was interested in exploring options for greater collaboration and less fragmentation of social services in the local area. Participants in the group were recruited from local agencies that were either already receiving or were applying for funding from the organization. The 2-hour focus group was recorded.

The facilitator explained the objective of the focus group and encouraged each participant to share personal experiences and perspectives regarding cross-system collaboration. Eight questions were asked that explored local examples of cross-system collaboration and the strengths and barriers found in using the model. The facilitator tried to achieve maximum participation by reflecting the answers back to the participants and maintaining eye contact.

To analyze the data, the researchers carefully transcribed the entire recorded discussion and utilized a qualitative data analysis software package issued by StatPac, which offers a product called Verbatim Blaster. This software focuses on content coding and word counting to identify the most salient themes and patterns.

The focus group was seen by the sponsoring entity as successful because every participant eventually provided feedback to the facilitator about cross-system collaboration. It was also seen as a success because the facilitator remained engaged and nonjudgmental and strived to have each participant share their experiences.

In terms of outcomes, the facilitator said that the feedback obtained was useful in exploring new ways of delivering services and encouraging greater cooperation. As a result of this process, the organization decided to add a component to all agency annual plans and reports that asked them to describe what types of cross-agency collaboration were occurring and what additional efforts were planned.

(Plummer 68-69)

Plummer, Sara-Beth, Sara Makris, Sally Brocksen. Social Work Case Studies: Concentration Year. Laureate Publishing, 10/21/13. VitalBook file.

Discussion 2: Social Work Agency Budgeting

Human services organizations cannot work in isolation because of the breadth and depth of social issues they address in their mission to provide services. By partnering with other organizations in the community, human services organizations can expand their service delivery. These community partners can complement the work of the social work agency and help raise additional funds for services. Strategic partnerships are not limited to nonprofit organizations; human services organizations may also work with local businesses. When considering community partners, administrators and leaders should keep an open mind and think about unique partnerships that will benefit the community.

For this Discussion, search for examples in your local community of partnerships between human services organizations and local businesses and/or nonprofits. (You may review the partnership example described at the top of page 306 in Understanding Your Social Agency, 3rd ed.) Consider how the human services organizations, nonprofits, businesses, and community benefit from these partnerships. Also consider limitations to these collaborative endeavors.

· Post a description of examples in your local community of partnerships between human services organizations and local businesses and/or nonprofits that benefit the community.

· Analyze the collaboration to identify aspects that provide benefits that go beyond the initial collaborative effort. 

· Explain how these aspects may benefit the human services organization.

· In addition, identify aspects of the collaboration that may lead to challenges, and explain how they may challenge the human services organization.

References (use 3 or more)

Lauffer, A. (2011). Understanding your social agency (3rd ed.). Washington, DC: Sage.

Chapter 9, “Fundraising and Development” (pp. 285–320)

Bowman, W. (2011). Financial capacity and sustainability of ordinary nonprofits. Nonprofit Management & Leadership, 22(1), 37–51.

LeRoux, K. (2009). Managing stakeholder demands: Balancing responsiveness to clients and funding agents in nonprofit social service organizations. Administration & Society, 41(2), 158–184.

Discussion 3: Financial Capacity and Sustainability in Human Services

Receiving funding from a grant or other source of funds is a great accomplishment. Once the funding is received, the human services organization must be able to manage the funds effectively. The organization must also develop a plan to sustain the program after the funding period ends or the potential for change from the funded program may be limited. One way to determine an organization’s capacity for fundraising and/or financial management is to assess its strengths and weaknesses in these areas and consider ways to improve. This type of assessment should be undertaken before the organization begins to actively seek funds.

For this Discussion, you will evaluate an aspect of financial management or fundraising efforts in a human services organization with which you are familiar. Refer to the inventory tool on page 319 of Understanding Your Social Agency, 3rd ed., for ideas on how to evaluate an organization’s fundraising efforts.

· Post your brief description of an organization with which you are familiar (e.g., a field placement, a previous employer) and evaluate one area of the organization’s financial management or fundraising that needs improving, and explain why. 

· Explain three strategies the organization could implement to improve its financial management or fundraising situation.

· Explain how each strategy supports improvement.

References (use 3 or more)

Lauffer, A. (2011). Understanding your social agency (3rd ed.). Washington, DC: Sage.

Chapter 9, “Fundraising and Development” (pp. 285–320)

Bowman, W. (2011). Financial capacity and sustainability of ordinary nonprofits. Nonprofit Management & Leadership, 22(1), 37–51.

LeRoux, K. (2009). Managing stakeholder demands: Balancing responsiveness to clients and funding agents in nonprofit social service organizations. Administration & Society, 41(2), 158–184.

Workbook

for Designing a Process Evaluation

 

Produced for the

Georgia Department of Human Resources

Division of Public Health

By

Melanie J. Bliss, M.A. James G. Emshoff, Ph.D.

Department of Psychology Georgia State University

 

July 2002

 

 

Evaluation Expert Session July 16, 2002 Page 1

 

What is process evaluation?

Process evaluation uses empirical data to assess the delivery of programs. In contrast to outcome evaluation, which assess the impact of the program, process evaluation verifies what the program is and whether it is being implemented as designed. Thus, process evaluation asks “what,” and outcome evaluation asks, “so what?”

When conducting a process evaluation, keep in mind these three questions:

1. What is the program intended to be? 2. What is delivered, in reality? 3. Where are the gaps between program design and delivery?

This workbook will serve as a guide for designing your own process evaluation for a program of your choosing. There are many steps involved in the implementation of a process evaluation, and this workbook will attempt to direct you through some of the main stages. It will be helpful to think of a delivery service program that you can use as your example as you complete these activities. Why is process evaluation important? 1. To determine the extent to which the program is being

implemented according to plan 2. To assess and document the degree of fidelity and variability in

program implementation, expected or unexpected, planned or unplanned

3. To compare multiple sites with respect to fidelity 4. To provide validity for the relationship between the intervention

and the outcomes 5. To provide information on what components of the intervention

are responsible for outcomes 6. To understand the relationship between program context (i.e.,

setting characteristics) and program processes (i.e., levels of implementation).

7. To provide managers feedback on the quality of implementation 8. To refine delivery components 9. To provide program accountability to sponsors, the public, clients,

and funders 10. To improve the quality of the program, as the act of evaluating is

an intervention.

 

 

Evaluation Expert Session July 16, 2002 Page 2

Stages of Process Evaluation Page Number

1. Form Collaborative Relationships 3 2. Determine Program Components 4 3. Develop Logic Model* 4. Determine Evaluation Questions 6 5. Determine Methodology 11 6. Consider a Management Information System 25 7. Implement Data Collection and Analysis 28 8. Write Report**

Also included in this workbook:

a. Logic Model Template 30 b. Pitfalls to avoid 30 c. References 31

 

Evaluation can be an exciting, challenging, and fun experience

Enjoy!

 

* Previously covered in Evaluation Planning Workshops. ** Will not be covered in this expert session. Please refer to the Evaluation Framework

and Evaluation Module of FHB Best Practice Manual for more details.

 

 

 

 

 

Evaluation Expert Session July 16, 2002 Page 3

Forming collaborative relationships

A strong, collaborative relationship with program delivery staff and management will likely result in the following:

Feedback regarding evaluation design and implementation Ease in conducting the evaluation due to increased cooperation Participation in interviews, panel discussion, meetings, etc. Increased utilization of findings

Seek to establish a mutually respectful relationship characterized by trust, commitment, and flexibility.

Key points in establishing a collaborative relationship:

Start early. Introduce yourself and the evaluation team to as many delivery staff and management personnel as early as possible.

Emphasize that THEY are the experts, and you will be utilizing their knowledge and

information to inform your evaluation development and implementation.

Be respectful of their time both in-person and on the telephone. Set up meeting places that are geographically accessible to all parties involved in the evaluation process.

Remain aware that, even if they have requested the evaluation, it may often appear as

an intrusion upon their daily activities. Attempt to be as unobtrusive as possible and request their feedback regarding appropriate times for on-site data collection.

Involve key policy makers, managers, and staff in a series of meetings throughout the

evaluation process. The evaluation should be driven by the questions that are of greatest interest to the stakeholders. Set agendas for meetings and provide an overview of the goals of the meeting before beginning. Obtain their feedback and provide them with updates regarding the evaluation process. You may wish to obtained structured feedback. Sample feedback forms are throughout the workbook.

Provide feedback regarding evaluation findings to the key policy makers, managers,

and staff when and as appropriate. Use visual aids and handouts. Tabulate and summarize information. Make it as interesting as possible.

Consider establishing a resource or expert “panel” or advisory board that is an official

group of people willing to be contacted when you need feedback or have questions.

 

 

 

 

 

Evaluation Expert Session July 16, 2002 Page 4

Determining Program Components

Program components are identified by answering the questions who, what, when, where, and how as they pertain to your program.

Who: the program clients/recipients and staff What: activities, behaviors, materials When: frequency and length of the contact or intervention Where: the community context and physical setting How: strategies for operating the program or intervention

BRIEF EXAMPLE: Who: elementary school students What: fire safety intervention When: 2 times per year Where: in students’ classroom How: group administered intervention, small group practice

1. Instruct students what to do in case of fire (stop, drop and roll). 2. Educate students on calling 911 and have them practice on play telephones. 3. Educate students on how to pull a fire alarm, how to test a home fire alarm and how to

change batteries in a home fire alarm. Have students practice each of these activities. 4. Provide students with written information and have them take it home to share with their

parents. Request parental signature to indicate compliance and target a 75% return rate. Points to keep in mind when determining program components Specify activities as behaviors that can be observed

If you have a logic model, use the “activities” column as a starting point

Ensure that each component is separate and distinguishable from others

Include all activities and materials intended for use in the intervention

Identify the aspects of the intervention that may need to be adapted, and those that should

always be delivered as designed. Consult with program staff, mission statements, and program materials as needed.

 

 

 

 

 

Evaluation Expert Session July 16, 2002 Page 5

Your Program Components

After you have identified your program components, create a logic model that graphically portrays the link between program components and outcomes expected from these components.

Now, write out a succinct list of the components of your program. WHO: WHAT: WHEN: WHERE: HOW:

 

 

 

 

 

Evaluation Expert Session July 16, 2002 Page 6

What is a Logic Model

A logical series of statements that link the problems your program is attempting to address (conditions), how it will address them (activities), and what are the expected results (immediate and intermediate outcomes, long-term goals).

Benefits of the logic model include:

helps develop clarity about a project or program, helps to develop consensus among people, helps to identify gaps or redundancies in a plan, helps to identify core hypothesis, helps to succinctly communicate what your project or program is about.

When do you use a logic model Use… – During any work to clarify what is being done, why, and with what intended results – During project or program planning to make sure that the project or program is logical and complete – During evaluation planning to focus the evaluation – During project or program implementation as a template for comparing to the actual program and as a filter to determine whether proposed changes fit or not. This information was extracted from the Logic Models: A Multi-Purpose Tool materials developed by Wellsys Corporation for the Evaluation Planning Workshop Training. Please see the Evaluation Planning Workshop materials for more information. Appendix A has a sample template of the tabular format.

 

 

 

 

 

Evaluation Expert Session July 16, 2002 Page 7

Determining Evaluation Questions

As you design your process evaluation, consider what questions you would like to answer. It is only after your questions are specified that you can begin to develop your methodology. Considering the importance and purpose of each question is critical.

BROADLY…. What questions do you hope to answer? You may wish to turn the program components that you have just identified into questions assessing: Was the component completed as indicated? What were the strengths in implementation? What were the barriers or challenges in implementation? What were the apparent strengths and weaknesses of each step of the intervention? Did the recipient understand the intervention? Were resources available to sustain project activities? What were staff perceptions? What were community perceptions? What was the nature of the interaction between staff and clients?

These are examples. Check off what is applicable to you, and use the space below to write additional broad, overarching questions that you wish to answer.

 

 

 

 

 

Evaluation Expert Session July 16, 2002 Page 8

SPECIFICALLY … Now, make a list of all the specific questions you wish to answer, and organize your questions categorically. Your list of questions will likely be much longer than your list of program components. This step of developing your evaluation will inform your methodologies and instrument choice. Remember that you must collect information on what the program is intended to be and what it is in reality, so you may need to ask some questions in 2 formats. For example:

How many people are intended to complete this intervention per week?” How many actually go through the intervention during an average week?”

Consider what specific questions you have. The questions below are only examples! Some may not be appropriate for your evaluation, and you will most likely need to add additional questions. Check off the questions that are applicable to you, and add your own questions in the space provided. WHO (regarding client): Who is the target audience, client, or recipient? How many people have participated? How many people have dropped out? How many people have declined participation? What are the demographic characteristics of clients?

Race Ethnicity National Origin Age Gender Sexual Orientation Religion Marital Status Employment Income Sources Education Socio-Economic Status

What factors do the clients have in common? What risk factors do clients have? Who is eligible for participation? How are people referred to the program? How are the screened? How satisfied are the clients?

YOUR QUESTIONS:

 

 

 

 

 

Evaluation Expert Session July 16, 2002 Page 9

WHO (Regarding staff): Who delivers the services? How are they hired? How supportive are staff and management of each other? What qualifications do staff have? How are staff trained? How congruent are staff and recipients with one another? What are staff demographics? (see client demographic list for specifics.)

YOUR QUESTIONS: WHAT: What happens during the intervention? What is being delivered? What are the methods of delivery for each service (e.g., one-on-one, group session, didactic instruction,

etc.) What are the standard operating procedures? What technologies are in use? What types of communication techniques are implemented? What type of organization delivers the program? How many years has the organization existed? How many years has the program been operating? What type of reputation does the agency have in the community? What about the program? What are the methods of service delivery? How is the intervention structured? How is confidentiality maintained?

YOUR QUESTIONS: WHEN: When is the intervention conducted? How frequently is the intervention conducted? At what intervals? At what time of day, week, month, year? What is the length and/or duration of each service?

 

 

 

 

 

 

Evaluation Expert Session July 16, 2002 Page 10

YOUR QUESTIONS: WHERE: Where does the intervention occur? What type of facility is used? What is the age and condition of the facility? In what part of town is the facility? Is it accessible to the target audience? Does public transportation access

the facility? Is parking available? Is child care provided on site?

YOUR QUESTIONS: WHY: Why are these activities or strategies implemented and why not others? Why has the intervention varied in ability to maintain interest? Why are clients not participating? Why is the intervention conducted at a certain time or at a certain frequency?

YOUR QUESTIONS:

 

 

 

 

 

Evaluation Expert Session July 16, 2002 Page 11

Validating Your Evaluation Questions

Even though all of your questions may be interesting, it is important to narrow your list to questions that will be particularly helpful to the evaluation and that can be answered given your specific resources, staff, and time.

Go through each of your questions and consider it with respect to the questions below, which may be helpful in streamlining your final list of questions. Revise your worksheet/list of questions until you can answer “yes” to all of these questions. If you cannot answer “yes” to your question, consider omitting the question from your evaluation.

Validation

Yes

No

Will I use the data that will stem from these questions?

 

 

Do I know why each question is important and /or valuable?

 

 

Is someone interested in each of these questions?

 

 

Have I ensured that no questions are omitted that may be important to someone else?

 

 

Is the wording of each question sufficiently clear and unambiguous?

 

 

Do I have a hypothesis about what the “correct” answer will be for each question?

 

 

Is each question specific without inappropriately limiting the scope of the evaluation or probing for a specific response?

 

 

Do they constitute a sufficient set of questions to achieve the purpose(s) of the evaluation?

 

 

Is it feasible to answer the question, given what I know about the resources for evaluation?

 

 

Is each question worth the expense of answering it?

 

 

Derived from “A Design Manual” Checklist, page 51.

 

 

 

 

 

Evaluation Expert Session July 16, 2002 Page 12

Determining Methodology Process evaluation is characterized by collection of data primarily through two formats: 1) Quantitative, archival, recorded data that may be managed by an computerized

tracking or management system, and 2) Qualitative data that may be obtained through a variety of formats, such as

surveys or focus groups.

When considering what methods to use, it is critical to have a thorough understanding and knowledge of the questions you want answered. Your questions will inform your choice of methods. After this section on types of methodologies, you will complete an exercise in which you consider what method of data collection is most appropriate for each question.

Do you have a thorough understanding of your questions?

Furthermore, it is essential to consider what data the organization you are evaluating already has. Data may exist in the form of an existing computerized management information system, records, or a tracking system of some other sort. Using this data may provide the best reflection of what is “going on,” and it will also save you time, money, and energy because you will not have to devise your own data collection method! However, keep in mind that you may have to adapt this data to meet your own needs – you may need to add or replace fields, records, or variables.

What data does your organization already have? Will you need to adapt it?

If the organization does not already have existing data, consider devising a method for the organizational staff to collect their own data. This process will ultimately be helpful for them so that they can continue to self-evaluate, track their activities, and assess progress and change. It will be helpful for the evaluation process because, again, it will save you time, money, and energy that you can better devote towards other aspects of the evaluation. Management information systems will be described more fully in a later section of this workbook.

Do you have the capacity and resources to devise such a system? (You may need to refer to a later section of this workbook before answering.)

 

 

 

 

 

Evaluation Expert Session July 16, 2002 Page 13

Who should collect the data?

 

Given all of this, what thoughts do you have on who should collect data for your evaluation? Program staff, evaluation staff, or some combination?

Program Staff: May collect data from activities such as attendance, demographics, participation, characteristics of participants, dispositions, etc; may conduct intake interviews, note changes regarding service delivery, and monitor program implementation.

Advantages: Cost-efficient, accessible, resourceful, available, time-efficient,

and increased understanding of the program. Disadvantages: May exhibit bias and/or social desirability, may use data for critical

judgment, may compromise the validity of the program; may put staff in uncomfortable or inappropriate position; also, if staff collect data, may have an increased burden and responsibility placed upon them outside of their usual or typical job responsibilities. If you utilize staff for data collection, provide frequent reminders as well as messages of gratitude.

 

Evaluation staff: May collect qualitative information regarding implementation, general characteristics of program participants, and other information that may otherwise be subject to bias or distortion.

Advantages: Data collected in manner consistent with overall goals and timeline

of evaluation; prevents bias and inappropriate use of information; promotes overall fidelity and validity of data.

Disadvantages: May be costly and take extensive time; may require additional

training on part of evaluator; presence of evaluator in organization may be intrusive, inconvenient, or burdensome.

 

 

 

 

 

 

Evaluation Expert Session July 16, 2002 Page 14

When should data be collected?

Conducting the evaluation according to your timeline can be challenging. Consider how much time you have for data collection, and make decisions regarding what to collect and how much based on your timeline. In many cases, outcome evaluation is not considered appropriate until the program has stabilized. However, when conducting a process evaluation, it can be important to start the evaluation at the beginning so that a story may be told regarding how the program was developed, information may be provided on refinements, and program growth and progress may be noted. If you have the luxury of collecting data from the start of the intervention to the end of the intervention, space out data collection as appropriate. If you are evaluating an ongoing intervention that is fairly quick (e.g., an 8-week educational group), you may choose to evaluate one or more “cycles.” How much time do you have to conduct your evaluation? How much time do you have for data collection (as opposed to designing the evaluation, training, organizing and analyzing results, and writing the report?) Is the program you are evaluating time specific? How long does the program or intervention last? At what stages do you think you will most likely collect data?

Soon after a program has begun Descriptive information on program characteristics that will not change; information requiring baseline information During the intervention Ongoing process information such as recruitment, program implementation After the intervention Demographics, attendance ratings, satisfaction ratings

 

 

 

 

 

Evaluation Expert Session July 16, 2002 Page 15

Before you consider methods

A list of various methods follows this section. Before choosing what methods are most appropriate for your evaluation, review the following questions. (Some may already be answered in another section of this workbook.)

What questions do I want answered? (see previous section)

Does the organization already have existing data, and if so, what kind?

Does the organization have staff to collect data?

What data can the organization staff collect?

Must I maintain anonymity (participant is not identified at all) or confidentiality

(participant is identified but responses remain private)? This consideration pertains to existing archival data as well as original data collection.

How much time do I have to conduct the evaluation?

How much money do I have in my budget?

How many evaluation staff do I have to manage the data collection activities?

Can I (and/or members of my evaluation staff) travel on site?

What time of day is best for collecting data? For example, if you plan to conduct

focus groups or interviews, remember that your population may work during the day and need evening times.

 

 

 

 

 

Evaluation Expert Session July 16, 2002 Page 16

Types of methods

A number of different methods exist that can be used to collect process information. Consider each of the following, and check those that you think would be helpful in addressing the specific questions in your evaluation. When “see sample” is indicated, refer to the pages that follow this table.

 

√ Method Description

 

Activity, participation, or client tracking log

Brief record completed on site at frequent intervals by participant or deliverer. May use form developed by evaluator if none previously exists. Examples: sign in log, daily records of food consumption, medication management.

Case Studies Collection of in-depth information regarding small number of intervention recipients; use multiple methods of data collection.

Ethnographic analysis

Obtain in-depth information regarding the experience of the recipient by partaking in the intervention, attending meetings, and talking with delivery staff and recipients.

Expert judgment Convene a panel of experts or conduct individual interviews to obtain their understanding of and reaction to program delivery.

Focus groups Small group discussion among program delivery staff or recipients. Focus on their thoughts and opinions regarding their experiences with the intervention.

Meeting minutes (see sample)

Qualitative information regarding agendas, tasks assigned, and coordination and implementation of the intervention as recorded on a consistent basis.

Observation (see sample)

Observe actual delivery in vivo or on video, record findings using check sheet or make qualitative observations.

 

Open-ended interviews – telephone or in person

Evaluator asks open questions (i.e., who, what, when, where, why, how) to delivery staff or recipients. Use interview protocol without preset response options.

Questionnaire Written survey with structured questions. May administer in individual, group, or mail format. May be anonymous or confidential.

Record review

Obtain indicators from intervention records such patient files, time sheets, telephone logs, registration forms, student charts, sales records, or records specific to the service delivery.

 

Structured interviews – telephone or in person

Interviewer asks direct questions using interview protocol with preset response options.

 

 

 

 

Evaluation Expert Session

July 16, 2002 Page 17

Sample activity log

This is a common process evaluation methodology because it systematically records exactly what is happening during implementation. You may wish to devise a log such as the one below and alter it to meet your specific needs. Consider computerizing such a log for efficiency. Your program may already have existing logs that you can utilize and adapt for your evaluation purposes.

Site:

Recorder:

Code

Service

Date

Location

# People

# Hours

Notes

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

Evaluation Expert Session July 16, 2002

Page 18

Meeting Minutes

Taking notes at meetings may provide extensive and invaluable process information that can later be organized and structured into a comprehensive report. Minutes may be taken by program staff or by the evaluator if necessary. You may find it helpful to use a structured form, such as the one below that is derived from Evaluating Collaboratives, University of Wisconsin-Cooperative Extension, 1998.

Meeting Place: __________________ Start time: ____________ Date: _____________________________ End time: ____________ Attendance (names): Agenda topic: _________________________________________________ Discussion: _____________________________________________________ Decision Related Tasks Who responsible Deadline 1. 2. 3. Agenda topic: _________________________________________________ Discussion: _____________________________________________________ Decision Related Tasks Who responsible Deadline 1. 2. 3. Sample observation log

 

 

 

 

 

Evaluation Expert Session July 16, 2002

Page 19

Observation may occur in various methods, but one of the most common is hand-recording specific details during a small time period. The following is several rows from an observation log utilized during an evaluation examining school classrooms.

CLASSROOM OBSERVATIONS (School Environment Scale) Classroom 1: Grade level _________________ (Goal: 30 minutes of observation)

Time began observation: _________Time ended observation:_________ Subjects were taught during observation period: ___________________

PHYSICAL ENVIRONMENT

Question

Answer 1. Number of students

 

2. Number of adults in room: a. Teachers b. Para-pros c. Parents

Total: a. b. c.

3. Desks/Tables a. Number of Desks b. Number of Tables for students’ use c. Any other furniture/include number (Arrangement of desks/tables/other furniture)

a. b. c.

4. Number of computers, type

 

5. How are computers being used?

 

6. What is the general classroom setup? (are there walls, windows, mirrors, carpet, rugs, cabinets, curtains, etc.)

 

7. Other technology (overhead projector, power point, VCR, etc.)

 

8. Are books and other materials accessible for students?

 

9. Is there adequate space for whole-class instruction?

 

12. What type of lighting is used?

 

13. Are there animals or fish in the room?

 

14. Is there background music playing?

Personality Chapter For General Psychology

Personality Chapter

 

 

Someone insults you and you spend the next two hours imagining the things you could have (or should have) said to humiliate them. According to Freud, you are engaging in free association.

Select one:

True

False

 

The BEST kind of personality test is one that is _______.

Select one:

a. reliable, but not necessarily valid

b. valid, but not necessarily reliable

c. neither reliable nor valid

d. both valid and reliable

 

Jenny is 15 years old. She is capable of mature sexuality, is able to postpone gratification, and handle responsibility. According to Freud, she is in the phallic stage.

Select one:

True

False

 

Johnny is 8 years old. He likes to play with other little boys, but has no interest in playing with little girls. According to Freud, he is in the latency stage.

Select one:

True

False

 

According to Hippocrates, if my temper is out of control, I need to check the balance of my green bile.

Select one:

True

False

 

According to Hippocrates, too much phlegm may make me sluggish and apathetic.

Select one:

True

False

 

Adler called the individual’s attempts to overcome physical weaknesses ________.

Select one:

a. Reciprocity

b. Fictional finalism

c. Compensation

d. Reaction formation

 

The Rorschach personality test relies on a subject telling stories based on cards depicting human figures in various poses?

Select one:

True

False

 

Freud believed that personality is formed around ________ structures.

Select one:

a. five

b. two

c. three

d. four

 

“Relative” is one of the three basic characteristics of personality as presented in your lecture notes.

Select one:

True

False

 

Without using the ego, one way the id can obtain gratification is through the reality principle.

Select one:

True

False

 

For Freud, the seething cauldron of unconscious urges and desires continuously seeking expression is the ego.

Select one:

True

False

 

For both Bandura and Rotter, a person’s expectancies become a critical part of his or her ________.

Select one:

a. explanatory style

b. self-actualizing tendency

c. ideal self

d. persona

 

For Horney, self-actualization is a stronger motivating force than is sex

Select one:

True

False

 

According to your text, personality is stable.

Select one:

True

False

 

Adler felt that some people become so fixated on their feelings of inadequacy that they develop ________.

Select one:

a. neurotic trends

b. an archetype

c. a fixation

d. an inferiority complex

 

According to Freud, a girl’s sexual attachment to her father and jealousy of her mother is called ________.

Select one:

a. the Electra complex

b. the Oedipus complex

c. countertransference

d. transference

 

For Freud, the term “sexual instinct” refers to ________.

Select one:

a. erotic sexuality

b. the personal unconscious

c. any form of pleasure

d. childhood experiences

 

According to Jung, people who base their actions on their perceptions, senses, and intuition are ________.

Select one:

a. introverts

b. irrational individuals

c. extraverts

d. rational individuals

 

Terry Francis, a world famous psychologist, argues that behavior is the end product of a variety of some conscious and mostly unconscious forces interacting within a person’s mind. Terry Francis is most likely a behavioral psychologist

Select one:

True

False

 

Collective memories of experiences people have had in common since prehistoric times, such as mothers, heroes, or villains are called ________ by Carl Jung

Select one:

a. archetypes

b. personas

c. parasymbols

d. animas

 

The best known and most influential psychodynamic theorist is ________.

Select one:

a. Watson

b. Maslow

c. Skinner

d. Freud

 

According to Jung, our repressed thoughts, undeveloped ideas, and forgotten experiences are contained in the collective unconscious.

Select one:

True

False

 

According to Jung, people who regulate actions by thinking and feeling are ________.

Select one:

a. irrational individuals

b. introverts

c. extraverts

d. rational individuals

 

According to Jung, a person who usually focuses on his or her own thoughts and feelings is a(n) introvert.

Select one:

True

False

 

According to Hippocrates, black bile could be responsible for a person’s depression.

Select one:

True

False

 

When an individual’s self-concept is closely matched with his or her inborn capacities, then that person is likely to become what Rogers calls a(n) ________ person.

Select one:

a. fully functioning

b. fully rational

c. actualizing

d. harmoniously integrated

 

Horney believed that sexual factors are the most important factors shaping personality.

Select one:

True

False

 

According to William Seldon, the lactomorphic body type signals an artistic, introverted temperament.

Select one:

True

False

 

Erikson’s stage of initiative versus guilt corresponds approximately with Freud’s ________ stage.

Select one:

a. oral

b. genital

c. genital

d. phallic

Erik is a newborn. According to Erikson, his main task will be to develop a sense of autonomy.

Select one:

True

False

 

Jack is 24 years old. His major task, according to Erikson, is to develop a sense of identity.

Select one:

True

False

 

According to William Seldon, the endomorphic body type is likely to display a relaxed and sociable demeanor

Select one:

True

False

 

In Rogers’s theory, the full acceptance and love of another person regardless of that person’s behavior is called ________.

Select one:

a. Psychological congruence

b. Self-actualization

c. Unconditional positive regard

d. Psychosynthesis

 

According to Rogers, the primary goal of life is to ________.

Select one:

a. understand one’s personal ancestral history

b. harmoniously unify the id, ego, and superego

c. successfully overcome the developmental challenges

d. fulfill one’s inborn capacities and potentialities

 

Gerald takes great pride in his sexual prowess and treats women with contempt. Yet, he feels worthless and has low self-esteem. In Freud’s view, he is probably fixated in the oral stage.

Select one:

True

False

 

The accuracy and usefulness of projective tests depends largely on ________.

Select one:

a. the age of the client

b. the type of disorder being diagnosed

c. whether the client likes taking tests

d. the skill of the examiner

 

According to William Sheldon, the mesomorphic body type indicates a courageous and assertive personality.

Select one:

True

False

 

According to Freud, a boy’s sexual attachment to his mother and jealousy of his father is called ________.

Select one:

a. countertransference

b. transference

c. the Electra complex

d. the Oedipus complex

 

Each of the following is one of the “Big Five” dimensions of personality EXCEPT ________.

Select one:

a. Emotional stability

b. Neuroticism

c. Agreeableness

d. Extraversion

 

According to Freud, the ego operates ________.

Select one:

a. only preconsciously

b. only unconsciously

c. partly consciously, partly preconsciously, and partly unconsciously

d. only unconsciously

 

Jerry has a very immature attitude about sex and sucks his thumb even though he is 32 years old. Freud would say these behaviors result from libido.

Select one:

True

False

 

Each of the following is an advantage of projective tests EXCEPT ________.

Select one:

a. because the tests are flexible, people can take them in a relaxed atmosphere

b. They have higher reliability and validity than objective tests.

c. some psychologists believe that projective tests can uncover unconscious thoughts and fantasies

d. since the person taking the test often doesn’t know its true purpose, responses are less likely to be faked

 

Barney is a person who believes he can control his own fate. He feels that by hard work, skill, and training it is possible to avoid punishments and find rewards. Rotter would say that Barney has a(n) external locus of control.

Select one:

True

False

 

In assessing personality, we are MOST interested in a person’s ________ behavior

Select one:

a. best

b. worst

c. typical

d. atypical

 

Ralph is a private person. He keeps other people at a distance by putting on a public “face” that is nearly the opposite of his inner, true self. Jung called this public “face” worn by Ralph the persona.

Select one:

True

False

 

A study of the “Big Five” dimensions of personality found that ________ were reliable predictors of job performance in sales.

Select one:

a. none of them

b. conscientiousness and extraversion

c. all five of them

d. agreeableness and neuroticism

 

Erikson stresses the quality of ________ in personality development

Select one:

a. stimulus motives

b. sexual motives

c. parent-child relationships

d. primary-process thinking

 

The unconscious is best defined as ideas, thoughts, and feelings of which we are not and cannot normally become aware .

Select one:

True

False

 

According to Bandura, standards people develop in order to rate the adequacy of their own behavior in variety of situations are called ________.

Select one:

a. reciprocal variables

b. conditions of worth

c. performance standards

d. self-efficacy standards

 

The proper chronological order of Freud’s psychosexual stages is ________.

Select one:

a. oral, anal, phallic, latency, genital

b. anal, oral, phallic, latency, genital

c. oral, anal, genital, phallic, latency

d. anal, oral, genital, latency, phallic

 

________ theory emphasizes that humans are positively motivated and progress toward higher levels of functioning.

Select one:

a. Social Learning

b. Humanistic

c. Psychoanalytic

d. Trait

 

The unique pattern of characteristic thoughts, feelings, and behaviors that persists over time and situations is called ________.

Select one:

a. personality

b. a habit

c. learning

d. a trait

 

Horney disagreed with many of Freud’s ideas, but particularly those regarding ________.

Select one:

a. his emphasis on unconscious processes affecting conscious functioning

b. the importance of early childhood in shaping adult personality

c. the importance of defense mechanisms in protecting the ego from anxiety.

d. his analysis of women and his emphasis on sexual instincts

 

Audrey is a normal, healthy two-year-old. Her main task, according to Erikson, will be to develop a sense of autonomy.

Select one:

True

False

 

Erikson suggested that success in each of the life stages he outlined depends upon adjustment during the previous stage .

Select one:

True

False

 

According to Bandura, the expectancy that one’s efforts will be successful is called self-esteem.

Select one:

True

False

 

In Rogers’s theory, the full acceptance and love of another person regardless of that person’s behavior is called archetypes.

True

False

 

According to Hippocrates, a surplus of blood may cause confusion and anxiety.

True

False

2

 

 

1

 

1

 

1

 

1

 

1

 

1

 

1

 

1

 

1

 

1

 

1

 

1

 

1

 

1

 

1

 

1

 

1

 

1

 

1

 

1

 

1

 

1

Cognitive psychology and its implications

Write a 1,050- to 1,200-word instruction paper on the processes involved with attaining expertise, reference the chapter in your text titled, “Expertise”. Anderson, J.R. (2009). Cognitive psychology and its implications (7th Ed.). New York, NY: Worth Publishers

 

Include the following salient points in your work:

1. Outline the stages in the development of expertise.

2. Outline the dimensions involved in the development of expertise.

3. Discuss how obtaining skills makes changes to the brain

4. EXAMPLE OF PAPER BELOW DO NOT COPY Plag FREE COPY ONLY

The Nature of Expertise

So far in this chapter, we have considered some of the phenomena associated

with skill acquisition. An understanding of the mechanisms behind these phenomena

has come from examining the nature of expertise in various fields of

endeavor. Since the mid-1970s, there has been a great deal of research looking

at expertise in such domains as mathematics, chess, computer programming,

and physics. This research compares people at various levels of development of

their expertise. Sometimes this research is truly longitudinal and follows students

from their introduction to a field to their development of some expertise.

More typically, such research samples people at different levels of expertise. For

instance, research on medical expertise might look at students just beginning

medical school, residents, and doctors with many years of medical practice.

This research has begun to identify some of the ways that problem solving

becomes more effective with experience. Let us consider some of these dimensions

of the development of expertise.

 

 

 

 

.

Tactical Learning

As students practice problems, they come to learn the sequences of actions

required to solve a problem or parts of the problem. Learning to execute such

sequences of actions is called tactical learning. A tactic refers to a method that

accomplishes a particular goal. For instance, Greeno (1974) found that it took

only about four repetitions of the hobbits and orcs problem (see discussion

surrounding Figure 8.7) before participants could solve the problem perfectly.

In this experiment, participants were learning the sequence of moves to get the

creatures across the river. Once they had learned the sequence, they could simply

recall it and did not have to figure it out.

Logan (1988) argued that a general mechanism of skill acquisition involves

learning to recall solutions to problems that formerly had to be figured out. A

nice illustration of this mechanism is from a domain called alpha-arithmetic. It

entails solving problems such as _ 3, in which the participant is supposed to

say the letter that is the number of letters forward in the alphabet—in this case,

_ 3 _ I. Logan and Klapp (1991) performed an

experiment in which they gave participants problems

that included addends from 2 (e.g., _ 2) through 5

(e.g., _ 5). Figure 9.9 shows the time taken by participants

to answer these problems initially and then

after 12 sessions of practice. Initially, participants

took 1.5 s longer on the 5-addend problems than on

the 2-addend problems, because it takes longer to

count five letters forward in the alphabet than two

letters forward. However, the problems were repeated

again and again across the sessions. With repeated,

continued practice, participants became faster on all

problems, reaching the point where they could solve

the 5-addend problems as quickly as the 2-addend

problems. They had memorized the answers to these

problems and were not going through the procedure

of solving the problems by counting.1

There is evidence that, as people become more

practiced at a task and shift from computation to

retrieval, brain activation shifts from the prefrontal

cortex to more posterior areas of the cortex. For

instance, Jenkins, Brooks, Nixon, Frackowiak, and

Passingham (1994) looked at participants learning to key out various sequences

of finger presses such as “ring, index, middle, little, middle, index, ring, index.”

They compared participants initially learning these sequences with participants

practiced in these sequences. They used PET imaging studies and found that

there was more activation in frontal areas early in learning than late in learning.2

On the other hand, later in learning, there was more activation in the hippocampus,

which is a structure associated with memory. Such results indicate that, early

in a task, there is significant involvement of the anterior cingulate in organizing

the behavior but that, late in learning, participants are just recalling the answers

from memory. Thus, these neurophysiological data are consistent with Logan’s

proposal.

Tactical learning refers to a process by which people learn specific procedures

for solving specific problems.

Strategic Learning

The preceding subsection on tactical learning was concerned with how students

learn tactics by memorizing sequences of actions to solve problems. Many small

problems repeat so often that we can solve them this way. However, large and

complex problems do not repeat exactly, but they still have

similar structures, and one can learn how to organize one’s

solution to the overall problem. Learning how to organize

one’s problem solving to capitalize on the general structure of

a class of problems is referred to as strategic learning. The

contrast between strategic and tactical learning in skill acquisition

is analogous to the distinction between tactics and strategy

in the military. In the military, tactics refers to smaller-scale

battlefield maneuvers, whereas strategy refers to higher-level

organization of a military campaign. Similarly, tactical learning

involves learning new pieces of skill, whereas strategic learning

is concerned with putting them together.

One of the clearest demonstrations of such strategic changes is in the domain

of physics problem solving. Researchers have compared novice and expert solutions

to problems like the one depicted in Figure 9.10. A block is sliding down an

inclined plane of length l, and u is the angle between the plane and the horizontal.

The coefficient of friction is m. The participant’s task is to find the velocity of the

block when it reaches the bottom of the plane. The typical novices in these studies

are beginning college students and the typical experts are their teachers.

In one study comparing novices and experts, Larkin (1981) found a difference

in how they approached the problem.

The novice’s solution typifies the reasoning backward method, which starts with

the unknown—in this case, the velocity v. Then the novice finds an equation for

calculating v. However, to calculate by this equation, it is necessary to calculate a,

the acceleration. So the novice finds an equation for calculating a; and the novice

chains backward until a set of equations is found for solving the problem.

The expert, on the other hand, uses similar equations but in the completely

opposite order. The expert starts with quantities that can be directly computed,

such as gravitational force, and works toward the desired velocity. It is also apparent

that the expert is speaking a bit like the physics teacher that he is, leaving

the final substitutions for the student.

Another study by Priest and Lindsay (1992) failed to find a difference in

problem-solving direction between novices and experts. Their study included

British university students rather than American students, and they found that

both novices and experts predominantly reasoned forward. However, their

experts were much more successful in doing so. Priest and Lindsay suggest that

the experts have the necessary experience to know which forward inferences are

appropriate for a problem. It seems that novices have two choices—reason forward,

but fail (Priest & Lindsay’s students) or reason backward, which is hard

(Larkin’s students)

Reasoning backward is hard because it requires setting goals and subgoals

and keeping track of them. For instance, a student must remember that he

or she is calculating so that can be calculated and hence so that can be

calculated. Thus, reasoning backward puts a severe strain on working memory

and this can lead to errors. Reasoning forward eliminates the need to keep

track of subgoals.

 

However, to successfully reason forward, one must know

which of the many possible forward inferences are relevant to the final solution,

which is what an expert learns with experience. He or she learns to associate

various inferences with various patterns of features in the problems. The

novices in Larkin’s study seemed to prefer to struggle with backward reasoning,

whereas the novices in Priest and Lindsay’s study tried forward reasoning

without success.

Not all domains show this advantage for forward problem solving. A good counterexample is computer programming (Anderson, Farrell, & Sauers, 1984; Jeffries, Turner, Polson, & Atwood, 1981; Rist, 1989). Both novice and expert programmers develop programs in what is called a top-down manner; that is, they

work from the statement of the problem to sub problems to sub-sub problems, and so on, until they solve the problem. This top-down development is basically the same as what is called reasoning backward in the context of geometry or physics. There are differences between expert programmers and novice programmers, however. Experts tend to develop problem solutions breadth first, whereas novices develop their solutions depth first. Physics and geometry problems have a rich set of givens that are more predictive of solutions than is the goal. In contrast, nothing in the typical statement of a programming

problem would guide a working forward or bottom-up solution. The typical problem statement only describes the goal and often does so with information that will guide a top-down solution. Thus, we see that expertise in different domains requires the adoption of those approaches that will be successful for

those particular domains. In summary, the transition from novices to experts does not entail the same

changes in strategy in all domains. Different problem domains have different structures that make different strategies optimal. Physics experts learn to reason forward; programming experts learn breadth-first expansion. Strategic learning refers to a process by which people learn to organize their

problem solving.

Problem Perception

As they acquire expertise problem solvers learn to perceive problems in ways

that enable more effective problem-solving procedures to apply. This dimension

can be nicely demonstrated in the domain of physics. Physics, being an intellectually

deep subject, has principles that are only implicit in the surface features

of a physics problem. Experts learn to see these implicit principles and represent

problems in terms of them.

Chi, Feltovich, and Glaser (1981) asked participants to classify a large set of

problems into similar categories. Figure 9.11 shows sets of problems that

novices thought were similar and the novices’ explanations for the similarity

groupings. As can be seen, the novices chose surface features, such as rotations

or inclined planes, as their bases for classification. Being a physics novice myself,

I have to admit that these seem very intuitive bases for similarity. Contrast

The Nature of Expertise | 255

Anderson7e_Chapter_09.qxd 8/20/09 9:49 AM Page 255

these classifications with the pairs of problems in Figure 9.12 that the expert

participants saw as similar. Problems that are completely different on the

surface were seen as similar because they both entailed conservation of energy

or they both used Newton’s second law. Thus, experts have the ability to map

surface features of a problem onto these deeper principles. This ability is very

useful because the deeper principles are more predictive of the method of

solution. This shift in classification from reliance on simple features to reliance

on more complex features has been found in a number of domains, including

mathematics (Silver, 1979; Schoenfeld & Herrmann, 1982), computer

programming (Weiser & Shertz, 1983), and medical diagnosis (Lesgold et al.,

1988).

A good example of this shift in processing of perceptual features is the interpretation

of X rays. Figure 9.13 is a schematic of one of the X rays diagnosed by

participants in the research by Lesgold et al. The sail-like area in the right lung is a

shadow (shown on the left side of the X ray) caused by a collapsed lobe of the

lung that created a denser shadow in the X ray than did other parts of the lung.

Medical students interpreted this shadow as an indication of a tumor because tumors

are the most common cause of shadows on the lung. Radiological experts,

on the other hand, were able to correctly interpret the shadow as an indication of

a collapsed lung. They saw counterindicative features such as the size of the saillike

region. Thus, experts no longer have a simple association between shadows

on the lungs and tumors, but rather can see a richer set of features in X rays.

An important dimension of growing expertise is the ability to learn to perceive problems in ways that enable more effective problem-solving procedures to apply.

 

Pattern Learning and Memory

A surprising discovery about expertise is that experts seem to display a special enhanced

memory for information about problems in their domains of expertise.

This enhanced memory was first discovered in the research of de Groot (1965,

1966), who was attempting to determine what separated master chess players from

weaker chess players. It turns out that chess masters are not particularly more

intelligent in domains other than chess. De Groot found hardly any differences between

expert players and weaker players—except, of course, that the expert players

chose much better moves. For instance, a chess master considers about the same

number of possible moves as does a weak chess player before selecting a move. In

fact, if anything, masters consider fewer moves than do chess duffers.

However, de Groot did find one intriguing difference between masters and weaker players.He presented chess masters with chess positions (i.e., chessboards with pieces in a configuration that occurred in a game) for just 5 s and then removed the chess pieces. The chess masters were able to reconstruct the positions of more than 20 pieces after just 5 s of study. In contrast, the chess duffers could

reconstruct only 4 or 5 pieces—an amount much more in line with the traditional capacity of working memory. Chess masters appear to have built up patterns of 4 or 5 pieces that correspond to common board configurations as a result of the massive amount of experience that they have had with chess.

Thus, they remember not individual pieces but these patterns. In line with this analysis, if the players are presented with random chessboard positions rather than ones that are actually encountered in games, no difference is demonstrated between masters and duffers—both reconstruct only a few chess positions. The masters also complain about being very uncomfortable and disturbed by such chaotic board positions.

In a systematic analysis, Chase and Simon (1973) compared novices, Class A players, and masters.

and to reproduce random positions such as those illustrated in Figure 9.14b. Figure 9.15

shows the results. Memory was poorer for all groups for the random positions and, if anything, masters were worse at reproducing these positions. On the other hand, masters showed a considerable advantage for the actual board positions. This basic phenomenon of superior expert memory for meaningful problems has been demonstrated in a large number of domains, including the game of Go

(Reitman, 1976), electronic circuit diagrams (Egan & Schwartz, 1979), bridge hands (Engle

& Bukstel, 1978; Charness, 1979), and computer programming (McKeithen, Reitman,

Rueter, & Hirtle, 1981; Schneiderman, 1976).

Chase and Simon (1973) also used a

chessboard-reproduction task to examine the

nature of the patterns, or chunks, used by

chess masters. The participants’ task was simply to reproduce the positions of

pieces of a target chessboard on a test chessboard. In this task, participants

glanced at the target board, placed some pieces on the test board, glanced back

to the target board, placed some more pieces on the test board, and so on.

Chase and Simon defined a chunk to be a group of pieces that participants

moved after one glance. They found that these chunks tended to define

meaningful game relations among the pieces. For instance, more than half of

the masters’ chunks were pawn chains (configurations of pawns that occur

frequently in chess).

Simon and Gilmartin (1973) estimated that chess masters have acquired

50,000 different chess patterns, that they can quickly recognize such patterns on

a chessboard, and that this ability is what underlies their superior memory performance

in chess. This 50,000 figure is not unreasonable when one considers

the years of dedicated study that becoming a chess master requires.What might

be the relation between memory for so many chess patterns and superior performance

in chess? Newell and Simon (1972) speculated that, in addition to

learning many patterns, masters have learned what to do in the presence of

such patterns. For instance, if the chunk pattern is symptomatic of a weak side,

the response might be to suggest an attack on the weak side. Thus, masters

effectively “see” possibilities for moves; they do not have to think them out,

which explains why chess masters do so well at lightning chess, in which they

have only a few seconds to move.

To summarize, chess experts have stored the solutions to many problems

that duffers must solve as novel problems. Duffers have to analyze different

configurations, try to figure out their consequences, and act accordingly.

Masters have all this information stored in memory, thereby claiming two

advantages. First, they do not risk making errors in solving these problems,

because they have stored the correct solution. Second, because they have stored

correct analyses of so many positions, they can focus their problem-solving efforts

on more sophisticated aspects and strategies of chess. Thus, the experts’

pattern learning and better memory for board positions is a part of the tactical

learning discussed earlier. The way humans become expert at chess reflects the

fact that we are very good at pattern recognition but relatively poor at things

like mentally searching through sequences of possible moves. As the Implications

box describes, human strengths and weaknesses lead to a very different

way of achieving expertise at chess than we see in computer programs for playing

chess.

260 | Expertise

chess in the 1960s, was beaten by the program of an

MIT undergraduate, Richard Greenblatt, in 1966 (Boden,

2006, discusses the intrigue surrounding

these events). However, Dreyfus was a

chess duffer and the programs of the

1960s and 1970s performed poorly

against chess masters. As computers

became more powerful and could search

larger spaces, they became increasingly

competitive, and finally in May 1997,

IBM’s Deep Blue program defeated the

reigning world champion, Gary Kasparov.

Deep Blue evaluated 200 million imagined

chess positions per second. It also

had stored records of 4,000 opening

positions and 700,000 master games

(Hsu, 2002) and had many other optimizations

that took advantage of special computer hardware.

Today there are freely available chess programs

for your personal computer that can be downloaded

over the Web and will play highly competitive chess at

a master level. These developments have led to a profound

shift in the understanding of intelligence. It once

was thought that there was only one way to achieve

high levels of intelligent behavior, and that was the

human way. Nowadays it is increasingly being accepted

that intelligence can be achieved in different ways, and

the human way may not always be the best. Also, curiously,

as a consequence some researchers no longer

view the ability to play chess as a reflection of the

essence of human intelligence.

Implications

Computers achieve computer expertise differently than humans

In Chapter 8, we discussed how human problem solving

can be viewed as a search of a problem space, consisting

of various states. The initial situation

is the start state, the situations on the

way to the goal are the intermediate

states, and the solution is the goal state.

Chapter 8 also described how people

use certain methods, such as avoiding

backup, difference reduction, and meansends

analysis, to move through the

states. Often when humans search a

problem space, they are actually manipulating

the actual physical world, as in

the 8-puzzle (Figures 8.3 and 8.4).

However, sometimes they imagine states,

as when one plays chess and contemplates

how an opponent will react to

some move one is considering, how one might react to

the opponent’s move, and so on. Computers are very

effective at representing such hypothetical states and

searching through them for the optimal goal state.

Artificial intelligence algorithms have been developed

that are very successful at all sorts of problem-solving

applications, including playing chess. This has led to a

style of chess playing program that is very different from

human chess play, which relies much more on pattern

recognition. At first many people thought that, although

such computer programs could play competent and

modestly competitive chess games, they would be no

match for the best human players. The philosopher

Hubert Dreyfus, who was famously critical of computer

Anderson7e_Chapter_09.qxd 8/20/09 9:49 AM Page 260

Experts can recognize patterns of elements that repeat in many problems,

and know what to do in the presence of such patterns without having to

think them through.

Long-Term Memory and Expertise

One might think that the memory advantage shown by experts is just a workingmemory

advantage, but research has shown that their advantage extends to

long-term memory. Charness (1976) compared experts’ memory for chess positions

immediately after they had viewed the positions or after a 30-s delay filled

with an interfering task. Class A chess players showed no loss in recall over the

30-s interval, unlike weaker participants, who showed a great deal of forgetting.

Thus, expert chess players, unlike duffers, have an increased capacity to store

information about the domain. Interestingly, these participants showed the

same poor memory for three-letter trigrams as do ordinary participants. Thus,

their increased long-term memory is only for the domain of expertise.

There is reason to believe that the memory advantage goes beyond experts’

ability to encode a problem in terms of familiar patterns. Experts appear to be

able to remember more patterns as well as larger patterns. For instance, Chase

and Simon (1973) in their study (see Figures 9.14 and 9.15) tried to identify the

patterns that their participants used to recall the chessboards. They found that

participants would tend to recall a pattern, pause, recall another pattern, pause,

and so on. They found that they could use a 2-s pause to identify boundaries

between patterns.With this objective definition of what a pattern is, they could

then explore how many patterns were recalled and how large these patterns

were. In comparing a master chess player with a beginner, they found large

differences in both measures. First, the pattern size of the master averaged

3.8 pieces, whereas it was only 2.4 for the beginner. Second, the master also

recalled an average of 7.7 patterns per board, whereas the beginner recalled an average of only

5.3. Thus, it seems that the experts’ memory advantage is based not only on larger patterns but

also on the ability to recall more of them.

The strongest evidence that expertise requires

the ability to remember more patterns as well as

larger patterns is from Chase and Ericsson (1982),

who studied the development of a simple but

remarkable skill. They watched a participant, S. F.,

increase his digit span, which is the number of

digits that he could repeat after one presentation.

As discussed in Chapter 6, the normal digit span is

about 7 or 8 items, just enough to accommodate a

telephone number. After about 200 hr of practice,

S. F. was able to recall 81 random digits presented

at the rate of 1 digit per second.

As people become more expert in a domain, they develop a better ability

to store problem information in long-term memory and to retrieve it.

 

The Role of Deliberate Practice

An implication of all the research that we have reviewed is that expertise comes

only with an investment of a great deal of time to learn the patterns, the problemsolving

rules, and the appropriate problem-solving organization for a domain.

As mentioned earlier, John Hayes found that geniuses in various fields produce

their best work only after 10 years of apprenticeship in a field. In another

research effort, Ericsson, Krampe, and Tesch-Römer (1993) compared the best

violinists at a music academy in Berlin with those who were only very good.

They looked at diaries and self-estimates to determine how much the two

populations had practiced and estimated that the best violinists had practiced

more than 7000 hr before coming to the academy, whereas the very good had

practiced only 5000 hr. Ericsson et al. reviewed a great many fields where, like

music, time spent practicing is critical. Not only is time on task important at

the highest levels of performance, but also it is essential to mastering school

subjects. For instance, Anderson, Reder, and Simon (1998) noted that a major

reason for the higher achievement in mathematics of students in Asian countries

is that those students spend twice as much time practicing mathematics.

Ericsson et al. (1993) make the strong claim that almost all of expertise is to

be accounted for by amount of practice, and there is virtually no role for natural

talent. They point to the research of Bloom (1985a, 1985b), who looked at the

histories of children who became great in fields such as music or tennis. Bloom

found that most of these children got started by playing around, but after a short

time they typically showed promise and were encouraged by their parents to

start serious training with a teacher. However, the early natural abilities of these

children were surprisingly modest and did not predict ultimate success in the

domain (Ericsson et al., 1993). Rather, what is critical seems to be that parents

come to believe that a child is talented and consequently pay for their child’s

instruction and equipment as well as support their time-consuming practice.

262 | Expertise

Anderson7e_Chapter_09.qxd 8/20/09 9:49 AM Page 262

Ericsson et al. speculated that the resulting training is sufficient to account for

the development of children’s success. There is almost certainly some role for

talent (considered in Chapter 13), but all the evidence indicates that genius is

90% perspiration and 10% inspiration.

Ericsson et al. are careful to note, however, that not all practice leads to the

development of expertise. They note that many people spend a lifetime playing

chess or some sport without ever getting any better.What is critical, according

to Ericsson et al., is what they call deliberate practice. In deliberate practice,

learners are motivated to learn, not just perform; they are given feedback on

their performance; and they carefully monitor how well their performance

corresponds to the correct performance and where the deviations exist. The

learners focus on eliminating these points of discrepancy. The importance of

deliberate practice is similar to the importance of deep and elaborative processing

of the to-be-learned material described in Chapters 6 and 7, in which

passive study was shown to yield few memory benefits.

An important function of deliberate practice in both children and adults

may be to drive the neural growth that is necessary to enable expertise. It had

once been thought that adults do not grow new neurons, but it now appears

that they do (Gross, 2000). An interesting recent discovery is that extensive

practice appears to drive neural growth in the adult brain. For instance, Elbert,

Pantev,Wienbruch, Rockstroh, and Taub (1995) found that violinists, who finger

strings with the left hand, show increased development of the right cortical

regions that correspond to their fingers. In another study already mentioned

in Chapter 4, Maguire et al. (2003) used imaging to examine the brains of

London taxi drivers. It takes at least 3 years for London taxi drivers to acquire

all of the knowledge necessary to navigate expertly through the streets of

London. The taxi drivers were found to have significantly more gray matter in

the hippocampal region than did matched controls. This finding corresponds to

the increased hippocampal volume reported in small mammals and birds that

engage in behavior requiring navigation (Lee, Miyasato, & Clayton, 1998). For

instance, food-storing birds show seasonal increases in hippocampal volume

corresponding to times of the year when they need to remember where they

store food.

A great deal of deliberate practice is necessary to develop expertise in any

field.

•Transfer of Skill

Expertise can often be quite narrow. As noted, Chase and Ericsson’s participant

S. F. was unable to transfer memory span skill from digits to letters. This example

is an almost ridiculous extreme of a frequent pattern in the development

of cognitive skills—that these skills can be quite narrow and fail to transfer

to other activities. Chess grand masters do not appear to be better thinkers

for all their genius in chess. An amusing example of the narrowness of expertise

Transfer of Skill | 263

Anderson7e_Chapter_09.qxd 8/20/09 9:49 AM Page 263

is aTransfer of Skill

Expertise can often be quite narrow. As noted, Chase and Ericsson’s participant

S. F. was unable to transfer memory span skill from digits to letters. This example

is an almost ridiculous extreme of a frequent pattern in the development

of cognitive skills—that these skills can be quite narrow and fail to transfer

to other activities. Chess grand masters do not appear to be better thinkers

for all their genius in chess. An amusing example of the narrowness of expertise

is a study by Carraher, Carraher, and Schliemann (1985). These researchers

investigated the mathematical strategies used by Brazilian schoolchildren who

also worked as street vendors. On the job, these children used quite sophisticated

strategies for calculating the total cost of orders consisting of different

numbers of different objects (e.g., the total cost of 4 coconuts and 12 lemons);

what’s more, they could perform such calculations reliably in their heads.

Carraher et al. actually went to the trouble of going to the streets and posing as

customers for these children, making certain kinds of purchases and recording

the percentage of correct calculations. The experimenters then asked the children

to come with them to the laboratory, where they were given written mathematics

tests that included the same numbers and mathematical operations that

they had manipulated successfully in the streets. For example, if a child had

correctly calculated the total cost of 5 lemons at 35 cruzeiros apiece on the

street, the child was given the following written problem:

5 _ 35 _ ?

Whereas children solved 98% of the problems presented in the real-world context,

they solved only 37% of the problems presented in the laboratory context.

It should be stressed that these problems included the exact same numbers and

mathematical operations. Interestingly, if the problems were stated in the form

of word problems in the laboratory, performance improved to 74%. This improvement

runs counter to the usual finding, which is that word problems are

more difficult than equivalent “number” problems (Carpenter & Moser, 1982).

Apparently, the additional context provided by the word problem allowed the

children to make contact with their pragmatic strategies.

The study of Carraher et al. showed a curious failure of expertise to transfer

from real life to the classroom, but the typical concern of educators is whether

what is taught in one class will transfer to other classes and the real world.

Early in the 20th century, educators were fairly optimistic on this matter. A

number of educational psychologists subscribed to what has been called the

doctrine of formal discipline (Angell, 1908; Pillsbury, 1908; Woodrow, 1927),

which held that studying such esoteric subjects as Latin and geometry was of

significant value because it served to discipline the mind. Formal discipline

subscribed to the faculty view of mind, which extends back to Aristotle and

was first formalized by Thomas Reid in the late 18th century (Boring, 1950).

The faculty position held that the mind is composed of a collection of general

faculties, such as observation, attention, discrimination, and reasoning, which

were exercised in much the same way as a set of muscles. The content of the

exercise made little difference; most important was the level of exertion (hence

the fondness for Latin and geometry). Transfer in such a view is broad and

takes place at a general level, sometimes spanning domains that have no content

in common.

Although it might be nice to believe that such general transfer is possible,

as envisioned by the doctrine of formal discipline, there has been effectively

no evidence for it, despite a century of research on the topic. Some of the

earliest research on this topic was performed by Thorndike (e.g., Thorndike &

Woodworth, 1901). In one study, no correlation was found between memory

264 | Expertise

Anderson7e_Chapter_09.qxd 8/20/09 9:49 AM Page 264

for words and memory for numbers. In another, accuracy in spelling was not

correlated with accuracy in arithmetic. Thorndike interpreted these results as

evidence against the general faculties of memory and accuracy.

There is often failure to transfer skills to similar domains and virtu

 

 

 

 

 

 

 

Attaining Expertise

 

Running head: ATTAINING EXPERTISE

1

 

 

 

ATTAINING EXPERTISE

16

 

 

 

Example DO NOT COPY THIS PAPER JUST a format example

Attaining Expertise

Encyclopedias describe an Expert as “one who is very skillful and well-informed in any given domain field, one who has acquired special skill in or knowledge about a particular subject through professional training and practical experience” (Webster’s, 1976, p. 800). It is said that humans are the only species to acquire such knowledge in attaining expertise (Anderson, 2009). Expertise refers to the skills, knowledge and characteristics needed to distinguish experts from novices and people with less experience. The process of attaining expertise begins with learning the fundamental skills needed in any particular domain, while learning to apply sound principles of that domain. Another part of processing is to apply critical thinking skills to find solutions to complex problems or situations. Before a person can excel in a particular domain they must first be willing to learn and absorb information. “There is a relatively widespread conception that if individuals are innately talented, they can easily and rapidly achieve an exceptional level of performance once they have acquired basic skills and knowledge” (Ericcon, Krampe, & Tesch-Romer, 1993, p.366). Outline the stages in the development of expertise.

Stages in the development of expertise

Acquisition is the act of acquiring, learning or developing of a skill. There are three stages of acquisition, cognitive, associative, and autonomous. In acquisition people are able to experience, learn, and refine motor skills. These skills greatly affect a person’s ability to perform in any physical activity. The three stages of acquisition discuss the processes that people undertake when learning a new skill. In addition, the processes from learning these skills can be adapted by individuals to help learn these types of skills more easily and quickly. These three stages allow the learner to gain a better understanding of a skill and what they learn. This also helps to improve an individual’s ability to execute what skill they have learned as they progress through each stage of acquisition.

The cognitive stage allows individuals to identify and understand the skill that is to be learned. In this stage individuals rehearse, and encode a set facts relevant to the skill into their memory (Anderson, 2009, pg. 244.). Individuals in this stage begin to watch, think, analyze, reason, judge, and visualize rather than practice. Individuals in this stage develop an in depth understanding of the acquired skill.

The second stage is the associative stage, in this stage two things happen. Any errors that occur in the initial understanding are detected in small degrees and eliminated (Anderson, 2009, pg. 244.). This stage is a hands on sort of stage, individuals practice this increases the ability to perform better at the skill or task. Meaning that the skills the individual learns are strengthened, this allows them to understand how to do the skill or task at hand.

The third stage is the autonomous stage. This stage focuses on skill acquisition that revolves around the individual executing the skill or task automatically without having to think about how to do it. The concept of automaticity is how central cognition drops out of performance of a task as we become more skilled at it (Anderson, 2009, pg. 245.). This stage allows the individual to perform the skill fluently, instinctively, and quickly and outside influence do not affect the outcome of the skill or task.

Dimensions involved in the development of expertise

Simon and Chase (1973) conducted research on the expertise of chess, they observed that nobody had attained the level of an international chess master (grandmaster) “with less than about a decade’s intense preparation with the game” (p. 402). Simon and chase estimated that the time it took a chess player to fully gain expertise is comparable in size to the vocabulary of an adult native speaker of English (Simon & Chase, 1973). It takes normal people approximately ten years or more to acquire this vocabulary. Kroguis (1976) stated that the time between chess players’ first learning the rules of chess and attaining international chess master status was 11.7 years for those who learned chess rules late (after age 11) and even longer for those who started early, that is, 16.5 years. Many researches such as J.R. Hayes (1981) also determined that in order to attained expertise in any domain of performance one must have at least ten or more years of experience. Reviews in research has also shown that the maximal level of performance for individuals in a given domain is not attained automatically as function of extended experience, but the level of performance can be increased even by highly experienced individuals as a result of deliberate efforts to improve (Ericsson, et. al. 1993).

How obtaining skills makes changes to the brain

The trainees can refine, experience, and learn his or her motor skills. His or her motor skills will strongly affect his or her ability to perform in any activity. In today’s society, industries come across challenges due to higher cost, worldly competition, and demands of the environment (Ericcson & Lehmann, 1996). I will explain to the trainees that the new problem in modern industries demands fast answers and problem-solving. When he or she are trained to attain expertise, he or she will understand it is important to ask his or herself how he or she can improve the success of his or her environment. By attaining expertise, individuals within an environment or workplace can face issues, offer support, plan efficiently, and optimize continuously (Ericcson & Lehmann, 1996).

When the trainees are attaining expertise, he or she can respond to his or her requirement quickly and gain the trust of customers, co-workers, and other in society. After the trainees attain expertise, he or she will be reliable and competent in the field he or she are a part. Attaining expertise allows an extensive portfolio to be offered to others showing services and solutions that are available and trustworthy. The expertise attained can be proven in global markets by the showing completed projects (Ericcson & Lehmann, 1996). When I train the individuals to achieve expertise, he or she can help productivity to be optimized and become highly available. The expertise the trainees obtain will allow him or her to possess security within his or herself and knowledge to offer to others.

 

Conclusion

Movie Worksheet

INTRODUCTION to “AWAKENINGS”

from www.filmeducaton.org/pdf/film/awakenings.pdf

 

In the winter of 1916-1917, an epidemic of a rare disease occurred, springing up, as virus

diseases sometimes do, seemingly out of nowhere. It spread over Europe and then to other

parts of the world and affected some five million people. The onset of the disease was sudden

and took different forms. Some people developed acute restlessness or insomnia or

dementia. Others fell into a trance-like sleep or coma. These different forms were recognised

and identified by the physician Constantin von Economo as one disease, which he called

encephalitis lethargica, or sleepy sickness.

 

Many people died of the disease. Of those who survived, some recovered completely. The

majority remained partly disabled, prone to symptoms reminiscent of Parkinson’s disease.

The worst affected sank into a kind of ‘sleep’, unable to move or speak, without any will of

their own, or hope, but conscious and with their memories intact. They were placed in

hospitals or asylums. Ten years after the epidemic had begun, it just as remarkably

disappeared. Fifty years later, the epidemic had been forgotten.

 

In 1966, when Dr. Oliver Sacks, a neurologist trained in London, took up his post at Mount

Carmel, a hospital in New York, he found there a group of eighty people who were the

forgotten survivors of the forgotten epidemic. It was clear that hundreds of thousands had

died in institutions. Dr. Sacks called them ‘the lepers of the present century’. In his book,

‘Awakenings’, he tells of his attempts to understand the nature of their affliction, but also of his

growing appreciation of them as individuals, with their own unique histories and experience.

 

In 1969, Dr. Sacks tried out a remarkable new drug, L-DOPA. For some of his patients, there

then followed a rapid and brief return to something like normality. They were suddenly

restored to the world of the late nineteen sixties. His book documents this remarkable

awakening, as experienced by twenty of his patients. L-DOPA was not, however, the magic

cure that it first seemed. The normality that it promoted soon broke down, with patients

subject to all kinds of bizarre behaviours.

 

In the film of ‘Awakenings’, Robert de Niro plays Leonard Lowe, someone affected by sleepy

sickness as a young man. He is in a state of near sleep, unable to move or speak. Every day,

his mother comes into hospital to care for him, as she has for many years. Robin Williams

plays Dr. Malcolm Sayer, the neurologist who, like Dr. Sacks himself in 1966, takes up a post

at a New York hospital, discovering there the forgotten survivors of the sleepy sickness

epidemic. He finds himself drawn to this group of chronically disabled people, and especially

to Leonard.

 

Robert de Niro’s Leonard is based on the Leonard L. who Sacks describes in his book – an

intelligent and courageous man with a wry sense of humour, who is able only to communicate

in a very limited way, using a letter board. Sacks says how thoroughly De Niro

prepared himself for his role, spending a great deal of time with post-encephalitic patients in

an effort to understand something of how it feels to be so chronically disabled, and to

represent as accurately as possible the quality of if disablement.

 

In the film, we are shown Leonard’s awakening under L-DOPA. Leonard sees the world to

which he has awoken truly wonderful. He has lost many years of his life. Now he wants to

live. He wants his independence. Briefly, we see him determined to achieve this before his

damaged nervous system pulls him back into a catatonic state.

 

 

In the book ‘Awakenings’, Dr. Sacks writes that Leonard says to him after the last futile trial of

another drug:

“Now I accept the whole situation. It was wonderful, terrible, dramatic and comic. It is finally –

sad, and that’s all there is to it. I’ve learned a great deal in the last three years. I’ve broken

through barriers which I had all life. And now, I’ll stay myself and you can keep your L-DOPA.”

 

A note about sleepy sickness:

Encephalitis lethargica (sleepy sickness, or sleeping sickness, as it is called in the U.S.A.) is

caused by a virus attacks the brain. In particular, it attacks a part of the mid-brain – the

substantia nigra – damaging the nerve cells this area and severely reducing their ability to

produce the chemical nerve impulse transmitter, dopamine. In respect, the disease is similar

to Parkinson’s disease. The cerebral cortex (the part of the brain concerned with conscious

awareness, thought and memory) is unaffected. When in the early 1960’s a substance (LDOPA) closely related to dopamine was found to alleviate the symptoms of Parkinson’s

disease, there was the hope that it would do the same for post-encephalitic patients, that is,

people suffering from the after-effects of sleepy sickness. In event, the effect of L-DOPA on

such people was variable and unpredictable. For some, except for a brief return something

close to normality, it was a failure. For others, its effects were beneficial over a longer period,

and for a few, there was a return to a long lasting near normality. The drug raised enormous

expectations in those who been worst affected by sleepy sickness, who for thirty or forty years

had been in a kind of catatonic sleep. Tragically, for some of them, their awakening was all

too brief

 

 

 

 

Leonard’s poem:

 

THE PANTHER by Rainer Maria Rilke (1875-2926)

 

His vision, from the constantly passing bars,

has grown so weary that it cannot hold

anything else. It seems to him there are

a thousand bars; and behind the bars, no world.

 

As he paces in cramped circles, over and over,

the movement of his powerful soft strides

is like a ritual dance around a centre

in which a mighty will stands paralysed.

 

Only at times, the curtain of the pupils

lifts, quietly -. An image enters in,

rushes down through the tensed, arrested muscles,

plunges into the heart and is gone.