Program Evaluation Review

Here we are in 2012 and I have a new batch of courses that I’m taking.  I just finished up Instructional Design and Designing for Distance Education.  My next two courses are Program Evaluation and Advanced Instructional Desgin.  Completeion of these two courses will put me over the 1/2 way mark in my Master’s program.  That’s hard to believe!  This first blog entry (in a LONG time) is a program evaluation review.  Over the coming months, I’ll be posting assignments here on my blog.  I encourage comments!

J.T.

Review of “The Brandon Middle Years Pilot Project Program Evaluation” (BMYPPPE)

The BMYPPPE was completed by David Patton, Ph D. and Jackie Lemaire B. Sc in August of 2002.  It is an evaluation of a Pilot Program taking place in two Brandon Junior High Schools whose goal it was to increase the knowledge of grade 7 and 8 students in alcohol and drug use, and to see if increased knowledge would change students’ decision making skills.  I found the report after a Google search of “middle years” and “program evaluation”.   Since middle years are my area of interest, I wanted to find a report that dealt with that target group.

What I liked about the Report:

The report was well organized and properly formatted.  It was official and professional.  It followed the general program evaluation format of the title page, table of contents, introduction, research results, summary, and appendices.  There were no grammatical errors and the findings were reported at an understandable language level.

Findings were presented in graphs.  The findings of each evaluated criteria were presented as text first, with a graph to unify the data.  The graphs were clear of any superfluous data.  They were well sized (they did not take up an entire page) and communicated the findings clearly.

The data was analyzed and compartmentalized according to specific criteria.  The program evaluation took into account the differences in:

  •         The two middle schools observed
  •          Gender differences
  •          Difference between Grade 7 and Grade 8 students
  •         Differences (in use and opinion) between alcohol and drug use

 

I liked how the program was evaluated using different question techniques.

  •         Questions on access to drugs and alcohol were asked using a Likert Scale.
  •         Questions on frequency of drug and alcohol use were asked using a Likert Scale.
  •         Students were also asked short answer and open ended questions when it came to questions about personal feelings.  I like this technique as it allows the respondent to speak as an individual and there are no ‘guiding’ questions or prompts.
  •         Questions regarding specific knowledge pertaining to the knowledge from the Pilot Project (such as the addictiveness of cigarettes, or the legality of certain drugs) were asked using True/False.  I liked how they also had a third option of “I don’t know” to allow for students to be honest in their knowledge.  This is something that I will take with me when I conduct my own program evaluation.

Here is what I did not like about the report:

I found that the report did not address ethnic diversity – cultural backgrounds or the socio-economic backgrounds of the students.  This could have been addressed in the initial introduction where a ‘snapshot’ of the two schools involved in the survey could have been given.  I was left asking questions such as, “Have more schools taken place in this pilot project?”, “Why were these two schools chosen?”, “Is there an existing drugs and alcohol problem which this pilot project was designed to address?”, and “Is there a senior high follow up to this pilot project?”.

Although I like the use of the Likert Scale, I was left wondering if questions regarding attitudes (and changing attitudes) would have been better addressed in a short answer format.  This led me to another question/idea of wondering if there was room in the evaluation to have completed some pre and post interviews with students.  This may not have been a possibility because of the sensitive nature of subject matter and the possibility exists of unauthentic responses in an in-person interview at this age.

J.T.

Here is a link to the Program Evaluation

http://bit.ly/xiEomK

Advertisements

One thought on “Program Evaluation Review

  1. Interesting choice Jeff. Glad to see that you chose a topic at the same grade level in which you work. Thanks for including a link to the document. I agree with the good variety of evaluation tools. Yes the report looked professional as well. Use of graphs and the idea of including an executive summary are useful to include. What I have the biggest concern about is that the report makes no recommendations. It only shares the outcomes and the effectiveness. For this reason I would not consider it a complete report. The authors provide no feedback to those running the program regarding next steps or true effectiveness of the endeavour.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s