Determining the Evaluation Model
7.d. Discuss chapter 5 in the course text and chapters 2 & 6 at the Internet site. What was the same with the two readings? Different? New knowledge? Insight into evaluations?
Chapter 5 of the Boulmetis & Dutwin text, Choosing an Evaluation Model, was a critical and informative piece of this book on evaluation. The differential between evaluation and research was very helpful. Research draws conclusions from observation through the scientific method; the purpose is to expand the knowledge base about a particular topic. An evaluation goes through a systematic process to reports results about whether objectives are being met to stakeholders. This process includes questions, design, collecting and analyzing data, drawing conclusions, making decisions, and reporting. Several evaluation models were discussed. Of particular interest to me were the Decision Making Model and the Art Criticism Model. The Decision Making Model by Stufflebeam, 1983, evaluates a program in order to make decisions about the existence and activities of the program in the future. It is concerned with the long-range view. It can use qualitative or quantitive data, and is best-suited to summative evaluations. The Art Criticism Model by Eisner, 1997, stipulates that the evaluator must be a qualified subject expert in a field relevant to what is being evaluated. The expert evaluator then conducts a critical review of the program. This is often used as preparation for accreditation or other critical review processes.
The Mixed Methods described in Chapter 2 and 6 of the User Friendly Handbook for Mixed Method Evaluation seemed to state that it is preferred that the evaluator use both qualitative and quantitative methods in an evaluation design. There were also many worksheets provided which I thought provided concrete ways to proceed with an evaluation such as developing evaluation questions. The Mixed Method differs from the text in preferring both qualitative and quantitative while the Boulmetis & Dutwin text have so far remained neutral on this issue. Given that my evaluation project will collect only quantitative data, I do not plan to proceed further with the Mixed Method approach.
7.e Use your new understanding about evaluations to address this question: Which evaluation model from chapter 5 would you choose for your own Evaluation Report-Course Project?
I must admit to feeling drawn to two evaluation models while reading through chapter 5 in the Boulmetis & Dutwin text. I thought that my evaluation project could benefit from either the Decision Making Model or the Art Criticism Model.
The Decision Making Model by Stufflebeam, 1983, specifies that the ultimate goal of the evaluation is to help the stakeholders make decisions about the program – either its very existence or future activities to improve meeting objectives. While it may provide data about how the program is meeting objectives currently, the focus is on the future. Summative evaluations are often best used in this model as the evaluator tries to decide how effective the program was in meeting goals. I feel that this model could be very helpful as I determine how effective the honors program is in providing information literacy to honors students. I hope that stakeholders would be provided with adequate data to make decisions about the future information literacy inclusion such as using liaison librarian expertise within the program.
On the other hand, the Art Criticism Model could also be a useful model when considering my project. This Model employs an evaluator who is a qualified subject expert in the topic of evaluation. The evaluator conducts a critical and objective review of the program. It is often used to prepare for accreditation status. Because I am an expert in information literacy, I am qualified to evaluate the information literacy in the Honors Program through the research projects of senior students. Objectivity will be easier to maintain due to the quantitative nature of the data. One purpose of this evaluation is to provide data to Administrative stakeholders who are working on a self-study for the accreditation process next year. Therefore, this Model is very applicable to my specific situation.
I have not yet decided which Model fits “best” and am uncertain that this decision is completely necessary. Could two models provide useful guidelines for an evaluation?