Skills Training å la Carte...Outcomes Assessment Driving Change?

Inside Higher Education

August 12, 2009

I ran across this article a few month’s ago and as I near completion of this course I felt a need to revisit the article from an assessment frame of mind. Kellogg Community College made the change from traditional credential-specific educational model to a skills-specific model—as one commenter coined the concept a “skills-centric” model.

The article delineates the case of outcomes-based assessment being driven by the needs of local industry. These needs are changing how skills development is delivered by a local community college. Kellogg Community College’s implementation of a skills training concept has totally shut down traditional classroom instruction in favor of a module-based system breaking down training into 1200 individual skill sets. Modules are sold to students or their sponsoring employer in fractions of a credit hour. An example was cited of one module on learning how to read a micrometer being worth .13 credit hours..

As a former business owner, these types of skills-based module trainings would have been much more palatable cost-wise as a pay-as-you-need training system. The executive manager of a tool manufacturer stated “The major difference is the Kellogg students did work focused on particular kinds of equipment, so that the things they learned in class they could apply immediately at work the next day”.

The next few years could be interesting if this type of model becomes the norm in community college training programs.

Assessment Article and Great Website

Many Colleges Assess Learning but May Not Use Data to Improve, Survey Finds
Chronicle – July 15, 2009
http://chronicle.com/article/Many-Colleges-Assess-Learni/47892

This discussion is based on how colleges are using the assessment for mainly accreditation purposes and not how on improving student learning.
The article discusses how colleges are doing more assessment on student learning, although they are not using the data gathered for improvement in teaching and learning. 80% of 1500 colleges surveyed found that the assessment gathered was often only used for accreditation purposes. The most common type of tool used to assess learning in the colleges were surveys to alumni, employers, and students. Ikenberry also found that there is strong resistance from faculty to measure student learning.
There is a website that I found when I was researching for my paper. This website has an enormous amount of information regarding program evaluation in Higher Education. I thought that I would share this website with everyone. http://www2.acs.ncsu.edu/UPA/assmt/resource.htm#hbooks
There are assessment rubrics, assessment resources, and examples of assessment to support faculty and departmental projects. This is a great resource as we move into our own career paths and begin looking for assessment ideas. Enjoy
Sheri

Union and Activities Professional Skills & Educational Needs

The Bulletin of the Association of College Unions International, January 2010. Report offers insights into union and activities professionals' skills and educational needs.

This article gives a report on the "first ever College Unions and Student Activities Professional Competency Assessment." The online survey was sent to over two thousand professionals with 585 responding.

The purpose of this study is to understand the knowledge base of union professionals. The data will be used to guide campus training efforts, graduate curricula and ACUI programs.

The survey consisted of three questions and respondents self-reported. The article highlights each questions and how respondents ranked. The demographic data also provided great insight in this area.

I found this article very exciting as the area I work in is taking strides in assessment. The research from this first survey will provide guidelines in our professional arena and I believe great opportunities for growth. I am very happy there was one article in our trade journal about assessment.

How's This For Closing The Loop?

(2010, February 24). Teachers fired for poor performance. State of Ohio Education. Retrieved on February 27, 2010 from http://www.stateofohioeducation.com/2010/02/teachers-fired-for-schools-poor.html

The school board at Central Falls High School, Rhode Island supported the superintendent’s recommendation and voted (5-2) to dismiss the entire faculty and staff members, over 100 individuals, due to chronic poor performance on Tuesday, February 23, 2010! This is the only public school in this small town, which is just west of the Massachusetts border. Central Falls is one of the lowest-achieving
high schools in the province and had a four-year graduation rate of 48%.

Evidently, when the emotional board meeting ended, the board went into closed session and board members were not available to comment on the issue. The plan also included a new structure for school governance, requiring all new high school teachers to participate in professional development activities aimed at meeting the federal standards.

Negotiations have been ongoing between Superintendent, Dr. Francis Gallo and the teachers’ union. Dr. Gallo stated that she chose a “turnaround” plan, one of four offered by the state, after the union rejected previously presented conditions calling for increased hours without salary increases. According to Dr. Gallo, “Union leadership went too far because I would not commit to monetary incentives.” Teachers and union leaders are claiming the Dr. Gallo and the board had not “bargained with them in good faith”.

This is an extreme example of closing the loop! Assessment and evaluation definitely made an impact on individual faculty and staff performance. It should be interesting to see how this story plays out in history.

Carla

College Teaching Needs to Get Better Now

The Chronicle of Higher Education, February 5, 2010. College Teaching Needs to Get Better Now, Educators Warn by David Glenn.

The author discusses the emphasis of the White House to increase the proportion of Americans who can earn college degrees and the impact this may have on higher education. This topic was the major focus of at the annual conferenc of the Association of American Colleges and Universities.

As stated in this article, "Effective assessment is critical to ensure that our colleges and universities are delivering the kinds of educational experiences that we believe we can actually provide to the students, said Richard Crutcher, president of Wheaton College." I believe that is the nuts and bolts of higher education and assessment.

The author pointed out how colleges may sidestep obstacles and motivating faculty. I thought this was especially interesting. We hear about faculty buy in weekly during our presentations and it appears to be a continuous theme.

I would believe that faculty would continue to improve their teaching on a regular basis and not become complacent in their teaching. The author touched on this briefly and stated that faculty members should not wait for some drastic restructuring before they try to improve their teaching.
I believe this article is relevant to all colleges and universities. We must provide a quality education and well worth the cost.

Academic Authority and accreditation

In the Chronicle of Higher Education, Date: February 25, 2010, Title: At a Tiny College, an Epic Battle Over Academic Authority By Don Troop wrote about the happenings of a small college and its struggles. The primary issue was the mission statement and its change and the power struggle on campus.. What I found interesting is how this fits in with accreditation. If you read this entry, nowhere does it indicate the college, specifically its leadership, does it indicate any input from the community and its constituents. I am wondering if there was any discussion on this. It is directly inline, or in this case out of line, with criterion 1 and how the institution relates to its mission.
Take a look at this article. I wonder what type of faculty buy in for assessment the faculty will or would have? Reading this from the outside is very interesting, on many fronts. Accreditation is not even discussed, but it is something I thought of when I read through it.
Another thing I wonder is how can an institution of higher education operate freely when someone donates large quantities of money to it. Would the donor not expect some sort of say in what happens on campus. {um UND???} Or wouldn't the college feel obligated to listen? I hope I never have to go through something like this. Happy reading.

The Quality Imparative

I found this report inters ting because it stresses that learning outcomes should be in line with what employers are looking for. Basically, we should a line the outcomes with the most important skills that employers say graduates are lacking. According to this report the two largest areas that employers would like more emphasis on are written and oral communication (89%) and critical thinking and analytical skills (81%).

I was surprised to see that science and technology (70%) was not one of the top two with the emphasis that has been on the STEM area. I was also a little disappointed that critical thinking was an area that was perceived to need more attention. To me that is the point of higher education..teach students to think critically regardless of the discipline.

The other part of the report that I connected with is that education should be global regardless of if it is a technical program our a PhD, and by only training job related skills we are not setting up students and ultimately our country to compete in the global environment we live in.

Data is Only Data, Unless...

The National Survey of Student Engagement (NSSE) aims to provide institutions with an understanding of "how undergraduates spend their time and what they gain from attending college." In October 2009 a group of college presidents, NSSE program leaders, and assessment experts gathered to discuss the widely used instrument ten years after its inception. The group agreed the NSSE collects valuable data regarding student behavior and student learning. The Wabash Study of Liberal Arts Education offered evidence confirming the validity of the instrument's findings. However, despite the positive support, there was concern that institutions are not utilizing NSSE results. In EDUC 721, I believe we have referred to this element of assessment as "closing the loop".

The theoretical side of me believes that there isn't any point to collecting data unless it is analyzed and then utilized to implement institutional improvements. However, the practical side of me, the one that recently received a 200+ page report from Concordia's participation in a national study, the Multi-Institutional Study of Leadership (MSL), understands the reality of this situation.

For the majority of us with careers in higher education, getting everything done in a day is a challenge in itself. We have multiple demands on what never seems like enough time in the day, the week, the year, etc... Some of us may be fortunate enough to have access to an institutional research and assessment office who will take on this project. But for the rest of us, finding time to analyze data all too often takes a back seat to the multiple deadlines and frequent "fires" that our daily work life entails. Implementing changes suggested by data, even if it is great data, is not something that we do not want to do, but something that we simply do not have time to do.

In the October discussion mentioned earlier by this esteemed group of colleagues, they "joked about how they would like to require colleges that participate (in the NSSE) to file reports on what they actually do with the data, and that they be required to do something." Perhaps some of us would benefit from a deadline to help us prioritize data analysis and utilization into our already busy lives, but shouldn't it be up to us to discern creative methods of holding ourselves accountable to utilizing the data? After all, without "closing the loop" in this manner, we have wasted the time, energy, and funding spent to participate in national studies such as the NSSE. I believe the solutions are out there, we just have to find and utilize them. After all, data is simply data, unless we take the time to put it to use.

Inside Higher Ed article about the NSSE discussion: http://www.insidehighered.com/layout/set/print/news/2009/10/26/nsse

National Survey of Student involvement: http://nsse.iub.edu/html/about.cfm

Inside Higher Ed--Community College Dean Blog

I read this daily, I enjoy his perspective on a number of things. His discussion on Monday related to assessment, and what it means, and the reasons why it's done came from a pragmatic perspective--which I enjoyed.

As schools in North Dakota continue to market themselves in order to garner the perspective students to join our campus', they may begin to see assessment data as more of a marketing tool rather than just a means to indicate that the students are learning. With parents who are concerned about their investment in their children's education, using assessment data to set one university apart from another certainly has a marketing flair.

What is Authentic Assessment?

"Assessment is authentic when we directly examine student performance on worthy intellectual tasks." (Wiggins, 1990)

The case for authentic assessment, left me wondering whether or not common assessment techniques in my classroom suffice. There appears to be an argument about ways to assess, no matter how you do it. In mathematics, I can teach "how to learn mathematics" to students, or I can teach, "how to learn", or I can simply teach, "mathematics" to my students. How I assess those students are going to be the deciding factor on whether or not they learned what I taught them...but is this good enough. If I can measure that learning has occured, isn't this what HLC has asked of me and my institution? If I can map this learning to a goal or objective of my program, I have satisfied the conditions set before me.

As does rubrics get student engaged in what it is that is expected of them in a particular assignment or project, a test review will also get a student involved...providing them with what is expected of them on the exam. For mathematics, I can see the difficulty in using authentic assessment versus traditional, but it would sure be interesting to see it in action.

http://www.educationoasis.com/curricuolum/assessment/case_authentic_assessment.htm

New OCED Project Compares Student Learning

New OECD Project will compare student learning across countries. Labi. A., February 5, 2010, Chronicle of Higher Education, Vol. 56. No. 21.

This was a very interesting article about comparing student learning across countries. The study, Assessment of Higher Education Learning Outcomes, is overseen by the Organisation for Economic Co-operation and Development.

Richard Yelland is leading the project and indicates the first phase of the project will be to develop learning measures across diverse institutions, languages and cultures. A this time the OECD is adapting the Collegiate Larning Assessment to measure skills such as problem solving, critical thinking and practical applicaiton of theory.

I found this really interesting as North Dakota State University continues to grow in the global community.

How Students Can Improve by Studying Themselves

How Students Can Improve by Studying Themselves. Glenn, D., Feruary 12, 2010, Chronicle of Higher Education, Vol. 56, No. 22.

This article sparked my interested because we study student learning and how that is achieved.
The premise of this article is based on students freshman students failing an introductory math course at City University in New York's mathematics entrance test. If students want to attend, they must pass this course.

Several instructors at the New York City College of Technology believe they have found an effective way for students to succeed with "self regulated learning." This type of learning includes a series of steps which encourage the student to evaluate how they study and identify areas where they go wrong.

The author explains the whole learning model and how it was developed throughout the article. The points that stuck with me are when the author states, "If you just talk about study habits, students resist that naturally... Once the course content was tied in, there was much less resistance. And I think that's key."

What particularly struck me about this is that we talk about faculty buy-in for assessment and the process of accreditation, however, I believe students are a key component of the process as well. I know faculty have many assessment pieces built into their courses as well as the final evaluation, but this article sparked my thoughts to student buy in and participation in pieces of assessment.

I just found this article very interesting and generated some reflective thinking on my part about how we can move forward, particularly, within our Department and Leadership programs to obtain that crucial student buy-in.

Supporting Classroom Assessment Practice: Lessons From a Small High School

I came across an article that I usually would overlook based on the focus and the population it relates to however it looked at assessment and the difficulty in getting instructor buy-in so I thought I would look it over. The article ‘Supporting Classroom Assessment Practice: Lessons From a Small High School” by David Allen, Suzanna Wichterle Ort, and Joseph Schmidt looked at the importance of using formative and summative assessment in the classroom and linked these concepts to course goals.

The authors of the paper took on a research project in which they worked with a Harlem Social Studies teacher who was reluctant to conduct assessment in the classroom giving traditional reasons of resistance including the time factor and the amount of work he felt he needed to cover. The teacher also indicated that he had tried assessment techniques in the past and had limited results indicating that students could not stay on track long enough to give him solid for amative assessment data.

The researchers worked with the teacher to develop a written assignment in which the students could meet the goals of the course and the instructor would receive feedback on the information presented in the course and the benefit, or lack of benefit, the students found in it.
The researchers reported that goals, criteria, support, process and product, and feedback to teachers and students were all strengthened through the assessment process.

Program Evaluation: A Varity of Rigorous Methods Can Help identify Effective Interventions

“Program Evaluation: A Variety of Rigorous Methods Can Help Identify Effective Interventions” United States Government Accountability Office. http://www.eric.ed.gov.proxy.library.ndsu.edu/ERICDocs/data/ericdocs2sql/content_storage_01/0000019b/80/45/f1/1c.pdfThe report seeks to provide assessment and evaluation regarding 63 human service programs relating to early education, public health, and human services.

What I found interesting about the article wasn’t what was determined in regards to what works and what doesn’t but the way in which that went about making those determinations. The method utilized for this study was a quasi-experimental design in which some programs were randomly assigned to experimental and control groups. Level of significance was set (value unknown0 and programs were essentially weighed against each other. The document that reports the findings was fairly thorough and complete however I think the information provided based on methodology is fairly useless.

From my perspective, using a quasi-experimental design to assess and evaluate programs may work for legislators but means very little to the program developers. Without qualitative data coupled with the quantitative data I find myself questioning what the report is representing. This is not the manufacturing of widgets or conducting experiments in the laboratory. Education and human service is only as strong as the understanding that comes with HEARING and understanding people’s experiences.

I found myself smiling and shaking my head when the report talked about attempts t limit other influencing variables. The whole notion of evidence based practice is merely the medical model revisited. We cannot quantify and assign value to every aspect of being human and in doing so we run the risk of turning human beings into simplistic, non-feeling research subjects.

In Academic Culture, Mental-Health Problems are Hard to Recognize and Hard to Treat

I came across an article, as I’m sure others have, that appeared in the Chronicle of Higher Education entitled In Academic Culture, Mental-Health Problems Are Hard to Recognize and Hard to Treat by Jennifer Ruark. The article focused on the shooting deaths of “three biology professors at the University of Alabama at Huntsville.”

The article lays out the multiple aspects that relate to the difficulty of the universities burden of assessing the mental health of its employees. I thought that he article was interesting because the focus was the difficulty of assessment and the burden falls on those who are not trained in this type of assessment.

The premise was that the university system needs to step up its procedures relating to the identification and mandated support of those struggling with mental health issues. The article identified that those who work in higher education experience more stress than their private sector peers. What I found interesting was that the author wrote of the responsibility falling on the department heads to assess the health of the individuals in the department.

I found it interesting that the world of higher education feels that it can assess an individual’s health without the necessary tools and training. Issues such as academic freedom and misunderstood brilliancy complicate an already delicate situation. The article made me uncomfortable to say the least however, I struggle to identify a better option.

Did you learn anything?

Did You Learn Anything? By: Lang, James M., Chronicle of Higher Education, 00095982, 3/9/2007, Vol. 53, Issue 27

I found this column to be timely as I am in the process of collecting midterm evaluations. The author uses assessment techniques by Angelo and Cross to enhance midterm data collection.

According to Lang, the most important thing we can ask students about their experiences in a course is did you learn anything? Like many faculty members, the author expressed frustration with student ratings of teaching, for three reasons:

• Many ratings forms ask questions that are irrelevant to identifying effective teaching, that push particular pedagogical agendas, or that may help create bias.
• Many administrators rely too heavily on the ratings. They let the forms do the work for them, rather than letting the forms serve as part of a package of methods for evaluating a teacher, including classroom observations and analyses of written instructional materials.
• The ratings forms come at the end of the semester, when it's too late to use the results to improve the course. That means you get summative judgments rather than constructive criticism.

The author recommends two possible methods for improving midterm assessment from your students, the first of which comes from Classroom Assessment Techniques: A Handbook for College Teachers, by Thomas A. Angelo and K. Patricia Cross (Jossey-
Bass, 1993). This book offers a host of methods that can be used to gather information from students. Their most well-known method is the "Minute Paper."

To use the Minute Paper," Angelo and Cross write, "an instructor stops class two or three minutes early and asks students to respond briefly to some variation on the following two questions: 'What was the most important thing you learned during this class?' and 'What important question remains unanswered?' Students then write their responses on index cards or half-sheets of scrap paper … and hand them in."

That technique, as the authors explain, provides "manageable amounts of timely and useful feedback for a minimal investment of time and energy." If every student mentions some trivial but entertaining point from your lecture as the most important thing they learned, you know you need to revisit the main idea. If a significant number of students list as an unanswered question one that you covered during a lecture, it's time to review.

Angelo and Cross describe the Minute Paper as an instrument designed to give feedback on a single course session, but it can be used to gauge student opinion on an entire unit or the course as a whole.

In any case, the student responses take just a few minutes to read and will help an instructor see whether the ideas, concepts, and skills you are teaching correspond with what the students are learning.

The second method, the author suggests to gather feedback from students is to take 10or 15 minutes at the end of a class and administer a survey with some version of these two questions: "What classroom activities or assignments have been most effective in helping you learn this semester, and why?" and "What classroom activities or assignments have been least effective in helping you learn this semester, and why?" Ask students to respond anonymously and write a paragraph for each question.

The author emphasizes the importance of explaining, both in writing and in class, that as an instructor you are interested in seeing what you can do that will help everyone learn as much as possible in the final weeks of the semester, and that the exercise will work only if they give honest responses.

Upon receipt of their responses, the author writes to pay attention to similar comments that come from multiple students. If 20 students say they are not getting much out of the group work, this area of the course should be re-evaluated.

The author also recommends that you discuss the results of your midterm survey with students, and find at least one or two ways to modify your course in order to show them that you actually listened to what they had to say.

The author closes by reiterating that it is your job as a teacher is to help your students learn. Who is more qualified than those very same students to tell you whether or not you are succeeding? The students may not have the qualifications to tell you what you should do in the classroom, but they can certainly tell you whether what you have been doing is helping them learn.

"With that information at your disposal, you are well equipped to determine whether you need to make changes in your course. Without that information, you're like an expedition leader who has a clear map to her destination, but who never looks behind to see whether anyone's following."

Embedded Assessment

Cummings R, Maddux CD, and Richamond A. (2008) Curriculum-embedded performance assessment in higher education: maximum efficiency and minimum disruption. Assessment & Evaluation in Higher Education, 22: 6, 599-605.

According to Cummings et al. institutions of higher education must develop more powerful learning environments that integrate instruction with assessment. Students then would be exposed to meaningful learning tasks requiring them to construct knowledge rather than reproduce information that others have discovered. Corresponding assessment activities would be directly related to the learning task and would tap higher level thinking and problem solving abilities. Additionally, they would measure the extent to which students can apply knowledge so solve real world problems and make appropriate decision.

The purpose of performance assessment practices should be to provide a more comprehensive picture of students' learning over the duration of their program of study and to evaluate overall program effectiveness. In spite of the advantage of performance assessment, faculty reactions to its implementation are frequently negative. The most common concerns are that such assessment practices will increase workloads, reduce time for scholarly activists, eliminate professional autonomy and reduce faculty work into component parts or discrete technical competences.

Curriculum embedded performance assessment integrates performance assessment requirements within individual courses. Curriculum embedded performance assessment tasks consist of any representation of student work that demonstrates mastery of course content and attainment of course and program learning goals. These may include written examination, essays, research papers, presentations, online bulletin board activities, or group projects. Student work on curriculum embedded performance assessment tasks is assessed as part of course requirements, but also can be used for portfolio assessment and program review.

The author lists advantages of using curriculum embedded performance assessment within individual courses as a component of the overall performance assessment process including: students are actively involved in the assessment process, assessment activities are part of regularly course requirements so that no additional faculty time is required for data collection, and data can be collected from students' assignments in a specific course that can be used for both individual student evaluation and assessment of overall program effectiveness.

Assessment and Accreditation—Luck of the Draw?

I was skimming through Inside Higher Education ran across an article by Herman Berliner on Feb. 14 titled Assessment. Berliner is an ABA reaccreditation team member and his musings from an assessment team member’s perspective hit a nerve with me.

Berliner states:

“I have a very high regard for the peer review process. Almost without exception, the teams I have served on and the teams that have visited Hofstra have been as diligent and objective as a group of individuals can be. Yes, on rare occasion I have had the sense that a visiting team has been overly picky. And yes, on rare occasion I have felt that a visiting team should have asked more questions.

From my recent service as well as my earlier service, I have also sometimes had a sense that the schools with the most prestigious reputations, at times, get a pass because reviewers feel that their overall reputation justifies overlooking some individual standards.”

It is well known that an institution’s success in a review hinges on the luck of the draw, i.e., the mix of accreditation team members assigned to the visit. Our institution faces an upcoming HLC visit. There are a number of us directly involved with the masters’ program who are concerned in regards to the inherent “research atmosphere” component of our accreditation process. We are a young institution when it comes to research as our master’s program is in its infancy. But, when you go back to the last visit and our institutions commitments to research and the research climate, we are a bit nervous. We can’t rely on our overall reputation in that regard—so it appears we are down to the luck of the draw. I hope we win the hand when we go “all-in”.

Do Universities Focus on Education or Business?

Kelderman, E. (February 17, 2010). Public opinion of higher education continues downward slide. The Chronicle. Retrieved on Febuary 19, 2010 from http://chronicle.com/article/Public-Opinion-of-Higher/64217/?sid=at&utm_source=at&utm_medium=en

This article presented facts about the declining public opinion of higher education, but I found the posted responses most enlightening and interesting. I think this is such an important conversation for us to have in higher education. The article and responses support the need for universities to place a higher focus on producing a quality education (teaching and learning) at public universities as opposed to what is perceived by the public to be the current focus – the financial well-being of the institution. I would take this a step further and challenge that faculty face more pressure to excel in research, grant writing, and producing publications than in teaching. This debate emphasizes the need to constantly assess how valuable teaching and students learning is in academia.

Kelderman quotes “a nationwide poll conducted in December found that 60 percent of respondents believed colleges are ‘like most businesses and mainly care about the bottom line’ compared with 32 percent who said colleges are mostly interested in ‘making sure students have a good educational experience’ ”. The report resulted from a national telephone interview of 1,000 adults was released by the National Center for Public Policy and Higher Education and Public Agenda, which is a nonpartisan, nonprofit policy-research organization. Evidently, the public ‘s view of higher education has been declining at a steady rate as the public perceives that colleges are not doing enough to keep costs down for students and provide a high quality education. At the same time, 55% of respondents agreed that “a college education is necessary for a person be successful in today’s work world”. This data is presented at a precarious time in history since colleges are facing the difficult challenge of maintaining their state appropriated funding.

As a faculty member at NDSU, I read this article with some skepticism and much dismay. But it was the responses that really fired me up. I absolutely agree that quality teaching needs to be more highly acknowledged and rewarded in both the tenure and nontenure track faculty positions. After all, the university leaders take every opportunity to boast our claim that “students are paramount”. If this is true, our highest priority should be teaching. In all fairness, I have seen a greater emphasis on teaching at this university over the past ten years. The reality is that universities are expected now to operate more like a business in securing and managing budgets and funding. At the same time, as NDSU has been vesting so much energy in progressing to Division I and Carnegie Institution status, there has been more and more pressure on faculty and departments to do research, obtain grants/ funding, and publish. It has been an exhilarating yet exhausting adventure. I also find the influence of the media on public perception extremely frustrating and think professionals in the media need to be challenged to be more accountable for the news that is presented.

Below are some excerpts from this conversation. I apologize for taking so many … I just thought it was fascinating.

“It is the administrators who have pushed the idea of the university as a "business" and the faculty as "employees" whom the administration "manages." Teaching at the real university level has suffered greatly from non-academic stuff such as teacher evaluations and the focus on "workloads," to cite just two examples. How this trend can be undone remains a mystery, at least to me.”

“Of course, this is all the fault of the big bad administrators. Where I come from, faculty treat tenure as a Constitutional right while millions of Americans are un or underemployed, and their contract stipulates that they get paid more money every time they're asked to lift a finger. Are there grotesquely overpaid presidents and coaches who contribute to this problem more than they do to education? There sure are, and something has to be done about that. But there are also grotesquely overpaid faculty members whose research and consulting come before improving any student's experience on campus.”
“The irony is that for decades pol
icymakers have sought to defund higher education and told it to act more like a business. They want the university to serve the market rather than students-- hence Bayh-Dole. Now that the universities have turned into businesses, the people finally realize that they were wrong to get what they asked for.”

“Higher education is suffering in large part because primary and secondary public education is failing. Students don't do homework in high school or if they do, they get an "A" just for turning it in regardless of quality (I've heard this many times from students). Parents don't help them with homework (or can't, for various reasons), and the culture as a whole encourages them to focus on getting rich quick (football, rap star, reality TV star), rather than on finding meaningful work, understanding the global world we live in, watching and reading a variety of news sources, and personal responsibility. Higher education should address its image problem by producing more evidence of what we are facing. For instance, if we were able to publish anonymously the kinds of work we get from our incoming freshmen the first week, might that raise awareness about the problems that originate earlier in the education chain? Of course, this probably can't happen for legal reasons, but it's the kind of evidence that might have some serious impact. Higher ed always thinks of marketing in terms of bright shiny images of sports and labs, but it should also think about sending the message about how it is serving the basic education needs of what is, frankly, a semi-literate population.”

Higher education is suffering in large part because primary and secondary public education is failing. Students don't do homework in high school or if they do, they get an "A" just for turning it in regardless of quality (I've heard this many times from students). Parents don't help them with homework (or can't, for various reasons), and the culture as a whole encourages them to focus on getting rich quick (football, rap star, reality TV star), rather than on finding meaningful work, understanding the global world we live in, watching and reading a variety of news sources, and personal responsibility.

“For years I've been told that my university is a "business" and my students are "customers." I've even had students argue that, since they pay for their classes, they shouldn't have to attend them--the customer, after all, is always right. But the real fallout from the business model is a definition of faculty productivity measured in student-contact hours. In this model, the faculty member who lectures to 200 students and gives machine-graded tests is more "productive" than the one who gives individualized attention to 25 students and reads and comments on their papers. And research isn't an issue where I work--we're supposed to be a teaching institution. It's the corporate model that's ruined higher education.”

“… when an institution measures faculty productivity in terms of student contact-hours. There are of course other ways to measure faculty productivity. My preference is to measure it in terms of measuring gains in student learning (the value-added model)….. Universities vary in the key metrics that they hold sacred. Higher education needs better metrics for faculty productivity, whether in a teaching or a research institution.”

“The reason the public is becoming disillusioned with higher education is not because of the vagaries of college governance and the tenure system, but because higher education has turned itself inward. In research institutions, faculty barricade themselves into their labs while administrators create layers of process to protect the academic environment from the chaos that is the "real world". College presidents extoll the virtues of their "public mission" while provosts speak about "public scholarship" while the public waits to see what will actually come of it all… From the public's perspective, the only thing that comes out of colleges right now are undergraduates who succeed despite the inattention of faculty and incompetence of TAs, and are now struggling to meet the debts imposed by the excesses of the pet programs of administrators… If my experience is in fact representative, then our whole industry needs to examine several questions. Are these perceptions are in fact reality, and if so, how do we change to correct that particular problem? If it is truly a perception problem, then how do we communicate the truth of the situation? … Ultimately, the choice to maintain status quo is not an option. When society views an institution as not meeting its needs, that institution looses support and eventually faces away.”

Carla

Alignment: $268.97

So you think this is easy.

My VP for Academic Affairs has recently tasked me with leading the development of a campus-wide end-of -course evaluation instrument to be delivered to all students in all courses by our new centralized online assessment software. As luck would have it, our Online Standards committee has been working on a set of questions designed to measure the effectiveness of design and delivery in our online courses. While the two efforts are obviously related they differ vastly in one crucial manner: Alignment.

The VCSU Standards of Excellence document lists nine principles of good design for our online courses as well as seven expectations of faculty in the delivery of their course offerings. Given these overarching expectations, it was fairly simple to develop survey questions to authentically measure these constructs. The Standards of Excellence provide us an unambiguous foundation of belief; guiding principles that provide clear evidence of what an effective course and instructor ‘look like’. In short, it paints a picture of what we want our courses (and instructors) to be.

At the university level, there is no shortage of documents and models describing what we strive to achieve as an institution of higher learning. There’s a mission and vision for the university (of course), as well as sets of both ‘guiding principles’ and ‘core values’. Additionally, VCSU has adopted eight student abilities that provide a framework for student outcomes. Furthermore, each division at the university marches to their own ‘orders’. For example, the School of Education and Graduate Studies boasts a mission, vision, a conceptual framework for both pre-service and advanced teachers, four ‘pillars’ of knowledge-based decision making, and three knowledge-base ‘domains’ for both the graduate and undergraduate programs. Many individual departments within divisions have developed their own guiding principles. The Tech Ed department for example, features their own, content-centered goals, commitments, and learning outcomes.

With all of these outcomes, it should be easy to develop assessment instrument right? In the words of the immortal Lee Corso: “Not so fast, my friend!”

When viewed individually, each of these documents represents meaningful and worthwhile aims for any institution. In the aggregate, however, they form a mountain of ‘objective-speak.’ For all of their altruism and earnestness, slick graphics, and flowery prose, they read like an avalanche of good intentions.
Clearly, the problem is not a lack of standards and objectives, nor is it a question of the quality of the ones we have. We, like many institutions, are simply struggling with a lack of alignment.

For all of its great ideas (and there are several), the website that lists our ‘defining commitments’ reads like an unfinished draft. The VCSU student abilities, while inarguably well considered, have not necessarily been universally accepted across campus as academic canon. Like the defining commitments, they exist ‘off to the side’, part of a long list of other ‘really good ideas’.

Some divisions, such as the School of Education mentioned above, maintain accreditations of their own, and therefore are well ahead of their peers in terms of internal goals and objectives. While their internal ducks are certainly queued, however, there is often a disconnect with the campus as a whole.

Absent a unified institutional assessment ‘voice’, how then are we to ascertain exactly what it is that a campus wide instrument is supposed to measure? The Online Standards of Excellence provided an invaluable toehold; a theoretical and ethical grounding on exactly what it is we consider quality in our course offerings. The campus-wide assessment, on the other hand is a whole different matter. Like a car out of alignment, every wheel is trying to go in its own direction. Not, it turns out, a very healthy proposition for the tires.

For me then, there is one question: Can institutional alignment be realized from the bottom up? Are we stuck in a holding pattern awaiting presidential edict, or can the lowly assessor lead the process necessary to affect authentic institutional assessment. Stay tuned…

The Impact of Attention on Learning

Glenn, D. (2010). Divided attention. The Chronicle Review. Retrieved February 9, 2010 from http://chronicle.com/article/Scholars-Turn-Their-Attention/63746/?sid=cr&utm_source

Can multi-tasking students in the classroom learn as well as those paying closer attention? There has been some debate among colleagues at NDSU as to whether or not a policy should be developed regarding the use of laptop computers by students in class or to include restrictions in the course syllabus. The thought is that many of the students are surfing the web or texting friends on Facebook, buying on e-bay, etc during class. One of my colleagues requires all students with laptops to sit in the front two rows of the room, hoping they will feel more compelled to limit computer activities to those on Blackboard/ in class documents. The other side of the argument is that instructors should just become resigned to the fact that the world has changed and today’s learners come with multitasking skills that allow them to learn while they are doing on-line activities. A recent article in The Chronicle addresses this debate.

Glenn quotes a researcher in psychology at Stanford University, Dr. Nass, “Heavy mulittaskers are often extremely confident in their abilities. But there’s evidence that those people are actually worse at multitasking then most people.” Research has shown that performance on cognitive and memory tasks is much worse in person’s that are self-described multitaskers than individuals who focus on single tasks. Much of the current research on cognitive function supports that attention that focuses on tasks requiring conscious efforts (controlled attention) and attention demanded by outside forces (stimulus-driven attention) are overlapping yet distinctly different neurological processes.

Research on human’s working memory dating back to 1956 by George A. Miller lends further support . He found that the ability to juggle facts and perform mental operations (working-memory capacity) is limited to about seven units e.g. if you asked someone to repeat a sequence of numbers they do well if there are seven or less. To compensate for this if the incoming information exceeds this amount, information is either “chunked” (organized in smaller groups) or unwanted stimuli or information is blocked from the working memory. The bottom line is that people who have higher levels of working member are better able to control their attention, avoiding distractions. “Strong attentional abilities produce stronger fluid intelligence” (p.4).

Furthermore, research by Foerde (2006)has shown that controlled attention improves t the ability to encode information into long term memory where the person is more likely to be able to use it to draw analogies and extrapolate from it. “When you’re teaching, you would like to see more than simple retention of the information that you’re providing people. You’d like to see some evidence that they can use their information in new ways.” (p.5)

Meyer, a professor of psychology, suggests going further than banning laptops in the classroom to refrain from taking notes! Thus, students should focus all attention on the instructor. His rationale is that multitasking degrades performance and that it takes the brain a significant amount of time to switch from the task of note taking to listening alone.

Other scholars retort that today’s media-rich culture has generated learners with “hyper attention” which is different from but not necessarily worse than controlled or deep attention. “Young people’s brains are getting better at making conceptual connections across a wide variety of domains.” They challenge teachers to recognize that hyper attention is not inferior and to focus on meeting the needs of the new learners. Goals should be set to cultivate both the hyper attention and deep attention.

I’m sure this debate is far from over. I tend to agree with the suggestion that multitaskers often over-estimate their ability to juggle all the incoming stimuli and learn at the same time. I often worry that today’s learners are so use to jumping from brief snap shots of information at a quick pace that they are not spending enough time really focusing and absorbing information in an in-depth manner. Are they taking any time to reflect critically? In my mind, a deep level of learning requires patience, real work, and silence … but then my mind may be wired completely different than these younger learners.This is certainly an issue that needs further assessment and research.

Carla

HLC Pathways

Many of you are aware of the proposed Higher learning commission changes. The link is: http://www.ncahlc.org/download/PathwaysOverviewv6013110Final.pdf , it discusses the open pathway to improve the value of the accreditiation by reaffirming the process and requires the institution to continually assess. One outcome goal is to improve credibility.
This new model separates the process into two parts: assurance and improvement. The assurance process is the accumulation of data and the improvement process consists of how the institution will improve.
I like the goals established by the open pathway, specifically the reduction of the reporting burden on the institution. The 5 goals are listed on page two if you are interested.
Since I am currently involved in our accreditation process, I have gained a unique perspective on the whole process. It is full of areas for improvement. This new process is very similar to the current process and should allow for an easy transition. I can easily see how this will become a routine in an institution thus alleviating the "panic" involved in the 10 year accreditation visit that is currently in place.
From what I can gather, all institutions will be phased in to this setup bythe 2015-2016 year with a timeline placed in each of the institutions schedule.
Kory

New Software Aids in Assessment

The article-New Software Aids in Assessment. By: Carnevale, Dan, was highlighted in the Chronicle of Higher Education, 00095982, 3/30/2007, Vol. 53, Issue 30.

Since almost everyone in our class is currently going through the accreditation process or will be soon, I thought this article was relevant at this time.

Mr. Carnevale indicated in the article how the demands for accountability for colleges is increasing, the need for technology is increasing as well. During class I heard many of my classmates talk about the tons of paperwork that is compiled when going through the accreditation process. This article mentions many computer programs that organizes and analyzes all the assessments of the students done on campus.

One such computer program called Weave which many of the faculty at the Virginia Commonwealth University used when they began gathering their information. Although faculty were reluctant to use it, it appears it was a way to solve their problem. Our class talks about getting the buy-in from their own faculty for assessment and what it entails by the mounds of paperwork required. Because the accreditation process is becoming so demanding it is better for colleges to utilize the resources that are out there to make their jobs easier.

It was interesting to read about another program and that was Blackboard. It appears that Blackboard has its own assessment software. Its focus is assessment at the classroom level and/or online collections of the students work. Since we currently use Blackboard in almost all of our classes, I thought is was important to point out that these programs mentioned as key programs for faculty to use.

The focus of assessment is towards student learning. How we improve on that all depends on the data we collect. The data collected needs to organized and accounted for. Systems mentioned earlier could be used as tools for this data collection.

Admission plan goes beyond numbers

This article was retrieved from the Chronicle of Higher Education.
Admissions Plan Goes Beyond Numbers. By: Hoover, Eric, Chronicle of Higher Education, 00095982, 9/14/2007, Vol. 54, Issue 3

I was most interested in this article because it addressed the fact that Northeastern University was actually evaluating their students' potential by looking beyond their test scores from the SAT. As we all know the SAT scores are scores that many colleges look at when accepting you into their program. Like Mr. Hoover indicated in the article, many people have trouble with tests and many times score low. In fact low enough that no college will look at them. For many Native people this is very true. From my reservation we have individuals who are very smart but when it comes to tests, they do poorly.

This article addressed how Northeastern had adopted a program which consisted of a 10-step evaluation process, which included extensive interviews and a personality test. This college took the initiative to look beyond the grades and made the effort to see what this individual could contribute. It was a very innovative approach to a non-traditional way of recruiting students.

It appears this way of identifying students from disadvantaged backgrounds is a success. These students are all talented but not in the traditional way. Many of the students would not qualify for the college's regular admissions process, but through this program, individuals are able to overcome many challenges and succeed.

For many of our Tribal colleges, this is sometimes the case. We have a program at United Tribes right now that is similar to this. We began a program for the underemployed and the unemployed. Admission into the program did not require these students to take an entrance exam, but did need to have minimum qualifications such as a high school diploma or a GED. These individuals are so far successful and hope to enter into the workforce by this summer.

Sample Size and Representation During Assessment

This article (http://pareonline.net/getvn.asp?v=9&n=11) published by Jason W. Osborne and Anna B. Costello from North Carolina State University I found interesting because the topic is something that I struggle with continually as I am sure most institutional researchers do.

Working in a smaller to medium size university gathering large sample sizes during assessments can be challenging. My previous postings discussed the CLA, they require a minimum sample size of 100 students per administration. In my perspective, finding 100 students willing to take a 90 minute writing assessment during regular class time has been somewhat of an issue. I have met the minimum requirement of 100 students, but narrowly.

The reason for a large sample size is to minimize the probability of errors and create the ability to generalize to the population at large. The assessments administered on the MSU campus are typically reported to the North Dakota University System for accountability reasons, these results are viewed by legislators and the State Board of Higher Education. Often times the results can lead to policy change and new criteria on campus....this is why the need for a solid study or assessment with a representative sample is necessary.

The author of the article notes that no sample is perfectly reflective of the population, but the higher the N the better the chances of representation. The author explains this notion by utilizing a number of variables to identify what form of mass would calculate the most return. The variables calculated were: subject-to-item ratios, variable-to-component ratios, extra matrices, and correct factor structure (conducted by factor analysis).

The major findings was that as mentioned previously size does indeed matter. The larger the N no matter what type of variables or analysis used.

Practical Assessment Research Evaluation Link

http://pareonline.net/
During my search of assessment tools for the development of my assessment plan I came across the above link. I found the link to provide a great deal of articles and authors known for their extensive backgrounds in relation to assessment.

A article titled, "A theoretical model for the authentic assessment of teaching," by
Peter Rennert-Ariev at Loyola College.

Rennert discussed the trend towards authentic assessment in recent years. Authentic assessment as defined in the article are those that: 1) sample the actual knowledge, skills, and dispositions of teachers in teaching and learning contexts; 2) require the integration of multiple types of knowledge and skill; 3) rely on multiple sources of evidence collected over time and in diverse contexts; and 4) are evaluated using codified professional standards. The most common characteristics apparent in the existing literature on authentic assessment suggest the importance of both teacher control over their assessment experience and context sensitivity of the instrument. The author purposes a need for revitalizing authentic assessment and suggests 5 new views for assessing teachers. The first suggestion was to provide the student with more control over assessment. The second is that in order to authenticate assessment the assessment should be conducted within the context of the student's work. Thirdly, allow for reflection on behalf of the student. Fourth, authentic provides the opportunity for the teacher to critique their own practice and teaching methods. Lastly, there must be dialogue between the assessor and those being assessed.

The author concludes that these are merely suggestions and that this article was written to provoke discussion with respect to authentic assessment.

Returning to Learning in the Age of Assessment

http://www.collegiatelearningassessment.org/files/ReturningToLearning.pdf
This article created by Benjamin Roger from the Council for Aid to Education introduces the rationale for the campus use of the Collegiate Learning Assessment. This article among others was utilized by the Assessment Committee at Minot State University (MSU) to determine the best possible tool to measure value-added learning for our students.

The section on importance of faculty in the age of assessment I found particularly interesting. This emphasizes the need to have faculty buy-in when taking on an assessment such as the CLA. The CLA is designed to measure a number of different learning objectives, these objectives align with that of MSU's General Education learning objectives as well. Without out valid results and faculty buy-in the assessment fails to provide data that is useful to the institution. This week I will be administering the CLA at MSU for exiting seniors. I have also been asked to speak during MSU's March Assessment Day on the importance of the CLA and how it is administered at MSU. As of recently I believe that only a few campuses within the state of North Dakota participate in the CLA (NDSU, UND and MSU). I know that the North Dakota University System has investigated the benefits of the administration to measure accountability across all NDUS campuses.

Florida family gives up on small-town North Dakota - Yahoo! News

Florida family gives up on small-town North Dakota - Yahoo! News

If you have touched on our book, Hollowing out the middle; The Rural Brain Drain... You might find this a bit interesting:)

Learning reconsidered 2

Keeling, R. P., (2006) Learning Reconsidered 2.

This publication builds upon the previous post. The author points out that it is expanded in two ways. The first is to provide some practical examples of how student affairs professionals are completing assessment of student learning. The second is to provide a more in depth look at the concepts from the first publication.

Chapter 4 focuses on creating program outcomes. During the mid to late 90's there was a push for student affairs to conduct assessment. Prior to than most data that had been collected had been about student satisfaction and had not focused on student learning. The text talks of 5 groups of outcomes that students should complete by the time they graduate. They include, educated persons, skilled workers, democratic citizens, self aware, and life skills managers.

The author points out that there are some key considerations when creating student outcomes. The first is that they must align with the Mission. What outcomes are valued needs to be clear. this can be a political process and not all parties may agree that life skills and inter and intra personal development are the responsibility of the institution. The final consideration is how student learning will be assessed.

There is a great table that lists sample student learning outcomes. I think this is a great chapter. I wish it would have gone more into what tools can be used to conduct the assessment. That is an area that I struggle with. The other part of the text that could be improved is that there are not examples about admission specifically. I feel that measuring student learning for is difficult when working with prospective students.

Law Schools Resist Proposal to Assess Them Based on What Students Learn

As I was perusing The Chronicle tonight, an article regarding assessment caught my eye. The article entitled, “Law schools resist proposal to assess them based on what students learn,” grabbed my attention and wouldn’t let go… It seems not only are two and four year institutions plagued by the ever demanding call for assessment, but so are prestigious and at times pretentious law schools. As I sat here reading the article, I couldn’t help but chuckle at the thought of some of the most notable and respected Law School Deans being asked to implement standardized assessment initiatives to prove their students are learning. Can hear them scoffing at such a lowly request? I sure can and this brings a smile to my face. You see, I don’t know about you, but is it too much to ask that your future lawyer, the person who may be defending you, in heaven-forbid a murder case, pass standardized assessments in order to prove they have mastered content and met their school’s objectives before representing you, your family, and your life in front of a jury? I would like to think that my lawyer, the person who I could potentially need to place my life and trust in, could pass some measly assessment standards to prove they have what it takes to represent me to the best of their ability. This concept seems like common sense to me, however as mentioned previously, the Deans scoff at such a request. Don’t believe me? Read the article.

The article explains that Law Schools across the country are preparing themselves for mandatory accreditation standards as proposed by the American Bar Association. Such standards are not currently in place, however the American Bar Association is concerned that lawyers graduating from law schools simply do not have the skills they need in order to perform basic job functions expected of a lawyer.

The standards, as proposed by the American Bar Association, and outlined in the article asks “…law schools to define learning outcomes that are consistent with their missions and to offer a curricula that will achieve those outcomes… Law schools will also be required to come up with a system to assess how well students are meeting those goals.”

The Bar Association said that lawyers today simply are not meeting standards, they are lacking in many areas which they should be educated and familiar with post-graduation. Thus, by enacting such standards the Bar Association would move the focus of accreditation from “input measures” like faculty-student ratio and number of library collections, to “outcome measures” proving what students will and should learn in their career at their respected institution.

However, as with all calls for assessment, Deans of many Law Schools are up in arms over such an assessment overhaul. There are many contentions people have with this proposed call for assessment standards; the most predominant, however, relate to the cost and other resources associated with implementing assessment, as well as how said outcomes can be measured in a tangible fashion to prove students are learning what they are expected to be learning in their collegiate career.

Unfortunately, the market for lawyers in our downward economy has plummeted with thousands of legal professionals being laid off in the last year. However, with that being said the rare companies and firms who happen to be looking for lawyers are tired of hiring new graduates that lack essential skills necessary for the profession. When companies and firms hire said new professionals, they are inherently wasting precious resources training and teaching skills and knowledge to new employees that they should have learned in their schooling.

As stated in the article, “Phillip A. Bradley, senior vice president and general counsel for Duane Reade, a large drugstore chain, likened law schools to car companies that are ‘manufacturing something that nobody wants.’" Mr. Bradley goes on to state that law firms today are generating their own list of “core competencies they expect from their lawyers,” so the question remains, why can’t law schools do the same?

This debate about the validity and need for standardized assessment initiatives in law schools is not over; in fact it has just begun. However, I do think it will be interesting to see how all of this plays out in the future, as some of the nation’s most prestigious law schools, including Harvard and Yale, may need to revise their curriculum and devise some learning outcomes they can prove their students are mastering, in order to turn out the most educated and sought after lawyers in the country.

Mangan, K. (2010, January 10). Law schools resist proposal to assess them based on what students learn. The Chronicle of Higher Education. Retrieved from http://chronicle.com/article/Law-Schools-Resist-Proposal/63494/.

Accrediting Bodies Consider New Standards for Distance-Education Programs

Distance learning has taken the higher education field by storm in the last decade with many online colleges and universities to which students can attend and earn their four year degree without ever stepping foot into a traditional collegiate classroom. Not even mention the number of traditional institutions, such as NDSU, that rely on IVN and online courses for distance education students. When the web grew and began to dominate the higher education field, students’ opportunities to earn college credit, along with college degrees, rapidly and dramatically increased. However, as this means to attaining an education has been popularized, the question remains is the education one earns online equivalent to that earned in a traditional college classroom?

In September of 2000, Dan Carnevale wrote an article in The Chronicle to address this very concern and acknowledge what accrediting bodies are doing to evaluate and ensure quality education is being produced and received in the online classroom.

The article explains that the six regional accrediting bodies have been struggling over how to evaluate distance education, as the classroom is very personal and usually is in the student’s homes. However, it goes on to explain that the accrediting bodies have created a set of standards that shifts the focus away from the faculty and traditional classroom and moves it to the individual student, their abilities, and of course their interests, performance and school work.

The accrediting bodies laid out regulations and standards that considered “… a number of factors, including: whether faculty members controlled the creation of content, whether the institution provided technical and program support for both faculty members and students, and whether the program had evaluation and assessment methods for measuring student learning.” The reason such strict standards needed to be agreed upon by all accrediting bodies is because unlike traditional classrooms, online and distance education programs cross many state lines – meaning a student in North Dakota could be enrolled in a college in Texas, but never even come close to the Lone Star state. Thus, in order to provide consistency and fairness, the bodies have agreed that an outlined standard would be the best means of ensuring consistent and quality online and distance education programs across the country.

Sounds great, doesn’t it? However, there are a few objections or contentions with the aforementioned standards. The most prominent contention is that because the focus of online education has become the student, their interactivity online, and their ability to learn, the accrediting bodies have naturally raised the bar for online courses far above that of the traditional classroom. For instance, in order for an online program to become accredited they must prove that the student is learning, producing their own work, interacting with their peers, meeting course standards, and that the professor is in fact controlling the content being broadcast in the course. However, in the traditional classroom, a program relying on lecture-based learning can also be accredited without the addition of the interactivity component. Thus, there are some critics who say that the proposed standards or regulations for online or distance education programs are too strict and in fact hold online programs to an unobtainable goal or standard.

Online and distance education has expanded and grown over the last decade. When the first set of distance education course standards were created in 1996, the focus was primarily on televised courses, but now hundreds and thousands of students are enrolled in online courses everyday and the standards need to be constantly updated to ensure quality education is still upheld and no “diploma-mills” become the predominant means of education across this country.

I am intrigued to see where the standards will go next as our country and the education system continues to evolve and change with the ever-updating and advancing technology which we rely on so heavily.

Carnevale, D. (2000, September 8). Accrediting bodies consider new standards for distance-education programs. The Chronicle of Higher Education. Retrieved from http://chronicle.com/article/Accrediting-Bodies-Consider/11731.

How Assessment works at one community college

This article was taken from the Chronicle of Higher Education, 5/29/2009, Vol. 55, Issue 38

I was most interested in the article because of the community college aspect. The article, written by Miami Dade College President Eduard J. Pardron, provided insight to how his college was working on meeting student learning goals. They created a top-10 list of what they wanted their students to learn. By assessing their students through a sit-down assessment process they are able to measure the results on a four-point rating scale.

I liked the fact that these results are compared each year. According to President Padron, Miami Dade College is able to improve in the areas that are suggested to have greater emphasis of need. An area that surprised me, as well as those at Miami Dade College, was that their recent study indicated they needed improvement in the global-perspectives goal. Miami Dade College has one of the most diverse student population in the nation. This indicates that even though a College may have a diverse student body, i.e. United Tribes Technical College, it should not be taken for granted that this is an area that would not need improvement.

To me, this article reiterates that even at a Community College level it is important to have ongoing assessment of all programs at the college and implement those changes for faculty and student improvement. In the end it is all about meeting the needs of the students and providing the best tools possible to them.

Educators Mull How to Motivate Professors to Improve Teaching

Educators Mull How to Motivate Professors to Improve Teaching

I know that Kory has already posted a blog about this article, however as I read the aforementioned article, I found myself growing very perturbed and annoyed with my fellow professionals and educators in the field of higher education. Thus, I decided despite a double post on this article, I needed to vent my frustration and author a blog about my sentiments and annoyance.

It seems that every Tuesday night we go to class and discuss, problem solve, and attempt to develop a means to generate faculty “buy-in” into the extremely valuable and quintessential assessment process, which would inherently assist us in our ability to “close the loop.” However, every class all we seem to do is come up with more road blocks or excuses as to why faculty should not or cannot buy into an assessment process. I understand that faculty are like all humans and are creatures of habit, but when it comes down to assessing the very thing that faculty are supposed to be facilitating – student learning – and they do not want to participate, it truly makes my blood boil. Students today are spending a lot of money or taking on a lot of debt to attend an institution of higher learning. Many of these students know that they will struggle to make ends meet while in college and even post-college, given the state of our current economy. However, these students put their trust in us as professionals, faculty, and an institution of higher learning, to take their tuition dollars and teach them all they need to know about their area of study, about life, and most importantly about themselves. We, as employees of an institution of higher learning, have been entrusted with student’s respect, money, and energy, and when faculty members or staff personnel at an institution fail to realize this and take their jobs lightly, quite frankly is makes me shake my head in disgust.

As a hall director, I relate to students in a way many faculty and staff never could, as I live with 300+ freshmen students. Every day, I have students in my office asking if college is right for them because their families need help paying bills, every day I have students complaining about how their professors don’t care even when students do go the extra mile, every day I see areas we, as in institution could improve our service to our students, yet nothing ever seems to change.

Thus, as I read the article written by David Glenn, entitled, “Educators Mull How to Motivate Professors to Improve Teaching,” I quite frankly was perturbed and decided to get on my soapbox. The article discussed how “…nearly 1900 presidents, provosts, and faculty members…” gathered at the annual Conference of the Association of American Colleges and Universities to discuss how to provide a better quality college education to students across the country. The focus naturally was on assessment and how colleges and universities need to embrace assessment in order to make any improvements to their programs and to the higher education field as a whole. The following old adage could have summed up the conference in a few simple words, “If you don’t know what’s wrong with your program, how do you know how to fix it?” More eloquently stated by Ronald Crutcher, president of Wheaton College, “Effective assessment is critical to ensure that our colleges and universities are delivering the kinds or educational experiences that we believe we actually provide for students.” So, if assessment is what the field of higher education needs to prove its worth and make improvements for our students’ sake, why were countless breakout sessions about faculty buy-in?

I guess I simply do not understand as an idealistic higher education professional, why we need to convince faculty that assessment is important. If assessment really is what we are taught it to be – a means of measuring student learning – then why do we need to prove its worth to the very people who hopefully entered the field of education because they WANTED to teach and facilitate student learning?

The article went on to explain some potential means of enticing faculty to buy-in to the assessment process, which have been researched through the association’s “Bringing Theory to Practice project” examining assessment and faculty buy-in. Such incentives to encourage assessment participation mentioned and discussed at the conference, included: bonus pay, faculty teaching awards, tenure or promotion, as well as other faculty-reward structures. Ashley Finley, the association’s director of assessment for learning and presenter at the conference, “…read a comment from one respondent (in the association’s Bringing Theory to Practice project) who wrote that teaching awards ‘sound good at first… until people start to feel overlooked, or until it’s obvious that the awards are used to make the award givers look good.’” As I read this comment, I was enraged, and kept asking myself, why do people need to be rewarded to do their job? These faculty and professionals are already receiving a paycheck from an institution of LEARNING and should care about their students LEARNING and if they are not or will not participate in assessment methods that will help their students, who in turn are paying them their salaries through tuition dollars to TEACH them, then I’d say it’s time to tell these faculty and professionals “goodbye.”

We don’t reward the custodian for taking out the garbage; he/she simply knows it needs to be done as a function of their job. We don’t reward the cook for making a delightful dinner and getting rave reviews, it is expected of them to do such a task as it is a part of their job. So, why do we need to entice, reward, or inspire faculty to prove, through assessment, that their students are learning, when that is EXACTLY what they have been hired to do – it is a function of their job?

As an employee at a higher education institution, I am tired of excuses and trying to figure out what would motivate someone to do their job, isn’t pay and the satisfaction of providing a well-rounded, quality education to our students, our future, enough? When faculty, staff, and the institution as a whole embraces assessment and providing “high-impact” practices, the kind of practices that teach life lessons and inspire students to want to learn, that is when we have done our job and given students what they are asking and paying for while attending our institutions. I think Ken O’Donnell, associate dean for academic-program planning in the California State University System’s office of the chancellor, said it best in the article, “High-impact [educational] practices can change student’s lives. Their brains open up. After they become engaged in this way, the hell they’ll drop out.” Thus, the question remains for each and every one of us employed in an education position, “What are you doing today to improve student learning and how are you going to prove it?”

Glenn, D. (2010, January 24). Educators mull how to motivate professors to improve teaching. The Chronicle of Higher Education. Retrieved from: http://chronicle.com/article/Educators-Mull-How-to-Motiv/63718/

CLA Project

Benjamin, R., & Chun, M. (2003). A New Field of Dreams: The Collegiate Learning Assessment Project. Peer Review, 5(4), 26-30. Retrieved February 14, 2010, from http://www.aacu.org/peerreview/pr-su03/pr-su03feature2.cfm

The article that I read for this blog posting is on the CLA (Collegiate Learning Assessment Project). The project was designed as a way to assess student learning in an authenticate manner. There are four differences between standard assessment measures and CLA assessment measures.
1. It uses direct measures of student learning.
2. It does not focus on disciplines, but on general education.
3. Uses a matrix-sampling approach. (How students are doing as a group, not individually)
4. It assesses the institution’s contribution to student learning.
The reasoning behind the project is that most institutions feature “General education as the knowledge, skills, and attitudes of an educated person” (Benjamin, p.26).
The assessment measurement has two parts in the CLA project. The first measures “A students ability to read data, make sense of a literature review, analyze an interview and newspaper report, and then synthesize the material and write a response” (Benjamin, p.27).
The second part of the assessment is now part of the GRE exam. It measures two kinds of analytical writing.
The authors found that the test has potential to help institutions make changes to programs based on student learning needs and institutional goals. The ability to analyze learning between institutions is a positive measure for faculty and administrators to identify the needs of their undergraduate students.
My opinion of this article and the CLA project is positive. We expect undergraduates to have credits in general education courses so that they are well prepared in basic knowledge, however many courses in the general education are not looking at whether students are able to synthesize the course’s information, rather they are looking at whether students can recall the information, and there is a huge difference between these two skills. For more information on the CLA project, the web site is: http://www.collegiatelearningassessment.org/

A Web Site Helps Faculty Members Assess Their Instructional Goals

Have you ever done a task so routinely in your daily life that you no longer need to think about the steps it takes to accomplish said task? For instance, think about the first time you made your favorite dish for dinner. You probably took out the recipe, went to the store to gather the necessary ingredients, and then step-by-step worked your way through the preparation steps. When you were finished and enjoyed the dish, you said, “I will make this again.” Thus, as human behavior allows, each and every time you made this dish, you got better at not needing to refer to the recipe after every step and eventually it became a task that needed no instruction, no forethought… it became habit. I would contend this sort of conditioning is not only present in our daily task accomplishment, such as cooking or driving to a destination, but also in our professional lives as educators and student affairs professionals.

A few weeks ago in class, when discussing the hot topic of “faculty buy-in,” it was mentioned that faculty would avoid assessment if it was imposed upon them. One reason for this hesitancy to “acclimate” was because faculty become comfortable in their teaching ways – they do not want someone else to come in and tell them how they need to teach, what they need to teach, or to impose assessment on them – thus they shy away from investment into a new program, whether that be a curriculum change or assessment additions.

This conversation in class got my intellectual wheels turning, asking myself, why faculty would be so blatantly opposed to adding new information and teaching techniques to their repertoire if it was to support better teaching and student learning? But then it hit me, just like the aforementioned recipe example, faculty members who have been teaching for x number of years, no longer need to think about what they’re teaching, as they’ve already set out course objectives and goals ages ago – and as the old adage goes, “If it isn’t broken, don’t fix it.” Thus, why would a faculty member or educator take time out of their already busy day to rework the wheel? It makes sense from a practical point of view related to time management and efficiency, and of course comfort.

However, as easy as it would be to say all professors or educators are lazy who disregard the addition of assessment means into their curriculum, I personally do not think that is the case. Why would someone commit years of their life to the study of education or a particular field with the intention of teaching only to become lazy and/or be unmotivated to enhance student learning? In my opinion, they just wouldn’t. However before you start rolling your eyes at my blissful ignorance, I would like to say, I do have a disclaimer: Although, I would like to be an idealist and think every educator across the country loves teaching and enjoys seeing students learn, I am aware that this is only idealism speaking and not reality; there are individuals who enter education so they can do research, there are educators who simply sought education after their perfect plan didn’t work out so perfectly, and then there are those who simply wouldn’t and don’t give a rat’s behind about student learning, it was just a job they qualified for and have stuck with it because it pays the bills. However, although acknowledging those anomaly educators, those that are in education for extraneous reasons, I would contend there is another “beast” out there preventing true educators from seeking the adoption of assessment into their classrooms. Thus, I set out to find potentially a more viable reason as to why educators would veer away from the adoption of assessment tools in their curricula.

As I was perusing Chronicle articles, I found one regarding faculty assessment of their personal instructional goals and then it hit me... What if faculty shy away from assessment because their frame of mind regarding what assessment is and how it can help them be better educators is dated and inaccurate as to what assessment really is today in the year 2010? Assessment, as a field, has only gained a lot of attention in the last decade or so, thus in the wonderful world of academia it is a fairly new area of study. And how many of us have had professors that seemed to have been teaching since the beginning of time? These faculty members may not know the importance of assessment or how one can assess without a standardized test because previously that was what assessment looked like – a paper and pencil fill in the bubble test.

So when I read Jessica Ludwig’s article, A web site helps faculty members assess their instructional goals, I was intrigued. Ms. Ludwig’s article explained how Tom Rocklin, an educational psychology professor at the University of Iowa, had taken the “Teaching Goals Inventory,” from our Angelo and Cross text, and made it an electronic survey for educators to take in order to assess and improve their instructional goals. The survey is available on the web at the following address: http://fm.iowa.uiowa.edu/fmi/xsl/tgi/data_entry.xsl?-db=tgi_data&-lay=Layout01&-view and allows educators to access it worldwide. The survey is a compilation of 52 educational goals that instructors are asked to rank from “not applicable” to “essential.” Once the survey is complete, the electronic survey categorizes or groups the results into “clusters” and shows the instructor which instructional goals they consider to be most important in their field of study, and inherently those that should be valued in their classroom objectives and course curriculum.

Rocklin stated in the article, “The exercise is not so much an evaluation as a ‘front-end planning tool.’” The site allows faculty and educators to anonymously evaluate their instructional goals without the pressure of someone else peering over their shoulder, and then after they receive their output or feedback they will be better equipped to create course goals and objectives with the “essential” components being capitalized upon. The site has served educators from a wide variety of disciplines ranging from nursing, to math, to special education.

Thus, maybe educators and faculty are not necessarily against or opposed to assessment plans, instead maybe they just don’t know what their goals or objectives should be anymore because the goals and objectives that used to work, when they first began teaching, are out of date and no longer match with the changing times and the ever-changing values associated with education or a specific field of study. Therefore, it might be time to break the monotony, and find a new “recipe” for assessment and instruction, one they need to follow step-by-step for awhile. When professors and educators create new instructional goals and objective plans, plans which place value on education and assessment so that our students can get ahead and meet the ever-demanding needs of our fast-paced world and changing economy, that is when education not only becomes informative, but also transformative.

Ludwig, J. (2000, November 3). A web site helps faculty members assess their instructional goals. The Chronicle of Higher Education. Retrieved from http://chronicle.com/article/A-Web-Site-Helps-Faculty-Me/36367/

Chronicle post: motivating educators

A January posting in the Chronicle discusses how to motivate educators to improve teaching. The link is: http://chronicle.com/article/Educators-Mull-How-to-Motiv/63718/
["Effective assessment is critical to ensure that our colleges and universities are delivering the kinds of educational experiences that we believe we actually provide for students," said Ronald A. Crutcher, president of Wheaton College, in Massachusetts, during the opening plenary. "That data is also vital to addressing the skepticism that society has about the value of a liberal education."]
An interesting statement to say the least. I would invite you to read this article if you have not done so already. It parallels many of the things we have been discussing in class and provides additional insight to other campuses with similar challenges.
The main theme of active and engaging students to improve student learning is very evident. I would have enjoyed attending the session the article refers to.
One burning question I continue to have is: "If industry wants a highly skilled independent graduate, how can we justify a graduate who comes from a well rounded liberally educated college?" Even if we are actively engaging, assessing learning, and providing the students with a quality education, it may not be enough to prepare them for the working world. I feel we have the best educational system available, but maybe we are missing something.
Any thoughts? Kory

Assessment and... Grocery Shopping?

Is assessment embedded into everything you do? Even your grocery shopping?

I have to admit, assessment and grocery shopping are not two activities that I normally think of as related. I tend to affiliate the concept of assessment with professional skills, learning outcomes and education. Needless to say, I don't think I have even used those same words when talking about grocery shopping. How could these concepts ever connect?

Gavin Henning, in a blog post for studentvoice.com explains the correlation by breaking down simple assessment concepts, "establishing a goal, developing strategies to reach that goal, and then determining how effectively and efficiently you've reached it." When you think of assessment in these terms, as Henning suggests, "you probably do a lot more assessment than you think."

When I thought about it, my own grocery shopping does fit into that definition of assessment

My Goal: Only go to the grocery store once every seven days at the minimum.

My strategy: Create a menu for the week before creating the grocery list.

Knowing if I succeeded: Not having to run back for an ingredient I am missing mid-week or, even worse, mid-recipe.

So, maybe this is like my professional thoughts about assessment regarding student leadership development:

Goal: For students to discover and develop their potential as leaders

Strategy: Write learning outcomes that align with institutional mission and goals and leadership development theory and then provide learning opportunities to meet those outcomes.

Knowing if we succeeded: Through outcomes from student interviews with faculty, reflection worksheets and periodic survey data.

Gavin Henning says that "Assessment isn't an activity. It's a state of mind." When you think about assessment in these basic terms, assessment can just as easily apply to our daily activities as it can to our professional activities. I bet we all are participating in assessment more often than we think. If this is true, we should be able to use the simplification of this concept to explain assessment to those who may be wary, and perhaps even to simplify our own assessment activities.

Something to think about...

Gavin Henning's Blog Post

http://www.studentvoice.com/app/Blogs/Gavin/post/Assessment-is-a-State-of-Mind.aspx