aiou course code 6507-1solved assignment autumn 2022

aiou course code 6507-1solved assignment autumn 2022

Course: Educational Measurement and Evaluation (6507)           

Semester: Autumn, 2022                                                   Level: MA/M.Ed.

 

 

Assignment No.1

 

 

 

Q.1 Keeping the item development rules in view develop 5 Multiple Choice Questions (MCQs) and 2 essay type questions from any secondary level textbook.

Multiple choice test questions, also known as items, can be an effective and efficient way to assess learning outcomes. Multiple choice test items have several potential advantages:

Versatility: Multiple choice test items can be written to assess various levels of learning outcomes, from basic recall to application, analysis, and evaluation. Because students are choosing from a set of potential answers, however, there are obvious limits on what can be tested with multiple choice items. For example, they are not an effective way to test students’ ability to organize thoughts or articulate explanations or creative ideas.

Reliability: Reliability is defined as the degree to which a test consistently measures a learning outcome. Multiple choice test items are less susceptible to guessing than true/false questions, making them a more reliable means of assessment. The reliability is enhanced when the number of MC items focused on a single learning objective is increased. In addition, the objective scoring associated with multiple choice test items frees them from problems with scorer inconsistency that can plague scoring of essay questions.

Validity: Validity is the degree to which a test measures the learning outcomes it purports to measure. Because students can typically answer a multiple choice item much more quickly than an essay question, tests based on multiple choice items can typically focus on a relatively broad representation of course material, thus increasing the validity of the assessment.

The key to taking advantage of these strengths, however, is construction of good multiple choice items.

A multiple choice item consists of a problem, known as the stem, and a list of suggested solutions, known as alternatives. The alternatives consist of one correct or best alternative, which is the answer, and incorrect or inferior alternatives, known as distractors.

Constructing an Effective Stem

1. The stem should be meaningful by itself and should present a definite problem. A stem that presents a definite problem allows a focus on the learning outcome. A stem that does not present a clear problem, however, may test students’ ability to draw inferences from vague descriptions rather serving as a more direct test of students’ achievement of the learning outcome.
2. The stem should not contain irrelevant material, which can decrease the reliability and the validity of the test scores
3. The stem should be negatively stated only when significant learning outcomes require it.Students often have difficulty understanding items with negative phrasing (Rodriguez 1997). If a significant learning outcome requires negative phrasing, such as identification of dangerous laboratory or clinical practices, the negative element should be emphasized with italics or capitalization 4. The stem should be a question or a partial sentence. A question stem is preferable because it allows the student to focus on answering the question rather than holding the partial sentence in working memory and sequentially completing it with each alternative (Statman 1988). The cognitive load is increased when the stem is constructed with an initial or interior blank, so this construction should be avoided.

Q.2 Which factors can be helpful in minimizing the test adminisration problems and how scoring of subjective type test items can be made more reliable at secondary level.

Assessing the quality and quantity of learning has been, and always will be, a regular feature of classroom practice in every public school. For teachers to establish whether their pupils have been learning, they have to set, administer, score and grade examinations. Testing provides information about the examinees’ abilities and performance. It also enables meaningful observations and comparisons to be made of the kind of behaviour learners acquire during the teaching-learning process

Similarly, performance by students should reflect similar grade, in the same test, and similar results should be obtained by groups of comparable candidates using the test on other occasions, even when marked by a different examiner. This kind of result may only be obtained if among others the test is carefully administered; implying that the quality of test management and administration ensures its validity and reliability

It is, therefore, incumbent upon persons entrusted with the management of tests to learn the principles and good practices of test administration to ensure these qualities of a test are upheld in testing. When test administrators are not conversant with the principles of test administration, the overall aim of examination process fails and more often than not, the examiner and the examinees suffer the consequences.

Across the world, a number of scholars have documented vast and interesting literatures regarding principles of test administration and good testing practices in schools. Gronlund and Linn (1990), for example, suggest that tests can be successfully administered by any conscientious teacher or test administrator, as long as the prescribed testing procedures are rigorously followed. They maintain that test administration becomes simple if:

1. the pupils are motivated to do their best,
   ii.   test administration directions are followed closely,
   iii.  time is accurately kept,
   iv.  any significant events that might influence test scores are recorded, and
   v.   testing materials are collected promptly.

In Nigeria, different approaches of test administration are adopted by different examination bodies. A number of stakeholders, including the federal state and local governments, Non-Governmental Organizations and concerned individuals amalgamate their efforts for successful test administration in the public schools. In addition, punitive measures for those individuals who commit offences during the examination process have been put in place. The punitive measures range from imprisonment to monetary fines, depending on the type and magnitude of the offence committed during the examination process. Related to punitive measures, according to Adewale (2008); if less emphasis is placed on paper qualification and continuous assessment encouraged, irregularities during examination administration can be eliminated, and consequently, the examination administration process can be more effective.

The situation in Uganda, regarding examination administration does not greatly differ from other countries. However, in Uganda, examination management, national assessment and administration depends on the level of learning and the purpose of the examination coordinated by a national body. At the highest level of learning, universities have the autonomy to manage their respective examinations. It is, therefore, the responsibility of each university to put guidelines for administration of its examinations in place and to follow such guidelines. For secondary schools and colleges, the examination process is entrusted with the examining body, the Uganda National Examinations Board (UNEB). Conscious of the need to obtain information on what learners actually learn in school, many countries now operate what are variously called national assessments, system assessments, learning assessments, or assessment of learning outcomes.

In Uganda the Education Policy Review Commission (1989), reported lack of reliable and up-to-date data on educational indicators. The only assessment information that was used for monitoring and evaluation was based on public examinations such as Primary Leaving Examination (PLE), the Uganda Certificate of Education (UCE) examination results. However, public examinations are done only at the end of a cycle of education and are designed to serve primarily as instruments for certification and selection of learners into institutions of higher learning.

In his study conducted on behalf of the UNEB, Ogwang (2007) stipulates that the process of examination administration is an uphill task, as it is sometimes marred by irregularities. This is why the UNEB concedes that tracking down examination irregularities is a management feat: that it requires a lot of additional resources, both human and monetary ones to curb examination malpractices. Kagoro (2008) in his study conducted for the UNEB agrees with Ogwang and contends that an examination supervisor is the overall officer responsible for the smooth and proper conduct and supervision of examinations. He asserts that the examination administration should ensure that the rules and regulations on the conduct and supervision of examinations are followed.

In any case, the examination processes in Uganda is tailored towards achieving high validity and reliability of any examination. This is why effective supervision of examinations is a very crucial element in the administration of public examinations. The officers involved in the administration of examinations must ensure that examinations are conducted in accordance with the laid down rules to maintain credibility of the system.

Principles of Test Administration

The paramount guiding principle in administering any classroom test is that all examinees should be given a fair chance to demonstrate their achievement of the learning outcomes intended or planned. This implies physical and psychological environment in which the examination is taking place has to be conducive for the examinee to facilitate the achievement of the testing outcome. The factors that might interfere with validity of the measurement also have to be controlled. Even though the evidence regarding the effects of physical and environmental conditions on test performance is inconclusive, examinees should be as relaxed as possible and distractions should be eliminated or minimized. Whereas distractions during testing are known to have little effect on the scores of students, they may have profound effect on especially young children.

Another principle is students having positive attitudes towards a test. People are likely to perform better at any endeavor, including test taking, when they approach the experience with a positive attitude. Unfortunately, teachers frequently fail to help students develop positive attitudes toward tests. Students are not likely to perform at their best when they are excessively tense. Thus, the experience of test anxiety among some students.

It is imperative that test administrators are qualified enough and trusted persons. This is to ensure that tests are properly managed to obtain valid and reliable results. Test administrators need to have the opportunity to learn their responsibilities as a prerequisite to accurate test results. It should also be noted that a well prepared test is easy to administer, and the reverse is true with a poorly prepared test. It is equally important to realize that a successful test administration exercise is a product of test planning. Cheating is most likely to occur in a poorly planned test, thus, proving a challenge to test administration. However, a good test administration irrespective of the test preparation is paramount.

Good Test Administration Practices

Good testing practices rest in the hands of the examiner, who should ensure the testing exercise, runs smoothly. The period before the test, during the test and after the test should be effectively managed to realize a highly efficient testing period.

Period before the test

Security of testing instruments:
All test materials used in the assessment process, whether paper-and-pencil or computer-based must be kept secure. Lack of security may result in some test takers having access to test questions before the test, thus, compromising the quality, and invalidating their scores. To prevent irregularities, test administrators should, for example, keep testing materials in locked rooms or cabinets and limit access to those materials to staff involved in the assessment process. Test security is also a responsibility of test developers to ensure the test is not compromised over time. To maintain their security, test developers should introduce new forms of tests periodically.

Related to security of tests, testing authorities should endeavor to open cartons containing test materials and inspect the contents to verify that appropriate test levels and quantities have been received. After inspection of the testing materials, they should be securely stored since examination monitors may during unannounced visits inspect these materials to ascertain the seals have not been tampered with before the due date.

After securing an adequate number of tests, the following considerations should be part of prior preparation checklist:

a.) examinees and parents have been notified regarding the test date and time.

b.) candidates have been reminded to bring materials necessary for the test.

c.) all students with special needs (e.g. glasses and hearing aids) have been considered before the start of the test.

d.) all adequate invigilation has been planned.

e.) examination administrators have read appropriate test administration procedures such as timing, examination regulations and test modifications.

f.) the rooms where the test is to be conducted have adequate ventilation and lighting and have been properly arranged.

g.) seats are arranged in such a way that candidates cannot look at each other’s work.

h.) candidates have been thoroughly prepared for the examination by suggesting to them ways of studying, giving them practice tests like those to be used, teaching them test-taking skills and stressing the value of tests as for improving learning.

i.) when all is set for the exam, secure the room, including the writing “Testing in Progress, Do not Enter”.

Period during the Test

The proper preparation for examinations may not produce the desired results if the conditions during the test are mishandled. It is the cardinal duty of the test administrators or institutions to ensure that conditions during testing ensure successful testing. The following are guidelines that need to be observed to ensure required conditions for successful testing are fulfilled:

Observe precision in giving instructions or clarifications

When an examiner announces that there will be “a full three hours” to complete the test and then talks for the first fifteen minutes, examinees feel that they are being unfairly deprived of testing time. Besides, just before a test is no time to make assignments, admonish the class, or introduce the next topic. In other words, examinees are mentally set for the test and will ignore anything not pertaining to the test for fear it will hinder their recall of information needed to answer the questions. Thus, the well intentioned remarks fall on “deaf ears” and merely increase anxiety toward the test and create hostility toward the teacher.

Avoid interruptions

At times, an examinee will ask to have an ambiguous item clarified, and it may be beneficial to explain the item to the entire group at the same time. All other distractions outside and inside the examination room should be eliminated, where possible. The challenge, however, is that more often than not, the distractions are beyond the test administrators’ reach!

Avoid giving hints to students who ask about individual items

If the item is ambiguous, it should be clarified for the entire group. If it is not ambiguous, refrain from helping the pupil to answer it. The challenge is that at times, refraining from giving hints to examinees who ask for help may be difficult especially for new comers in the field of testing. Nevertheless, giving unfair aid to some students decreases the validity of the test results and lowers class morale.

Discourage cheating
When there is good teacher-student rapport and students view tests as helpful rather than harmful, cheating are usually not a problem. Under other conditions, however, it might be necessary to discourage cheating by special seating arrangements and careful supervision. Candidates receiving unauthorized assistance from other examinees during an examination have the same deleterious effect on validity of test results and class morale as does receiving special hints from the teacher. We are interested in pupils doing their best; but for valid results, their scores must be based on their own unaided efforts.

Careful proctoring of the testing session, such as, periodically walking around the room and observing how the students are doing is also of paramount importance in preventing cheating. The obstacle is that many teachers define proctoring as “being present in the examination room”. They consequently become physically present but spend their time reading a novel, writing a letter or marking and scoring previous tests. The best way to proctor an examination is to observe students doing the test and not being preoccupied at one’s desk.

Another way of discouraging cheating is discouraging students from using any form of communication devices, either in the room where the test is being administered or while on a supervised break, such as a bathroom visit. It would be better if students are reminded earlier that they may not use any devices including but not limited to cellular telephones, pagers, audiocassette players, radios, personal digital assistants, video devices, associated headphones, headsets, microphones, or earplugs while taking an examination.

Ensure that no eating takes place in the examination hall
Students should not be allowed to bring any food items in the examination room, unless it is on proven medical grounds. Under such circumstances, it is advisable that special arrangements are made in advance for purposes of securing a designated area where the food items could be kept, to avoid distracting those who do not require the food items.

Identify each examinee to prevent a situation where someone may attempt to take the examination on someone else’s behalf
Students should, therefore, be informed in advance to bring with them their identity cards and/or examination cards.

Handle emergencies appropriately
If an examinee becomes ill during the examination, and must leave the examination hall, they should not be allowed to return. The test administrator is advised to make a comprehensive report about the candidate’s situation to make it possible for authorities to consider a retest for such a candidate, to be scheduled for another time.

Inform students on progress of testing
It is the responsibility of the test supervisor or invigilator to keep the students informed of the time remaining

 

 

Q.3 What is the link of course objectives with test items? Keeping the principles of content selection in view develop a table of specification for twenty items from any secondary level textbook.

A course objective specifies a behavior, skill, or action that a student can demonstrate if they have achieved mastery of the objective. As such, objectives need to be written in such a way that they are measurable by some sort of assessment. Course objectives form the foundation of the class. Everything in the course should work together to ensure students master the course objectives.

WHAT DO GOOD COURSE OBJECTIVES LOOK LIKE

Good course objectives will be specific, measurable, and written from the learner’s perspective. Here’s a good formula for writing objectives:

• Start your course objectives with: By the end of the course, students will be able to:
  • Choose an action verb that corresponds to the specific action you wish students to demonstrate
  • Explain the knowledge students are expected to acquire or construct
  • [Optional]: explain the criterion or level students are expected to reach to show mastery of knowledge

You will also want to make sure that you have thought of a way to assess students’ learned knowledge when writing course objectives. For example, if you always test students’ knowledge of content matter with a multiple choice test, the course objectives cannot ask that students evaluate or create something, as multiple choice tests cannot assess those levels of learning with a high level of accuracy.

If you cannot logically fill in the last blank of this example (assessment), then the objective is not measurable. You won’t include the assessment in the text for your actual course objective that you publish on your syllabus, but you need to know if what you’re asking students to know can be assessed. Otherwise, how can you know or prove that students have learned what they’re supposed to learn in your course?

DIFFERENCE BETWEEN OBJECTIVES AND ASSESSMENTS:

One thing that can be confusing when creating course objectives is the difference between what students are being asked to know and the assessment that is used to “prove” that students know the information. In the example above, it may be tempting to write something like this: If students have learned U.S. History since 1865, then they should be able to write an essay comparing and contrasting the effect of two immigrant populations on American culture from 1865-1900.

This objective seems measurable, but it’s saying that by the end of the class “students should be able to write an essay”. That makes it sound like one of the objectives of the course is for students to write an essay. What students really need to know though is the effect of immigration on American culture. If you use the assessment in the “course objective slot,” the knowledge students need in order to complete the assessment is assumed rather than explicitly stated.

This could cause issues with the creation of materials and activities in your units because the focus may be on teaching students how to write essays rather than teaching them about the effect of immigration on American culture during a specific time period.

BLOOM’S TAXONOMY:

Using a taxonomy that explains different levels of learning can be helpful for selecting the appropriate action verbs for your course objectives. These will help prevent you from choosing lower order actions when you really want students to demonstrate higher order thinking.

Bloom’s Taxonomy is broken into six knowledge dimensions: Remembering, Understanding, Applying, Analyzing, Evaluating, and Creating and range from lower order thinking skills to higher order thinking skills.

By their very nature, higher order thinking skills are more difficult and build on the previous lower order thinking skills. An oversimplified explanation of this would be the following: A student can not be expected to create a design brief (Creating) if they can’t remember what a design brief is (Remembering).

Traditionally, entry level courses ask students to demonstrate remembering, understanding, and applying thinking skills with a few higher order thinking skills while graduate level courses ask students to demonstrate analyzing, evaluating, and creating thinking skills with a few lower order thinking skills.

 

HOW DO COURSE OBJECTIVES DIFFER FROM MODULE OBJECTIVES?

Course objectives are much broader in scope than module level objectives. Where module objectives break down skills and knowledge into very specific, discrete skills, course objectives point more to overarching student understanding and higher level thinking skills. In a unit, you may have 10 or more objectives explaining all of the steps/tasks involved in learning a concept. For a course, you will only want 3-6 course objectives.

 

 

Q.4 Why Multiple Choice Questions (MCQs) are widely used types of test items? Criticize the frequent use of MCQs in high stake testing.

Who knew that a question type could be so shrouded in controversy? The multiple choice question (MCQ) may be a “Who Wants to Be a Millionaire” favorite, but it’s also the most widely debated question type when it comes to efficacy and outcomes reporting. Why all the buzz? The multiple choice question is forever associated with standardized tests, Scantron sheets, #2 pencils, and all of the above. But like any question type, there are benefits and downfalls, there’s a time and place, and there are a slew of best practices. Let’s weigh the pros and cons and figure out when to best use this traditional testing favorite.

The Multiple-Benefit Question Type

Like any question type, the format alone is useless without proper usage, wording, and subject pairing to make it effective. The following benefits make multiple choice an attractive option for fact-based content.

• Easy on the GraderThink about the instructor with no TA and 500 students in their 101 course. Essays and short answer questions, while effective, will inevitably delay grading. Auto-graded multiple-choice questions allow instructors to test their students quickly and efficiently, without hiring additional graders.
• Time and Scope:There’s a reason why MCQs are a default for most standardized testing. By nature, MCQs allow for fast testing across a vast expanse of content. According to Vanderbilt University, “because students can typically answer a multiple choice item much more quickly than an essay question, tests based on multiple choice items can typically focus on a relatively broad representation of course material, thus increasing the validity of the ”
• FlexibilityPerhaps it isn’t the nature of the question but what we are asking that allows us to think of this question type as so rigid. There are options to expand to different Bloom’s Taxonomylevels in am MCQ. While many default to questions that test both understanding and remembering facts, a well-worded question can test on application and analysis.
• Single/Multiple AnswersA single answer allows for simple weeding out of incorrect answers. However, with multiple correct answers (and this doesn’t mean “D. All of the above”) present, you can eliminate the process of elimination.
• Measurable and ReliableWith the focus on efficacy measurement in schools increasing, being able to have large amounts of objective testing data that show students’ grasp and retention of content is pivotal for an institution.

The Multiple Layers of Controversy

All benefits aside, MCQs are widely debated for their efficacy and often considered a poor question type to gauge a student’s level of critical thinking, making them far better suited for lower-level Bloom’s questioning. The following are some of the pitfalls mentioned by leading MCQ opponents.

• Development TimeFor the question author, a well-crafted MCQ isn’t always just about writing the best correct answer, it’s creating deeply convincing false answers, or distractors. This takes more time than a simple fill-in-the-blank or essay question. Too many sloppy questions have been written in the past that give the correct answer away or that give a freebie distractor away as a definitely wrong answer.
• Working Backwards from WrongAccording to the National Center for Fair and Open Testing, “Multiple choice items can be easier than open-ended questions asking the same thing. This is because it is harder to recall an answer than to recognize it. Test-wise students know that it is sometimes easier to work backwards from the answer options, looking for the one that best fits. It also is possible to choose the ‘right’ answer for the wrong reason.”
• Beating the OddsYou may have heard these question types called “multiple guess questions.” Of course, guessing is present in any question, though MCQs allow for even the most clueless learner to have a 25% chance. If they can remove even one distractor, their odds have immediately increased to 33%. The option for guessing is present in plenty of question types… but here, the right answer is literally on the page. May the odds be ever NOT in their favor.
• Diversify Your Question TypesWith the majority of standardized tests heavily reliant on multiple-choice, deliberate choices must be made as to when to use MCQs and MCQs should be interspersed with other question types that assess students on their abilities to create, evaluate, and formulate their own responses to situational questions.

Q.5 Explain the qualities of a good test? In which situations equivalent form of reliability can be a good measure of reliability?

One of the major goals of education is to prepare students for the next step in their future. They have to make sure that their learners have acquired enough knowledge about the field of study. Only good tests ensure this. A good test is not only a score that learners struggle to ace.

It’s feedback a student receives to improve his skills and knowledge and a good teacher loves to get back to, always, to make sure their teaching strategies are on point and whether they need development or not.

It’s also a feedback for decision-makers in all educational institutions and governmental positions who need good data to get to the next step of the institution or the State’s education plan.

It’s not something centric that students spend days of anxiety on, wondering how well they will do in a given test and how well the test questions are actually written and whether they are questions they do know the answer to or not.

What is a good test in education? It is an evaluation through which teachers measure learners’ abilities and points of weaknesses and strengths. It gauges their knowledge in the field of study and provides both sides with real feedback.

A good test should ensure that learners are ready to move to the next step whether this step is a high school, college, or even the military.

 

In our previous event, the first free online webinar, “Ensuring Effective E-Assessment for Higher Education,” Qorrect e-assessment team, discussed the complete cycle of a good test in detail focusing on higher education examination.

The team discussed how to analyze, design, develop, implement, and evaluate the phases that together comprise the e-assessment life cycle, going through the e-assessment life cycle and its importance to higher education, edtech role in the evolution of the digital assessment process.

That’s plus considering the contribution of edtech in improving assessment quality, analysing the examinees’ responses, assessing the exam’s quality and the effectiveness of the involved questions in measuring what they are designed to measure.

An assessment is a process through which students can share their educational experiences. In order for a test to be a good tool for measuring students’ knowledge and skills, it should have the following characteristics of examination that are essential for the success of any test.

 

Reliability or Consistency

Reliability or consistency of a test means that learners should perform the same or get the same score if they are exposed to different questions in different times and places. A test is considered reliable when the same result is achieved over different tests.

As James Carlson mentions in his research memorandum, “The reliability of test scores is the extent to which they are consistent across different occasions of testing, different editions of the test, or different raters scoring the test taker’s responses.” He also mentions some statistics to describe how a test can be reliable.

 

How to Make Sure Your Test Is Reliable

1. Score Distribution: The percentage of test takers at each score level.
2. Mean Score: The average score, computed by summing the scores of all test takers and dividing by the number of test takers.
3. Standard Deviation: A measure of the amount of variation in a set of scores. It can be interpreted as the average distance of scores from the mean. (Actually, it is a special kind of average called a “root mean square,” computed by squaring the distance of each score from the mean score, averaging the squared distances, and then taking the square root.)
4. Correlation: A measure of the strength and direction of the relationship between the scores of the same people on two tests.

Reliability is the ratio of the true score and the observed score variance. To measure a test’s reliability, we may administer a test to the same group more than once.

However, errors may occur as students may forget or have some physical problems. Thus, it is crucial to administer the same test in identical conditions to ensure that we will get the same results.

 

Validity

A validity of a test can be achieved when the test measures what it is really intended to measure. Therefore, a certain criteria must be selected.

Validity is very important to gauge the quality of a given test as questions must be in line with the selected criteria and measures.

 

Here are some of the top different types of validity:

Content Validity: A test should fairly represent the content of the course or the field of study.

Criterion Validity: It is used to predict the performance of a job applicant or a student.

Convergent validity: This is mostly used in the field of sociology or psychology.

Discriminant Validity: Discriminant validity means that a test of a concept is not highly correlated with other tests that are set to measure theoretically different concepts.