Please select your primary account
The account you select will be tagged to your registered mobile number 7865487696
A deep dive into Adaptive Assessment’s transformative journey
August 29, 2024
Education, the birthright of every child, is their primary means of achieving success. Examination and education go together. Examinations judge a student’s progress at every stage, thus revealing any existing gaps.
However, data shows that even after 12 years of extensive schooling and exams, several students (data shows more than one in three) do not achieve the socially expected levels. The chief reason behind this failure is not always the learner; the school, our education system, also has a massive role to play in this.
Traditional education's one-size-fits-all approach often fails to meet individual learning needs, hindering engagement and performance. Adaptive Assessment, however, adjusts evaluations to match each student's pace and knowledge level, pinpointing strengths, and weaknesses for targeted support. Implementing Adaptive Assessment methodologies can effectively overcome the limitations of standardized approaches, improving students' learning outcomes.
Adaptive Assessment is a dynamic method for evaluating an individual's skills and knowledge, wherein question difficulty adjusts based on their prior responses. Also known as Computerized Adaptive Testing (CAT), these assessments are personalized for each student, presenting a tailored set of questions based on their performance. For instance, a correct answer prompts a more challenging follow-up question, while an incorrect response leads to an easier one. Unlike conventional assessments, the score calculation is influenced by both the number of correct answers and the difficulty of those questions. Central to Adaptive Assessment is an extensive question bank encompassing variations in difficulty, content, and format.
Within Adaptive Assessment, consistently measuring student performance poses challenges due to its dynamic nature. As assessments adapt to individual learning paths, traditional metrics may not always apply, requiring educators to continually refine evaluation strategies to accurately gauge student achievement. Geoff Masters, Chief Executive of the Australian Council for Educational Research, writes that the future of education will be based on “identifying and responding to individual progress and learning needs, rather than on traditional group-based solutions.” As schools look to improve personalized learning, they should consider switching to Adaptive Assessment, as they offer more advantages over traditional assessments. Research by HAL Open Science reveals that Computerized Adaptive Tests (CATs) have been extensively researched and implemented in recent years. For example, the Graduate Management Admission Council administered 238,536 adaptive tests via the GMAT during 2012-2013 (Graduate Management Admission Council, 2013). The purpose of these tests is to provide a precise measurement of a new student’s parameters while asking the fewest questions possible, a challenge known as test-size reduction (Lan etal., 2014). This concept is also related to predicting student performance (Bergner et al., 2012; Thai-Nghe et al., 2011). In machine learning, a similar technique is called active learning, which involves adaptively querying the most informative labels of a training set to optimize the learning process.
The benefits of Adaptive Assessment include personalized learning experiences, accurate evaluation of individual abilities, efficient use of time, targeted feedback, and the ability to identify areas for improvement more effectively.
• Personalization: Adaptive Assessment personalize the testing experience, tailoring questions to the test-taker's proficiency level. This ensures a more accurate representation of their true abilities.
• Efficiency: By dynamically adjusting question difficulty, Adaptive Assessment optimize testing time. Participants are not burdened with questions that are too easy or too challenging, leading to a more efficient evaluation process.
• Engagement: The interactive nature of Adaptive Assessment engages test-takers actively. The challenge level aligns with their competency, maintaining interest and motivation throughout the assessment.
• Precision in Measurement: Traditional assessments may fall short in precisely measuring a person's abilities. Adaptive Assessment, on the other hand, provide a granular view of strengths and weaknesses, offering a more accurate evaluation.
• Immediate feedback: Adaptive Assessment often provide real-time feedback, offering participants insights into their performance immediately after completing the test.
• Adaptive learning paths: Facilitates personalized learning paths based on assessment results, directing individuals to targeted educational resources and interventions.
• Enhanced motivation: Boosts motivation through a dynamic and challenging assessment experience. It also fosters a positive outlook towards learning and evaluation.
• Resource optimization: Maximize the use of resources by focusing on areas that require evaluation by reducing redundancy in the testing process.
Multistage Adaptive Test:
This type of Adaptive Assessment adjusts each future item based on an average ability score, called competency score. It has many uses. For example, an ideal application of this technique would be a vocabulary test in which students are presented with words and asked to define them (or provide definitions and ask them to indicate the correct term). If the students answer incorrectly, lower-level vocabulary is provided. As students answer questions correctly, terms and/or definitions may become increasingly difficult and complex.
The above diagram illustrates a multistage adaptive testing process in which the test adjusts based on the test taker's performance; correct answers result in more difficult questions, while incorrect answers lead to easier ones. This creates a customized assessment tailored to the individual's ability level.
Variable Length Adaptive Test:
This adaptive testing method allows item authors to create mini tests called testlets, which are made up of groups of questions or activities to create larger blocks of questions that follow an adaptive algorithm. In this scenario, each test is selected based on an assessment of the student’s current abilities. This approach allows teachers to test topics incrementally rather than with individual questions.
An example of variable length adaptive testing is a language proficiency assessment where the difficulty of questions adjusts based on the test-taker's responses, leading to a personalized test duration tailored to their skill level.
The above diagram illustrates an adaptive testing process where the selection of each testlet is based on the test taker's performance on the preceding testlet. This personalizes the test length and difficulty to match the individual's capability.
Every student is unique. The tests should be too. Now it is time to say goodbye to the limitations of traditional assessments and welcome the power of multi-stage assessment approach. In any kind of educational assessments or exams our next-gen personalized testing helps to access the true potential of a student’s capabilities.
Today in TCS iON we have multi-stage adaptive testing using Rasch model for Internet-based Assessments. The Rasch model is a probabilistic framework used in psychometrics to measure hidden traits, like abilities or attitudes, through the analysis of assessment responses. Here, navigation rules can be configured at the backend and assessment configurator can choose a preferred model to create the assessment. Items can also be calibrated using Rasch model for LAN-based and Internet-based assessments.
Click here for more details on Rasch model! In general, during the test, if a person’s estimated ability is high (that is, they have answered well on the more difficult questions), the Computerized Adaptive Test will have estimated their ability to be “high” and select and present an item from a “difficult range” of items in the pool. The process is the same for every estimated ability level from low to high, and for any number of levels from narrow to broad.
When enough questions have been asked and answered—generally not as many as an equivalent traditional test—a reliable score for the test taker is calculated. The score is not based on the number of questions answered correctly, but on the level of difficulty of items the person is able to answer correctly. While the details change slightly, this is functionally like how a high jumper’s score is obtained in track and field (jump to this section to learn more).
How can we check to help identify and solve the cause? By providing individualized student/study report, which shall,
• be fair to the diversity of students,
• help them all succeed,
• trust user numbers and learn about scalability issues, and
• reduce carbon footprint.
Comparing Linear and Adapting testing methods:
Currently, we are familiar with the configuration of linear tests in assessments or exams. However, adaptive testing introduces significant changes to this process. There are several distinctions between these testing methods, prompting educational organizations to consider implementing adaptive testing to enhance learning outcomes.
TCS iON has improved the functionality of Adaptive Assessment in both LAN-based and Internet-based assessments. Employing the Multistage Test (MST) approach, TCS iON ensures seamless services for its customers. The images below illustrate the appearance of MST and its configurable settings.
Multistage testing adjusts the difficulty of presented items based on the student's performance. However, while adaptive testing employs *Item Response Theory (IRT) to adapt each item individually, multistage testing operates in blocks of items. In Computerized Adaptive Test, one item is delivered, scored, and then a new item is selected, whereas in multistage testing, a block of items, such as 10, is delivered, scored, and then another block of 10 is provided.
*Item Response Theory (IRT) is a statistical framework used in adaptive testing to model the relationship between a test taker's ability and their performance on test items, allowing for precise ability estimation and tailored item selection. It ensures efficient and accurate assessments by adapting questions to the individual's proficiency level.
Source: https://assess.com/multistage-testing/
The structure of a multistage test is often termed as "panels." Typically, there is a single routing stage to initiate the exam, after which students proceed to different panel levels for subsequent stages. The design may be described by the number of levels, such as the 1-3-3 design example provided. Unlike Computerized Adaptive Test, which offers numerous potential paths, multistage testing usually has only a few paths unless each stage has a pool of available testlets.
Let’s understand how Adaptive Assessment works through the below use case,
Name: Tom Moody, College Student
Interest: Applying for MBA and preparing for standardized admission test.
Use Case:
Tom is preparing for the admission test and decides to use a Computerized Adaptive Test (CAT) based on the Rasch model to assess his readiness and pinpoint areas for improvement.
Tom logs into the testing platform, which uses the Rasch model for its CAT. He enters his information and starts the test.
Scenario:
1. Initial Setup: Tom logs into the testing platform, which uses the Rasch model for its CAT. He enters his information and starts the test.
2. Adaptive Questioning: The test begins with a medium-difficulty question. Tom answers correctly, indicating a higher ability level to the system.
3. Dynamic Adjustment: Following his correct response, the CAT algorithm presents a slightly harder question. Tom finds this one challenging and answers incorrectly, prompting the system to offer an easier question next.
4. Real-Time Estimation: As Tom answers more questions, the CAT continuously recalculates his ability using the Rasch model, adjusting the difficulty of questions to match his estimated ability level, ensuring the test remains appropriately challenging.
5. Precision and Efficiency: The CAT efficiently determines Tom’s true ability level by selecting questions that provide the most information about his skills, requiring fewer questions than a traditional fixed test.
6. Results and Feedback: Upon completing the test, Tom receives immediate feedback, including a detailed report that highlights his strengths and areas for improvement, along with his estimated ability level based on the Rasch model.
7. Targeted Study Plan: Using the insights from the CAT, Tom can concentrate his study efforts on specific topics that need improvement, optimizing his admission test preparation.
By employing the Rasch model in a computerized adaptive test, Tom experiences a personalized and efficient assessment that accurately reflects his abilities and helps him prepare effectively for college entrance exams.
In future, TCS iON is looking forward to bringing variable length Computer Adaptive Testing where assessment can be configured with only one group and assessment exit will be based on the candidate’s continuous performance. To support all the functionalities there will be more enhancements in item analytics as well.
Adaptive Assessment marks a shift in fundamental assumptions regarding the evaluation of knowledge and skill assessments. The ability to adapt to individual learning paths provides a glimpse into a more personalized and effective learning environment. Challenges remain, but the potential benefits make Adaptive Assessment a transformative force in shaping the future of assessment in education and beyond.
In summary, Adaptive Assessment is beneficial for students and teachers. Students can improve their learning, take responsibility for organizing themselves, and identify their problems. It is easier for teachers to work in large, mixed classes, create effective lesson plans for students and make better use of classroom time and resources.