exam answers 10 extra questions that are actually pilot items and do not count toward their final score. Pilot-testing on the exam is ideal, but there is a limit to the number of questions that can be included during each exam administration. We will still continue to do pilot testing with certificants whenever we have a large number of new items to try out.
Even after an item gets the go-ahead to become part of the exam, it may still be subject to review and revision, if necessary. Each time the exam is given, exam review meetings are conducted here in our Tallahassee offices. Pre-exam review sessions are conducted before every exam administration. Once the items for the next exam are selected, a group of certificants reviews them. They check for a little bit of everything; from making sure the answer key is correct to identifying spelling errors.
Post-exam reviews are also conducted after each administration of BACB exams. Data on how test-takers responded to each question are used to identify any items that didn’t perform well. A group of BACB certificants then reviews all of the poorly performing items. Their objective is to review each item in light of its performance and to determine what revisions might improve it. It is also their job to determine if an item is flawed to such an extent that the recent test-takers should be given credit for it.
|
We strive not only to continuously improve the exam, but our exam development activities, as well. In the past, pre- and post-exam reviews were one-day meetings held before and after each exam administration. Since we have begun offering the exam three times per year, we have been able to successfully combine the pre- and post-exam reviews, allowing us to have three two-day meetings each year rather than six one-day meetings. Not only is this more cost-efficient, we’ve found that the longer meetings promote lively and productive discussions that carry throughout the course of the two days – often through lunch and dinner, as well!
In the future, we plan to continue to conducting item writing, pilot testing and exam review activities. Our plans for improving item writing activities include such things as targeted item writing and “item rehab.” In order to conduct targeted item writing, we will identify content areas where we have the most need for more items and focus item writing sessions on those areas. Fae conducted the first “item rehab” sessions at the FABA conference this fall. There, she presented volunteers with items that have been identified in past exam review meetings as in need of major revisions. These sessions were very successful and we expect to hold similar ones in the future.
An area for improvement that is very important to us is ensuring that the individuals who participate in exam development activities are diverse and representative of all of our certificants. In the past year, we have made a concerted effort to make exam development activities available to more certificants and to be certain that our participants include individuals that represent a wide variety of geographic locations and educational backgrounds.
We hope to continue improve all of these activities to make them more accessible to certificants. In our survey of certificants last year, we asked a few questions about whether you would be interested in participating in exam development activities. We were thrilled to see that 69% of the people responding said they would be willing to help! We used the answers to combinations of questions on this section (e.g., interest in different activities, willingness to travel, etc.) to include a large number of individuals in our invitation for the December pre-post exam meeting. The number of
|
|
