• Home
  • /
  • Topic
  • /
  • Other
  • /
  • Test
  • /
  • Automated Test Form Generation Critique of the Research Paper
Verified Document

Automated Test Form Generation Critique Of The Research Paper

Automated Test Form Generation Critique of the Journal Article "Automated Test Form Generation" by Wim J. van der Linden and Qi Diao

Van der Linden, J.M. & Diao, Q. (2013). Integrating test-form formatting into automated test assembly. Applied Psychological Measurement, 37(5) 361 -- 374

Retrieved: http://www.utwente.nl/gw/omd/Medewerkers/artikelen/vdLinden/artikelen%20vanaf%20juni%202011/APM%202013,%20361-374.pdf

The ordering of items in test forms and the selection of test items from a database can be extremely tedious and laborious when done manually. It would be extremely advantageous to be able to do both simultaneously using automated test form generation (ATFG). But the ATFG process can be unwieldy and cumbersome given the number of variables the creator must tackle. For example, "for an item bank of size 2,000, test length of 50 items, and upper limit of the number of pages in the test form equal to 15, the number of variables in the model is equal to 1,500,000" (Van der Linden & Diao 2013: 361). According to the article, "Integrating test-form formatting into automate test assembly" from Applied Psychological Measurement, "one of the goals of this research is to find out if test-production problems of realistic sizes can be solved" and make the simultaneous...

Parts of this document are hidden

View Full Document
svg-one

The first is the selection of items from a database and ordering and formatting the item into the actual test form itself. "ATA involves the selection of the items for a test form from an item bank subject to a potentially large set of content and statistical specifications while optimizing their measurement properties" through the use of mixed integer programming (MIP)" (Van der Linden & Diao 2013: 361). While MIP is mainly used for test selection, it can also "be used to optimize the second step (automated test-form generation [ATFG]" (Van der Linden & Diao, 2013: 361). The purpose of the study is to determine how this can be accomplished in an efficient manner. The objective is to explore whether it is possible to use ATA and ATFG simultaneously for more efficient test generation and "then evaluate their performances against the case of separate models for ATA and ATFG for a series of applications with realistic item bank sizes and test lengths" (Van der Linden & Diao, 2013: 361).
Previous authors have found this quite challenging so rather than a formal literature review, the authors construct two paradigmatic examples…

Sources used in this document:
To argue that to optimize ATFG is possible, the authors simulated two different test forms: one computerized and the other paper based, according to various "content, statistical, and formatting specifications. More specifically, they used an existing set of specifications for the assembly of test forms, along with an existing item pool from a large-scale mathematics achievement testing program" (Van der Linden & Diao, 2013: 368). This was to show that ATFG was both hypothetically possible as well as applicable to the needs of an actual testing program

In their conclusion, the authors noted that it was possible to deploy a simultaneous modeling approach that allowed an integration of both selection of problems and generation of test forms, considerably streamlining existing available formats. "For future applications, the performance of the solver needs to be assessed on a case-by-case basis. The total number of decision variables is only one of the factors with an impact on the performance of MIP solvers; others are the structure and sparseness of the coefficient matrix for the constraints" (Van der Linden & Diao, 2013: 368). Thus, the simultaneous approach is recommended, based upon their calculations although this may vary according to circumstances on occasion.

While the research provided by the authors is valuable, in terms of structure, the article is somewhat problematic. First of all, the authors assume a great deal of knowledge on the part of the reader, automatically beginning with a discussion of the different forms of test-generation without setting up the reader with an extended example or grounding the problem in real-world analysis. Also, the equations used to substantiate the hypothesis are included in a separate section rather than explained. This article is thus largely unreadable by a layperson in the field and must accept the authors' conclusions on faith if they are not mathematically fluent. Even someone in the field might find it problematic that they must evaluate the author's mathematical conclusions to support or take issue with their analysis.
Cite this Document:
Copy Bibliography Citation

Related Documents

Other Cultures Xenophobia and Understanding
Words: 954 Length: 3 Document Type: Essay

The world is more interconnected now than ever before, both with technological tools and also with population migrations that are occurring on an unprecedented scale. People who grow up in pluralistic, heterogeneous societies like the United States, India, or Canada take cultural diversity for granted. However, many people around the world live in homogenous societies with little contact with other cultures other than through what they read about or see

Other Disciplines Relate to Nursing
Words: 726 Length: 3 Document Type: A-Level Outline Answer

Nurse How to gain a comprehensive and well-founded knowledge of nursing as a student? As a student, it is important to keep an open mind when gathering knowledge of this profession. This attitude is most important in my opinion because of the continual changes the medicine world offers on a fairly regular basis. New treatments and new ailments arise consistently and it is up to a nurse's education to guide this process

Other Wind by Ursula K. Le Guin
Words: 1267 Length: 4 Document Type: Term Paper

Wind -- Science Fiction for Adults, a Drama of the Human Heart and Mind rather than Light-Sabers Ursula Le Guin, the Modern Female Conscience of Science Fiction Ursula Le Guin is one of the most highly respected authors of fantasy and science fiction of the 20th century. The award-winning Le Guin has long been praised for combining traditional elements of literary fiction, science fiction; with philosophical and ethical speculations on ways

Media in Other Countries Many
Words: 1295 Length: 4 Document Type: Essay

In the Arab world, it can get that woman killed…literally. That is not to say that is the norm, but it's a commonly discussed and accepted precept with much (but notably not all) of the Arab sphere. Similarly, honor killings are considered acceptable, even with some people that have immigrated to countries that consider it cold-blooded murder. As for whether anything has changed in the recent past that might change

Engagement Phase Examination of Self and Others
Words: 1505 Length: 5 Document Type: Term Paper

Self and Others The term engagement has been defined as being actively involved in, a part of. When applied in a social work context, the same definition holds true. For a client to be engaged it means they must be actively involved in and a part of the process from the beginning until the very end. The National Association of Social Worker's Code of Ethics bespeaks the importance of client

Humanities and Other Modes of Human Inquiry
Words: 1118 Length: 3 Document Type: Essay

Humanities and Other Modes of Human Inquiry and Expression • Define the term humanities Humanities are a term that encompasses many individual study and sciences. There can be a two way classification of all human knowledge. First is the knowledge of the space around us, but not directly linked to humans. For example, the study of physics, botany or astronomy does not involve expressions from human emotion and nor do they reflect

Sign Up for Unlimited Study Help

Our semester plans gives you unlimited, unrestricted access to our entire library of resources —writing tools, guides, example essays, tutorials, class notes, and more.

Get Started Now