Title 05 PsyAs - Test Development.pdf ✅

05 – Test Development PSYAS | 2024 - 2025 | NOT FOR SALE OUTLINE 1. Defining Test Development 2. Test Conceptualization 3. Test Construction 4. Test Tryout 5. Item Analysis 6. Test Revision DEFINING TEST DEVELOPMENT DEFINING TEST DEVELOPMENT Test Development – an umbrella term for all that goes into the process of creating a test ● Occurs in 5 stages: ○ Test Conceptualization ○ Test Construction ○ Test Tryout ○ Item Analysis ○ Test Revision ● Some tests are conceived of and constructed but neither tried-out, nor item-analyzed, nor revised TEST CONCEPTUALIZATION DEFINING TEST CONCEPTUALIZATION Test Conceptualization – process of conceptualizing the construct, items included, and design of the test ● Step 1: Describe purpose and rationale for test ● Step 2: Describe the target population for the test ● Step 3: Clearly define the key variable of interest ● Step 4: Create item specifications ● Step 5: Choose item format ● Step 6: Specify administration and scoring procedures Preliminary Questions ● What is the test designed to measure? ● What is the objective of the test? ● Is there a need for this test? ● Who will use this test? ● Who will take this test? ● What content will the test cover? ● How will the test be administered? ● What is the ideal format of the test? ● Should more than one form of the test be developed? ● What special training will be required of test users for administering or interpreting the test? ● What types of responses will be required of test takers? ● Who benefits from the administration of this test? ● Is there any potential for harm as the result of an administration of this test? ● How will meaning be attributed to scores on this test? Target Population – relevant characteristics to consider ● Age ● Educational Status ● Language (which language and level of language ability) ● Literacy Level ● Disabilities Variable of Interest – developed by referencing ● Theory ● Empirical Literature ● Cultural Definitions ● DSM-V or other diagnostic criteria Item Specifications ● List major content areas to be included in the test ○ Can include number of items ● For clinical tests, the DSM-V (or other guidelines for diagnosis) can provide guidelines for item specifications ● Otherwise, develop item specifications from theory or definitions of variable TYPES OF INSTRUMENTS Self-Report – participants are asked to report their own attitudes, beliefs, knowledge, feelings, and behavior ● Self-report can be either a questionnaire or interview ● Typically assess attitudes, beliefs, values, interests, knowledge, feelings, and overt behavior PROS CONS Questionnaire Easy to administer and score Can be administered to larger numbers of test takers The data is typically not as in-depth as an interview Requires a high level of literacy Interview Gathers rich, in-depth information Can ask test-taker clarify certain responses Can be used with any test-taker (e.g., cannot Can be time-consuming compared to questionnaires Typically subject to issues of inter-rater reliability 1 | @studywithky