Report CopyRight/DMCA Form For : Non Experimental Desing
Non Experimental Designin EducationUmmul Ruthbah A way around RCT Experiments are difficult to run . Is there a way around There are several techniques Difference in Difference Regression discontinuity Propensity score matching. Methodology 1 Difference in Difference Suppose we want to evaluate the impact ofsupplementary tutoring on primary school students Ruthbah Rabbani Hossain Sarwar 2012 One way to do it is to assign students randomly to the. What if the program is already in place and it did notfollow the RCT protocol How do we create a control group now Evaluating the Education SupportProgram of the CDIP. The Center for Development Innovation and Practices provide 2hours of supplementary tuition to nursery grade 1 and 2students in many districts of Bangladesh Operating learning centres adjacent to primary schools since Supplementary tuition about 10 hours per week to primary.school students in nursery grade 1 and grade 2 1 750 learning centres adjacent to the primary schools We want to estimate the effect of the program on the participantstest score and dropout rate Treatment and Control. We could compare students who participated in theprogram with those who did not But it could be that only the weak studentsparticipated into the program and therefore thetreatment and control students are not similar. MethodologyTreatment group Control groupPre treatment 2007 Grade I Students Studentsobservation Pre treatment attending attendingobservation primary schools primary schools.Post treatment 2008 Grade II Students Studentsobservations CDIP attending attendingintervention primary schools primary schoolsand CDIP LCs only2009 Grade Students Students. III attending attendingprimary schools primary schools2010 Grade IV do do 2011 Grade V do do Methodology.Test Scores in Final Difference in testscores between2007 grade 1 2008 grade 2 2007 and 2008Students who XT2008 XT2007participated in XT2007 XT2008.ESP Treatment a Students whodid not XC2007 XC2008 XC2008 XC2007participate in b ESP Control .DifferenceXT2007 XC2007 a b DIDbetween XT2008 XC2008treatment andcontrol groups. Sampling Strategy 304 learning centres in 2008 in 33 unions of 8 upazilas inBangladesh Only 262 centres had students from grade 2 A sample of 1900 students 950 in each of treatment and.control groups from 159 learning centres and theassociated primary schools Sampling Strategy Multistage sampling Select the learning centres. Select students who were in grade 2 in 2008 andparticipated in the program Select control students 6 on average from theschools who were in grade 2 in 2008 but did notparticipate in the program . The Surveys Three sets of questionnaire on Performance of the treatment and control studentsin the final exams Background of students. School information The Field Experience Could not get the complete list of students who were in gradetwo in 2008 and attended the ESP We collected data on 2147 students of whom 1078 students.attended 144 different CDIP learning centers in 2008 The schools could provide the marks test scores for 2007 foronly 1215 students Figure 1 total marks obtainedMarks out of 300 .151 37 148 12150 139 522007 2008 2009 2010 2011 Year 12 Figure 2 difference in marks betweenpre post treatment years.2008 2009 2010 2011Marks out of 300 Difference 40 between 50 53 91 and control. 80 Control Results class performance did estimates Total Bengali English Math 1 2 3 4 Grade 2 0 072 0 12 0 01 0 05. 06 06 0 06 0 07 Grade 3 013 01 040 002 06 05 07 07 Grade 4 06 06 09 02 07 06 07 08 .Grade 5 0 01 0 03 0 01 0 01 06 06 06 06 Methodology 2 Regression Discontinuity Jacob and Lefgren 2002 examine the effect ofsummer school and grade retention on students .achievement The ideal situation would be to randomly assignstudents with poor grades to summer school or retainin the same grade And compare them with those whodid not go to summer school or repeated the grade. the control group But it is not possible to ethical and or other reasons How do we find the counterfactual the control Measuring the Impact of Remedial Chicago Public Schools introduced an accountability policy. Schools should decide who goes to summer school and whoshould repeat the grade depending on the student sperformance in a standardized test on Math and Reading Figure 3 The Design Treatment Control Groups. Students just below the cut off in June test constitute thetreatment group and those at the cut off belong to thecontrol group for assessing the impact of summer school onfuture achievement Students just below the cut off in August test constitute the.treatment group and those at the cut off belong to thecontrol group for assessing the impact of grade retentionon future achievement Figure 4 the Relationship between June Reading Scores andthe Probability of Attending Summer School or being Retained. Figure 5 Relationship between June reading and next yearreading and math performance for third grade students Figure 6 Relationship between June reading and next yearreading and math performance for sixth grade students Figure 7 Relationship between August reading and next year.reading and math performance for third grade students Figure 8 Relationship between August reading and next yearreading and math performance for sixth grade students The DID Estimate can use the following DID estimator to find the impact of.summer school Where mean achievementc student at the cut offc 1 student just below the cut offT the probability of attending the summer school.t time period RDD Main Idea There is a continuous variable that determines treatment The assignment to treatment is a discontinuous function ofthat variable . Where S selection variablec the cut offT 1 if treated 0 otherwise Methodology 3 the Propensity Score If the program affects the treatment group in a different way.then it would have affected the control group The DID estimates are of no use It happens is selection into the program depends on factorsthat also affect the outcome of interest Example the decision to attend the leaning centers may.depend on the parents years of education and parentseducation is believed to have influence on students test scores How to create a treatment control group is this case For the same level of parental education there are somestudents who attend the LCs and some who do not . For each level of parental education those who attend theLCs belong to the treatment group and those who do notbelong to the control group We take the average difference in test scores of treatmentand control students for each level of parental education . The average of the differenced test scores over all parentaleducation level is out treatment effect More than One Determinants What if there are more than one variable factor that affectboth selection and outcome variable For example parental.education and income Use propensity scores Propensity score probability of getting treatment f parentaleducation income Students with same parental education and income will have.the same probability of getting treatment propensity score Treatment an Control Groups For the same propensity score students who went to the LCsbelong to the treatment group and those who did not belongto the control group .Evaluating the Education Support Program of the CDIP. The Center for Development Innovation and Practices provide 2 hours of supplementary tuition to nursery, grade 1 and 2 students in many districts of Bangladesh.