The Importance of Implementation in ICT

With the various drives for information technology in the developing world, one necessary lesson remains true: there is no way to leapfrog the necessity for a well-trained teacher; and without a well-trained teacher implementing the ICT program and integrating it into the curriculum, the ICT revolution, despite fantastic claims of "self-learning revolutions" by some, will remain a myth. Thus, with the proliferation of efforts occuring around the world, I am apt to focus on success with the key ingredient: a focus on curriculum development, curriculum integration, and teacher training.

The good folks at the Poverty Action Lab cited this issue with a recent study of a massive, nationwide program in Colombia. Distinctive in this effort, the drive was indigenous, with the Colombian Ministry of Education working to recycle donated machines to school locations around the nation; however laudible an indigenous ICT program might be, if the program does not give the intended outputs, it is still just as big of a waste of resources as an externally-funded and driven ICT project. And again, the key issue here was not scale or breadth; it was the lack of focus on the key determining factors mentioned. Because of a lack of curriculum development and integration, the machines were only used to teach technology skills and not in other core learning areas; there were no significant increases in test scores in any core subject areas despite the availability of machines.

To unlock both the potential of technology and the potential of students to utilize this technology, key tasks must be focused on, or this push risks being another blind drive to obscurity.


Details of the report:

http://www.povertyactionlab.org/evaluation/use-and-misuse-computers-education-evidence-randomized-controlled-trial-language-arts-pro

How to Control Absenteeism in the Classroom-MIT

In furthering my research into MIT's work in educational randomized trials, in the summer of 2009, a report entitled, "Showing Up is the First Step," was published. The results of this report point towards impersonal, direct incentives for teachers to improve attendance. As already reported, during my school visitations amongst the developing world nations in Africa and South Asia, simple abscenteeism can be seen as the biggest impediment to educational advancement, as is a key factor in some students sitting in classes for 7 years and not being able to read or write. Teacher effectiveness surely plays a role, but without their attendance (the report cites that on any given day, 27% of teachers in Uganda do not show up, and the figure is 23% in India;  further, in India, only 2/3 of teachers who showed up were actually teaching!), there is simply nothing being learned, and this contributes to a further breakdown of the entire process, as absent teachers also lead to demoralization and behavior problems that do not end when the teacher shows up for work, eventually. The details from the report, on what works best:

"Seva Mandir, an NGO in Rajastan, India, introduced a simple mechanism that recorded teacher attendance and directly determined teacher pay. This combination of external monitoring with credible rewards (and punishments) cut absence rates in half. Moreover, providing incentives for service provider attendance improves outcomes for poor people without undermining providers’ intrinsic motivation." 

MIT-The Incentivation of Student Test Scores in Kenya

The push for standardized testing around the world as one of the key metrics evaluating student progress, and thus, teacher effectiveness has been in full swing for some time now. In the NYC public schools where I taught for two years, as well as the Pohnpei State School system where I taught in the Peace Corps for two years, teachers were constantly faced with state and national testing for students to determine a wide range of "competencies," with testing becoming so proliferate in NYC that it seemed every other week we had at least one day dedicated to evaluation programs, significantly eroding actual, valuable class time. Although as a teacher I strive to measure progress through alternative means, such as portfolio/project and group/peer assessments, the standardized test always looms large on the periphery. While there are often perils in using standardized testing as a means of incentivising teacher motivations (the largest being that teachers will "teach to the test" and relegate actual learning to the back burner), in some instances, there are few alternatives, and providing incentives for improvements in testing can be a viable way of increasing teacher effectiveness. Thus, the folks at MIT have been looking at, and conducting randomized trials in this focal area in Western Kenya (actually in the same areas where their other work as been held in dealing with increasing teacher effectiveness through community teachers, which raises a bit of a red flag in my mind).
In Kenya, as in most of the developing world, the teacher absenteeism rate is alarmingly high (over 20%) and although universal primary education has been implemented, the largest hurdle faced is the actual quality of the education that is being offered. In most instances, it is alarmingly poor, and is often a net loss for many students who trade off helping their families with agriculture or in the household for a completely inadequate education. The biggest issue in Kenya is, as stated, "Teachers’ salaries depend on their education and experience, with no opportunity for performance-based promotion, which appears to result in a system with no incentives to teach well." 
Merit pay was offered to teachers in the test groups for improvements in their schools as a whole for the national testings. The program ran for two years, and penalized teachers for student dropouts, which is a serious issue in the region, as well.

The result was less than encouraging. Test scores did improve, but not through the means that one would have hoped (such as increased teacher attendance or decreased student dropout rates, the real core issues that were being addressed in the study). What occurred is that schools offered test-prep classes outside of the normal school day to improve the testing scores; additionally, there was no spillover when the project was ended, and thus, the sustainability factor was negligible. 

Thus, there are many issues to analyze with this trial in Kenya and the greater implications of what can be done to improve teacher absentee rates and student performance minus the critical elements of strong supervision, parental involvement, and social drive for educational advancement. Without actual, tangible societal shifts and community focused shifts, any approach will be non-sustainable. Without the larger scope of societal progress being a critical input into the process, incentives will be hollowed out and meaningless for the intended beneficiaries, ie: the students. And so, we are faced with the hard questions yet again. What is to be done? The shift needs to occur at the personnel level of the school bureaucracies; without strong incentives to work, people will not work, it is simple, whether these be intrinsic/societal/cultural or extrinsic/pay factors. Either will do; both are especially advantageous. But the lack of either is crippling, and non-permanent, non-indigenous approaches, as seen in this test case, will only offer temporary solutions, at best, and encourage deviance, at worst.

The full report can be seen here, courtesy of the MIT Poverty Action Lab Website:

http://www.povertyactionlab.org/evaluation/teacher-incentives-based-students-test-scores-kenya

Teacher Attendance: What Works Most?

 "One can build schools and clinics and stock them with books, drugs, and equipment, but if the teachers, nurses, and other providers are chronically absent, these investments will be wasted."

In looking at the core issues facing educational development and the reformation/enhancement of educational systems in the developing world, the key issues of teacher quality and teacher attendance are the most critical variables in this complex puzzle. Teachers are the human elements that have the most direct impact on the educational process and the ability to make or break an educational system, bar none. One bad student and a good teacher, no problem. Five bad students and a good teacher, no problem. Thirty great students and a bad teacher, problem. This is an issue that transcends national and demographic barriers; in the richest nations, bestowed with the most efficient educational systems, the heaviest weight and concern goes into ensuring teacher training and quality. Thus, how can quality be ensured in nations where training often doesnt exceed 6 months, and teachers are thrown into foreign communities with little support and little incentive? I have seen these trends over and over again, from the US, where I taught in the inner city of New York for two years, to Africa and Asia, where I have conduced numerous school visitations and evaluations for this project. In addressing teacher quality as the key aspect of educational reform, teacher attendance is a core, vital pillar. When visiting Mozambique this summer, I was told repeatedly by both teachers, community members, and NGO personell in the field that teachers could be, and thus often were, absent for extended periods of time with no penalization and oversight. In one case, the school principal was absent for the better part of a year, leaving the school to wither in her absence, as any hierarchical organization would most likely do, especially in a lax cultural environment. The Poverty Action Lab at MIT has been looking into the issue of teacher attendance, and have come up with some very interesting results after their randomized trials have been conducted and assessed. I am going to be digging into the results of this overall study, first looking at the most effective cost-ratio solutions to teacher attendance and quality http://www.povertyactionlab.org/policy-lessons/education/teacher-attendance                                    MIT Poverty Action Lab Site:

Innovations for Poverty Action conducted a successful trial in which supplementary community-based teachers, who make much less than full-sector teachers, were brought into the schools, to be monitored by the local community-boards. The results were positive; these contract teachers had the initial impact of lowering class sizes, which is a huge problem in developing-world nations with large scale literacy and primary education drives, which are often pushed by multi-lateral organizations (and is another subject for discussion in itself...) as well as improving overall test scores.The biggest gains are seen when the local school committees are empowered to monitor the supplementary teachers

The issue of supplementary teachers, as used in this study, brings up some red flags. The first is that the approach seems to be a band-aid; it is not dealing with the actual problem of teacher quality and teacher training, it is, rather, providing an additional fix without curing the root problem. While not negative in itself, if the educational process can be improved in such nations as Kenya where there has been free-universal primary education drives, the fix needs to come from the national level, which needs to be more focused on producing quality teacher training academies and a culture of competition and incentivization. This could be seen as a great model moving forward, a model to be built upon, and works to address two of the most critical areas, in terms of class size and teacher motivation/quality.

In the area of incentives for increasing the educational effectiveness, the randomized trials that were seen in the most positive light by the folks at MIT came in a merit-based girls scholarship program in Kenya: http://www.povertyactionlab.org/evaluation/incentives-learn-merit-based-girls-scholarship-program-kenya               Kenya Girls Scholarship Trials:
The researchers in this case looked at student motivations and progress in the classrooms of Western Kenya. In these trials, the researchers were looking to address the key issues of high primary school dropout rates (a result of the push for universal primary education and the lack of local resources to actually implement this) and scholarships to cover the small primary school fees that students must pay to cover educational and classroom expenses, which is often a cause of the high dropout rate as well. The scholarships were merit-based, and given to sixth-grade girls who scored in the top 15% of national tests. The results were skewed, and show the importance of local customs and norms in the implementation of any aid project.  In one community where there was a large pool of disadvantaged students and high public skepticism, the program did not show any effectiveness; however, in another community where there was higher rates of receptivity from the onset, the program was shown to both improve teacher effectiveness and attendance, as well as increasing test scores and parental involvement. 
What is the takeaway from this set of trials? Community involvement, that old cliche of aid and development, is absolutely essential to effectiveness of any community/school project. However, how to gain this involvement is the great puzzle; motivation must be seen in the structure of the local community, elders and chiefs must be on-board and motivated, and the leadership as a whole must have ownership, as I have seen and stated time and time again. Funding and supervision can easily come from the outside, but if there is no buy-in to the programs, there will be no net impact, which can also further the risk of an increase in future failure rates. 

MIT Poverty Action Lab/Education Technology

A great bit of research courtesy of the MIT Poverty Action lab on the effectiveness of technology training on primary school students in India. Students were tested with both after school, additional computing classes, a pull-out model of during-school classes, and no extra computer classes at all; the results were quite interesting....Niranjan Rajadhyaksha reported on the findings in the Wall Street Journal: 


Computers or classrooms?
The role of the teacher is restricted to switching on the computers and allocating them to different batches of children
Cafe Economics | Niranjan Rajadhyaksha

Ever since fears erupted about a decade ago that the world could be divided into digital haves and have-nots, policymakers and do-gooders have assumed quite correctly that this digital divide needs to be bridged. The most obvious first step was to give children from poor families access to computers, in school and at home. From that followed ambitious programmes as One Laptop Per Child (OLPC), which is funded by some of the world’s best firms such as Google. Some visionaries even dream of an education system where the teacher is replaced by a computer programme.

Does this plug-and-study idea really work in poor neighbourhoods? Not necessarily, it seems.

True, the initial findings were encouraging. Many studies showed that poor kids improved their exam scores when they had access to computers. But more recent studies cast some doubts on the assumption that the academic performance of children from poor families improves with access to computers. In other words, plonking a computer in front of a kid does not necessarily do the trick.

In one recent study in Gujarat, Leigh Linden, an economist with Columbia University, and the MIT Jameel Poverty Action Lab evaluated how academic performance changed when computers were introduced in classrooms. The data was collected from schools in the slums of Ahmedabad and some other towns and villages in Gujarat that are run by Gyan Shala, an NGO. Children in these schools get one hour of computer time each day. The role of the teacher is restricted to switching on the computers and allocating them to different batches of children.

Linden found that a lot depends on how the computers are used — as complements or substitutes for the teacher and the regular curriculum. The programme of computerized learning does not work too well when it is used to substitute the teacher in the normal school day. Math scores actually dropped in schools that took this path. The “out-of-school” alternative — when students sat at the computers either before or after school — showed better, though modest, improvements in academic performance. Here, the learning software is a complement rather than a substitute for the usual curriculum. Further, Linden says the worst students benefited the most in this case.

The Gujarat study shows that merely providing computers in schools is not much of an answer. A lot depends on how they are used, when they are used, and who uses them.

Another study from across the world has an even more sobering lesson. Economists Ofer Malamud and Cristian Pop-Eleches turned their eyes on what happens when poor children in Romania get computers at home. As part of a programme, called Euro 200, some poor Romanian families were given euro 200 to buy computers for their children. Other families with similar income levels did not get this subsidy because of budget constraints. The two economists compared what happened in the two groups of families which were alike in almost every other respect.

There is much to be learnt: Kids with computers saw less television, but they also had less time for their homework. Grades dropped. “The lesson from Romania’s voucher experiment is not that computers aren’t useful learning tools, but that their usefulness relies on parents being around to assure they don’t simply become a very tempting distraction from the unpleasantness of trigonometry homework. But this is a crucial insight for those tasked with designing policies to bridge the digital divide,” writes Ray Fisman, in a June article for online magazine Slate, where Malamud and Pop-Eleches’ research was cited.

Does this mean that computers have no role in classrooms? Does this mean that the age-old talk and chalk teaching routine is irreplaceable? There is no need to draw such dark conclusions. (And these are dark conclusions, since schools do need reform. Peter Drucker​ once pointed out that our schools are the only social institutions around us that have not changed at all since the Industrial Revolution​. Everything else — from governments to workplaces to families — has been radically transformed.)

The more limited point is that it’s not just an issue of lavish funding and putting computers in classrooms. The OLPC mission statement reflects this belief: “To eliminate poverty and create world peace by providing education to the poorest and most remote children on the planet by making them more active in their own learning, through collaborative and creative activities, connected to the Internet, with their own laptop, as a human right and cost free to them.”

In the Gujarat study, Linden draws attention to several more cost-effective ways to improve the academic performance of children from poor families — cash incentives for teachers, scholarships for girls and access to textbooks. And good libraries, too. Computers are part of the answer — but perhaps not the most important part.

MIT Poverty Action Lab-Madagascar Education Development Trials

I have been reading into the MIT Poverty Action Lab's work since seeing their test results published in the book Economic Gangsters. In looking specifically at their randomized trials in the educational setting, I was drawn to recent work done in Madagascar, which looked at the effects of both a "top down" and a "bottom up" approach to school interventions. Madagascar is filled with the same issues as most of the developing world in terms of lack of truly progressive educational policy and a stagnant public education system riddled with huge systemic problems. The details are as followed for the "intervention:"

Researchers, in collaboration with The Ministry of Education in Madagascar, ran a randomized experiment in 3,774 primary schools in 30 public school districts. These districts represented all geographic areas in the country, but were focused on schools with the higher rates of grade repetition.

All district administrators in treatment districts received operational tools and training that included forms for supervision visits to schools, and procurement sheets for school supplies and grants (district-level intervention). In some of these schools, the subdistrict head was also trained and provided with tools to supervise school visits, as well as information on the performance and resource level at each school (subdistrict-level intervention).

Lastly, several districts also introduced a school level intervention which involved parental monitoring through school meetings. Field workers distributed a ‘report card’ to schools, which included the previous year’s dropout rate, exam pass rate, and repetition rate. Two community meetings were then held, and the first meeting resulted in an action plan based on the report card. One example of the goals specified in the action plans was to increase the school exam pass rate by 5 percentage points by the end of the academic year. Common tasks specified for teachers included lesson planning and student evaluation every few weeks. The parent’s association was expected to monitor the student evaluation reports which the teachers were supposed to communicate to them. These tools allowed parents to coordinate on taking actions to monitor service quality and exercise social pressure on the teachers.

What is so interesting about the results is that the top-down approach, which is the traditional development approach of dealing with school reform, did practically nothing to actually improve the conditions on the ground. What showed large results were the "bottom up" trials, in which parental monitoring, field workers, and community meetings following specific action plans. Here are the details from the MIT site: 



Impact from Bottom-Up Approach: The interventions at the school level led to significantly improved teacher behavior. Teachers were on average 0.26 standard deviations more likely to create daily and weekly lesson plans and to have discussed them with their director. Test scores were 0.1 standard deviations higher than those in the comparison group two years after the implementation of the program. Additionally, student attendance increased by 4.3 percentage points compared to the comparison group average of 87%, though teacher attendance and communication with parents did not improve.