Evidence-based treatments for students with autism have tended to be derived almost solely from the conceptual model of Applied Behavior Analysis (ABA). For reviews of such evidence, please see The National Autism Center’s Standard’s Report, 2009 and the Maine Administrators Report, 2009. Since 1993-94 school year through the 2011-12 school year, the number of students receiving individualized education plans under the Individual with Disabilities Education Act (IDEA) disability definition of autism has increased in Pennsylvania from 498 students to 23,405 students (PDE Child Count data, 2012.) This trend is reflected in national numbers documenting the incidence of individuals with autism, which is now stated to occur at a rate of 1 in 88 live births (CDC, 2011.) Together these two phenomena, the rising prevalence numbers and evidence base for treatments, suggest a need for school-based programs for students with autism that are well structured and based on the principles of ABA. In Pennsylvania, The Pennsylvania Training and Technical Assistance Network Autism Initiative (PATTAN AI) has provided ABA-focused training and on-site technical support to a wide range of public school autism support programs. The efforts are a collaboration of PATTAN, Tuscarora Intermediate Unit 11 (providing administrative support), the Pennsylvania Department of Education's Bureau of Special Education, and many school districts and intermediate units. This brief review will discuss the history of PATAN AI ABA Supports and its basic training structure and methodology.
During the 2012-2013 school year, approximately 240 participating school based Autism Support providers received consultative support and training from PATTAN AI. In the current school year (2013-2014) there are over 335 classes receiving consultative support and training.
PATTAN’s AI ABA Supports has been in place since 2002-03 school year; however, through the period of 2003-2009 the effort was known as the Pennsylvania Verbal Behavior Project. The effort originally started with consultation provided in two classes in the Wilkes-Barre area. Currently, PaTTAN consultants guide training efforts often in collaboration with Intermediate Unit Technical Assistant Consultants (TACs). Direct collaboration occurs with public education settings across the state including school district elementary, middle and high school classes, early intervention classes, Intermediate Unit classes, Approved Private School settings, and Charter Schools.
The needs of children with autism and their families are quite diverse. The current Diagnostic and Statistical manual of the American Psychiatric Association (DSM V) describes individuals with autism as presenting “Autism Spectrum Disorders (ASD)”. The word ‘spectrum’ is used to emphasize that, although individuals with this disability all share certain characteristics such as social-communicative deficits and repetitive or stereotyped behavior, the level at which individuals function across intellectual, social, adaptive and communication skills vary widely. One thing all of these children share, however, is the need for effective instruction.
A goal of PATTAN’s AI ABA Supports efforts is to assist teams in delivering evidence based interventions for children with autism. A unifying aspect of effective intervention is a basis in clearly defined performance standards, reliable progress monitoring, procedural descriptions of instructional practices, and teacher responsiveness to student progress within a well-sequenced list of skills targeted for instruction. In short, the characteristics of applied behavior analysis clearly overlap with the qualities of effective instruction. Applied Behavior Analysis is a systematic process to achieve socially significant outcomes based on a scientific approach to human behavior. Behavior is defined as observable and measureable events that occur in relation to regular aspects of the environment in which individuals act. The environment includes any observable aspect of the social and physical world. The analysis involves use of methods of measuring behavior and the antecedent and consequent events that correlate with predictable patterns of responding. Skills related to thinking, reasoning, and emotion are defined through their overt characteristics, in other words what the students do to indicate their covert activity.
The core deficits characteristic of autism are, as mentioned earlier, related to social-communicative differences and the presence of repetitive or stereotypical behaviors. Well-designed instruction should address those core deficits and also assist students in accessing general education curricula. Through use of the systematic application of ABA, teachers receiving consultation through PATTAN’s AI ABA Supports have learned how to develop effective programs to promote social-communicative skills for students with ASD. In this model, communication is seen as something individuals do: it is behavior and therefore is observable and measureable. Communication is also seen as something that is inherently social. Systematic instruction to develop skills allowing a full set of appropriate social-verbal responses includes teaching students to make requests (mand training), to label things that they see, to respond verbally when people speak to them in conversations, and to respond as listeners by following directions. The theoretical constructs of B. F. Skinner’s model of verbal behavior and current verbal behavior research is used to design and guide language interventions across participating sites.
The consultation model includes multiple key components. Instructional staff is provided guidance related to organizing the classroom environment, materials, classroom schedules, and data systems. Consultation also focuses on building school staff capacity to deliver instruction in least restrictive settings and support effective inclusion. Central to the model is the provision of support related to building effective processes of instructional delivery. Teams are trained in how to teach using errorless teaching processes, discrete trial instruction, Direct Instruction methods, vocal training procedures, error correction, generalization procedures, natural environment training, social skills instruction, effective group instruction and fluency training. Pairing instruction with improving conditions, in other words, positive reinforcement is a central focus of consultation. In order to establish instructional control, so that students enjoy learning, key variables including motivation and pace of instructional delivery are addressed.
Participating sites receive consultation from PATTAN consultants on a regular basis. About 35 consultants are involved in this effort. Most consultants are Board Certified Associate Behavior Analysts or have extensive experience in ABA-guided instruction for children with autism. Participating educational agencies also provide consultative support to the sites. Each participating school district or agency provides an internal coach. The role of the internal coach is to work with the PATTAN consultants so that they can acquire the technical aspects necessary to provide consultation based on the principles of ABA. The internal coaches receive ongoing training through PATTAN’s AI and work collaboratively with PATTAN AI ABA Support consultants so that independent implementation of applied behavior analytic programming can occur in their school districts or Intermediate Units.
Often interventions fail to be effective because they are not implemented consistently or in the way they are designed. Efforts are ongoing within this consultation model to establish and maintain consistent instructional fidelity. This is accomplished through the use of various procedural checklists as well as through methods of transcribing instructional delivery. Teachers are provided ongoing and direct feedback as to the degree to which they are delivering instruction as designed. Instructional procedures are spelled out in training manuals that are written or provided in video form
Overall implementation within participating sites is measured through a rigorous site review process. The site review is a checklist of 61 measureable items. The site review checklist is used to assess all sites in the beginning of each school year and then again late in the school year, thus it is serves as a pre-post measure of consultative implementation. Specific site review items are grouped into several main categories:
- Classroom organization, including scheduling, data systems, and materials organization
- Inclusive practices
- Consultation Processes, including degree to which consultation guides practice and the degree to which treatment integrity processes are used.
- Parent and Family Engagement
- Instructional practices including intensive teaching, mand training, natural environment training, fluency training, Direct Instruction, social skills instruction, vocal training, and group instruction.
- A function-based approach to behavior interventions to reduce problem behavior
- The initial site review is used to set priorities and guide consultation for regularly scheduled sessions.
Over the past several years, for a total of 998 site reviews, the average level of implementation at the beginning of the school year was 49% items implemented. The average level of implementation for those 998 site reviews at the end of the school year was 79% of items implemented. Such significant increases in evidence-based instructional practices suggest that consultation and training has been effective.
Prior to beginning consultation most instructional teams attend a three day intensive skill training session. The training is competency based and involves active participation of those in attendance. This training has been provided for the past four years to over 2000 participants. Over 97% of participants have demonstrated competency outcomes including the ability to identify teaching trial type (i.e. mand, tact, intraverbal) at a rate of over 25 per minute. Over 99% of participants are able to demonstrate basic errorless teaching procedures, errorless correction procedures, and basic data collection processes. Over 97% of participants are able to demonstrate intensive teaching (i.e. discrete trial teaching) and mand teaching procedures. Additionally, the mean score of participants on conceptual tests in which they define key terms related to behavior analysis including reinforcement, discriminative stimulus, and motivating operation is 93%.
Consultation is structured through a guided practice format. The consultants explain procedures, model the instructional techniques and then have the instructional staff practice the techniques with specific feedback. The staff then adopts the practices as appropriate for individual students’ needs as determined by Individualized Education Program (IEP) teams.
Assessing the effectiveness of the model on direct student outcomes has been a challenge given the diversity of students served across consultative settings. However, almost all students are administered the Verbal Behavior Milestone Assessment and Placement program (M. L. Sundberg, 2008). During the 2009-2010 school year, VB-MAPP scores for 122 students were compiled. The students had a mean score of 58 on the fall VB-MAPP assessment. The students had a mean score of 73 on the spring re-assessment. This indicates an average gain of 14.7 milestones. Each milestone may include a number of discrete skills. During the 2012-2013 School year, VB-MAPP outcomes were reviewed for 822 students. The fall average score was 78 milestones and the spring average was 92 milestones, with an average change of 15 milestones. During both study years, most students demonstrated gains beyond what would have been predicted by their estimated baseline rate of skill acquisition.
Additionally, student outcomes are evidenced by gains in particular skills within instructional programs. Ongoing mastery of skills is measured through data collection. Graphs showing cumulative skills acquired for individual students consistently indicate mastery of several hundred significant skills acquired each school year. Please see the video of case studies completed within the Harrisburg School District during the 2011-2012 school year for a sample of such data. (Sample Data)
In summary, PATTAN’s Autism Initiative Applied Behavior Analysis Supports is assisting teams across the Commonwealth in developing and maintaining effective instructional practices for students with Autism Spectrum Disorders. The approach includes an inter-locking system of staff training, student assessment, formative assessment for instructional decision making, materials organization, instructional procedures, and treatment integrity. Actual implementation of instructional practice has been verified through the site review process. Instructional consultation is context based in that it focuses on actual teaching practices and the relation of that practice to student learning. The efforts represent a positive approach guided by the principles of reinforcement and skill building. Because instruction is focused on key social-communicative skills for students with ASD, teachers are reporting meaningful outcomes for students related to language skills, social skills, academic skills, reasoning skills, and quality of life. Data reviewed in this report are suggestive of meaningful student outcomes.
Effective educational programming requires a specification of practices that will result in student achievement. While practices related to the delivery of instructional content is important, effective practice involves much more. Teachers and other instructional staff need to address organizational variables that make teaching practices efficient and verifiable. Data systems allowing responsiveness to student performance must be part of the mix. Additionally, for students to be able to access instruction, they need to have developed an ability to cooperate and respond to instruction. Preparing the teaching circumstances so that learning has value to the student is critical. The development of clear, positive steps to reduce any problem behavior that may occur is an important part of classroom management. Additionally, processes need to be in place to insure that teaching is delivered in consistent patterns and that all staff are trained to a high level of competence.
Making evidence-based practice common can be facilitated by having a list of specific practices that promote well-defined outcomes. The PaTTAN Autism Initiative ABA Supports effort has developed a Site Review as a system that allows a criterion based assessment of implementation of effective practice. The Site Review form is designed to provide direct feedback on implementation of consultative support provided by PaTTAN staff and internal staff at the local level. Bi-annual pre and post school-year completion of the Site Review by independent reviewers provides feedback to participating sites on their level of implementation of effective practices and response to focused consultation. Inter-observer agreement has been established for over 8% of all site reviews and has yielded agreement levels at about 93% across criteria.
We are providing a copy of the PaTTAN Autism Initiative Site Review to allow access to the criteria specified on the form in hopes that it will help guide practices in school based autism support programs.
Annotated Site Review