||Augmented Reality Based Social Stories Training System for Promoting the Conversational Social Skills of Children with Autism
||Department of Industrial Design
Profound social reciprocity deficits are a core feature of the autism spectrum disorders (ASD). Children with autism often lack appropriate social skills when they would like to communicate with other people, and then result in a frustrated and embarrassed condition. Conversational skills offer guidelines toward various aspects. Having good conversational skills means the ability to get a better life. There have been many social skills training for autism children. One of the training methods is called Social Stories, which use accurate words with clear images to describe and explain a specific social situation. Although Social Stories is successful in autism treating, its interface and presentation still have room for improvement. We found that augmented reality (AR) technology could solve problems of the weaknesses when a social story was presented and created better user experiences. In this research, we would explore the potential of augmented reality to visually conceptualize social stories for twenty conversational skills.
We had three participants with autism. The research would use multiple baseline design and include several parts. First, we raised a new framework to transform original social stories into AR structures. The social meanings of different social relationships were also analyzed and transformed into corresponding control behavior. Then, we applied the framework to twenty social stories that described the conversational skills separately. All the stories were inspected and supported by experts. The AR social stories system, which followed the AR structures from the framework, involved tangible markers and virtual models with identical appearances of the participants, their parents, classmates and a stranger. The participants would interact with this AR system and watch AR animations of the twenty social stories for learning conversational skills in the experimental phase.
In this research, we demonstrated a new approach for social behavior intervention using the AR social stories system. Results have verified that the children with autism could improve their conversational skills. The new way to present social stories has an improvement of tool’s attraction and enhances effects of social skills learning. We expected to give future studies a potential direction and bring more developmental possibilities to the field of autism treating and augmented reality technology.
TABLE OF CONTENTS v
LIST OF TABLES vii
LIST OF FIGURES viii
CHAPTER 1 INTRODUCTION 1
1.1 Background 1
1.2 Motivation of this Research 3
1.3 Purpose 4
1.4 Importance of this Research 5
1.5 Overview of this Research 6
CHAPTER 2 LITERATURE REVIEW 7
2.1 Conversational Deficits of Autism and Visual Strategies 7
2.2 Computer-Based Training Ways for Autism 9
2.3 Augmented Reality and Its Use in Intervening Autism Learning 10
CHAPTER 3 METHOD 12
3.1 A Framework for Transforming Social Stories into AR Scenarios 13
3.2 Experimental Method 18
3.3 Experimental Design 19
3.3.1 Participants 20
3.3.2 Stimuli 21
3.3.3 System Design 31
3.3.4 Procedure 40
3.4 Evaluation Tools 45
CHAPTER 4 RESULTS AND DISCUSSION 47
CHAPTER 5 CONCLUSION 57
Appendix A THE SOCIAL SKILLS MENU OF JED BAKER 65
Appendix B QUESTIONS FOR EXPERIMENTAL EVALUATION 67
Appendix C THE SOCIAL STORIES IN THIS RESEARCH 74
Appendix D THE BASELINE DATA OF VINELAND ADAPTIVE BEHAVIOR SCALE 83
Appendix E THE FOLLOW UP DATA OF VINELAND ADAPTIVE BEHAVIOR SCALE 98
Alessandrini, A., Cappelletti, A., & Zancanaro, M. (2014). Audio-augmented paper for therapy and educational intervention for children with autistic spectrum disorder. International Journal of Human-Computer Studies, 72(4), 422-430.
Azuma, R. T. (1997). A survey of augmented reality. Presence: Teleoperators and virtual environments, 6(4), 355-385.
Baker, J. (2003). Social skills training for children and adolescents with Asperger syndrome and social-communication problems: Autism Asperger Publishing Company.
Bauminger, N. (2007). Brief report: Individual social-multi-modal intervention for HFASD. Journal of autism and developmental disorders, 37(8), 1593-1604.
Blacher, J., Kraemer, B., & Schalow, M. (2003). Asperger syndrome and high functioning autism: research concerns and emerging foci. Current opinion in Psychiatry, 16(5), 535-542.
Bosseler, A., & Massaro, D. W. (2003). Development and evaluation of a computer-animated tutor for vocabulary and language learning in children with autism. Journal of autism and developmental disorders, 33(6), 653-672.
Charlop‐Christy, M. H., Carpenter, M., Le, L., LeBlanc, L. A., & Kellet, K. (2002). Using the picture exchange communication system (PECS) with children with autism: Assessment of PECS acquisition, speech, social‐communicative behavior, and problem behavior. Journal of applied behavior analysis, 35(3), 213-231.
Chen, C. H., Lee, I. J., & Lin, L. Y. (2015). Augmented reality-based self-facial modeling to promote the emotional expression and social skills of adolescents with autism spectrum disorders. Research in developmental disabilities, 36, 396-403.
Cohen, S. (1998). Targeting autism: what we know, don't know, and can do to help young children with autism and related disorders. Berkeley and Los Angeles: California: University of California Press.
Cooper, M. J., Griffith, K. G., & Filer, J. (1999). School intervention for inclusion of students with and without disabilities. Focus on Autism and Other Developmental Disabilities, 14(2), 110-115.
Cumine, V., Dunlop, J., & Stevenson, G. (2009). Asperger Syndrome: A practical guide for teachers: Routledge.
Dalryaple, N. (1989). Learning to be independent and responsible: Functional programming for people with autism. ERIC Document Reproduction Service No. EC, 302, 520.
Di Serio, Á., Ibáñez, M. B., & Kloos, C. D. (2013). Impact of an augmented reality system on students' motivation for a visual art course. Computers & Education, 68, 586-596.
Ehrlich, J. A., & Miller, J. R. (2009). A virtual environment for teaching social skills: AViSSS. IEEE computer graphics and applications, 29(4), 10-16.
Escobedo, L., Nguyen, D. H., Boyd, L., Hirano, S., Rangel, A., Garcia-Rosas, D., . . . Hayes, G. (2012). MOSOCO: a mobile assistive tool to support children with autism practicing social skills in real-life situations. Paper presented at the Proceedings of the SIGCHI Conference on Human Factors in Computing Systems.
Farr, W., Yuill, N., & Hinske, S. (2012). An augmented toy and social interaction in children with autism. International Journal of Arts and Technology, 5(2-4), 104-125.
Frith, U. (2004). Emanuel Miller lecture: Confusions and controversies about Asperger syndrome. Journal of child psychology and psychiatry, 45(4), 672-686.
Frost, L., & Bondy, A. (2002). The picture exchange communication system training manual: Pyramid Educational Products, Incorporated.
Gray, C. (1993). The social story kit and sample social stories. Future Horizons.
Gray, C. (2010). The new social story book: Future Horizons.
Gray, C. A., & Garand, J. D. (1993). Social stories: Improving responses of students with autism with accurate social information. Focus on Autistic Behavior.
Gutstein, S. E., & Whitney, T. (2002). Asperger syndrome and the development of social competence. Focus on Autism and Other Developmental Disabilities, 17(3), 161-171.
Hagiwara, T., & Myles, B. S. (1999). A multimedia social story intervention teaching skills to children with autism. Focus on Autism and Other Developmental Disabilities, 14(2), 82-95.
Jordan, R. (1995). Computer assisted education for individuals with autism. Paper presented at the Autisme France 3rd International Conference, Nice.
Juan, M. C., Alcaniz, M., Monserrat, C., Botella, C., Baños, R. M., & Guerrero, B. (2005). Using augmented reality to treat phobias. IEEE computer graphics and applications, 25(6), 31-37.
Kaufmann, W. E. (2012). DSM-5: The new diagnostic criteria for autism spectrum disorders. Paper presented at the Research Symposium-Autism Consortium, Boston, MA.
Kawakita, J. (1991). The original KJ method. Tokyo: Kawakita Research Institute.
Kim, M. S., Blair, K. S., & Lim, K. W. (2014). Using tablet assisted Social Stories to improve classroom behavior for adolescents with intellectual disabilities. Res Dev Disabil, 35(9), 2241-2251. doi:10.1016/j.ridd.2014.05.011
Kratochwill, T. R. (1978). Single subject research: Strategies for evaluating change: Academic Press.
Laugeson, E. A., Frankel, F., Gantman, A., Dillon, A. R., & Mogil, C. (2012). Evidence-based social skills training for adolescents with autism spectrum disorders: The UCLA PEERS program. Journal of autism and developmental disorders, 42(6), 1025-1036.
McConnell, S. R. (2002). Interventions to facilitate social interaction for young children with autism: Review of available research and recommendations for educational intervention and future research. Journal of autism and developmental disorders, 32(5), 351-372.
Murray, D. (1997). Autism and information technology: therapy with computers. Autism and learning: a guide to good practice, 100-117.
Parsons, S., Leonard, A., & Mitchell, P. (2006). Virtual environments for social skills training: comments from two adolescents with autistic spectrum disorder. Computers & Education, 47(2), 186-206. doi:10.1016/j.compedu.2004.10.003
Prizant, B., Schuler, A., Wetherby, A., & Rydell, P. (1997). Enhancing language and communication development: Language approaches. Handbook of autism and pervasive developmental disorders (2nd ed., pp. 195-225). USA: John Wiley & Sons.
Putnam, C., & Chong, L. (2008). Software and technologies designed for people with autism: what do users want? Paper presented at the Proceedings of the 10th international ACM SIGACCESS conference on Computers and accessibility.
Quill, K. A. (1995). Teaching children with autism: Strategies to enhance communication and socialization: Cengage Learning.
Rao, S. M., & Gagie, B. (2006). Learning through seeing and doing: Visual supports for children with autism. Teaching Exceptional Children, 38(6), 26-33.
Richard, E., Billaudeau, V., Richard, P., & Gaudin, G. (2007). Augmented reality for rehabilitation of cognitive disabled children: A preliminary study. Paper presented at the 2007 Virtual Rehabilitation.
Sampath, H., Agarwal, R., & Indurkhya, B. (2013). Assistive technology for children with autism-lessons for interaction design. Paper presented at the Proceedings of the 11th Asia Pacific Conference on Computer Human Interaction.
Scattone, D., Tingstrom, D. H., & Wilczynski, S. M. (2006). Increasing appropriate social interactions of children with autism spectrum disorders using Social Stories™. Focus on Autism and Other Developmental Disabilities, 21(4), 211-222.
Slater Software, I.-P. I. a. P. (2016). Retrieved from from http://www.slatersoftware.com/
Smilansky, S. (1968). The effects of sociodramatic play on disadvantaged preschool children.
Sotiriou, S., & Bogner, F. X. (2008). Visualizing the invisible: augmented reality as an innovative science education scheme. Advanced Science Letters, 1(1), 114-122.
Spool, J. M. (2004). The KJ-technique: A group process for establishing priorities. User interface engineering.
Stokes, S. (2001). Autism: Interventions and strategies for success: Cooperative Educational Service Agency# 7 (CESA# 7).
Thornbury, S., & Slade, D. (2006). Conversation: From Description to Pedagogy: Cambridge University Press.
Villafuerte, L., Markova, M., & Jorda, S. (2012). Acquisition of social abilities through musical tangible user interface: children with autism spectrum condition and the reactable. Paper presented at the CHI'12 Extended Abstracts on Human Factors in Computing Systems.
Weiss, M. J., & Harris, S. L. (2001). Teaching social skills to people with autism. Behavior modification, 25(5), 785-802.
Wellman, H. M., Cross, D., & Watson, J. (2001). Meta‐analysis of theory‐of‐mind development: the truth about false belief. Child development, 72(3), 655-684.
Williams White, S., Keonig, K., & Scahill, L. (2007). Social skills development in children with autism spectrum disorders: a review of the intervention research. J Autism Dev Disord, 37(10), 1858-1868. doi:10.1007/s10803-006-0320-x
Wing, L., & Gould, J. (1979). Severe impairments of social interaction and associated abnormalities in children: Epidemiology and classification. Journal of autism and developmental disorders, 9(1), 11-29.
Wu, H.-K., Lee, S. W.-Y., Chang, H.-Y., & Liang, J.-C. (2013). Current status, opportunities and challenges of augmented reality in education. Computers & Education, 62, 41-49.
Zhen, B., Blackwell, A. F., & Coulouris, G. (2015). Using Augmented Reality to Elicit Pretend Play for Children with Autism. IEEE Trans Vis Comput Graph, 21(5), 598-610. doi:10.1109/TVCG.2014.2385092