Final Year Project Proposal

 

ARGuS for Autism

 

 

Submitted By

Aamir Shahzad                       14-SE-97

Mudassar Saeed                      14-SE-91

Muhammad Abdul Mueed      14-SE-33

 

Supervised

Mam Kanwal Yousaf

Lecturer SED UET Taxila

 

DEPARTMENT OF SOFTWARE ENGINEERING

  FACULTY OF TELECOMMUNICATION
AND INFORMATION

 ENGINEERING

UNIVERSITY OF ENGINEERING AND TECHNOLOGY

 TAXILA

 

December 21,2017

 

Contents
Introduction. 2
Project Goals. 2
Aims & Objectives. 3
Deliverables. 3
Literature Review.. 4
Literature Survey. 4
Marketing Survey. 6
 

 

Chapter 1

Introduction

 

Project Goal

Autism Spectrum Disorder is a prevalent
development disorder that causes impairment in person’s hearing ,feeling
,speaking, thinking and social interaction1. All over the world out of 88 children 1
child is victim of ASD. For this reason, children suffering from autism need to
follow special training in order to increase their ability to learn new skills
and knowledge. These children have propensity to be attracted  by technology devices especially virtual
animations. the interest of this app is to provide an environment for children
to get education and social skills. Using Augmented Reality and 3D models with
animations gives the children a real-life experience which will create a
positive improvement in terms of enjoyment, involvement, development of social
and learning skills to the autistic children. This application will build
social skills and non-verbal behavior in effected children. It will provide an
effective way of learning. Application can also be used to teach normal kids.

Aims & Objectives

 

·       
Help Autistic
children to improve social skills using augmented reality.

·        
Provide a more better way
for education to children with autism.

·        
Help autistic children to
make better their communication with people and social interactivity.

·        
To ease the Families with
autistic children and social care centers with providing them with virtual
assistant.

 

 

 

    

Deliverables

 

•        
Android Application

 

Android application that uses 3d models which
user can interact with it and play games providing education and social skills

 

•        
Augmented reality:

 

            
3D models augmented in camera that interacts with user.

 

•        
Documentation

 

Detailed documentation on working procedures

 

•        
  3D
models and Animation

 

•        
   User
Manual & Guideline

 

       Guide on how to use the application

 

Chapter 2

Literature Survey

Lakshmiprabha N. S. 
introduced a system to help children with
Autism. This system teaches about new objects and pictures with their
associated keywords. The System is based on the idea that any correct behavior
can probably be repeated if it is followed by some reward. The tutoring
begins by letting the kid view a couple of pictures (usually three) or
objects(like elephant)  which the kid
should learn. the kid is requested to join every object and picture with their
respected keywords.. Steps are repeated various times changing the pictures or
objects and soliciting the kids to show the related item. This process with the
rewards give the kids motivation to memorize the keywords with their related
pictures and objects.

 The main aim of the project was to
develop AR system for training the kids with ASD. This system encourages the
kid to interact, copy a model, and give response to the primary requests such
as keyword connection and matching the sentences to its related pictures and
objects.1

 

Richard E proposed an application
called ARVe (Augmented Reality applied to Vegetal field). It is an augmented
reality based application which allows kids to handle 2D and 3D plant entities.This
system consist of task of pairing with providing visual and audio cues for
help.The objective of the system was to examine kids performance in using AR
System, Also to check attitude of cognitive disabled children towards the AR.  It provides the option of playing and
interacting with the virtual entities and their supports Also, it offers the
children to uncover some characteristics of the plants but also to adjust the
gestures according to an intention and handle the entities.

Cues which were provided to kids
were:

·        
Visual cues: Entities were surrounded by red
color when misplacing the object and with blue color when correct.

·        
Auditory cues: The recorded audio of fruit
names were provided as kid brings the given entity near his face.

·        
Olfactory cues: Oder of related fruit was displayed
as as kid brings the given entity near his face.2

 

 

 

 

 

 

Pedro Cunha suggested a  Gamebook to help kids with Autism to recognize
and obtain emotions by engaging their attention and motivation was developed.
The aim was to promote the interaction between the kid and his/her fascination. Also help the kid to
recognize right emotion at right situation.

This Gamebook
shows the story of Tobias’s adventure during a visit to a zoo park. The story
will present five stories which contains scenarios and provide interaction. The
kid will interact with the stories by playing with Tobias and learning his
various facial expressions.it can be played on any mobile technology such as
tablet.

The stories represent
five different emotions which are joy, sadness, fear, anger and disgust. To involve the child on real life
situation these different scenarios was developed .when the chapter ends the kid
have to give a simple quiz before going to next chapter ,the question will
about what the emotion Tobias felt on the situation described  before.3

 

 

 

Zen bai proposed a system that uses Augmented Reality (AR) visually
conceptualize the representation of pretense within an open-ended play
environment(lack of imagination).it is based on idea that kids with autism
suffer from development delays in symbolic thinking. In this System ,various
blocks were used to represent different shapes (like cars, ships).when kid
places the block in front of camera it changes the block and present an augmented
shape of the block. Kid can play easily by moving these object and making an environment.
To help kid in making environment this system provided shapes of school, petrol
,bridge, carriage, light , station, hanger, runway.4

 

Chien-Hsu Chen proposed an Augmented Reality -based video
modeling Storybook (ARVM) to attract kids having Autism to help them with switching
and adjusting their attentional focus. The System was
used to teach the kids to recognize patterns as they focused on the non moving  visual image printed in the storybook and then
extended their attention to an animation of the story.

The ARVMS learning system takes video materials that depict everyday
life activities and it focuses on social cues and special moments  suitable for children with ASD.The story book
consist of frozen images captured from video.when the kid moves the device on
the storybook and the kid sees the story into a moving video.This feature attracts
the kid to watch the movies of story book.5

 

 

 

 

Marketing Survey

·        
Empower Me:

Empower Me is an electronic life
coach that children (and adults) with autism love! Developed by neuroscientists
and doctors from MIT and Harvard.6

·        
Movements:

Moments App  uses augmented reality to help teach various
emotions and role-playing social scenarios to children with ASD.7

·        
IPrompts:

IPrompts is an
app for supporting organizational skills, setting expectations for performance,
and setting up subtle supports and reminders.Users can create and modify visual
schedules, as well as use a countdown timer with picture supports to indicate
how much longer a task or activity will last.

·        
Autism
Track:

This
customizable data tracking tool allows parents to easily track behaviors,
interventions and symptoms in one place. Parents can also review trends in
their child’s data and share these data with school and medical providers to
inform treatment planning.

 

 

 

iPrompts

“iPrompts is an app for supporting
organizational skills, setting expectations for performance, and setting up
subtle supports and reminders. Users can create and modify visual schedules, as
well as use a countdown timer with picture supports to indicate how much longer
a task or activity will last.

Autism Track

This app is brought to you by the creators of
iPrompts and is designed for parents of children with ASD to help them track
data. “This customizable data tracking tool allows parents to easily track
behaviors, interventions and symptoms in one place,” says Aguayo.
“Behaviors and symptoms can be rated, as well as particular medications
and their doses, diet changes and therapies. Parents can also review trends in
their child’s data and share these data with school and medical providers to
inform treatment planning. This app is especially helpful for children with
challenging behaviors, psychiatric conditions or both.”8

 

 

Marketing
Plan

 

Hospitals and Autism Care Centers

Visiting different hospitals, Autism Care
Centers and offering them the product.

 

Online Health pages

We will advertise our product in online health
pages and specially related to autism, so we can approach different people
globally.

 

Families with autistic children

Visiting different families having autistic
children and pursue them to buy our product.

 

Customizing our Product

We will customize our product for specific
organization based on what are their needs and requirements.

 

Government Health Organization

We will offer our app to government health
organization to buy this product and use it in government hospitals.

 

 

   Play Store

       
i.           
Free App with advertisement and limited
features.

     
ii.           
Purchased App with no advertisement.

   
iii.           
Additional features and packages.

 

Chapter 3

Methodology

Literature Survey

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

Chapter 3

Proposed Solution

Methodology:

ARGuS is based on two types of augmented reality
feature based and environment based. Feature based augmented reality provide
ability to participant to play using flashcard using indoor environment. In
Environment based augmented reality participant can play in outdoor environment
with need of flashcard.

Feature
Tracking

1)    Initialize
play material (Put flash cards on table/floor).

2)    Run
mobile app and select feature tracking.

3)    Focus
camera on the flash card.

4)    App
will capture image frame.

5)    Performa
feature extraction on image based on texture distribution, contrast
distribution and corner points of shapes in the image.

6)    Do
compare extracted features with decoded image in the app database.

7)    After
comparison it will place a virtual plane on the image.

8)    Decode
the encoded 3D model.

9)    Perform
realistic rendering and place 3D model on the image.

10)                       
Show the result on
output.

 

Environment
Tracking

1)    Run
mobile app and select environment tracking.

2)    Initialize
the environment tracker.

3)    Select
point to place model in environment.

4)    App
will capture image frame.

5)    Simultaneous
Localization and Mapping is an algorithm that is used to track the environment
from extracted frames.

6)    After
comparison it will place a virtual plane on the image.

7)    Decode
the encoded 3D model.

8)    Perform
realistic rendering and place 3D model on the image.

9)    Show
the result on output.

 

 

 

 

 

 

 

Development
Tool

   Android Studio

Android Studio is an IDE for developing for the Android
platform. Android Studio is freely available under the Apache License 2.0 7. Based on JetBrain’s
IntelliJ IDEA software, Android Studio is designed specifically for Android
development.

   Features

Ø  Gradle-based build support.

Ø  Android-specific refactoring and quick fixes 8.

Ø  Lint tools to catch performance, usability, version
compatibility and other problems. ProGuard integration.

Ø  Template-based wizards to create common Android designs and
components.

Ø  A rich layout editor that allows users to drag-and-drop UI components,
option to preview layouts on multiple screen configurations.

 Auto Desk Maya

Maya is a 3D animation, modeling, simulation, and
rendering software helps artists work faster with enhancements to existing
workflows and new tools.

   Features

Ø Make
high-level animation edits with a nondestructive, clip-based, nonlinear editor.

Ø Fine-tune
your characters and animation more quickly and easily.

Ø New
system speeds rig playback and manipulation.

Ø And
more

 

Project time-line:

Experimental/ Simulation Setup:

 

 

 

 

 

 

 

 

 

 

 

 

 

 

Details of Work packages
completed/milestones achieved:

Task

Achieved

Requirements Gathering

?

Feasibility Study

?

Requirements Analysis

?

Requirements Validation

?

Android App Design

?

Concept Modeling

?

3D Modeling

?

Scenario Developing

?

3D Models Animation

?

Scenario Execution in App

?

Integrating models into App

?

Java coding

?

Unit Testing

?

Setting up Experimental Setup

?

Execution of Experimental Setup

?

System Test

?

Bug fixes

?

Results Evaluation

?

Documentation & User Manual

?

Release version

?

Requirements Gathering

?

 

 

Evaluation parameters:

Since our system is designed  for
kids with Autism ,so our evaluation parameters will be:

·       
Did this app improved social skills
of kids?

·       
Did this app provided education for
kids?

·       
Did this app improved behavior and interactivity
of kid with others?

·       
Does the kid likes the technology used
in this app (Augmented Reality)?

 

 

Chapter 4

Work Plan

Utilization
(End users/Beneficiaries):

•        
Our target
users/beneficiaries are Autistic children.

•        
Autistic
children can improve their social behavior.

•        
Autistic
children families will be assisted by ARGuS app to teach their children.

•        
Autistic
children will be educated in an innovative and attractive way.

•        
Autistic
children will enjoy playing games also learn many things.

•        
This app will
serve as a special kind of tutor in hospitals and ASD Centers.

•        
Our market
audience includes Specially hospitals, Autism Care Centers, families having
Autistic children, health groups, Government Health Organizations.

 

Detailed
Work Plan:

The plan we have proposed to complete the entire project is
based on 270 days.

·        
75
days will be served for Requirements Specification including requirements
gathering, feasibility study, requirements analysis, requirements validation,
proposal report and SRS.

·        
 90 days will be served for Designing and
Development including android application design, concept modeling,3D Modeling,
Scenarios developing ,3D Models Animation  .

·        
60
days will be served for Coding and Environment setup including Java coding, unit
testing ,Setting up experimental setup and Execution of experimental setup.

·        
 35 days will be served for testing and Evaluation
of results including System testing ,Bugs fixing,results evaluation. , android
application final version and integration. Then, 20 days will be consumed doing
Testing and Deployment that will include alpha testing, beta testing, system
testing and system deployment. Remaining 30 days will be utilized for
Documentation composed of user guide and thesis.

·        
Last
25 days will be served for Documentation , making user manual and Releasing the
final version of app.

 

 

 

 

 

 

 

Budget
Requirement:

Total budget
we need is 22000 Rs for as follows:

 

Requirements:

·        
Research:
Rs.1000 ,since we need a subscription of websites that have e-researches in
order to download them.

·        
ASD
Center:in order to visit ASD Center which is far away we need money estimated
Rs.1000 (vehicle rent.ASD consultant fees).

·        
Survey:Rs.1000
,since we need to print papers ,vouchers and assign works to different teams
and pay them.

 

Design:

·       
Modeling:Rs.1000,
we need to purchase licensed account for modeling softwares.

·       
Rigging:Rs.1000
, we need to buy plugins to rig the models.

·       
Animation:Rs.500,we
need to hire person for animation of models

·       
Interface
Design:we need to buy widgets libraries in order to import them.

 

 

Implementation:

·       
Add
model to app:Rs 500, we need to buy sdk in order to import them.

·       
Give
interactivity to model:Rs 500, we need to buy sdk in order to import them.

·       
Adding
Chat bot:Rs.1000 ,we purchase to buy AI Microsoft library for this purpose.

 

 Testing:

·       
Rs.5000
,we need to go the ASD centers and pay them in order to use their resources and
we need to purchase testing tools.

 

 

Publishing:

·       
PlayStore:Rs.6000,
pay to google to create account at play store

·       
SDK:
Rs.4000 need to buy license to publish the app

 

 

 

 

 

 

 

 

Market
Forecasting:

 

 

 

Chapter 5

Conclusion

 

 

 

 

 

References:

1.            Lakshmiprabha, N., et al. Poster An augmented and virtual reality
system for training autistic children. in Mixed and Augmented Reality (ISMAR), 2014 IEEE International Symposium
on. 2014. IEEE.

2.            Richard, E., et al. Augmented reality for rehabilitation of
cognitive disabled children: A preliminary study. in Virtual Rehabilitation, 2007. 2007. IEEE.

3.            Cunha, P., et al. Augmented reality for cognitive and social
skills improvement in children with ASD. in Remote Engineering and Virtual Instrumentation (REV), 2016 13th
International Conference on. 2016. IEEE.

4.            Bai, Z., A.F. Blackwell, and G.
Coulouris, Using augmented reality to
elicit pretend play for children with autism. IEEE transactions on
visualization and computer graphics, 2015. 21(5):
p. 598-610.

5.            Chen, C.-H., I.-J. Lee, and L.-Y.
Lin, Augmented reality-based video-modeling
storybook of nonverbal facial cues for children with autism spectrum disorder
to improve their perceptions and judgments of facial expressions and emotions.
Computers in Human Behavior, 2016. 55:
p. 477-485.

6.            https://www.indiegogo.com/projects/world-s-first-augmented-reality-glasses-for-autism-education#/.

7.            www.rachelmadel.com.

8.            http://www.parenting.com/gallery/autism-apps.