Final Year Project Proposal
ARGuS for Autism
Submitted By
Aamir Shahzad 14-SE-97
Mudassar Saeed 14-SE-91
Muhammad Abdul Mueed 14-SE-33
Supervised
Mam Kanwal Yousaf
Lecturer SED UET Taxila
DEPARTMENT OF SOFTWARE ENGINEERING
FACULTY OF TELECOMMUNICATION
AND INFORMATION
ENGINEERING
UNIVERSITY OF ENGINEERING AND TECHNOLOGY
TAXILA
December 21,2017
Contents
Introduction. 2
Project Goals. 2
Aims & Objectives. 3
Deliverables. 3
Literature Review.. 4
Literature Survey. 4
Marketing Survey. 6
Chapter 1
Introduction
Project Goal
Autism Spectrum Disorder is a prevalent
development disorder that causes impairment in person’s hearing ,feeling
,speaking, thinking and social interaction1. All over the world out of 88 children 1
child is victim of ASD. For this reason, children suffering from autism need to
follow special training in order to increase their ability to learn new skills
and knowledge. These children have propensity to be attracted by technology devices especially virtual
animations. the interest of this app is to provide an environment for children
to get education and social skills. Using Augmented Reality and 3D models with
animations gives the children a real-life experience which will create a
positive improvement in terms of enjoyment, involvement, development of social
and learning skills to the autistic children. This application will build
social skills and non-verbal behavior in effected children. It will provide an
effective way of learning. Application can also be used to teach normal kids.
Aims & Objectives
·
Help Autistic
children to improve social skills using augmented reality.
·
Provide a more better way
for education to children with autism.
·
Help autistic children to
make better their communication with people and social interactivity.
·
To ease the Families with
autistic children and social care centers with providing them with virtual
assistant.
Deliverables
•
Android Application
Android application that uses 3d models which
user can interact with it and play games providing education and social skills
•
Augmented reality:
3D models augmented in camera that interacts with user.
•
Documentation
Detailed documentation on working procedures
•
3D
models and Animation
•
User
Manual & Guideline
Guide on how to use the application
Chapter 2
Literature Survey
Lakshmiprabha N. S.
introduced a system to help children with
Autism. This system teaches about new objects and pictures with their
associated keywords. The System is based on the idea that any correct behavior
can probably be repeated if it is followed by some reward. The tutoring
begins by letting the kid view a couple of pictures (usually three) or
objects(like elephant) which the kid
should learn. the kid is requested to join every object and picture with their
respected keywords.. Steps are repeated various times changing the pictures or
objects and soliciting the kids to show the related item. This process with the
rewards give the kids motivation to memorize the keywords with their related
pictures and objects.
The main aim of the project was to
develop AR system for training the kids with ASD. This system encourages the
kid to interact, copy a model, and give response to the primary requests such
as keyword connection and matching the sentences to its related pictures and
objects.1
Richard E proposed an application
called ARVe (Augmented Reality applied to Vegetal field). It is an augmented
reality based application which allows kids to handle 2D and 3D plant entities.This
system consist of task of pairing with providing visual and audio cues for
help.The objective of the system was to examine kids performance in using AR
System, Also to check attitude of cognitive disabled children towards the AR. It provides the option of playing and
interacting with the virtual entities and their supports Also, it offers the
children to uncover some characteristics of the plants but also to adjust the
gestures according to an intention and handle the entities.
Cues which were provided to kids
were:
·
Visual cues: Entities were surrounded by red
color when misplacing the object and with blue color when correct.
·
Auditory cues: The recorded audio of fruit
names were provided as kid brings the given entity near his face.
·
Olfactory cues: Oder of related fruit was displayed
as as kid brings the given entity near his face.2
Pedro Cunha suggested a Gamebook to help kids with Autism to recognize
and obtain emotions by engaging their attention and motivation was developed.
The aim was to promote the interaction between the kid and his/her fascination. Also help the kid to
recognize right emotion at right situation.
This Gamebook
shows the story of Tobias’s adventure during a visit to a zoo park. The story
will present five stories which contains scenarios and provide interaction. The
kid will interact with the stories by playing with Tobias and learning his
various facial expressions.it can be played on any mobile technology such as
tablet.
The stories represent
five different emotions which are joy, sadness, fear, anger and disgust. To involve the child on real life
situation these different scenarios was developed .when the chapter ends the kid
have to give a simple quiz before going to next chapter ,the question will
about what the emotion Tobias felt on the situation described before.3
Zen bai proposed a system that uses Augmented Reality (AR) visually
conceptualize the representation of pretense within an open-ended play
environment(lack of imagination).it is based on idea that kids with autism
suffer from development delays in symbolic thinking. In this System ,various
blocks were used to represent different shapes (like cars, ships).when kid
places the block in front of camera it changes the block and present an augmented
shape of the block. Kid can play easily by moving these object and making an environment.
To help kid in making environment this system provided shapes of school, petrol
,bridge, carriage, light , station, hanger, runway.4
Chien-Hsu Chen proposed an Augmented Reality -based video
modeling Storybook (ARVM) to attract kids having Autism to help them with switching
and adjusting their attentional focus. The System was
used to teach the kids to recognize patterns as they focused on the non moving visual image printed in the storybook and then
extended their attention to an animation of the story.
The ARVMS learning system takes video materials that depict everyday
life activities and it focuses on social cues and special moments suitable for children with ASD.The story book
consist of frozen images captured from video.when the kid moves the device on
the storybook and the kid sees the story into a moving video.This feature attracts
the kid to watch the movies of story book.5
Marketing Survey
·
Empower Me:
Empower Me is an electronic life
coach that children (and adults) with autism love! Developed by neuroscientists
and doctors from MIT and Harvard.6
·
Movements:
Moments App uses augmented reality to help teach various
emotions and role-playing social scenarios to children with ASD.7
·
IPrompts:
IPrompts is an
app for supporting organizational skills, setting expectations for performance,
and setting up subtle supports and reminders.Users can create and modify visual
schedules, as well as use a countdown timer with picture supports to indicate
how much longer a task or activity will last.
·
Autism
Track:
This
customizable data tracking tool allows parents to easily track behaviors,
interventions and symptoms in one place. Parents can also review trends in
their child’s data and share these data with school and medical providers to
inform treatment planning.
iPrompts
“iPrompts is an app for supporting
organizational skills, setting expectations for performance, and setting up
subtle supports and reminders. Users can create and modify visual schedules, as
well as use a countdown timer with picture supports to indicate how much longer
a task or activity will last.
Autism Track
This app is brought to you by the creators of
iPrompts and is designed for parents of children with ASD to help them track
data. “This customizable data tracking tool allows parents to easily track
behaviors, interventions and symptoms in one place,” says Aguayo.
“Behaviors and symptoms can be rated, as well as particular medications
and their doses, diet changes and therapies. Parents can also review trends in
their child’s data and share these data with school and medical providers to
inform treatment planning. This app is especially helpful for children with
challenging behaviors, psychiatric conditions or both.”8
Marketing
Plan
Hospitals and Autism Care Centers
Visiting different hospitals, Autism Care
Centers and offering them the product.
Online Health pages
We will advertise our product in online health
pages and specially related to autism, so we can approach different people
globally.
Families with autistic children
Visiting different families having autistic
children and pursue them to buy our product.
Customizing our Product
We will customize our product for specific
organization based on what are their needs and requirements.
Government Health Organization
We will offer our app to government health
organization to buy this product and use it in government hospitals.
Play Store
i.
Free App with advertisement and limited
features.
ii.
Purchased App with no advertisement.
iii.
Additional features and packages.
Chapter 3
Methodology
Literature Survey
Chapter 3
Proposed Solution
Methodology:
ARGuS is based on two types of augmented reality
feature based and environment based. Feature based augmented reality provide
ability to participant to play using flashcard using indoor environment. In
Environment based augmented reality participant can play in outdoor environment
with need of flashcard.
Feature
Tracking
1) Initialize
play material (Put flash cards on table/floor).
2) Run
mobile app and select feature tracking.
3) Focus
camera on the flash card.
4) App
will capture image frame.
5) Performa
feature extraction on image based on texture distribution, contrast
distribution and corner points of shapes in the image.
6) Do
compare extracted features with decoded image in the app database.
7) After
comparison it will place a virtual plane on the image.
8) Decode
the encoded 3D model.
9) Perform
realistic rendering and place 3D model on the image.
10)
Show the result on
output.
Environment
Tracking
1) Run
mobile app and select environment tracking.
2) Initialize
the environment tracker.
3) Select
point to place model in environment.
4) App
will capture image frame.
5) Simultaneous
Localization and Mapping is an algorithm that is used to track the environment
from extracted frames.
6) After
comparison it will place a virtual plane on the image.
7) Decode
the encoded 3D model.
8) Perform
realistic rendering and place 3D model on the image.
9) Show
the result on output.
Development
Tool
Android Studio
Android Studio is an IDE for developing for the Android
platform. Android Studio is freely available under the Apache License 2.0 7. Based on JetBrain’s
IntelliJ IDEA software, Android Studio is designed specifically for Android
development.
Features
Ø Gradle-based build support.
Ø Android-specific refactoring and quick fixes 8.
Ø Lint tools to catch performance, usability, version
compatibility and other problems. ProGuard integration.
Ø Template-based wizards to create common Android designs and
components.
Ø A rich layout editor that allows users to drag-and-drop UI components,
option to preview layouts on multiple screen configurations.
Auto Desk Maya
Maya is a 3D animation, modeling, simulation, and
rendering software helps artists work faster with enhancements to existing
workflows and new tools.
Features
Ø Make
high-level animation edits with a nondestructive, clip-based, nonlinear editor.
Ø Fine-tune
your characters and animation more quickly and easily.
Ø New
system speeds rig playback and manipulation.
Ø And
more
Project time-line:
Experimental/ Simulation Setup:
Details of Work packages
completed/milestones achieved:
Task
Achieved
Requirements Gathering
?
Feasibility Study
?
Requirements Analysis
?
Requirements Validation
?
Android App Design
?
Concept Modeling
?
3D Modeling
?
Scenario Developing
?
3D Models Animation
?
Scenario Execution in App
?
Integrating models into App
?
Java coding
?
Unit Testing
?
Setting up Experimental Setup
?
Execution of Experimental Setup
?
System Test
?
Bug fixes
?
Results Evaluation
?
Documentation & User Manual
?
Release version
?
Requirements Gathering
?
Evaluation parameters:
Since our system is designed for
kids with Autism ,so our evaluation parameters will be:
·
Did this app improved social skills
of kids?
·
Did this app provided education for
kids?
·
Did this app improved behavior and interactivity
of kid with others?
·
Does the kid likes the technology used
in this app (Augmented Reality)?
Chapter 4
Work Plan
Utilization
(End users/Beneficiaries):
•
Our target
users/beneficiaries are Autistic children.
•
Autistic
children can improve their social behavior.
•
Autistic
children families will be assisted by ARGuS app to teach their children.
•
Autistic
children will be educated in an innovative and attractive way.
•
Autistic
children will enjoy playing games also learn many things.
•
This app will
serve as a special kind of tutor in hospitals and ASD Centers.
•
Our market
audience includes Specially hospitals, Autism Care Centers, families having
Autistic children, health groups, Government Health Organizations.
Detailed
Work Plan:
The plan we have proposed to complete the entire project is
based on 270 days.
·
75
days will be served for Requirements Specification including requirements
gathering, feasibility study, requirements analysis, requirements validation,
proposal report and SRS.
·
90 days will be served for Designing and
Development including android application design, concept modeling,3D Modeling,
Scenarios developing ,3D Models Animation .
·
60
days will be served for Coding and Environment setup including Java coding, unit
testing ,Setting up experimental setup and Execution of experimental setup.
·
35 days will be served for testing and Evaluation
of results including System testing ,Bugs fixing,results evaluation. , android
application final version and integration. Then, 20 days will be consumed doing
Testing and Deployment that will include alpha testing, beta testing, system
testing and system deployment. Remaining 30 days will be utilized for
Documentation composed of user guide and thesis.
·
Last
25 days will be served for Documentation , making user manual and Releasing the
final version of app.
Budget
Requirement:
Total budget
we need is 22000 Rs for as follows:
Requirements:
·
Research:
Rs.1000 ,since we need a subscription of websites that have e-researches in
order to download them.
·
ASD
Center:in order to visit ASD Center which is far away we need money estimated
Rs.1000 (vehicle rent.ASD consultant fees).
·
Survey:Rs.1000
,since we need to print papers ,vouchers and assign works to different teams
and pay them.
Design:
·
Modeling:Rs.1000,
we need to purchase licensed account for modeling softwares.
·
Rigging:Rs.1000
, we need to buy plugins to rig the models.
·
Animation:Rs.500,we
need to hire person for animation of models
·
Interface
Design:we need to buy widgets libraries in order to import them.
Implementation:
·
Add
model to app:Rs 500, we need to buy sdk in order to import them.
·
Give
interactivity to model:Rs 500, we need to buy sdk in order to import them.
·
Adding
Chat bot:Rs.1000 ,we purchase to buy AI Microsoft library for this purpose.
Testing:
·
Rs.5000
,we need to go the ASD centers and pay them in order to use their resources and
we need to purchase testing tools.
Publishing:
·
PlayStore:Rs.6000,
pay to google to create account at play store
·
SDK:
Rs.4000 need to buy license to publish the app
Market
Forecasting:
Chapter 5
Conclusion
References:
1. Lakshmiprabha, N., et al. Poster An augmented and virtual reality
system for training autistic children. in Mixed and Augmented Reality (ISMAR), 2014 IEEE International Symposium
on. 2014. IEEE.
2. Richard, E., et al. Augmented reality for rehabilitation of
cognitive disabled children: A preliminary study. in Virtual Rehabilitation, 2007. 2007. IEEE.
3. Cunha, P., et al. Augmented reality for cognitive and social
skills improvement in children with ASD. in Remote Engineering and Virtual Instrumentation (REV), 2016 13th
International Conference on. 2016. IEEE.
4. Bai, Z., A.F. Blackwell, and G.
Coulouris, Using augmented reality to
elicit pretend play for children with autism. IEEE transactions on
visualization and computer graphics, 2015. 21(5):
p. 598-610.
5. Chen, C.-H., I.-J. Lee, and L.-Y.
Lin, Augmented reality-based video-modeling
storybook of nonverbal facial cues for children with autism spectrum disorder
to improve their perceptions and judgments of facial expressions and emotions.
Computers in Human Behavior, 2016. 55:
p. 477-485.
6. https://www.indiegogo.com/projects/world-s-first-augmented-reality-glasses-for-autism-education#/.
7. www.rachelmadel.com.
8. http://www.parenting.com/gallery/autism-apps.
