Designing of marker-based augmented reality learning environment for kids using convolutional neural network architecture

Abstract

This paper focuses on using the augmented reality (AR) technology to create visual-aids through display for early childhood learning. The proposed methodology works on the principle of augmenting 3D virtual objects over the English alphabets that are used as printed markers. The important steps of a typical marker-based AR application are, (i) detection of markers in the field of view (FOV) of the camera, (ii) identification of the marker, (iii) estimating the pose of the marker, and (iv) rendering 3D virtual content over the marker in a live video stream. We have formulated the marker identification process as a classification problem which has been accomplished with the help of convolutional neural networks (CNN). The effectiveness of the marker identification process using CNN is validated by comparing its identification accuracy with support vector machine (SVM) classifier. The marker identification by the CNN model shows better accuracy than SVM. After successful marker identification and pose estimation, virtual objects are rendered over the 2D projection of the alphabets. The seamless augmentation of the virtual objects over the markers are rendered on display. The setup has been tested on a large dataset and it is believed to create engaging experience for the kids, especially the kindergarten age group. � 2018 Elsevier B.V.