Currency Notefor Visually Challenged People through Voice Message
Abstract
In present digital world, to get survive there is a need for an independent life, in case of visually challenged people who face lot of problems in their day-to-day life. They feel very strange when they are new to an environment. Visual information is the basic need to do a task, so visually challenged people get struggle because necessary information about the environment is not available for them. With the advanced technology, we can support the people who are visually challenged. This project is proposed to support those people who are visually challenged. Using spectacles currency image is captured, this captured image is sent to the mobile application, then using the deep learning and machine learning algorithm currency is recognized, then it compares the captured currency Features with the already trained data set, After Recognizing it passes the information of the recognized currency by assisting them through voice command. This is more efficient in which visually challenged people able to do payments in Supermarkets, Shopping malls and all other sectors with the help of technology.
Keywords:Visually challengedMobile applicationCurrency imageDeep learningAssisting through voice
command.
1. Introduction
Visual is the important thing which help us to see the eternal beauty of world. But it is very much difficult for visually challenged people, without seeing they even can’t feel the emotions of other people. Visual impairment is a major problem faced by many people all over world [1]. This makes them to depend on others for their daily activities. They need to depend on others for doing payments in shopping mall, supermarkets, etc. Our aim is to help them to make payments on their own without depending on others. The currently existing system has accuracy of 90% and it captures currency image through mobile application and also need internet connection to work with. The major problem is they cannot do the payments in remote places. The proposed system uses Deep Learning and Machine Learning techniques to assist visually challenged people through a voice command and also use spectacles with camera to capture the currency image. It does not need any internet connection to work i.e. it can also work in remote areas where there is no network. The other sections are organized in following ways: in section II discussed about various other works done in this area. The proposed work like problem statement, complete system architecture description is given in this section III. In section IV discussed about the results and discussion of the work was done. In section V performance is analyzed. Finally, in section VI concluded the work and also given the future direction.
2. Related Works
A Visually Impaired AI assistant for object recognition to make visually challenged people to recognize the object. This system is a great support to visually challenged people [2]. A hybrid algorithm is used in this with the help of Artificial Neural Network (ANN) [3]. This system captures the image through camera and it then process this captured data.
The data is then categorized in many frames and each frame gets compared with the immediate previous frame. After many comparisons the data gets stored and it produces the response for the captured data. This response is given to the visually challenged people in the form of a voice command. This system has an accuracy of 90% and it needs an internet connection to work with. In this currency recognition is done edge detection. It is one of the preprocessing steps involved in this process. Then after detecting the edges it then extract the frames and it compares and produces the output. This model completely based on ANN and it helps to recognize the object which is been captured.
This system gives object classification for visually challenged people. It also gives the correct recognition of captured object and assist visually impaired persons through voice which tells what the captured object is [4]. The main disadvantage of this system is currency recognition accuracy is low and also more features of currency are not trained in this system.
3. Proposed System
3.1 Problem Statement
Blind people are facing many problems in their day to day life. Among those one of the major problems is making payments in supermarkets, shopping malls, etc [5–7]. They depend on others for doing their payments. As they depend on others they face many consequences in handling their money.
In this paper we have tried to give solution for making their payments without depending on others in shopping malls, supermarkets, and all other financial sectors.
3.2 System Architecture
Camera
Camera is fixed to the Spectacles. Spectacles and mobile application connected through Bluetooth. Currency image is captured through the Camera which is fixed at the Spectacles.
Currency image input
Currency image is given as the input to our mobile application.
Mobile application
The mobile application is made to recognize the currency [8]. The Currency image is compared with the given data set and it recognizes the currency. The recognized currency is given as audio output through the Bluetooth device. This application is made of deep learning algorithms [9].
Here the captured image is preprocessed using tensor flow lite platform which further segmented, features are extracted and classified using convolution neural network. After classifying the currency image it is then given to the application and assisted through voice command.
The app classifies frames in real-time, displaying the top most probable classifications. It allows the user to choose between a floating point and quantized model. This allows user to work in any kind of device such as smart phones, tablet, android, I phone, etc.
Training data set
We have collected the data set such as Rs.10, 20,50,100,500,1000, we have collected more than 1000 data’s in total and we have trained our system. Thus by using these trained data set, currency is recognized with greater accuracy. It uses resnet 50 Architecture to train the dataset using Tensor Flow Machine Learning.
Here we have used 256 pixel sizes for image, and resnet 50 as the pre trained model. With resnet frozen, we did 50 epochs training the added top layers and did more training with sliced learning rates on 50 epochs. In each rupee we have collected various notes such as old and new notes which are currently in use, and we trained it with the help of the collected sample data in various angles (unfolded front horizontal, unfolded back horizontal, unfolded front vertical, unfolded back vertical, single folded in 90°, single folded in 45°, single folded in 30°and double folded images).
Preprocessing
The main goal of the pre-processing to enhance better visual appearance of images and improve the manipulation of data sets [10]. Image pre-processing, also called image restoration, and involves the correction of distortion and degradation introduced during the imaging process. Interpolation is the technique mostly used for tasks such as zooming, rotating, shrinking, and for geometric corrections.
The application may be implemented in any device, the camera quality of some device may be low, so when the currency image is captured using those devices it is must we need to enhance the image, and thus we use the preprocessing technique like pixel brightness transformation and geometric transformations. In pixel brightness transformation, the brightness is corrected in which the output value of pixel depends on the value of input pixel. Computer enhancement is the important thing in preprocessing.
In geometric transformations, position of pixel in an image is modified but colors are unchanged. Here in our project we use scaling and rotation transformations. In scaling the image is resized to 250 pixels. In rotation, as we capture the currency in different angle it is rotated to 90° and then it is segmented.
a. input b. pixel brightness c. geometric transformation transformation
Segmentation
Image segmentation is the process of partitioning a digital image into multiple Segments. It separates the currency image from the background image. The goal of segmentation is to simplify and change the representation of an image into something that is easier to analyze.
Here the image is segmented using U-Net model. It masks the background image by identifying the target image.
a. preprocessed b. segmented image image
Feature extraction
Feature extraction is a special form of dimensional reduction. When the input data to an algorithm is too large to be processed and it is suspected to be very redundant then the input data will be transformed into a reduced representation set of features [11]. Transforming the input data into the set of features is called feature extraction.
If the features extracted are carefully chosen it is expected that the features set will extract the relevant information from the input data in order to perform the desired task using this reduced representation instead of the full size input.
Here we consider the features such as color of the currency note, size of the currency note, edge which has the denomination of the currency and shape identification.
In color of the currency, each currency has different colors as mentioned in the below table.
Table 1. Color of the currency.
This table is shown in the PDF version of the article.
In size of the currency, the size is identified only if the captured input currency image is either vertically or horizontally unfolded. The following table gives the height and width of each currency.
Table 2. Size of the currency.
This table is shown in the PDF version of the article.
In edge identification the edge of the currency note is detected as the edge consists of the denomination of the currency. It checks both the edges as the old notes has the denomination in the upper right edge and the new note has the denomination in the bottom right edge.
In shape identification, this is done only when the captured image is unfolded either horizontally or vertically. For 10 rupee note, 20 rupee new note and 50 rupee new note it is not applicable. Other than this it is applicable for all the other currency notes. The shapes of each note are mentioned in the table below. For all new notes the shape is identified on the right and for all the old note shape is identified on the left. We have trained our system to identify in both left and right of the currency note.
Table 3. Identification shape of the currency.
This table is shown in the PDF version of the article.
Classification
The classification is done using Convolution Neural Network. Our project consists of 4 layers of convolution operations. They are convolution layer, pooling layer, dense layer and logit layer.In convolution layer the image gets preprocessed and segmented. In pooling layer the features are extracted. In dense layer it declares if the model needs to be trained or evaluated. In Logit layer the image gets predicted and checked for accuracy.
Testing and validation
In currency verification we check the currency accuracy. If the accuracy is greater than 95% after checking all features it produces the output. If the accuracy is less than 95% then again it checks and predicts using the trained sample data.
Audio output
The audio output of the recognized currency is given through the bluetooth device.
4. Modules
We have implemented this system using the Package tensorflow.lite. It uses RESNET 50 Architecture to train the dataset using Tensor Flow Machine Learning. Then we have imported all the android library files and all our trained datasets are stored as metadata in assets file. Then on CaptureProgressed and onCaptureCompleted functions are called. Through this the captured image is preprocessed, features are extracted and classified. The classified image is then checked for correct accuracy. Then the voice for the classified image is given through Bluetooth device to the visually impaired people.
5. Discussion
This system has achieved above 95% accuracy in recognizing the currency. The audio output is accurate and clear without any distortion.
The application which is created through android studio also shows the accuracy at the bottom in the application.
We have tested the application in various devices. We worked on all kinds of Indian Currency and extracted 4 features of the currency for its recognition [12]. The given database for identifying the currency is enough large i.e. we have given the different form of currency as sample data such as dirty note, torn note, clean note and also scribbled notes.
We have also added the currency image having different angles like, front, back, front clock wise angles, back clock wise angles and front anti clock wise angles, back anticlockwise angles to increase our accuracy in recognizing the currency.
With all our observations we have made a confusion matrix which describes the performance of the classification model. The confusion matrix is given below.
The confusion matrix is plotted through with the help of assessment table. This helps us to find the actual accuracy of our application i.e., 97.4%. This accuracy is found by summing all the diagonal values and dividing it with the total values.
This also shows the accuracy of the out class from each target class. We have tested our classification with 700 images in total (100 from each currency).
The Results and assessment parameters are shown in the below table.
Table 5. Result.
This table is shown in the PDF version of the article.
Table 6. Assessment parameter.
This table is shown in the PDF version of the article.
With this assessment parameter we have constructed the confusion matrix.
Each currency is given a class from 1-7. By using this we have constructed the matrix.
Fig. 10 Gives us the process in which this system works. This explains the complete process through which we get the output.
Fig. 11 Tells us how the application looks like. At the bottom there’s a table that shows accuracy.
6. Performance Analysis
This system given a high accuracy compared to other system. The graph below gives the actual performance of our system after plotting the confusion matrix.
The graph gives average accuracy in detail for each data.
7. Conclusion
Currency Recognition is very much helpful for visually challenged people to make payments easily. Currency image can be recognized with greater accuracy. This would be a satisfactory solution for visually challenged people to make payments as like a normal people.This system produce more accuracy compared to other systems as we considered many features of currency and given a large database for recognizing the currency.
Future scope
Every system has its own advantages and limitations as well. We have discussed all the pros in our system but this system also has some limitations like:
I. This system does not recognize the watermark, latent features of the currency note. So we can improve by adding these features to the system II. This system does not identify the fake currency note, This can also be added by improving the system with Ultraviolet light to determine the fake note.
III. This system recognizes only the Indian currency note. This can be improved to recognize other currencies of different countries.
We can also add features like GUI interface and the portability with this system.
References
- [1] Semary, N. A., Fadl, S. M., Essa, M. S., & Gad, A. F. (2015). Currency recognition system for visually impaired: egyptian banknote as a study case. 2015 5th international conference on information & communication technology and accessibility (ICTA) (pp. 1–6). IEEE. https://doi.org/10.1109/ICTA.2015.7426896
- [2] Felix, S. M., Kumar, S., & Veeramuthu, A. (2018). A smart personal ai assistant for visually impaired people. 2018 2nd international conference on trends in electronics and informatics (ICOEI) (pp. 1245–1250) IEEE. https://doi.org/10.1109/ICOEI.2018.8553750
- [3] Jyothi, C. R., SundaraKrishna, Y. K., & SrinivasaRao, V. (2016). Paper currency recognition for color images based on artificial neural network. 2016 international conference on electrical, electronics, and optimization techniques (ICEEOT) (pp. 3402–3406). IEEE. https://doi.org/10.1109/ICEEOT.2016.7755335
- [4] Krishnan, K. G., Porkodi, C. M., & Kanimozhi, K. (2013). Image recognition for visually impaired people by sound. 2013 international conference on communication and signal processing (pp. 943–946). IEEE. https://doi.org/10.1109/iccsp.2013.6577195
- [5] Yousry, A., Taha, M., & Selim, M. M. (2018). Currency recognition system for blind people using ORB algorithm. The international arab journal of information technology, 5(1), 34–40. https://B2n.ir/pg3209
- [6] Sabab, S. A., & Ashmafee, M. H. (2016). Blind reader: an intelligent assistant for blind. 2016 19th international conference on computer and information technology (ICCIT) (pp. 229–234). IEEE. https://doi.org/10.1109/ICCITECHN.2016.7860200
- [7] Jabnoun, H., Benzarti, F., & Amiri, H. (2014). Object recognition for blind people based on features extraction. International image processing, applications and systems conference (pp. 1–6). IEEE. https://doi.org/10.1109/IPAS.2014.7043293
- [8] Bagwan, S. M. R., & Sankpal, L. J. (2015). VisualPal: a mobile app for object recognition for the visually impaired. 2015 international conference on computer, communication and control (IC4) (pp. 1–6). IEEE. https://doi.org/10.1109/IC4.2015.7375665
- [9] Laavanya, M., & Vijayaraghavan, V. (2019). Real time fake currency note detection using deep learning. The international journal of engineering and advanced technology (IJEAT), 9(1S5), 95–98. 10.35940/ijeat.A1007.1291S519
- [10] Tian, F., & Shi, P. (2014). Image mosaic using orb descriptor and improved blending algorithm. 2014 7th international congress on image and signal processing (pp. 693–698). IEEE. https://doi.org/10.1109/CISP.2014.7003867
- [11] Yi-bo, L., & Jun-Jun, L. (2011). Harris corner detection algorithm based on improved contourlet transform. Procedia engineering, 15, 2239–2243. https://doi.org/10.1016/j.proeng.2011.08.419
- [12] Garc’ia-Lamont, F., Cervantes, J., & López, A. (2012). Recognition of Mexican banknotes via their color and texture features. Expert systems with applications, 39(10), 9651–9660. https://doi.org/10.1016/j.eswa.2012.02.132