Skip to article
AHFE AHFE Discoveries AHFE International Open Access Journal
AHFE OPEN ACCESS OPEN ACCESS CONFERENCE PROCEEDINGS PAPER

COMM-G: A Communication Glove for Smart Communication

1 School of Automation, Banasthali Vidyapith, Tonk, India
2 School of Automation Banasthali Vidyapith, Tonk, India
3 Department of Home Science, Banasthali Vidyapith, Tonk, India
Publication year
Digital Object Identifier10.54941/ahfe100906

Abstract

This paper proposes a smart communication process using google assistant for sign language identification with the help of the components majorly comprising of flex sensors, MPU6050 sensors, and microcontrollers. The basic idea evolved overtime to make the process simpler and more efficient. The conversion from gesture to speech is carried in two sub-processes, firstly gesture is converted into text with the help of flex sensors and the text is converted into speech with the help of default functionality of google assistant. The test results are prudently evaluated with the respective output of the sensors. This proposed idea can assist people with impaired learning and people working in industries.

Human Interaction & Emerging Technologies (IHIET-AI 2022) Artificial Intelligence & Future Applications https://openaccess.cms-conferences.org/#/publications/book/978-1-7923-8989-4

INTRODUCTION

Communication is the exchange of information, ideas, thoughts, sentiments, and emotions by verbal, written, or behavioral gestures. In our daily lives, communication is crucial. Hand signals, gestures, facial emotions, and body language are used in sign language. Sign language is mostly used by the deaf and mute community, but it can also be beneficial to other groups of people and in industry. In India, there are nearly 2.4 million people who are deaf and mute, which constitutes around 20% of the world population. As per the World Federation of the Deaf (WFD), there are 5% of the world’s population are deaf including adults and children . Aiding the fluent communication needs of deaf and mute people has remained a challenge for researchers over the years.

Researchers have deployed innovative ways to help such people by developing contraptions such as robotic arms and prosthetics (Raghavan et al., 2016, O’Toole et al., 2009, Loper et al., 2009) as well as communication gloves (Singh 2019, Dipietro et al. 2008, Jacobson et al., 1997, Fortnum et al., 2016, Mo et al., 2005). Many researchers have developed this idea to make communication easy using cameras to recognize the 3D hand trajectory by a Finite State Machine (FSM) and using it to predict the statement (Singh, 2019). Researchers have focused on many applications using hand gestures like drawing using vision-based methods (Mo, 2005). The primary objective of this paper is to focus on communication with the help of a flex sensor-based glove while making this product efficient by using google assistant. This product is in the form of a glove and that’s why this product is named COMM-G (Communication-Glove).

COMM-G is a glove fitted with an advanced microcontroller that can detect hand gestures and translate them to speech with the help of Google assistant. It uses flex sensors attached to the fingers which can detect the deflection of fingers depending on its resistance. Flex sensors are connected to Arduino Uno to configure the resistance values in terms of voltage. EEPROM is fed with the data and the output of Arduino Uno is matched with its data. Once the voltage is matched, the corresponding word is printed corresponding to that gesture. With the help of Google assistant, which is readily available on all android mobile phones, the presented text is converted into speech. COMM-G can be beneficial in filling the communication gap which exists between deaf-mute people and other people.

COMM-G DESIGN To make COMM-G more accurate and technically feasible, the major components used are MPU6050 for measuring the rotation and acceleration of hand for various applications. Flex sensors (Dipietro et al., 2008) and sensing applications using flexible antennas (Shastri et al., 2021), have become extremely lucrative recently (Njogu et al., 2021). The flexibility of the device provides reliable results and can be deployed in compact areas and still provide optimum performances.

Human Interaction & Emerging Technologies (IHIET-AI 2022) Artificial Intelligence & Future Applications https://openaccess.cms-conferences.org/#/publications/book/978-1-7923-8989-4

MPU6050 sensor used here works on the principle of change in capacitance and Coriolis effect. It consists of a micro-machined structure built on top of a silicon wafer; this structure is suspended by polysilicon springs which allow the structure to deflect at the time when acceleration is applied on the axis. Due to deflection, the hand capacitance between the two fixed plates is changed where the change in capacitance is proportional to the acceleration on that axis which is converted into an analog output voltage.

Figure 1. Visual representation of COMM-G Flex sensor works on the principle of voltage-divider circuit, in which two components are connected to a single voltage source and the output voltage is taken between the two components. The output voltage can be calculated as:
Original PDF Figure 1.

Vout =Vin(R2/R1+R2) (1) where R1 is the Flex resistance, R2 is Constant resistance, Vin is the input voltage and Vout is the output voltage.

HC-05 is the Bluetooth module that was used as per the availability in every device, and it operates at a much slower rate of around 720 kbps. The module requires 4 V to 6 V for operation and this module provides two-way communication so, further, in the future continuation of this work, the speech can also be converted into sign language. The default band rate in command mode is 38400 bps and in data mode is 9600 bps, hence it is easy to interface with any microcontroller. The brain of COMM-G is the Arduino UNO 8-bit ATmega 328P microcontroller. The microcontroller on the board is programmed using the Arduino programming language and the Arduino development environment. Arduino can receive the input from various sensors and can do the desired task for the user. The google assistant is used to convert the text into speech which is the inbuilt function of Google assistant. In today’s day and age, everyone is familiar with google assistant so that anybody can communicate with deaf and mute people by just using their smartphones.

GLOVE DESIGN AND INTEGRATION A glove is designed as a wearable device for ease. A flex sensor is used to measure the deflection of fingers or joints of the hand. Flex sensor can be used as a variable

Human Interaction & Emerging Technologies (IHIET-AI 2022) Artificial Intelligence & Future Applications https://openaccess.cms-conferences.org/#/publications/book/978-1-7923-8989-4 resistor in which resistance is increased with the increase in bending as there is a direct proportional relationship between resistance of flex sensor and bending of finger and joint of the hand. An MPU6050 sensor which is a combination of accelerometers and gyroscopes is commonly referred to as an Inertial Measurement Unit or IMU using which we can implement many applications by knowing the rotation of the hand as well.

MEASUREMENTS AND RESULTS Working of flex sensors were tested to obtain the relation between resistance and voltage and to determine the output voltage with respect to the bending of fingers in hand which is shown in Fig. 2. The test was conducted using MultiSimTM live making a voltage divider circuit and tested the result with some random values of flex resistance and assuming a constant resistance value, using the equation (1).

When the finger is straight and flex sensors are not deflected then the flex resistance was 25kohm which gives the output voltage 3.2639 V practically as shown in Fig. 3. Theoretical output voltage when the resistance of flex sensor is 25kΩ: Vout=5× (47kΩ /47kΩ + 25kΩ) (2)

This gives Vout as 3.26 V and when the fingers bend at approximately 90 degrees then flex resistance was 100kΩ which gives the output voltage 1.5986 V practically as shown in Fig. 4. Theoretical output voltage when the resistance of flex sensor is 100kΩ: Vout=5× (47kΩ /47kΩ +100kΩ) (3)

Figure 2. Relation between Deflection in hand (in degrees) and output voltage of flex sensor (in volts)
Original PDF Figure 2.

Human Interaction & Emerging Technologies (IHIET-AI 2022) Artificial Intelligence & Future Applications https://openaccess.cms-conferences.org/#/publications/book/978-1-7923-8989-4

Which then gives Vout as 1.59 V. In conclusion, flex resistance increases with an increase in bend angle while output voltage decreases.

Figure 3. Unbent flex sensor having a resistance of 25kΩ.
Original PDF Figure 3.
Figure 4. Flex sensor bent up to 90 degrees.
Original PDF Figure 4.

The output voltage is directly fed to Arduino UNO for the comparison with pre- assigned data in EEPROM and the word assigned to the respective voltage will be generated in the text form. This text is then delivered to Google assistant by using the HC-05 Bluetooth module which is then converted into speech and transmitted for communication.

Graphical representation of flex sensors is performed on LabVIEW software. The gesture box for the letter ‘H’ and the corresponding letter is shown in Fig. 5. The hand gesture is made according to the American Sign Language (ASL). The block diagram for the above front panel is shown in Fig. 6. The loops are used to change the letter. As an example, we have demonstrated the word ‘HELLO’. So, each loop will iterate five times for each alphabet of the word ‘Hello’ and a time gap of 1000 ms is observed between two consecutive letters for clear understanding.

Human Interaction & Emerging Technologies (IHIET-AI 2022) Artificial Intelligence & Future Applications https://openaccess.cms-conferences.org/#/publications/book/978-1-7923-8989-4

The output voltage obtained after making a particular hand gesture will be used to determine the letter through Arduino Uno Code and the letter or the word will then be displayed on the phone screen.

Figure 5. Front panel of graphical representation showing gesture for letter H and the corresponding letter.
Original PDF Figure 5.
Figure 6. Block diagram of graphical representation showing the connections done.
Original PDF Figure 6.

Simulation of flex sensor has been done on an online platform to get the output accordingly. Unbent flex sensor resistance is 25kΩ similarly, every bent angle of flex sensor has different resistance. The supply voltage is provided to Arduino UNO is 5V and the constant resistor of the voltage-divider circuit is taken as 47kΩ. Initialization of the input pin of the flex sensor is done by using a variable that defines the purpose of the flex sensor pin. The maximum voltage of Arduino UNO is 5 V and there are 1024 (for 10-bit ADC 210) states available in Arduino UNO and the range starts from 0. The corresponding output voltage of the flex sensor is calculated.

Human Interaction & Emerging Technologies (IHIET-AI 2022) Artificial Intelligence & Future Applications https://openaccess.cms-conferences.org/#/publications/book/978-1-7923-8989-4

APPLICATIONS The corresponding responses to the hand gesture are derived as per the ASL gesture to text conversion. The highly flexible program can also be modified to convert the hand gesture to a specific text-speech that can be extremely useful on industry floors and in combat drills of security forces where the details can be communicated more clearly manner without making much noise. Some other applications may include Gesture control car or bike driving in which advanced safety and comfort features can enhance the overall driving experience. Gesture control not only makes using various in-built systems easier but also reduces the probability of causing a distraction-related accident by minimizing the need to take one’s eyes away from the road. Medical Surgery and monitoring of human hands can deploy COMM-G which will help in tracking the improvement in our hands, how our fingers are moving and how our wrist is functioning.

Industry 4.0 and Industry 5.0 are well known in today’s time and to create the processing smarter with smart warehouses and smart factories. Robots are used to help humans to work faster in an efficient manner.

CONCLUSION

In this paper, a smart glove design that can be used for non-verbal communication purposes was presented that can convert signs into speech. Using gestures, sign languages we can easily communicate seamlessly. Sign language is a way of communicating by using the hands and other parts of the body. Through this glove, it would become very easy for deaf and mute people to efficiently communicate with everyone. It has been designed by using sensors and microcontrollers.

Next, the working of flex sensors was presented and their output when connected to a glove and made a gesture to test it on an online platform was displayed. The output voltage obtained from Arduino UNO is configured according to the predefined sign language and the respective word is printed on the screen. Graphically, using the LabVIEW software, we presented how the words will change upon changing of voltage. The voltage depends on the gesture made.

A major part of this work may continue as there are numerous future possibilities which consist of going deeper into the analysis and development of mechanisms, new suggestions, and trying different methods and using them in various applications comprising of Artificial Intelligence (AI) integration. COMM-G can convert gestures into speech, but as future development, the process can also be reversed or bidirectional for smoother communication. COMM-G can also be used as a translator device for different languages, can be used as a unit converter device and for complex calculations. MPU6050 can also detect the surface inclination or the flatness whatever would be written by moving the fingers and the movement of hands could be recorded in EEPROM and then could be made accessible through a smart device.

Human Interaction & Emerging Technologies (IHIET-AI 2022) Artificial Intelligence & Future Applications https://openaccess.cms-conferences.org/#/publications/book/978-1-7923-8989-4

ACKNOWLEDGMENTS

We would like to express our sincere gratitude to Banasthali Vidyapith, India for providing us this wonderful opportunity. Their profound insights and attention to details have been true inspirations to our research.

REFERENCES

Dipietro, L., Sabatini, A. M., and Dario, P. (2008), “A survey of glove-based systems and their applications”. IEEE Transactions on Systems, Man, and Cybernetics, Part C (Applications and Reviews), 38(4), 461–482. Fortnum, H., Ukoumunne, O., C., Hyde, C., Taylor, R., S., Ozolins, M., Errington,

S., Zhele, Z., Pritchard, C., Benton, C., Moody, J., Cocking, L., Watson, J., Roberts, S. (2016), “A programme of studies including assessment of diagnostic accuracy of school hearing screening tests and a cost-effectiveness mosel of school entry hearing screening programmes”. Health Technology Assessment, 20(36), 1-178. Jacobson, B. H., Johnson, A., Grywalski, C., Silbergleit, A., Jacobson, G., Benninger, M. S. and Newman, C. W. (1997), “The voice handicap index (VHI),”. American Journal of Speech-Language Pathology, 6(3), 66–70. Loper, M. M., Koenig, N. P., Chernova, S. H., Jones, C. V., and Jenkins, O. C.

(2009), “Mobile human-robot teaming with environmental tolerance”. Proceedings of the 4th ACM/IEEE International Conference on Human Robot Interaction - HRI '09, 157-163. Mo, Z., Lewis, J. P., and Neumann, U. (2005), “SmartCanvas: a gesture-driven intelligent drawing desk system”. Proceedings of the 10th International Conference on Intelligent User Interfaces - IUI '05, 239-243. Njogu, P., Jablonski, P., Shastri, A., and Sanz-Izquierdo, B. (2021), “Origami boat

sensing antenna”. 15th European Conference on Antennas and Propagation (EuCAP) 2021. O’Toole, K. T., McGrath, M. M., and Coyle, E. D. (2009), “Analysis and evaluation of the dynamic performance of SMA Actuators for Prosthetic Hand Design”. Journal of Materials Engineering and Performance, 18(5-6), 781–786. Raghavan, A., Joseph, S. (2016), “EMG analysis and control of Artificial Arm”.

International Journal on Cybernetics & Informatics (IJCI), 5(2), 317–327. Shastri, A., Ullah, I., Sanz-Izquierdo, B. (2021), “Alternating current sensing slot antenna”. IEEE Sensors Journal, 21(7), 9484–9491. Singh, M. S. (2019), “Wirefree glove: Gesture vocalizer for deaf and dumb people”.

International Journal for Research in Applied Science and Engineering Technology, 7(4), 1184–1188.

Human Interaction & Emerging Technologies (IHIET-AI 2022) Artificial Intelligence & Future Applications https://openaccess.cms-conferences.org/#/publications/book/978-1-7923-8989-4

Cover of the proceedings volume Human Interaction & Emerging Technologies (IHIET-AI 2022): Artificial Intelligence & Future Applications

Explore the complete digital volume

Human Interaction & Emerging Technologies (IHIET-AI 2022): Artificial Intelligence & Future Applications

Read this individual research paper free through the original publisher, or explore the full digital proceedings book.

ISBN 978-1-7923-8989-4

Explore this ISBN in the AHFE Bookstore Listed in the July 2026 bookstore catalogue; search by ISBN to confirm current availability · Digital book purchase optional

Explore Related Research

Related research 8 matches

More papers from this volume

← Back to Research ExplorerBrowse the full library →