Real-time fall detection on roads using transfer learning-based granulated Bi-LSTM

Article Type

Research Article

Publication Title

Knowledge Based Systems

Abstract

In this study, a new deep learning-based model, namely, transfer learning-based granulated Bi-LSTM (TLG-LSTM) is developed for fall detection on roads. The TLG-LSTM can handle the uncertainty issue arising between various ‘Fall’ and ‘No Fall’ events in complex scenarios concerning both indoor (i.e., home) and outdoor (i.e., road) areas. The TLG-LSTM consists of four phases: (i) object detection and tracking, (ii) MoveNet-Lightening and object-level feature(s) computation for all the detected objects, (iii) granule formation using these features, and (iv) temporal self attention mechanism-based Bi-LSTM for granule classification as ‘Fall’ or ‘No Fall’. Unlike state-of-the-art models, TLG-LSTM uses both MoveNet-Lightning and object-level features, enabling better modeling of both indoor and outdoor falls. For each detected object, two MoveNet-Lightning features, namely head-hip distance and hip-ankle distance are defined and used for obtaining a granule, namely pose granule. Whereas, three object-level features, namely change in aspect ratio, speed variation, and change in area are used for obtaining another granule, namely object granule. The commonality between these two granules represents the approximate regions concerning fall scenarios. Instead of the entire frame, these common granules are fed to the Bi-LSTM network for fall classification, thereby increasing speed as well as accuracy. Moreover, temporal self attention mechanism-based transfer learning is used to re-train the Bi-LSTM network, enhances the training speed and accuracy. Characteristics of TLG-LSTM are demonstrated over several real-time traffic videos acquired from ‘YouTube8M’. The superiority of the developed TLG-LSTM is also claimed over several state-of-the-art models.

DOI

10.1016/j.knosys.2025.113038

Publication Date

2-28-2025

Link to Full Text

Share

COinS