Deep Reinforcement Learning-Enhanced Levenberg-Marquardt Neural Network for Improved Energy Efficiency in Wireless Sensor Networks

1. Introduction

In applications like environmental monitoring and tracking, Wireless Sensor Network (WSN) are widely employed. One of the major challenge for WSN is the energy limitations [1]. For the purpose of optimizing EC and ensuring long-term functionality, these WSN networks depend upon the Energy Efficiency (EE). Recently, these issues can be resolved by suggesting several Routing Protocols (RP). Here, considering the limited capacities of sensors and extending the Network Lifetime (NL) still became a challenge. For balancing the network performance with EC, effective Energy Management System (EMS) are needed. Thus, in Real-Time (RT) energy-constrained applications, prolonging NL and ensuring reliable Data Transmission (DT) were considered to be the objective of this EMS.

In remote or wide areas, with minimal human intervention, WSN are widely employed for communication [2]. These networks communicate through the radio signals to form a network for data collection and DT. Here, these networks have SN. The environmental settings, sensor deployment, and RP greatly impacts the EC, and it is considered to be a major challenge in WSN. Optimizing EC and extending NL can be attained by several RP. WSN’s long-term functionality in various applications can be attained by these protocols, and these protocols helps in balancing the EE For DT to Base Station (BS), batteries were employed by the low-power SN in the WSN [3]. Anomalies refers to faulty sensors, as it generates the irregular data readings. Anomalies also caused by limited resources or communication issues. These networks are suspectible to these networks. To detect the background anomalies, the EE RP was established. Extending NL and network stability are considered to be the main objective of the suggested method. Anomaly Detection (AD) methods like Local Outlier Factor (LOF), K-Nearest Neighbours (KNN), and Random Forest (RF) was utilized by this method.

Recently, WSN have gained attention due of its accessibility and small size [4]. Thus, precision agriculture, building security, and environmental monitoring were few of applications for WSN. Especially in unsecured or hostile settings, WSN are suspectible to security risks. The most crucial thing is to ensure secure DT. For preserving data privacy, several techniques have been introduced. Focussing Attack Detection (AD) in this study is crucial for preserving network security. An Intrusion Detection (ID) System (IDS) is suggested by researchers and tested on a WSN-DS dataset. It uses the online aggressive classifier for identifying Denial of Service (DoS) attacks and the Information Gain Ratio (IGR) for Feature Selection (FS).

In WSN [5], strategically placing a few heterogeneous nodes with higher computational capabilities can significantly enhance NL and performance. While various clustering algorithms have been developed to optimize EE and network coverage, determining the optimal network structure remains a challenging task due to the vast number of possible sensor cluster configurations. This complexity arises from the need to balance EC, communication efficiency, and the overall stability of the network. Despite advances, there is still a need for more effective approaches to determine the ideal placement and structure of heterogeneous nodes in WSNs as shown in Fig. 1.

The reliability and security of WSN depend on AD [6]. Improving resource efficiency is essential in considering EC difficulties. Online Anomaly Detection with Energy Efficiency (OAD-EE) techniques with Cloud-Based (CB) model aggregation and Ensemble Federated Learning (EFL) with cloud integration are two sophisticated algorithms created for AD in WSNs. The FL and Ensemble Learning (EL) is employed by EFL with cloud integration for the purpose of enhancing the detection accuracy and securing data privacy. For improving the EC in SN, online learning and EE techniques are used, and it is considered to be the main aim of the OAD-EE. These algorithms collaborate in creating an effective and comprehensive method for WSN.

The Green Internet of Things (GIoT) mainly emphasis on EE, and it have raised its applications. In sensing technologies, GIoT has a major part. Many sectors like healthcare, smart cities, and transportation were revolutionized by IoT. Then, an excessive EC has become the outcomes of the rapid growth of SN. Problems over these networks' effects on the environment are being raised. Thus, researchers and engineering professionals must aim to overcome those issues. For sustainable operations, and maintaining security over a range of domains, the EE and security is crucial for IoT development.

This research focuses on improving EE through an integrated approach that combines advanced techniques for the RL-LEACH protocol to enhance clustering and Energy Management (EM). It combines Deep Reinforcement Learning (DRL) with the ELMNN for AD. The proposed method is benchmarked against existing approaches, showing notable improvements in tumor detection accuracy, sensitivity, and processing efficiency. Experimental outcomes show that the suggested framework surpasses traditional methods in EE, AD accuracy, and network stability, highlighting its potential for RT applications in energy-constrained WSNs.

The remaining sections of the study are organised in the following order: A brief overview of some of the research in the area of disease classification is provided in Section 2. Descriptions of the suggested approach for the DRL-ELMNN scheme are given in Section 3. The results of the experimental performance analysis are described in Section 4. And finally, the results are summarised in Section 5.

2. Related Work

Wang et al. [7] introduced a greedy algorithm for inter-cluster communication and weight-based Cluster Head (CH) election for optimal routing. The method integrates clustering with sink mobility to enhance EE. To validate the effectiveness of the suggested method, the suggested approach is compared to CCMAR and ECDRA algorithms. The synthetic dataset is used for performing this analysis, and this synthetic dataset is generated for sensor networks, various events was also simulated. From the outcomes of the simulation, it is clear that the suggested method exhibits superior improvements in EC, NL, and DT. Then, the impact of network parameters like node density and mobility patterns were explored in this study. This analysis also attains better efficiency and scalability in dynamic WSN backgrounds.

To improve ID in WSN, a GWOSVM-IDS was suggested by Safaldin et al. [8]. Optimizing FS and reducing False Alarms (FA) will pay way for improving ID. A popular benchmark dataset named NSL-KDD'99 dataset was employed for ID. For performance analysis, these datasets are used. From the analysis, the outcomes indicate that the suggested method performs better than the current methods by accuracy, lower FA Rates (FAR), fewer selected features, and quicker execution time. The Grey Wolf Optimizer (GWO) with the configuration of 7 wolves helps in generating favorable outcomes. This GWO attains higher Detection Rates (DR) and overall efficiency. Thus, it is clear that, for robust and effective WSN IDS, this GWO is considered as an effective method.

The mutual user authentication system was suggested by Gope et al. [9]. For Industrial Wireless Sensor Network (IWSN), these systems are designed. Here, privacy-preserving and lightweight methods for safe access are the main focus of IWSN. The SN security can be attained by utilizing the cryptographic methods like one-way hash functions, physically unclonable functions, and bitwise exclusive operations. The protocol ensures security even in the event that a sensor node is hacked. This protocol is secure, effective and it was validated by the outcomes of the analysis. This model is considered to be an effective method for sensor devices with a limited resource. Then, it also provides a comprehensive method for user authentication in IWSN applications. This method also ensures the balance among the security and efficiency.

An EE RP for Underwater Wireless Sensor Network (UWSN) was utilized for improving the EC and it has been suggested by Lilhore et al. [10]. An energy-balanced and depth-controlled method was employed by the suggested method. For Data Fusion (DF), BackPropagation (BP) NN with an Improved Genetic Algorithm (GA) was integrated. Improved encoding, crossover, and mutation approaches are also employed. Selection of high-energy nodes, reducing DT, and optimized EE were considered to be the main focus of this method. From the simulation outcomes, an 86.7% Packet Delivery Ratio (PDR), 12.6% EC, and a 10.5% Packet Loss Ratio (PLR), was attained by the suggested method, it is clear that the suggested method is effective for UWSN and it performs better than the current methods.

According to Deebak and Al-Turjman [11], a secure routing and monitoring protocol was used to identify and stop attacks in WSN. This suggested approach used the Two-Fish (TF) symmetric key encryption technique. This integration will help the protocol choose safe sensor guard nodes. The Authentication and Encryption Model (ATE) with Eligibility Weight Function (EWF) was used. Then, complex symmetric keys are used to secure these nodes. The hybrid RP combines multipath Optimised Link State Routing (OLSR) with Adhoc On-Demand Multipath Distance Vector (AOMDV) protocols. It is evident from the simulation results that the suggested method outperforms the existing approaches in terms of robust multipath delivery, robust security against mobile attacks, and node percentage monitoring.

Dual-side charging techniques for Mobile Robot (MR) traversal planning are suggested by Chen et al. [12]. By doing this, the MR traversal path's length, EC, and completion time are reduced. According to MR dual-side charging, MR may wirelessly charge neighboring sensors on either side of a specified channel while concurrently sending sensory data to MR. Based on the sensor power levels and distances the power diagram is constructed. The path planning and charging methods was facilitated by the power diagram. The MR movement, EC and completion time were additionally optimized by a clustering-based approach. Thus, simulation analysis was conducted, from the outcomes, it is clear that the suggested method performs better than the current methods by the EC, reduced total distance, and minimized completion time in WRSN. An effective Cluster Head Election (CHE) was introduced for IoT applications like environmental monitoring, smart cities, and systems, and it was suggested by Behera et al. [13]. Based on the initial and Residual Energy (RE) levels, the method replaces the CH location for the purpose of selecting nodes with high energy. Then, the simulation outcomes indicate that the suggested method performs better than the Low-Energy Adaptive Clustering Hierarchy (LEACH) methods by a 60% increase in Throughput (T), a 66% improvement in NL, and a 64% increase in RE. The algorithm's efficiency in enhancing the performance of IoT-based networks was highlighted in the outcomes.

For optimizing the crop irrigation, an effective agricultural watering system based WSN was designed, and it was suggested by Muangprathub et al. [14]. A hardware control box with soil moisture sensors, a web-based application for Data Analysis (DA) and Data Management (DM), and a mobile application for user interaction were the3 main components that is integrated by the system. The hardware collects the RT data regarding the soil moisture, temperature, and humidity. To predict the optimal conditions for crop growth, this collected data was then analyzed by the support of web application. Based on the sensor data, this irrigation system can be automatically, or manually managed by the users with the support of the mobile app. This will also facilitate the adaptability. Notifications via the LINE API are also incorporated into the system for the purpose of informing users. Then, the optimal soil moisture levels for vegetable growth was effectively maintained by the system when tested in Makhamtia District, Suratthani Province, Thailand. From the outcomes of the analysis, the reduction in irrigation costs was attained, improved agricultural productivity was demonstrated. The way the digital innovation enhances the sustainable farming practices and resource management was also demonstrated by the outcomes.

By utilizing software-defined radios, a realistic SigFox communication model, and a tested SigFox traffic generator was offered by Lavric et al. [15]. This method facilitates the performance evaluation in large-scale, high density WSN. When 360 channels are accessible, about 100 sensors can transmit data simultaneously at an optimal performance. If this limit is exceeding then there will be a congestion, and it may cause significant performances degradation named "avalanche effect". In case of large-scale, high-density backgrounds, several methods were suggested in this study to lessen this problem and also for enhancing the SigFox networks performances. The comprehension regarding the limitations SigFox was offered by this study. For improving the SigFox efficiency and scalability in IoT applications, this study offers methods.

To ensure the sensor Data Integrity (DI), An integrated IoT platform was suggested by Hang and Kim [16]. This integrated IoT platform utilizes Blockchain Technology (BCT). A beneficial application was offered by this platform to the device users, as it helps in managing devices over several domains. A n immutable repository was also offered for easy access. The essential components of conventional IoT systems were incorporated, by doing this, the RT monitoring and control between end users and devices are facilitated. The business logic is controlled by the Smart Contracts (SC), and these SC offers terms and conditions. The proposed approach is demonstrated as a proof of concept using Raspberry Pi devices in the Hyperledger Fabric permissioned network. Several performance metrics and a benchmark study were utilized for evaluating the capabilities of the platform.

Bu et al. [17] explored the filtering problems associated with nonlinear systems in sensor networks characterized by dynamic topologies. Their research focused on the implementation of the Round-Robin protocol within finite horizon contexts. The study provides insights into the complexities of managing nonlinear filtering in evolving network structures. The RR communication method was utilized by this method. Switching topologies caused by weak node-to-node connections are considered for enhancing the Bandwidth (BW) and reducing resource utilization. The primary goal is to develop distributed filters that maintain a consistent average H∞ performance within a specified range. To ascertain the parameters of the desired filters, the study uses a recursive approach based on matrix inequalities. It creates the necessary conditions for these filters to exist in the parameters of the study. A numerical simulation is then employed for assessing the effectiveness of the suggested filter design method. This study makes a substantial contribution to distributed filtering techniques in adaptable network topologies.This distributed filtering methods is applicable to sensor networks where robust filtering under dynamic situations and effective resource utilisation are crucial.

3. Proposed Methodology

In this paper [8], the performance of EE in WSNs categorization, the Enhanced Levenberg-Marquardt Neural Network (ELMNN) technique is suggested. Classification is the primary process of the suggested method. The general block diagram of the suggested system is shown in Fig. 2.

3.1 RL-LEACH

In this study [18], the RL-LEACH protocol improves the efficiency of WSN by leveraging Reinforcement Learning (RL) to optimize CH Selection (CHS). This approach considers key factors such as EC, coverage area, and the distance to the sink node, ensuring a more EE network.

Initially, each node generates a random number rNum within the range [0,1]. This value is compared against a calculated threshold Tnn(m), which is determined as shown in Eq. (1):

p Eresidual Tnn(m) = { . , m ∈ G, 1 − p. (m mod initial

p (1)

0, otherwise.

Here, p is the desired percentage of CH nodes, m is the node index, Eresidual is the node’s remaining energy, Einitial is its initial energy, and represents the set of eligible nodes for CHS.

In Fig. 3, RL is used to evaluate each node based on the following metrics:

I. Energy Consumption (EC) (Reward(i,1)): energy required for data aggregation and communication.

II. Coverage area (Reward(i,2)): the communication range of the node.

III. Distance to the sink (Reward(i,3)): determines proximity to minimize transmission energy.

Nodes are assigned reward values based on these criteria, and those with the highest rewards and satisfaction rNum < Tnn(m) are selected as CHs.

DT efficiency is further optimized by using selected CHs as intermediate nodes. The RL algorithm analyzes reward metrics and CH properties to determine the most efficient route from source nodes to the sink, minimizing energy usage and improving data delivery.

Algorithm 1: RL-LEACH algorithm

Start

I. During cluster formation, each node generates a random number rNum in [0,1].

II. The given formula is employed to compute the threshold Tnn(m) for each node.

III. To evaluate nodes based on EC, coverage area, and distance, the RL was applied.

IV. Initialize a reward matrix reward []:

− Reward(i,1): energy consumption (i).

− Reward(i,2): coveragearea (i).

− Reward(i,3): distance (i).

V. Identify nodes with the highest rewards.

VI. If rNum < Tnn(m) and the node has the maximum reward:

− Select the node as a CH.

VII. Otherwise, exclude the node from the CHS.

VIII. Optimize routing by choosing intermediate CHs based on their rewards and properties.

IX. Return the optimized CHs for effective cluster monitoring and DT.

End

Here, this protocol supports in ensuring an effective CHS and routing. The NL has been extended by this protocol, as it also improves the overall performance.

This RL-LEACH protocol integrates the threshold-based CHS with a RL method, and this integration will assist in improving the overall performance and lifetime of the WSN [19]. The nodes are evaluated based on the EC, coverage area, and distance to sink for the purpose of attaining an optimal CHS and effective routing. This method facilitates in reducing the EC, improving the DT reliability, extending NL. So, this method is considered to be effective for robust EE WSN management.

3.2 Deep Reinforcement Learning

The complex Decision Making (DM) problems was managed by DRL, as it integrates RL and DL. It is visualized in the Fig. 4 [20]. The Deep Neural Network (DNN) was utilized for learning the optimal features from High-Dimensional (HD) data. DRL can optimise processes like network traffic or EC in Resource Allocation (RA) by processing incoming data and then passing it via a DNN for DM.

Deep Q-Learning (DQN) and Deep Policy Gradient (DPG) are two important DRL methods. DQN is value-based, where the agent learns a Q-function that estimates the expected cumulative reward for a given state-action pair. The Q-value is updated using the Bellman Eq. (2):

max ′ ′

Q(s, a) = R(s, a) + γ Q(s , a ), (2) where R(s, a)is the immediate reward, γ is the discount factor, and is the next state.

DPG [21] is policy-based, where the agent directly optimizes the policy, π(a|s; θ), by performing gradient ascent to maximize expected rewards. In Eq. (3), The objective is:

J (θ) = πθ[R(s, a)], (3)

where R(s, a) is the reward after action a in state. DQN is best for discrete action spaces, while DPG is suited for continuous action tasks, such as resource allocation or robot control.

4. Experimental Result

Here, the performance of the suggested method and the current methods are compared for assessing the effectiveness of the model [26]. System setup for the suggested system, implemented in MATLAB, consists of an Intel i3 processor running at 3.0 GHz, 8 GB of RAM, and 1TB of hard disk storage. The simulation efficiency of the suggested method is compared to other standard methods for determining the suggested methods efficiency.

Problems like neural network classification are effectively resolved by these classification tasks. So, these classification tasks are well-suited for addressing those issues. Adjusting the parameter greatly impacts the model’s accuracy. From the analysis, it is clear that the suggested method is beneficial in enhancing the computational efficiency and accuracy of the model. A crucial benchmark was offered for DRL-ELMNN by these models [27]. The performance comparison outcomes among the suggested method are presented in

5. Conclusion

A sophisticated method for enhancing EE and reliability in WSN was presented in this paper, which is essential for power-constrained applications like tracking and EM. The suggested approach optimises clustering and EM by combining DRL with the LEACH technique. Furthermore, intelligent categorisation is improved, EE is increased, and the NL is extended by the integration of DRL and ELMNN. Simulation findings demonstrate the model's superiority over conventional techniques, with notable gains in throughput, EC, and PDR. The DRL-ELMNN method obtains a PDR of 90.8%, boosts throughput to 80.6%, and lowers EC to 75.3%. It is perfect for RT WSN applications since it also efficiently handles packet loss and E2ED. These results show that DRL and ELMNN can optimise WSN operations and offer a reliable solution for areas with limited energy. In contemporary IoT and sensor-based applications, this model raises the standards for improving WSN performance and helps create more sustainable and effective networks.

Author Contributions

Hashem Saberi Najafi: conceptualizing, methodology, development of the DRL-ELMNN Model, Writing– initial draft.

Szabolcs Fischer: designing simulations, assessing performance, visualizing data, writing–review & editing.

Isametova Madina Esdauletova: reviewing literature, validating findings, structuring the manuscript, interpreting results.

All authors have reviewed and sanctioned the final manuscript.

Funding

This study was carried out without any external funding. All necessary resources were provided by the respective institutions of the authors.

Data Availability

The datasets and simulation results produced or assessed during this study are available from the

simulations can also be provided upon request for academic and research purposes.

Conflicts of Interest

The authors state that there are no conflicts of interest. The study was conducted impartially and free from any commercial or financial ties that might be perceived as a possible conflict.

Figures