Wireless sensor networks (WSNs) are open to false data injection attack when they are deployed in hostile scenarios. Attackers can easily deceive the sink by compromising sensing nodes or by injecting phoney data into the network. Such attacks can deplete the energy resources of the network by providing wrong information which in turn can affect the proper network functioning or sometimes can shut the network from further functioning. The existing schemes that deal with this problem focus on only a few aspects of the false data injection attack. To resolve this problem, we propose a Rank-based Report Filtering Scheme (RRFS), a holistic and group verification scheme for the identification of compromised nodes and the filtering of false data injected into the network. The proposed scheme verifies report among clusters, en-routers, and sink. Hence, the RRFS, a holistic scheme that is composed of three-tier verifications, successfully rejects the false data before the attackers falsify the whole environment, and this makes the system unique. Reliability Index (RI) is calculated by the nodes for fellow cluster members, and the cluster head (CH) provides the score for a node based on its RI. This, in turn, strengthens the scheme by assisting the en-routers to detect the compromised nodes. The RRFS scheme has been verified and validated by extensive simulation and meticulous performance evaluation of filtering efficiency and energy consumption against various schemes. The scheme gives high filtering efficiency against the multiple compromised nodes and also improves the network’s lifespan. The sustainability of RRFS against numerous attacks that are launched in the sensor environment is thoroughly investigated.