| Abstract: |
Collecting highly accurate scientific measurements asks for highest data quality and yield. But, satisfying these requirements is non-trivial, when considering phenomena common to wireless
sensing systems such as clock drift, packet duplicates, packet loss and device reboots. Previous experience shows that these problems have not been resolved sufficiently by system design. In this
paper, we introduce an offline approach to improve data quality by (a) providing a formal system model, (b) verifying conformance of packets received to this model, (c) providing the corrected
packet sequence, and (d) providing additional information on packet generation inferred from temporally adjacent packets. We apply this method to a substantial amount of data from a real-world
deployment and show the usefulness of this new intermediate packet processing step. In our validation of the proposed algorithm, we find that our approach successfully reconstructs the correct
order of packet data streams. On application of the proposed data cleaning only a single violation is found when cross-validating a sequence of more than 4.6 million packets with ground truth
derived from duplicate sensor data recovered from external storage post-deployment. The proposed method is thus suitable for both enhancing data accuracy on the occurrence of faults as well as
the validation of data integrity. |
