diffrepro: tool to measure reproducibility of mismatches detected by msgdiff
Some mismatches detected by msgdiff are not readily reproducible and sieving output from diffsum by hand is time consuming. The diffrepro tool walks through all differences in sub-database "diffs" and re-queries specified resolvers. Answers are compared using the very same algorithm as in msgdiff and diff from new answers is compared with respective diff stored in "diffs" DB. Information whether new answers from other resolves match and if comparison with target resolver lead to the same diff as stored from previous runs is stored into "reprostats" sub-database. This tool can be re-run multiple times to gather better statistics. Results from current run will be added to results from all the previous runs. Data in the "reprostats" DB can be later read by "reprosum" tool which produces list of queries leading to reproducible failures. This ability will be later folded into "diffsum" tool.
Showing with 177 additions and 29 deletions