• Petr Špaček's avatar
    diffrepro: tool to measure reproducibility of mismatches detected by msgdiff · 2b2a03b4
    Petr Špaček authored
    Some mismatches detected by msgdiff are not readily reproducible and
    sieving output from diffsum by hand is time consuming.
    
    The diffrepro tool walks through all differences in sub-database "diffs"
    and re-queries specified resolvers. Answers are compared using the very
    same algorithm as in msgdiff and diff from new answers is compared
    with respective diff stored in "diffs" DB.
    
    Information whether new answers from other resolves match
    and if comparison with target resolver lead to the same diff as stored
    from previous runs is stored into "reprostats" sub-database.
    
    This tool can be re-run multiple times to gather better statistics.
    Results from current run will be added to results from all the previous
    runs.
    
    Data in the "reprostats" DB can be later read by "reprosum" tool which
    produces list of queries leading to reproducible failures.
    This ability will be later folded into "diffsum" tool.
    2b2a03b4