Commit 965d19b8 authored by Jan Kiene's avatar Jan Kiene
Browse files

add explanation and better sort

parent ac4c0d3d
Loading
Loading
Loading
Loading
+45 −9
Original line number Diff line number Diff line
@@ -34,8 +34,37 @@ Comparing:
</ul>

<br>
<b>Table is sorted by Difference in MLD with ERRORs or missing values ("None", "") being on top additionally.</b>
<b>How is the table sorted?</b>
<ul>
    <li>Cases with result ERROR or invalid/missing values for the numerical measures are given first</li>
    <li>Next are all cases with MLD(current) != MLD(previous), sorted in descending order (biggest MLD increase comes first)</li>
    <li>All cases with no difference in MLD are at the bottom</li>
</ul>
<br>
<b>What do the colours indicate</b>
<ul>
    <li>
        <span style="background-color:#ff5500;">Red background</span>:
        <ul>
            <li>This testcases either triggered an ERROR in the pytest run (probably a crash in the codec) or did not give valid values for the numerical measures (probably an error in the scripts).</li>
            <li>For this testcase the MLD increased wrt the previous run - MLD(current) > MLD(previous)</li>
        </ul>
    </li>
    <li>
        <span style="background-color:#acff00;">Green background</span>:
        <ul>
            <li>For this testcase the MLD decreased wrt the previous run - MLD(current) < MLD(previous)</li>
        </ul>
    </li>
</ul>
<br>
<b>How to interpret the Result column?</b>
<ul>
    <li>ERROR: An error occured during test run. It should be checked if a crash in the codec occured.</li>
    <li>FAIL: An MLD value > 0 was reported. This is to be expected, the test just reports a failure due to how the tests are currently implemented.</li>
    <li>PASS: MLD value of 0 was reported. This should only be the case for some special operating points and could indicate that parts of the codec are not being converted yet.</li>
</ul>
<b>

<table class="tbase"><thead>
  <tr>
@@ -59,8 +88,6 @@ TH_TMPL_SECOND_ROW = '<th class="tcenter">{}</th>'

ARROW_UP = '<span class="arrowup">&#11008;</span>'
ARROW_DOWN = '<span class="arrowdown">&#11010;</span>'
# ARROW_UP = '&#11008;'
# ARROW_DOWN = '&#11010;'

# expected columns. actual columns are filtered from the incoming data later, this
# is mainly for controlling the order in the output table
@@ -183,15 +210,24 @@ def merge_and_cleanup_mld_reports(
    mld_col_curr = f"MLD-{id_current}"
    mld_col_prev = f"MLD-{id_previous}"

    # sort based on difference in MLD between current and previous run
    # put cases with "None" at the top of the list
    def sort_func(x):
        vals_missing = ["None", ""]

        if x[mld_col_curr] in vals_missing or x[mld_col_prev] in vals_missing:
        """
        Sort function for the rows. Puts missing or invalid values on top as those usually
        indicate crashes. Then sorts by MLD difference in descending order. MLD diffs of zero
        are uninteresting and are put last.
        """
        try:
            float(x[mld_col_curr])
            float(x[mld_col_prev])
        except ValueError:
            # Value is no valid floating point value
            return float("inf")

        return float(x[mld_col_curr]) - float(x[mld_col_prev])
        diff = float(x[mld_col_curr]) - float(x[mld_col_prev])
        if diff == 0:
            diff = float("-inf")

        return diff

    merged = sorted(merged, key=sort_func, reverse=True)