0
votes

In Power Query, given a table with cumulative values (Cumulative Unique Count) based on dates (Start of Week) which require filling down, how can the missing rows be generated if it has multiple dimension/attribute columns? Other value columns (Count and Unique Count in Week) in the table should be filled in as null or 0.

Example table:

Criteria One Criteria Two Start of Week Count Unique Count in Week Cumulative Unique Count
A C 3/22/2020 4 1 1
A C 3/29/2020 9 3 4
A C 4/5/2020 3 2 5
A C 4/12/2020 10 8 9
A D 3/22/2020 3 3 3
A D 4/5/2020 8 5 6
A D 4/12/2020 2 3 9
B C 3/29/2020 6 4 4
B C 4/12/2020 12 9 10
B D 3/29/2020 8 4 4
B D 4/5/2020 6 6 7
B D 4/12/2020 9 7 9

The result should be something like this (without the Comment column):

Criteria One Criteria Two Start of Week Count Unique Count in Week Cumulative Unique Count Comment
A C 3/22/2020 4 1 1
A C 3/29/2020 9 3 4
A C 4/5/2020 3 2 5
A C 4/12/2020 10 8 9
A D 3/22/2020 3 3 3
A D 3/29/2020 0 0 3 Fill down
A D 4/5/2020 8 5 6
A D 4/12/2020 2 3 9
B C 3/22/2020 0 0 0 Fill
B C 3/29/2020 6 4 4
B C 4/5/2020 0 0 4 Fill down
B C 4/12/2020 12 9 10
B D 3/22/2020 0 0 0 Fill
B D 3/29/2020 8 4 4
B D 4/5/2020 6 6 7
B D 4/12/2020 9 7 9

Pivoting, filling down, then unpivoting like suggested here and here won't work because there's more than one criteria and the criteria sometimes contains nulls.

2

2 Answers

0
votes

See if this works for you

I am creating all combinations of dates and criteria, appending that to the original table, then using an index to compare to prior row to see if I need to fill down or not

let Source = Excel.CurrentWorkbook(){[Name="Table1"]}[Content],
#"Changed Type" = Table.TransformColumnTypes(Source,{{"Criteria One", type text}, {"Criteria Two", type text}, {"Start of Week", type date}, {"Count", Int64.Type}, {"Unique Count in Week", Int64.Type}, {"Cumulative Unique Count", Int64.Type}}),

// get all unique combinations of Criteria1 and Criteria2 and merge it to all combination of dates between max and min dates
#"Removed Other Columns" = Table.SelectColumns(#"Changed Type",{"Criteria One", "Criteria Two"}),
Unique_Critera = Table.Distinct(#"Removed Other Columns"),
Max= Number.From(List.Max(#"Changed Type"[Start of Week])),
Min= Number.From(List.Min(#"Changed Type"[Start of Week])),
Date_Range={ Number.From(List.Min(#"Changed Type"[Start of Week])) .. Number.From(List.Max(#"Changed Type"[Start of Week]))},
#"Added Custom" = Table.AddColumn(Unique_Critera, "Start of Week", each Date_Range),
#"Expanded Custom" = Table.ExpandListColumn(#"Added Custom", "Start of Week"),
#"Changed Type1" = Table.TransformColumnTypes(#"Expanded Custom",{{"Start of Week", type date}}),
#"Added Custom1" = Table.AddColumn(#"Changed Type1", "DayWeek", each Date.DayOfWeek([Start of Week])),
#"Filtered Rows" = Table.SelectRows(#"Added Custom1", each ([DayWeek] = 0)),
#"Removed Columns" = Table.RemoveColumns(#"Filtered Rows",{"DayWeek"}),

// remove those on list that exists alrady
#"Merged Queries" = Table.NestedJoin(#"Removed Columns",{"Criteria One", "Criteria Two", "Start of Week"},#"Changed Type",{"Criteria One", "Criteria Two", "Start of Week"},"Table2",JoinKind.LeftOuter),
#"Expanded Table2" = Table.ExpandTableColumn(#"Merged Queries", "Table2", {"Criteria One"}, {"Criteria One.1"}),
#"Filtered Rows1" = Table.SelectRows(#"Expanded Table2", each ([Criteria One.1] = null)),
#"Removed Columns1" = Table.RemoveColumns(#"Filtered Rows1",{"Criteria One.1"}),

// append that list to original and sort
#"Appended Query" = Table.Combine({#"Changed Type", #"Removed Columns1"}),
#"Sorted Rows" = Table.Sort(#"Appended Query",{{"Criteria One", Order.Ascending}, {"Criteria Two", Order.Ascending}, {"Start of Week", Order.Ascending}}),

// refer to previous row to see if we need to fill down or leave as null
#"Added Index" = Table.AddIndexColumn(#"Sorted Rows", "Index", 0, 1),
#"Added Custom2" = Table.AddColumn(#"Added Index", "Custom", each if [Cumulative Unique Count] <> null then [Cumulative Unique Count] else if #"Added Index"{[Index]-1}[Criteria One]=[Criteria One] and #"Added Index"{[Index]-1}[Criteria Two]=[Criteria Two] then #"Added Index"{[Index]-1}[Cumulative Unique Count] else null),
#"Removed Columns2" = Table.RemoveColumns(#"Added Custom2",{"Cumulative Unique Count", "Index"}),
#"Renamed Columns" = Table.RenameColumns(#"Removed Columns2",{{"Custom", "Cumulative Unique Count"}})
in #"Renamed Columns"

You could I suppose also merge the original data into the new table and proceed from there saving a few rows

0
votes

Here is a different Power Query Approach:

  • Create a list of ALL the possible weeks
  • Group by the Criteria
  • Create a subTable for each group
    • Create a list of the Missing Weeks
    • For each missing week, create a Record whereby
      • The Criteria are carried over from the group
      • The Date is incremented according to the missing week list
      • the various counts are set to Null (you could set the to zero if you prefer, or anything else by changing the value in the List.Generate function)
  • Then expand the subTables and set the Type

M Code

let
    Source = Excel.CurrentWorkbook(){[Name="Table6"]}[Content],
    #"Changed Type" = Table.TransformColumnTypes(Source,{
        {"Criteria One", type text}, {"Criteria Two", type text}, {"Start of Week", type date}, 
        {"Count", Int64.Type}, {"Unique Count in Week", Int64.Type}, {"Cumulative Unique Count", Int64.Type}}),

//create a list of all the possible weeks
    wkFirst = List.Min(#"Changed Type"[Start of Week]),
    wkLast = List.Max(#"Changed Type"[Start of Week]),
    numWeeks = Duration.Days(wkLast-wkFirst)/7+1,
    allWeeks = List.Dates(wkFirst,numWeeks,#duration(7,0,0,0)),

//Group; then add missing weeks to each sub-table    
    #"Grouped Rows" = Table.Group(#"Changed Type", {"Criteria One", "Criteria Two"}, {
        {"All", each _, type table [Criteria One=nullable text, Criteria Two=nullable text, Start of Week=nullable date, Count=nullable number, Unique Count in Week=nullable number, Cumulative Unique Count=nullable number]},
        {"with All weeks", each if Table.RowCount(_) = List.Count(allWeeks) then _

            //create the subtables whereby we fill in the missing weeks
            //and create a record for the missing weeks whereby the Criteria are carried over
            //and the Counts are "nulled".
            //if you want to make the counts Zero, just change that within the List.Generate function
                else Table.Sort(
                        Table.Combine({_,   
                    let 
                        missingWeeks = List.RemoveMatchingItems(allWeeks, [Start of Week]),
                        startDts = List.Generate(()=>
                            [Counter =0, Criteria One= [Criteria One]{0}, Criteria Two= [Criteria Two]{0}, Start of Week=missingWeeks{0}, Count=null, Unique Count in Week=null, Cumulative Unique Count=null],
                            each [Counter] < List.Count(missingWeeks),
                            each [Counter = [Counter]+1, Criteria One=[Criteria One], Criteria Two=[Criteria Two], Start of Week=missingWeeks{[Counter]+1}, Count=null, Unique Count in Week=null, Cumulative Unique Count=null])
                    in

                        //Remove the "Counter" column from the Combined subtable
                        //Then Sort by "Start of Week"
                        Table.RemoveColumns(Table.FromRecords(startDts),"Counter")}),
                        {"Start of Week"})
                        }
        }),

//Remove Extra Columns, then expand the subtable, and type it
    #"Removed Columns" = Table.RemoveColumns(#"Grouped Rows",{"Criteria One", "Criteria Two", "All"}),
    #"Expanded with All weeks" = Table.ExpandTableColumn(#"Removed Columns", "with All weeks", {"Criteria One", "Criteria Two", "Start of Week", "Count", "Unique Count in Week", "Cumulative Unique Count"}, {"Criteria One", "Criteria Two", "Start of Week", "Count", "Unique Count in Week", "Cumulative Unique Count"}),
    #"Changed Type1" = Table.TransformColumnTypes(#"Expanded with All weeks",{{"Criteria One", type text}, {"Criteria Two", type text}, {"Start of Week", type date}, {"Count", Int64.Type}, {"Unique Count in Week", Int64.Type}, {"Cumulative Unique Count", Int64.Type}})
in
    #"Changed Type1"

Orig Data
enter image description here

Results
enter image description here