Merge two data frames with out loosing column values

Question

I have the following problem: I have two pandas data frames of different length containing some rows and columns that have common values and some that are different, like this:

df1: df2:

   Date       NOofIM's   Shift              Date     Shift Engineers
0    3/1/2017  2        3            0    3/1/2017     3       4
1    3/1/2017  3        2            1    4/1/2017     2       4
2    4/1/2017  5        1            2    5/1/2017     2       3
3    5/1/2017  6        1            3    7/1/2017     1       2
4    6/1/2017  4        1
5    7/1/2017  5        1

What I want to do now is merging the two dataframes so that if Date and shift have the same value then the rows from df2 shoul merge/appended to the corresponding row in df1, like this:

df1:

          Date       Shift       Engineers   NOofIM's
  0    3/1/2017        3            4         2
  1    3/1/2017        2            Nan       3
  2    4/1/2017        2            4         Nan
  3    4/1/2017        1            Nan       5
  4    5/1/2017        1            Nan       6
  5    5/1/2017        2            4         Nan
  6    6/1/2017        1            Nan       4
  7    7/1/2017        1            2         5

piRSquared piRSquared · Accepted Answer · 2017-04-12T12:03:45

merge combines on all common rows by default.
how='outer' accommodates missing from the first and second dataframe

df1.merge(df2, 'outer')
# more explicit with `on`
# df1.merge(df2, on=['Date', 'Shift'], how='outer')

       Date  NOofIM's  Shift  Engineers
0  3/1/2017       2.0      3        4.0
1  3/1/2017       3.0      2        NaN
2  4/1/2017       5.0      1        NaN
3  5/1/2017       6.0      1        NaN
4  6/1/2017       4.0      1        NaN
5  7/1/2017       5.0      1        2.0
6  4/1/2017       NaN      2        4.0
7  5/1/2017       NaN      2        3.0

Merge two data frames with out loosing column values

1 Answers