How to transform long to wide data in Stata?

Question

I have this data:

id  test    test_date   value
1   A   02/06/2014 12:26    11
1   B   02/06/2014 12:26    23
1   C   02/06/2014 13:17    43
1   D   02/06/2014 13:17    65
1   E   02/06/2014 13:17    34
1   F   02/06/2014 13:17    64
1   A   05/06/2014 15:14    234
1   B   05/06/2014 15:14    646
1   C   05/06/2014 16:50    44
1   E   05/06/2014 16:50    55
2   E   05/06/2014 16:50    443
2   F   05/06/2014 16:50    22
2   G   05/06/2014 16:59    445
2   B   05/06/2014 20:03    66
2   C   05/06/2014 20:03    77
2   D   05/06/2014 20:03    88
2   E   05/06/2014 20:03    44
2   F   05/06/2014 20:19    33
2   G   05/06/2014 20:19    22

I would like to transform this data into wide format like this:

id    date              A   B   C   D   E   F   G
1   02/06/2014 12:26    11  23  43  65  34  64  .
1   05/06/2014 15:14    234 646 44  .   55  .   .
2   05/06/2014 16:50    .   .   .   .   443 22  445
2   05/06/2014 20:03    .   66  77  88  44  33  22

I am using reshape command in Stata, but it is not producing required results:

reshape wide test_date value, i(id) j(test) string

Any idea how to do this?

UPDATE: You're right that we need this missvar. I try to create this by programming, but failed. Let say with-in 2 hours of test date the batch will consider same. We have only 7 tests (A,B,C,D,E,F,G). First I try to find the time difference;

 bysort id: gen diff_bd = (test_date[_n] - test_date[_n-1])/(1000*60*60)    

 bysort id: generate missvar = _n if diff_bd <= 2

Not the question, but such reshape wide usually makes analyses harder! — Nick Cox
"not producing required results" is a poor problem report. You need to explain precisely what is wrong or puzzling. Does Stata produce an error message? etc. — Nick Cox

Roberto Ferrer Roberto Ferrer · Accepted Answer · 2015-04-09T13:23:19

@jfeigenbaum has given part of the answer.

The problem I see is that you are missing a variable that identifies relevant sub-groups. These sub-groups seem to be bounded by test taking values A - G. But I may be wrong.

I've included this variable in the example data set, and named it missvar. I forced this variable into the data set believing it identifies groups that, although implicit in your original post, are important for your analysis.

clear
set more off

*----- example data -----

input ///
id  str1 test    str30 test_date   value missvar
1   A   "02/06/2014 12:26"    11 1
1   B   "02/06/2014 12:26"    23 1
1   C   "02/06/2014 13:17"    43 1
1   D   "02/06/2014 13:17"    65 1
1   E   "02/06/2014 13:17"    34 1
1   F   "02/06/2014 13:17"    64 1
1   A   "05/06/2014 15:14"    234 2
1   B   "05/06/2014 15:14"    646 2
1   C   "05/06/2014 16:50"    44 2
1   E   "05/06/2014 16:50"    55 2
2   E   "05/06/2014 16:50"    443 1
2   F   "05/06/2014 16:50"    22 1
2   G   "05/06/2014 16:59"    445 1
2   B   "05/06/2014 20:03"    66 2
2   C   "05/06/2014 20:03"    77 2
2   D   "05/06/2014 20:03"    88 2
2   E   "05/06/2014 20:03"    44 2
2   F   "05/06/2014 20:19"    33 2
2   G   "05/06/2014 20:19"    22 2
end

gen double tdate = clock( test_date, "DM20Yhm")
format %tc tdate

drop test_date
list, sepby(id)

*----- what you want ? -----

reshape wide value, i(id missvar tdate) j(test) string
collapse (min) tdate value?, by(id missvar)

rename value* *

list

There should be some way of identifying the groups programmatically. Relying on the original sort order of the data is one way, but it may not be the safest. It may be the only way, but only you know that.

Edit

Regarding your comment and the "missing" variable, one way to create it is:

// one hour is 3600000 milliseconds
bysort id (tdate): gen batch = sum(tdate - tdate[_n-1] > 7200000)

For your example data, this creates a batch variable identical to my missvar. You can also use time-series operators.

Let me emphasize the need for you to be carefull when presenting your example data. It must be representative of the real one or you might get code that doesn't suit it; that includes the possibility that you don't notice it because Stata gives no error.

For example, if you have the same test, applied to the same id within the two-hour limit, then you'll lose information with this code (in the collapse). (This is not a problem in your example data.)

Edit 2

In response to another question found in the comments:

Suppose a new observation for person 1, such that he receives a repeated test within the two-hour limit, but at a different time :

1   A   "02/06/2014 12:26"    11 1 // old observation
1   B   "02/06/2014 12:26"    23 1
1   A   "02/06/2014 12:35"    99 1 // new observation
1   C   "02/06/2014 13:17"    43 1
1   D   "02/06/2014 13:17"    65 1
1   E   "02/06/2014 13:17"    34 1
1   F   "02/06/2014 13:17"    64 1
1   A   "05/06/2014 15:14"    234 2
1   B   "05/06/2014 15:14"    646 2
1   C   "05/06/2014 16:50"    44 2
1   E   "05/06/2014 16:50"    55 2

Test A is applied at 12:26 and at 12:35. Reshape will have no problem with this, but the collapse will discard information because it is taking the minimum values amongst the id missvar groups; notice that for the variable valueA, new information (the 99) will be lost (so too happens with all other variables, but you are explicit about wanting to discard that). After the reshape but before the collapse you get:

. list, sepby(id)

     +--------------------------------------------------------------------------------------------------+
     | id   missvar                tdate   valueA   valueB   valueC   valueD   valueE   valueF   valueG |
     |--------------------------------------------------------------------------------------------------|
  1. |  1         1   02jun2014 12:26:00       11       23        .        .        .        .        . |
  2. |  1         1   02jun2014 12:35:00       99        .        .        .        .        .        . |
  3. |  1         1   02jun2014 13:17:00        .        .       43       65       34       64        . |
  4. |  1         2   05jun2014 15:14:00      234      646        .        .        .        .        . |
  5. |  1         2   05jun2014 16:50:00        .        .       44        .       55        .        . |
     |--------------------------------------------------------------------------------------------------|
  6. |  2         1   05jun2014 16:50:00        .        .        .        .      443       22        . |
  7. |  2         1   05jun2014 16:59:00        .        .        .        .        .        .      445 |
  8. |  2         2   05jun2014 20:03:00        .       66       77       88       44        .        . |
  9. |  2         2   05jun2014 20:19:00        .        .        .        .        .       33       22 |
     +--------------------------------------------------------------------------------------------------+

Running the complete code confirms what we just said:

. list, sepby(id)

     +--------------------------------------------------------------------------+
     | id   missvar                tdate     A     B    C    D     E    F     G |
     |--------------------------------------------------------------------------|
  1. |  1         1   02jun2014 12:26:00    11    23   43   65    34   64     . |
  2. |  1         2   05jun2014 15:14:00   234   646   44    .    55    .     . |
     |--------------------------------------------------------------------------|
  3. |  2         1   05jun2014 16:50:00     .     .    .    .   443   22   445 |
  4. |  2         2   05jun2014 20:03:00     .    66   77   88    44   33    22 |
     +--------------------------------------------------------------------------+

Suppose now a new observation for person 1, such that he receives a repeated test within the two-hour limit, but at the same time:

1   A   "02/06/2014 12:26"    11 1 // old observation
1   B   "02/06/2014 12:26"    23 1
1   A   "02/06/2014 12:26"    99 1 // new observation
1   C   "02/06/2014 13:17"    43 1
1   D   "02/06/2014 13:17"    65 1
1   E   "02/06/2014 13:17"    34 1
1   F   "02/06/2014 13:17"    64 1
1   A   "05/06/2014 15:14"    234 2
1   B   "05/06/2014 15:14"    646 2
1   C   "05/06/2014 16:50"    44 2
1   E   "05/06/2014 16:50"    55 2

Then the reshape won't work. Stata complains:

values of variable test not unique within id missvar tdate

and with reason. The error is clear in signalling the problem. (If not clear, go back to help reshape and work out some exercises.) The request makes no sense given the functioning of the command.

Finally, note it's relatively easy to check if something will work or not: just try it! All that was necessary in this case was to modify a bit the example data. Go back to help files and manuals, if necessary.

How to transform long to wide data in Stata?

2 Answers

Edit

Edit 2