There is a lot of inconvenience going around the answers.
First: Visual Studio 2010 can install its VSTO package even on machines where Office 2010 is not installed, for me it was done when upgrading VS2008/VSTO3 project.
Second: upgraded project retained links links to Office 11, Visual Studio 9 assemblies. (I honestly doubt this will pass with VS2012 though, if these assemblies aren't part of separate VSTO3.0/4.0 runtime packages).
Also, it is impossible to create Office 2003 project from VS2010, but https://stackoverflow.com/a/5325505/438039 points us to the wide possibilities of making VSTO4.0 (e.g. Interop for Office 12, Visual Studio 10) project to run within Office 2003. I think it's also possible to run it from Studio by changing project settings like in here: C# - can't debug office word add-in - this way, I recommend starting without debug (since external application isn't attached to debugger when starting..). This also revealed the existence of VSTOWord2003Adaptor.dll
which is loaded from VS2008 directory on my system. - hope it's a part of either VSTO3 / VSTO4 runtime packages. Another way is to edit .csproj, providing path to WinWord.exe - https://stackoverflow.com/a/12584772/438039 - both aren't working right.
It's harder to set up a proper VS2010/VSTO4 development environment than to build an addin targeting Office 2003.