This sounds exactly like the problem I was having. I couldn't find a consolidated answer so had to piece one together.
First, any CSS that appeared different on portrait and horizontal had to be put into it's own @media tag. It's important to right out the whole class into each @media selector, not just the bits that are different. CSS that is common to both views can got at the top. My device width was showing at 580 so I set the cut-off at 600 - you can set it to what you feel is right for you.
// All CSS that is common to both
@media all and (min-device-width:601px)and (orientation:landscape){
// All CSS for Landscape and Desktop
}
@media only screen and (max-device-width:600px)and (orientation:portrait){
// All CSS for Portrait view
}
Next was the viewport settings. I put this code as standard into each of my page heads (the size is my Mobile Phone Portrait size). It needs the meta there so that the javascript can get to it later.
<meta name="viewport" id="viewport" content="width=480, initial-scale=0.25, maximum-scale=1.0;" />
Finally I had to use some Javascript to re-write the viewport settings when the page detected a rotation of the phone (thanks to Vinayak.B Original Article)
//Code to display on mobiles
//========================================
var swidth = window.screen.width;
//These were the values of my website CSS container for portrait and landscape
var vpwidth = 480;
var vlwidth = 960;
updateOrientation();
window.addEventListener('orientationchange', updateOrientation, false);
function updateOrientation() {
var viewport = document.querySelector("meta[name=viewport]");
switch (window.orientation) {
case 0: //portrait
//set the viewport attributes to whatever you want!
viewport.setAttribute('content', 'width=' + vpwidth + ', initial-scale=0.25, maximum-scale=1.0;')
break;
case 90: case -90: //landscape
//set the viewport attributes to whatever you want!
viewport.setAttribute('content', 'width=' + vlwidth + ', initial-scale=0.25, maximum-scale=1.0;')
break;
default:
//set the viewport attributes to whatever you want!
viewport.setAttribute('content', 'width=' + vpwidth + ', initial-scale=0.25, maximum-scale=1.0;')
break;
}
//alert(swidth + ' lead to an initial width of ' + vpwidth + ' and a rotate width of ' + vlwidth);
}
After HOURS of trying things out, this is what worked for me. For some reason on my phone, the initial-scale=1 screwed it up but 0.25 worked?! I hope it works for you or at least offers a good starting point.