Ensure that ScrollViewer.CanContentScroll is set to false.
<ScrollViewer PanningMode="Both" CanContentScroll=false>
<TextBlock FontSize="15" TextWrapping="Wrap">
(a block of random text)
</TextBlock>
</ScrollViewer>
Physical vs. Logical Scrolling
Physical scrolling is used to scroll content by a predetermined
physical increment, typically by a value that is declared in pixels.
Logical scrolling is used to scroll to the next item in the logical
tree. Physical scrolling is the default scroll behavior for most Panel
elements. WPF supports both types of scrolling.
http://msdn.microsoft.com/en-us/library/system.windows.controls.scrollviewer.cancontentscroll%28v=vs.110%29.aspx
Alternatively, you could try this fella:
SurfaceScrollViewer Class
Users can flick the content of a SurfaceScrollViewer control by
touching the control, rapidly moving their finger a short distance,
and then lifting their finger. When their finger is lifted, the
content of the SurfaceScrollViewer control continues to move. The
content then decelerates and comes to a stop. You can programmatically
stop the movement caused by flicking by calling the StopFlick method.
You can manipulate the content of a SurfaceScrollViewer either by
moving a scroll bar (like with the WPF SurfaceScrollViewer) or by
direct touch interaction with the content itself. The ability to
manipulate the content directly is called panning. By default, panning
is enabled. You can disable it by setting the IsManipulationEnabled
property to false.
http://msdn.microsoft.com/en-us/library/microsoft.surface.presentation.controls.surfacescrollviewer.aspx
(Archived link)