There are a few reasons this can occur:
Internet Explorer Memory Usage
This is the default value for caching in the grid on your server. You can lower this all the way to 0 which turns off caching completely. The '3' means your end users are caching 4 total documents as they move forward in linear review, i.e. it will always be cached 3 ahead. So if your users jump around the grid, lower this option to reduce memory consumption on Internet explorer.
This is located C:/Program Files/Accessdata/Map/Web.config
<add key="GridCacheCount" value="3" />
Loading Large Native into Alternate Viewer
Another value that affects end user experience is the native viewer max which you would adjust down to prevent large files from even attempting to load into the Alternate Viewer. By default it is on 50 MB.
Error Message: The current document’s file size is larger than the current settings allow to display.
<add key="NativeViewerMaxFileSizeMegaBytes" value="50" />
If you have a specific format that causes the Alternate Viewer issue, you can add the extension.
<add key="ExcludeFileExtensionsFromNativeViewer" value="...
Browser Cache Issue
Lastly if you never have cleared your Internet Explorer cache, especially after an upgrade, please use the following script.
ECHO Clearing IE cache...
RunDll32.exe InetCpl.cpl,ClearMyTracksByProcess 9
RunDll32.exe InetCpl.cpl,ClearMyTracksByProcess 2
ECHO Job complete