Overnight I managed to get a copy of a Game of Thrones file that exhibits the same symptoms (I think - its not exactly the same file). Interestingly, this file plays fine on my Panasonic but not on the PS3 (unusual, its normally the other way around).
If it is the same as the one I have then there is something wrong with the video stream within the mkv container (I re-encoded audio only and that didn't fix - even though it has that weird stereo to 5.1 transition - then re-encoded video and that works). As such Illico's solution should work for you, as it transcodes the video stream. The problem though is this will transcode video for
all h264 encoded video (including all the mkv's and mp4's that work for you) - so you will either need to live with that or you will need to swap profiles for each file.
As I am not a big fan of transcoding video unless absolutely necessary, I tried remuxing the file to a new mkv container - didn't work, and then I tried remuxing into an MP4, and bingo, worked. Remuxing is prefered as there is no loss in video quality. Unfortunately Serviio cannot remux into an MP4 container automatically, so you need to do this outside of Serviio. If you'd like to go down this path, then from the command prompt type:
"C:\Program Files\Serviio\lib\ffmpeg.exe" - i "G:\Serien\Game of Thrones\S1\gamethrones.s01e01.720-pretail.mkv" -vcodec copy -acodec copy "G:\Serien\Game of Thrones\S1\gamethrones.s01e01.720-pretail.mp4"
I stress this worked for me with my test file - am hoping your file has the same problem. It copies both video and audio without loss, but you will likely lose your chapters and maybe your other audio track (let me know). I have used your file with header compression in the above command line - if this doesn't work, try it with the file that you removed compression from. The above assumes you are a Windows 32 bit users - you may have to adjust for the location of your ffmpeg.exe file if not.
I have no idea why this works - there is an issue in the video stream which only appears in an mkv file and only some renderers (eg the PS3). Put the same streams into an MP4 container and it works!? The added advantage though is that the PS3 prefers MP4 - you will likely get better support for fast forwarding and rewinding as a result.