I am struggling as well.
So far Jellyfin seems to be struggling with playing x265 video's. I get the same effect as you.
On the other hand x264 seems to be working well, but I didn't extensively test it yet.
Any one can explain what we are exactly missing for x265? From what I read the chip should be able to handle it, so I am hopeful.