Another thing came to mind right now: My final instruct is at the bottom of the prompt, which I changed very early on when I was still using qwen2.5 and it was sometimes ignoring initial instructions (attention shift) when i fed it large content. This may actually also help, because the last instruction is: "summarize the above".
qwen2.5and it was sometimes ignoring initial instructions (attention shift) when i fed it large content. This may actually also help, because the last instruction is: "summarize the above".