Can it be that when you print checks, Stub Lines is set to something less than 17 on that screen?

No. It's 17. One particular vendor it does it on always has 12 lines and it prints an extended stub.

What version?  Are the invoice numbers for the affected vendors longer than 10 characters?  If so, is the check form one that was upgraded from version 4.5 or below?  Long story short, could be that each invoice number longer than 10 character is counted as 2 stub lines.

Version 5.3

They are longer than 10 characters.

I'm not sure about the check form.

Now that we know why, do you know how to stop it?

Version 5.3

They are longer than 10 characters.

I'm not sure about the check form.

Now that we know why, do you know how to stop it?

Try testing with a form template from the current version and see if you have the same experience as with your current form.  I haven't resolved the issue for anyone to date, and didn't have success resolving the problem several versions ago (2013 or 2014).  Haven't looked at it in later versions.