When you say it appears to have been sent do you mean you can see it in your Sent Items folder in Outlook?

Is the old version of Outlook still on your system and still marked as your Default email program in Windows?  If so, disconnect from the internet in some manner and open the old version of Outlook to see if they are still in the Outbox.  Disconnecting will make sure they don't get sent until you are ready to send them.

Now that I am rereading your message, am I reading this wrong?

Alive said:

I am using Microsoft Exchange and Outlook and I just upgraded to the 2023 version

This tells me you upgraded Outlook.