Click Help, Help Topics, type NSF in the search field.

When I've had this happen, one of the ways I have done this is to create an invoice under the vendor for the amount of the NSF, but use the account number of the bank account instead of an expense account.  Use the same invoice number with "NSF" at the beginning or end for traceability.  This will create the appropriate entry in the bank to credit that cheque back to the bank and will adjust the AP amount.  Then, just go into the payments module and reissue payment.  

https://goo.gl/e7vGFT Sage 50 Knowledge Base as seen below:

• How to manually reverse a payment for vendors?
• How to manually adjust a payment for vendors?
• How to manually reverse a payment for customers?
• How to manually adjust a payment for customers?
• How to handle a NSF payment for customers or vendors?
• NSF adjustment
• How to handle a stale-dated cheque for customers or vendors?
• How to handle a stale-dated cheque for customers or vendors older than 2 years?
• How to void a payment cheque that was cleared in data maintenance?
• I can't adjust an invoice/receipt/payment, it is greyed out.
• I get an error: "The amount owing for prepayments cannot be less than zero. Enter an amount greater than or equal to ..."
• These solutions are provided that the invoice has been created already.Method I: Reverse Vendor payment
• In the Home window, under the Payables (Vendors & Purchases) module, click on Payments.
• In the To the order of drop-down box, select the vendor for whom you need to reverse the payment.
• You will need to show fully paid invoices by clicking on the menu Payment, and Include Fully Paid Invoices/Prepayments if the invoice was paid in full.
• Change the Paid By box to Cash
• Change the From box  to select the correct account in which to record the reversal.
• Add info to the receipt number (i.e. NSF, stale-dated, reversed, void,...)
• Change the date to the date your bank reversed the payment.
• In the Payment Amount column, enter the amount as a negative on the line where the invoice appears.
• Enter any Comment you may want to include.
• Click Process/Post. 
• Method II: Reverse Customer receipt
• In the Home window, under the Receivables (Customers & Sales) module, click on Receipts.
• In the From drop-down box, select the customer for whom you need to reverse the payment.
• You will need to show fully paid invoices by clicking on the menu Receipts, and Include Fully Paid Invoices/Deposits if the invoice was paid in full.
• Change the Paid By box to Cash
• Change the Deposit to box to select the correct account in which to record the reversal.
• Add info to the receipt number (i.e. NSF, stale-dated, reversed, void,...)
• Change the date to the date your bank reversed the deposit.
• In the Amount Received column, enter the amount as a negative on the line where the invoice appears.
• Enter any Comment you may want to include.
• Click Process/Post. 
• Note: If you want to charge the customer for any bank fees, you must do so on a separate invoice. Apply the charges to Miscellaneous Revenue in the Account column.How do I void a vendor payment without reversing it? ID:10617...Last Modified Date:12-29-20171514580232177   ​

  How to manually reverse a payment?
  Products
  Sage 50—Canadian Edition
  Description
  Resolution
  Need help? Chat with support
  Related resources
  Category
  Adjustments
  adjustments, corrections, and reversal of entries
  Processing
  Entitlement
  Open