Non-Stock items

We're a construction company that the majority of our sales come from large purchases. The product does not come to our warehouse (drop-shipped) and we do not stock the items.

Currently all the items are being placed on a PO and order to inventory. Then sold by adding the items on a Sales Invoice.

Can we simply create PO's that reference a quote number, or another such reference and Invoice the client without having to add all the items to and from Inventory?