With a LAN to LAN connection (via VPN), can the two (or three or four) offices share printers and other resources as if they were in the same building?

Yes, they can all share resources as long as all the sites are setup with VPN tunnels to communicate with each other.