Even when no one is trying to get out, the modem starts dialing.

Make sure always reconnect or always stay connected is not selected. If it isn't and it is still dialing, remove everything from the four port switch and see if it tries to dial. If it doesn't, something on the lan is triggering it.