Never said it was easy, but not as difficult as you pretend. An ISP will have so much more volume from express that he can hire an extra driver or drivers just to handle anything that makes it too difficult for his other drivers. Fedex has so much opportunity to reduce costs that they could afford to pay the ISP double for the guaranteed deliveries/pickups.. I don't think that express drivers even realize the total cost of a delivery driver. All they know is what they make, and don't have a clue about all the other costs involved. The ISP know those costs- the vehicle, the insurance, the cost of payroll, employment taxes, and so on.
I don't see how fedex CANNOT have ground deliver almost all, if not all express packages. Everything from the terminal up would be the same- planes, trucking to the terminal, upper mngmt, etc, but once the package is at the ground terminal, there is no reason for fedex to hire drivers, managers, twice the number of package handlers, and so on. It will violate their legal duty to stockholders to maximize profit.