The thing about contractual rights, is you have to consistently enforce them in order for them to be maintained. So, if you want to be guaranteed hours, you have to show up on time ready to work. The flip side of this, is that if you DO show up on time ready to work, management can't lay you off that day without paying guaranteed hours (unless you agree). So this works both ways.
As for looking around and seeing who else is working, I'm not trying to be a jerk, but by deciding to be a Teamster, you decided NOT to take a position running the company. Management runs the company as they see fit, and if you believe they're in violation of the contract, you demand your rights under the contract, or file a grievance.
Showing up anyway is a nice gesture, but in the scheme of things it doesn't mean anything because the manager was clear in telling you your position was filled before you left home.