I'm assuming you meant that you volunteered to go home, and it appeared that your services were not needed thus you were relieved for work early for the day; on your way out, you were told that your services would be needed after all but you choose to continue on home away, resulting in management working & a grievance being filed. The building manager has every right to terminate you... but I doubt it'd stick, although it'd probably qualify as a cardinal infraction and you'd receive a few days (weeks?) of stressful unscheduled time off.
Instead of popping an attitude, you should've worked. At the very least, you're going to be on the @$!$ list and it's likely management will turn toward an ALO system rather than giving voluteers the day off. Thus, you probably blew it for everyone.
Call your business agent and union steward this afternoon for advice on tomorrow. Personally, I'd refrain from being aggressive/assertive, play it off as an irrational move for being fatigued & profundy apologize for the situation (don't admit guilt).