That is a question you need to ask your shop steward in your hub. There a subtle differences in seniority language between the different Contract Supplements/Riders, that makes a big difference in how that question is answered. Also, what is the agreed-on practice for your hub.
Another question to ask your steward is what do you get paid for unloading? Around here: if they ask you to unload, you keep the sort pay; if you ask them to assign you to unload, they don't have to pay you the sorter premium.
If you don't have easy access to a shop steward, call your Local Union's office and ask the Business Agent for your hub.
Most importantly, have you heard of the "Work now, Grieve later" rule?