I have to say that out of the 3, sorting is the easiest. Hard for the first month or so, but once you know where everything goes, and your body adjusts, its cake. Now I say that when it's a good night. Meaning I got paired with an unloader who actually cares about his job, has a good attitude, listens, and does it right. There are quite a few unloaders in my hub who just want to go balls to wall all night long, not caring about lables being up or out, sending retapes, irregs....And give us the stink eye when we have to hold em up. Also unloaders who have bad friend'ing attitudes and won't stop the belt when you've yelled 5,6,7 times at them to stop. Then start the belt back up before you are ready or being told to. I've heard on a few occasions the unload supervisors tell the unloaders to "make the sorters want to quit." So maybe that has something to do with it also.
Now with that said, in my case, it all depends on who you get paired up with that determines if you're going to have a good night or a horrible night. Too many unloaders in my hub seem to have bad attitudes, so I'd say that unloading is the 2nd hardest. Loaders I just feel bad for every time I walk by them. They are at the mercy of the flow.
Good Luck to you. Try and go for sort. If you become an unloader, have a little compassion for the sorters. You get paid by the hour, not how many trucks you unload.
p.s. The faster you go, the less money everyone makes.