There are a limited number of offices in the Capitol building itself. Most senators and House reps have their offices in the buildings surrounding the Capitol. Those limited "hideaway" offices were handled semi-unofficially and were given to senior people in both parties.
- Either people like Whips, committee chair etc that need to be nearby so they can get to the chambers right away.
- Older members so they don't have to walk as far.
- Committee heads that involve security, defense etc. Areas that need to be more secured than the office buildings.
The point wasn't that McCarthy took the office back from Pelosi; its his power to do so. Just that he did it within like 2 hours, while Pelosi was at a funeral and gave her less than 24 hours to get out.