That’s fair, though you could leave a zigbee router plugged in to bridge things even if you don’t really need the device for example a smart plug.

The other disadvantage is that for a zigbee device to control a matter device it will have to route through your home assistant/smart home hub instead of being able to pair a switch directly to lights for example with zigbee.