Random blocks that if I break will crash the game

Every time I attempt to break blocks like glass, fence, and glow lichen it crashes the game for everyone but not all, it depends on how far the other players are. So far its only been glass fence, and glow lichen that we found that crashes the game and I tried doing it in single player but its completely fine I also don't have any mod or plugins installed.