As first, I'm surpressed that your friend playing on a PS4 is able to join at all. Normally console players can't join custom servers due to licensing/terms regulations.
I don't really know why that problem appears, you might have more luck asking on the Geyser discord/forum
It is possible for my friend on PS4 to join. You are right that the PS4 version of Minecraft doesn't normally allow you join custom servers, but that wouldn't stop us. There is an app called BedrockTogether (on iOS and Android). Essentially, what it does is it creates a LAN network and you connect to that and you'll end to the server.