Cant connect to a modded server

Hi! me and my friend decided to make a server with the origins mod and some other mod that expands origins but when i tried to join it just said "connection lost". my friend tried to join as well but got the same thing. can you please help? we already have the fabric loader, api and the mod in our pc.

do we have to download all the mods i put in the server? if so, we already have those.

i just got back from my friend and it apparently didnt register the mod pekhui and i removed the mod and the other mod that is dependent on it, it works now. sorry if i didnt reply back :,)