Can't join the server

Hello guys, some weeks ago I started a server and some of us can join but some can't. This is the error I get in the console:

[17:43:58] [Server thread/INFO] [net.minecraft.server.dedicated.DedicatedServer]: alhoju99 joined the game
[17:43:59] [Netty Epoll Server IO #2/ERROR] [FML]: NetworkDispatcher exception
io.netty.channel.unix.Errors$NativeIoException: syscall:write(..) failed: Broken pipe
at io.netty.channel.unix.FileDescriptor.writeAddress(..)(Unknown Source) ~[FileDescriptor.class:?]
[17:43:59] [Server thread/INFO] [net.minecraft.network.NetHandlerPlayServer]: alhoju99 lost connection: Disconnected
[17:43:59] [Server thread/INFO] [net.minecraft.server.dedicated.DedicatedServer]: alhoju99 left the game

And that is the error he gets in the launcher:

Failed to connect to server

Internal Exception: io.netty.handler.codec.EncoderException: java.lang.OutOfMemoryError

He tried adding more memory to JVM but getting at this point... I think it is a server problem, that allows some of us to join but not the others. Hope you can help me resolve this!

Quote from Mc88Donalds|Julian

sounds like his jvm doesnt have enogh memory lol

Well, he tried adding more than 1G to the JVM... because for example I have only added 1024M to JVM in my laptop, just to try if it works and it does. So that brings me back to the console error code, making me question about if it is the server the one that doesn't allow some players to join the server. Cause I'm having the same trouble with 3 of my friends trying to join the server