Changing difficulty on paper server breaks the game

I own a aternos server to play with my friends on regular survival, and due to the limited distance, we have paper to increase it. It's good, but when I change the difficulty, the server still thinks its on the previous difficulty, and starts to glitch out the game. The hunger bar never decreases, but I can still eat food. When I go to options, the difficulty says the previous difficulty, and zombies either spawn or not spawn. We changed back to vanilla because it was too awkward and interrupting. Any help is appreciated!

Ohh, but is there any way to change the difficulty in game without it breaking? If I can't, it's okay. I change the difficulty with the difficulty commands.

It works, thank you so much! I wish changing the difficulty is easier like it is in Vanilla.