Can't start a 1.6.4 MagicWorld2 Server

I was able to successfully start a vanilla 1.7.10 server and play with friends. Now that we decided to install MagicWorld2, we ran into a problem.
Forge1.6.4 only starts when the "mods" folder in the minecraft directory is empty. But when it's empty, the server won't let us in because it's missing the mods. We are not very smart. What do we do?

Edit: actually, the problem seems to be that our client doesn't see the modpack we're trying to install. We get the error "Forge Mod Loader could not connect to this server"

No, still nothing. Everyone who tries to connect to the server has the same problem, the same error message. I have a feeling we're just not installing the mods correctly.
We download MagicWorld 2 from the FTB page (the one that was linked in the mod description on Aternos), we get an archive and.. what do we do next? Because we tried several things, extracting, not extracting, extracting in different folders, the client doesn't see the mods, even on single player.

I have installed the mods through Curse and everything works. Except that one person gets a "bad login" error. Our server is set to "cracked" mode, he still can't enter. He could on vanilla.

Sorry for being such a pain.

It's fine, we're good, it worked. Everything seems to be working now

Another quick question. I noticed some mods are disabled like ThaumicTinkererKAMI. If I enable them will it mess with the generation, will I need to generate a new map?