How to remove this shower head? It just rotates on both sides

Most likely inside the tiny hole is a grub screw. Most likely a small allen key is needed.

I'm not familiar with this head. I wonder if the flat plastic inside the hole should be pushed in with a small screwdriver. The head itself that you already removed looks like a mira head. Maybe Google mira fixed head

This is you shower. It's a Mira Response

You can send them a question in that link.

I would try inserting a flat head screwdriver into the hole and pulling at the same too.