I suspect that you are using water straight from the tap. Our tap water is not so good for saltwater tanks. I suspect that even WITHOUT fishes, our tap water has enough phrophate and nitrate to cause you your algea problem.

Switch immediately to RO water!!

Also, don't stir the algea into the sand. Remove it manually from the tank. If not, they will decompose, and release more phrosphate and nitrates, and the situation will become worse!!