Try feeding him flakes again. I had the opposite problem with my goldfish a while ago. He kept floating to the top. But he got over it. Have you made any drastic water changes lately? How long have you had the tank running? When my tank was new my goldfish did the same thing. We later figured out that there were no benificial bacterias in the tank and that was killing him. You can buy a liquid that will add these bacterias to a new tank. Hope he gets better soon! Poor little guy!

-Mandy- P.S. don't change 100% of the water> That will definately kill him.