Sounds like he's definitely getting the worst of the algae. You've possibly had an algae bloom. I hope he's not to far gone.
Here's what I would do (and mind you, I only have a 10 gallon
Goldfish tank and a 30 gallon cichlid tank and have NO experience w/Bettas). I'd put the fish in another tank (a hospital tank if possible) and quarantine him thus giving him a tank of freshwater to recoup in. Then I would completely clean my small tank w/warm water and white vinegar (if the tank was glass and not plastic I may consider using a very heavily diluted bleach otherwise stick w/white vinegar). Clean the tank and everything in the tank including plastic plants, gravel, etc. Then when you are finished cleaning, add freshwater (treated with Chlor-Out if using tap water) and then re-add your decor. When the tank is full, treat the water w/a chemical algae product (usually found at your LFS or even Wally World in tablet form). After that, you could leave the fish in the hospital tank for a while or opt to put him back in the freshly cleaned tank.
Other than that...I don't know what to tell you. The only other suggestion I'd have for you is to get a bigger tank w/a filter system. I think you'd see a happier fish in a 5 or 10 gallon system (and perhaps be able to add some other fish too, just not another Betta).
Good luck and I hope this helps you.