Well, if your watter is treated with chlorine, then yes, leaving it out will diffuse the chlorine (but not heavy metals, which dechlorinator removes most of the time.)
Unfortunately, most municipalities don't use chlorine. They've switched to Chloramines, which prefers to stay attached to water.
If you have chlormines, then running an airstone in the bucket for about 24 hrs will help remove some of the chloramines, and definately remove chlorine and chloramine faster than if it was just sitting.
Then again, I'm a bit confused... Unless you live in the middle of no where, you can get water conditioner in small bottles really cheap...
Most of the gas stations and grocery stores around me have a bottle for 2-4 dollars... When would you be able to buy it?
And what kind of fish do you have? (some are a bit more tolerable, but shrimp do not do well with any amount of chloramine)