It is true that at very low pH the nitrifying bacteria slow down and will even cease. However, the irony is that little harm will occur, due to the fact that in acidic water the ammonia changes into ammonium which is basically harmless. And without bacteria, nitrite will not occur in levels as it would in basic water. Live plants also help all this, by assimilating most of the ammonia/ammonium which they do faster than the bacteria anyway. You can read more detail in my article on bacteria:
http://www.tropicalfishkeeping.com/freshwater-articles/bacteria-freshwater-aquarium-74891/
Depending upon the fish species, the pH should be somewhat controlled. Some species occur in very acidic waters (pH 4-5), but other soft water fish will have problems this low.
I have a lot of Malaysian driftwood in my tanks. While wood will acidify water it is usually very minimal. The most I have read about is .2 or .3 degrees, example from a pH of 6.5 down to 6.3 due to the wood. This of course can be impacted by the hardness, and you can read about that relationship here:
http://www.tropicalfishkeeping.com/freshwater-articles/water-hardness-ph-freshwater-aquarium-73276/
As mentioned, too low a pH can cause trouble for some fish. The low GH and KH associated with a low pH as explained in that last linked article can be detrimental to some plants. We have a current thread [I think its in the Aquarium Plants section] on the problem of excess iron in swords caused by too low a GH.
You should contact your water supplier and ascertain the GH (general hardness) and KH [Alkalinity]. A pH as high as 8.4 is usually associated with hard water, though not always; and lowering to 6.4 without fish in the tank is not normal. Again, this is explained more in those articles. My point here is that it needs investigating before fish enter the tank.
Byron.