Ok. So I just spent the last however long it has been... emptying the tank (20G) and rinsing new sand.
I did just as described above by JohnnyD. I put the amount of sand I would need into my 5 gallon bucket. With my kids' beach bucket (I think about 1.5 gallons) I dumped in on top of the sand.
Then I went back to the sink to fill the small bucket giving the sand enough time to settle.
After it was small bucket was full of water, I dumped out the water from the 5G assuming it would also get rid of the dirt and the real sand had settled. Then repeated by dumping the small bucket on top of the sand and refilling it.
I think I did that 5 times.
So when I finally got it into the 20 gallon aquarium............. did it make a huge difference? For sure.
The time that it took to rinse the sand versus the time that I gave it to settle is enormous.
It "settled" over night and about 2 hours with the filter on. It still was not as clear as it is right now after about 30 minutes of rinsing the sand.
So even for the lazy and impatient like me, rising is definitely the way to go. Thanks guys for recommending that.