Dollar tree around us has a huge selection of banners, borders, cut outs, stickers, scissors, etc.. as well as educational books for the younger grades, (flash cards, etc)
The teacher suppy stores sometimes have a bin of things between 30-75% off.
There is a huge Christian book and gift store in parts of Missouri called Mardels, and once a year (I think august or july?) they have a gigantic sale for their homeschool stuff. I've gotten stuff that was 90% off in there and paid pennies to dollars for things.
oh, and I've ebayed a complete Rod & Staff english set for like $7.00 plus $4.00 shipping. It was in good condition, the binding on the one book was getting loose from being used so much, but none of the pages were falling out and there was almost no marks in it. It's normally like $55.00 for the set plus shipping.
I also found things on Amazon. However, amazon is tough to try to buy a bunch of stuff because you end up paying a ton of shipping fees because the stuff comes from a variety of sellers. I can sometimes find a few pieces from the same seller and shipping can be combined then.