If you can find them:
Heathkit put out a set of educational materials that I heard was very good. They came in 3-ring binders.
My introduction to electronics was through a coorespondence course put out by Bell and Howell Schools in the early years of DeVry. This was some of the best elementary engineering educational materials I have ever seen independent of discipline. They really developed the concepts from scratch. If you could find a copy I strongly recommend them. The Heathkit stuff may be easier to find. See a few on ebay. I have never used them.
I love Horowitz and Hill. However, I would not recommend it for a newbie. I went to grad school at what some consider a top tech school. The undergrads did not like it. They felt it required a certain level of electronics knowledge to begin with. Horowitz and Hill are great guys. Hill helped me directly find a replacement device for one of his circuits. The part called out in the book went out of production. H and H were trying to address the need you expressed in you post. They felt frustrated with the book and trying to address a broad audience in a single volume. I constantly refer back to it.
Some recommend the ARRL big book. It can be useful. Helps with antenna theory. However, because it is basically a collection of articles, it lacks a consistent development of concepts.
