I also figured that, with the CB inside the console, I wouldn't be able to hear the speaker. So I mounted an old marine-grade exterior speaker to the back of the console and ran the wire in (terminating it in a 1/8" mini-plug for the CB that I also picked up at Radio Shack. Now I can hear the CB very well.
When the whole thing is closed and secured, except for that speaker, one can't tell there's a stereo, CB or mic connection inside.