First, you'll need to make the choice. Is this for personal use (as a 'custom machine'), or do you want to try and sell it? The reason behind this question, is things like: Production costs, parts... etc..
Your budget will influence the kind of hardware you will use. There was a similar thread on the newsgroup rec.games.video.classic a long time ago (when I read it about a year ago), which became very detailed and would be of interest to you.
I wish I could help you more, but all I can do is point you in the right direction. However, I can suggest you consider the following factors when designing this system:
- Ease of use
- Display quality (ok, so you want a 'nostalgic feel'... but from what era? Are we talking about 2600 nostalgia? 7800 nostalgia? or maybe even slightly later systems like the master system?)
- Production Cost/Ease of Production (I strongly suggest you read the earlier threads about mass producing atari carts with regard to manufacturing issues.)
And the big one...
-Ease of Programming - You don't want to make a system no one wants to write games for (ie: you wouldn't want to be writing at the same level as the 2600). Various chips/processors will have different levels of programming skill requirements. My first suggestion, is that you go and get your hold on all the programming notes/resources you can get your hands on, and realise *what you are up against*. The thing to remember is: if you want to produce this console for private use, you'll have to do *all* the programming yourself. If you want it to be commercially manufactured and available, you'll be able to bring in others who may wish to develop for the system, but you'll bring on more headaches with regard to manufacturing issues.
Basically, if you want to avoid manufacturing problems, you'll be the only one with the console, and the only developer for it. If you want to overcome programming problems, you'll have to deal with manufacturing issues.
Sound like a headache? It is. I've been long considering doing the same thing, but I'm a software kind of person, not hardware. You'd be wise to consider all factors
That said, I wish you the best of luck. Consider everything, and if you still want to do it, make sure to talk to the people around the community. I'm sure they'll have some tips/ideas on the development of the system. I hope you succeed :)
Kale