I use an old Mac G4 I bought used for £80 (it came loaded with Traction, which is a fairly decent DAW), a Tapco Link firewire interface which cost around £150, and a Rhode M3 condensor mic which cost around £90. Everything worked a treat straight away, no latency issues or anything like that, although it does struggle to run plugins like Amplitube unless I render the tracks first so its not computing everything in real time. I find recording with a computer far, far easier than using an all in one unit, being able to edit waveforms on screen with a mouse is fantastic.
So, you dont need to spend a fortune. I recorded this the other day with my setup, just to give you an idea of the sound quality:
[url="http://wilmiles.bandcamp.com/track/unfinished-track"]http://wilmiles.bandcamp.com/track/unfinished-track[/url]