There’s a few things to unpack here so sorry if I miss something (and am on my phone, so a little awkward lol).
Feedback loop won’t really be a problem, just a case of routing whatever channel in your DAW to say something like output 3, through the hardware and monitor the input when you come back in again. If you want output 3 to form part of your main monitor mix (for some reason that I can’t think of), then you’ll just not monitor the signal coming back in again. So easy to avoid feedback. It’s a pretty common use-case, so quite normal to set an interface up that way.
Drivers are certainly a component of latency, but yes, as is the rest of the computer. The easy way around any latency issues (which I don't think you’ll encounter), is simply to avoid any processing until you’re done tracking, when you’re fully done with tracking, up your buffer size all the way to 1024 or whatever, basically giving you all the CPU power ready for processing (because don’t need to worry about latency, because tracking is done).
And yes, everything above relates to Scarlett’s as well as Claretts. Just be aware that Scarlett 18i8 is the first in the range that has ADAT In (If channel expansion is important to you in the future), whereas all Claretts have ADAT in. While Scarlett’s sound great, Claretts do have improved audio specs, hence the price differences
Cheers
Si // Focusrite