If you are starting a family, they should be the priority. Do what is right for them. If that means buying a house with a fiat loan, do it. There is nothing inherently immoral about using the fiat system when it makes sense. There is, however, something immoral about neglecting the needs of your new family.