The big problem is the difference in the marginal cost of inputs. Comparative advantage theory says that there is some point to buying even high cost goods depending on the advantage you gain by pursuing your own advantageous activity over doing everything autarkically.