Does that not make the hair oily?

no, you just use a bit ( a few drops ) and mix with a carrier oil, then massage into your scalp.

I wanna move away from chemical shampoo and conditioner.

take baby steps! try to use soap replace chemical shampoo first.