Try looking into SNARKs and STARKS and languages like Cairo, but I'm not sure if that matches what you will need...