There are options to download some parts of it only and trust the rest. But by downloading all blocks, you can verify all transactions without needing to trust someone else. You need all previous transactions because how the Blockchain technology works with each block referencing the previous one.