There might be an edge case in which the time stops for too long.

For example, imagine a period of massive increase in difficulty given exponential growth of miners followed by a massive destruction of technology, i.e. global war, natural disaster, etc.

There is a change in difficulty, but you need to get 2016 blocks first. If you only have primitive miners, say Pentium1-level compute power, then time will be expanded into pretty much infinity