Straight from the official source.  It is time sensitive though and videos can be off when their validity expires.

https://olympics.com/tokyo-2020/en/replay/

However, the videos could be blocked in countries like US and China, where they have exclusive broadcasts.

Replays from past Olympics events are also available on the official site.  I have checked out some of them and the videos quality could vary.  And they come with ads.

By the way, there is also an official YouTube channel, where they will share selected past events.  Just early this month, they share a group gymnastics event from Beijing 2008.  The video quality is solid.

7th September 2021 EDIT:

The Tokyo 2020 Paralympics replays are also available (but again time sensitive I believe).  Here is the link from the official source.  Some territories are blocked from the view though.

https://olympics.com/tokyo-2020/en/paralympics/watch/