The Jira or Tempo Administrator permission does not automatically give access to view the worklogs of other users.
If you only need to see the worklogs of your team, but not of all users in your instance, then you should only grant yourself the "View Worklogs" (Cloud or "View Team Worklogs" Server) for the teams you need to review the time logged. In this case, you do not need the View All Worklogs permission for the projects that your team is working on.
However, if you need to see the worklogs of all users, you will need the "View all worklogs" permission for each of the JIRA projects you are reviewing. The 'View all Worklogs' permission for the Project is required to view other users' worklogs.
Cloud instances: Please, bear in mind that the permission could take up to an hour to kick in permanently even if it looks like it has already happened.