...
If you only need to see the worklogs of your team, but not of all users in your instance, then you should only grant yourself the "View Worklogs" (Cloud) (Cloud or "View Team Worklogs" (Server) for Team Permissions for the teams you need to review the time logged. In this case, you do not need the View All Worklogs permission for the projects that your team is working on.
...