The information displayed in the Issue Activity section in the Issue view is provided by JIRA, not Tempo
On Server, you can disable the Work Log tab in the issue view (JIRA Administration > Add-ons > Manage Add-ons > System add-ons > Issue Tab Panels Plugin), but the worklogs will still appear in the Activity tab.
On Cloud, when Tempo is set as time tracking provider, the worklog tab is provided by Tempo. Other tabs (History/Activity) are provided by Jira.
Atlassian has suggestion issues in their JIRA support system, requesting a change to the information displayed in the Jira provided tabs: