Is there any difference in setting up the integration in Jira for the multi-site feature?

No, you can follow the Setting up your integration in Jira documentation for each Jira site that you are setting up for your Salesforce configuration. 

Related content

• How do I connect more than one Jira connection in my Salesforce environment?

• How to view, create and associate Jira Issues from Salesforce by switching the Jira connection?

• Why can't I switch or view the Jira connection dropdown for the Lightning Aura and Visualforce components?

• Are attachments sync configurations configured individually with multiple Jira site integration?

• What are the limitations of the multiple Jira site integration?