Flow Agent installation using the server installer

Flow’s Agent is client-side software that sends commit-related metadata to Flow from your git servers. This ensures that source code never leaves your environment and that you can still able to take advantage of Flow’s insights and metrics. 

The Agent is an optional add-on to your Flow plan. Reach out to your Flow contact for information on how to get started with an Agent installation.

What does the Agent installer do?

The Agent installer is an Ansible installer that:

• Installs dependencies

  • ContainerD

  • Nerdctl

  • Pip

  • Python3

    • If Python is already installed on the host, ensure it's version 3.7 or above. If not, update to version 3.7 or later before running the installer. The Flow Agent is not compatible with earlier Python versions. If Python is not installed on the host, the installer will install a compatible version for you.

• Creates a user for runtime, unless a user is specified at the time of installation.

• Creates a repo cache folder where cloned copies of repos are stored.

  • This location is configurable. Default: /var/lib/flow/agent/repoCache

• Creates a configuration directory where the Agent configuration file is stored.

  • This location is configurable. Default: /etc/flow/agent/config

• Imports the Agent container image.

• Creates a cron job to run the image at regular intervals.

  • This interval is configurable. The default interval is 3 hours and can be increased as needed based on how much data you have.

System requirements

To successfully install and run the Agent with the Agent installer, your system must meet the following requirements.

Network requirements

The host must have:

• Access to the git repositories you want to connect to.

• Egress configured to access the Flow API gateway, as listed in the downloaded config file

  • The link for multi-tenant cloud is https://agent-api.appfireflow.com. If you’re on a single tenant instance, the link will be custom to your instance.

• Internet access at the time of installation to successfully locate updates.

Compute resource recommendations

OS (including two workers)

• CPU: 4 processor cores

• Memory: 8 GB

For every two additional workers, add 2 processor cores and 4 GB of memory.

These recommendations fit most customer needs, but requirements may vary depending on the number of workers and commit threads in your configuration.

Storage requirements

• The sum of the total size of all of your git repositories multiplied by at least 1.3 to account for growth.

• 2 GB for log files

• File system must support fsync. Please avoid NFS mounts.

Operating system

The Flow Agent can be run on compatible Linux operating systems, including:

• Debian 11, 12

• Ubuntu 20.04, 22.04, 22.10, 23.04

• CentOS 8 (stream), 9 (stream)

• RHEL 8.6, 8.7, 8.8, 8.9, 8.10, 9.0, 9.1, 9.2, 9.3

Agent vendor and authentication requirements

In Flow, you can set up integrations using the Agent for the following vendors. You must use one of the following authentication methods in the Agent to ingest git data:

• GitHub

  • Access token with the required permissions for GitHub Cloud

• GitHub Enterprise Server

  • Access token with the required permissions for GitHub Enterprise Server

• GitLab and GitLab self-hosted

  • Access token

• BitBucket Server

  • Access token

  • Username/password

• BitBucket

  • Username/password

• Azure DevOps Services

  • Access token

• Azure DevOps Server(TFS)

  • Access token

Only git data is collected through the Agent. Project lists, PR data, and ticket data are not collected through the Agent; instead, they and collected via the authentication method provided in Flow.

Agent versioning and supported versions

Flow supports N-2 minor versions of the Agent. Unless otherwise specified, we recommend updating to the latest version, which includes the most recent patches.

Agent installation with the Agent installer

Before running the installer, create all your git integrations in Flow. This will ensure your Agent config file contains all the correct information for each integration. If you add additional integrations after downloading the config file, you must redownload it and update it in your system.

1. After you’ve created your integrations, go to the Integrations page and click Download metadata collector assets. In the modal, choose the installer option if prompted.

   Open

   

    

2. Next, click Download Agent config file to access the file containing all integration and authentication information to add to the Agent.

3. Click Generate installer link, then click Copy link. For security reasons, this link expires after one hour, so make sure you’re ready to download the installer inside your host environment. Otherwise, return to complete this step when you’re ready.

4. Create an API key in Flow, using a Flow user service account if possible.

5. Download the Agent installer inside your host environment using the installer link. Then run curl -o flow-ingestion-agent.tar.gz "{installer link}".

6. Extract the tar file by running tar -xzvf flow-ingestion-agent.tar.gz.

7. Execute the Agent by running the installer file as sudo on {executable path}/flow-ingestion-agent-install.sh. As an example, this could look like sudo ./flow-ingestion-agent-install.sh. The install script must be run as sudo.

8. Put the Agent config.toml file in this path on your server: /etc/flow/agent/config/config.toml.

9. Update your Agent config file to include your authentication values. 

10. Add your API key to the Agent config file so it can access the Flow API gateway.

At this point, your Agent installation should be complete, and your data should begin ingesting and processing. If you add any integrations in the future or want to modify your credentials, download the Agent config file from Flow again and replace your current config file with the new one. Alternatively, edit the config file you’re already using.