Standalone Lineage Scanner

This guide is intended for admin users. To make sure you have the necessary permissions to run the scanner, see Lineage Permissions.

Ataccama standalone lineage scanner is deployed as an independent application using Docker Compose. One of the most important security aspects of the solution is to have the hosting environment of the Docker compose service secured on the OS and networking level.

System requirements

OS: Linux, MacOS, or Windows.
Docker Compose version 2.32.0 or higher is installed on the OS.
- To check your Docker Compose version, run docker-compose --version or docker info.
To install and run the scanner, you need to have the necessary permissions to run Docker.
- To check this, run docker image ls. The command should return no error.
On Windows, Git BASH is installed.

Installation

When using Windows, you need to use a BASH terminal. We recommend using Git BASH, which comes with the Git client.

Depending on your OS, start the following:
- Linux: Start the Docker daemon.
- Windows or MacOS: Start the Docker Desktop.
Open the terminal and change your current directory to the folder with the downloaded installation file.
Modify the file to be executable. For this purpose you can run chmod 775.
Run the installer: ./lineage-scanners-installer.sh.
When prompted, choose whether you want to use the current directory for installation. If you select No, enter the target installation directory.

In the following steps, we’ll refer to this folder as <install>.

Start the lineage service

Open the terminal and change your current directory to <install>/bin.
Run the following command:
```
./lineage-scanners system start
```
Check whether the service is running:
```
./lineage-scanners system status
```

If you receive a similar message, it is safe to ignore it and proceed with the installation:

! lss The requested image's platform (linux/amd64) does not match the detected host platform (linux/arm64/v8) and no specific platform was requested 0.0s

Configure and run the scanner

Configure a scan plan

Place your scan plan to <install>/user-data/scan-plans directory. The plan must be in valid JSON format.

If you are deploying on Windows, restart the service after you have copied the scan plan for first time.
Open the terminal and change your current directory to <install>/bin.
Run the following command to check if your scan plan was successfully registered.

Provide the scan plan name without the .json extension.
```
./lineage-scanners plan ls
```

Configure scan plan secrets

We recommend encrypting any sensitive information such as passwords, secrets, and other credentials information. For this purpose, you can use secret placeholders that ensure actual values are safely stored in the encrypted form in the service.

The syntax for secret placeholders is as follows:

"@@ref:ata:[<secret-name>]"

For example: "clientSecret" : "@@ref:ata:[powerbi.client.secret]".

If a secret is used in the scan plan, you need to set its value before you run the scan. Use the following command:

./lineage-scanners secret create <secret-name>

When prompted, provide the actual value.

The <secret-name> is a unique identifier of the secret across all scan plans. The same secret can therefore be used in multiple scan plans.

To list all commands available for secret management, run:

./lineage-scanners secret

The list is as follows:

  create <name> Create a new secret
  ls            List available secrets
  rm <name>     Remove a secret

Run a scan

Open the terminal and change your current directory to <install>/bin.
Run the followingc command:
```
./lineage-scanners plan exec <scan-plane-name>
```
The scan should now be running and you should see the scan <scan-run-id> and status. The <scan-run-id> is a unique identifier of the scan. For a list of possible scan statuses, see Scan statuses.
```
Scan run id: 0295650d-7302-43ec-9d7e-8fa799cf7ca7
Scan run status: CREATED

[start=5]
```
To check if the scan successfully finished, run the following command:
```
./lineage-scanners scanrun status <scan-run-id>
```

Scan statuses

A scan be in one of the following states:

CREATED: The status that is assigned right after the scan is initiated.
RUNNING: The scan is running.
FINISHED: The scan finished successfully.
FAILED: The scan failed.
REJECTED: The scan was not started as the scan plan JSON file is corrupt or a secret value used in the scan plan is not defined.

ID                                      STATUS
0295650d-7302-43ec-9d7e-8fa799cf7ca7    FINISHED

Retrieve lineage zip file

You can find the lineage ZIP file that the scan produced at the following location:

<install>/user-data/work-area/scan-runs/<scan-run-id>/exec_<timestamp>/exec_<timestamp>.zip

This is the file you need to import into ONE. For details, see Next steps.

Troubleshooting

In case the scan finished with en error, check the content of the <install>/user-data/work-area/scan-runs/<scan-run-id> folder. To find out more about the folder structure and learn what to look for, see Scanner Output File Structure.

Remove the lineage scanner

Uninstalling the lineage scanner completely removes the scanner from the system, including scan plans, secrets, logs, and scan results. If you want to keep the scan plans, create a backup of the folder <install>/user-data/scan-plans.

To remove the lineage scanner:

Stop the lineage scanner:
```
./lineage-scanners system shutdown
```
Change the current directory to the following one:
```
cd compose
```
Execute the lineage scanner environment variables file.
```
source ../bin/lineage-scanners.env
```
Stop and remove the Docker containers and associated files:
```
docker compose down --rmi local -v
```
Delete the <install> folder.

Next steps

Once you generate the lineage file, it needs to be uploaded to Ataccama ONE.

Was this page useful?