BigQuery Connection
This article describes how to connect to BigQuery for data processing and lineage extraction.
For lineage scanner configuration, see also BigQuery Lineage Scanner.
Prerequisites
-
Review how sources and connections work.
-
Create a source to add this connection to.
-
Create a service account and service account key in your Google Cloud project. See Create service accounts and Create and delete service account keys in the Google documentation.
Add a connection
-
Go to [your source] > Connections and select Add Connection.
-
In Connection type, select BigQuery.
-
Fill in the following:
-
Name: A meaningful name for your connection. Used to indicate the location of catalog items.
-
Description (Optional): A short description of the connection.
-
Edge instance (Optional): The edge instance used to process this connection. Available if edge processing is configured for your environment. See Edge Processing.
-
Secret management service (Optional): The secret management service used to retrieve connection credentials at runtime. Available if a secret management service is configured. See secret-management-service.adoc.
-
JDBC: A JDBC connection string pointing to the IP address or the URL where your data source can be reached. See JDBC connection string format.
-
JDBC connection string format
The expected syntax is as follows:
jdbc:bigquery://https://www.googleapis.com/bigquery/v2:443;ProjectId=<project_name>;OAuthType=0;OAuthServiceAcctEmail=<account_id>.iam.gserviceaccount.com;Timeout=3600;EnableSession=1Replace the following values:
-
<project_name>: Your Google Cloud project ID. -
<account_id>: The service account ID (the part before@in the service account email).
Add credentials
Only Google service account key authentication is supported.
Always use the dedicated credential fields for authentication details such as passwords, secrets, and tokens. This ensures credentials are handled with the appropriate level of protection and reliably preserved across environments.
To keep credentials in your existing infrastructure and have ONE retrieve them at runtime, configure a secret management service instead. See secret-management-service.adoc.
| One set of credentials must be defined as default for each connection. Otherwise, DQ evaluation fails and previewing data in the catalog is not possible. |
-
Select Add Credentials.
-
Fill in the following:
-
Name: Provide a clear name for this set of credentials.
-
Description (Optional): Explain what the credentials are used for or provide other useful information.
-
Service account key: The service account key (in JSON format) for the Google Cloud project containing your BigQuery data. See Prerequisites.
-
-
To use this set of credentials by default when connecting to the data source, select Assign as default.
Add write credentials
If you want to export data to this source, add write credentials. Select Add Credentials and follow the instructions in Add credentials.
Was this page useful?