Google Cloud Storage Connection
Google Cloud Storage is a service provided by Google that lets you store data assets in Google Cloud.
Data assets can consist of a file of any format. They are stored in containers called buckets, and all buckets are associated with a project.
For more information, see the official product description: Google Cloud Storage.
This article describes how to create a Google Cloud Storage connection in ONE.
Obtain a service account key
A connection to Google Cloud Storage requires the use of a service account key (in the form of a JSON file) that you can obtain from your Google Cloud project.
To learn how to create a service account and obtain the service account key from your Google Cloud project, see the official Google documentation:
After you create and download a service account key associated with the project containing the Google Cloud Storage bucket you want to connect to, you can use the generated JSON file to authenticate the connection.
Create a source
To connect to Google Cloud Storage:
-
Navigate to Data Catalog > Sources.
-
Select Create.
-
Provide the following:
-
Name: The source name.
-
Description: A description of the source.
-
Deployment (Optional): Choose the deployment type.
You can add new values if needed. See Lists of Values. -
Stewardship: The source owner and roles. For more information, see Stewardship.
-
| Alternatively, add a connection to an existing data source. See Connect to a Source. |
Add a connection
-
Select Add Connection.
-
In Connection type, select Google Cloud Storage from the list.
-
Provide the following:
-
Name: A meaningful name for your connection. This name is used to indicate the location of catalog items.
-
Description (Optional): A short description of the connection.
-
Dpe label (Optional): Assign the processing of a data source to a particular data processing engine (DPE) by entering the DPE label assigned to the engine. For more information, see DPM and DPE Configuration in DPM Admin Console.
-
Bucket name: The name of the Google Cloud Storage bucket you want to connect to.
-
Project ID: The ID of the project containing the Google Cloud Storage bucket you want to connect to.
-
-
In Additional settings, select Enable exporting and loading of data if you want to export data from this connection and use it in ONE Data or outside of ONE.
If you want to export data to this source, you also need to configure write credentials as well. Consider the security and privacy risks of allowing the export of data to other locations.
Add credentials
-
Select Add Credentials.
Repeat the following steps as needed to add multiple sets of credentials. -
Provide the following:
-
Name (Optional): A name for this set of credentials.
-
Description (Optional): A description for this set of credentials.
-
Service account key: Upload the service account key (in JSON file format) associated with the project where the Google Cloud Storage bucket you want to connect to is located.
-
-
Set the credentials to be used as default (Set as default).
Add write credentials
Write credentials are required if you want to export data to this source.
To configure these, in Write credentials, select Add Credentials and follow the corresponding step depending on the chosen authentication method (see Add credentials).
| Make sure to set one set of write credentials as default. Otherwise, this connection isn’t shown when configuring data export. |
Test the connection
To test and verify whether the data source connection has been correctly configured, select Test Connection.
If the connection is successful, continue with the following step. Otherwise, verify that your configuration is correct and that the data source is running.
Save and publish
Once you have configured your connection, save and publish your changes. If you provided all the required information, the connection is now available for other users in the application.
In case your configuration is missing required fields, you can view a list of detected errors instead. Review your configuration and resolve the issues before continuing.
Next steps
You can now browse and profile assets from your Google Cloud Storage connection.
In Data Catalog > Sources, find and open the source you just configured. Switch to the Connections tab and select Document. Alternatively, opt for Import or Discover documentation flow.
Or, to import or profile only some assets, select Browse on the Connections tab. Choose the assets you want to analyze and then the appropriate profiling option.
Was this page useful?