Amazon S3 Connection
Create a source
To connect to Amazon S3:
-
Navigate to Knowledge Catalog > Sources.
-
Select Create.
-
Provide the following:
-
Name: The source name.
-
Description: A description of the source.
-
Deployment (Optional): Choose the deployment type.
You can add new values if needed. See Lists of Values.
-
| Alternatively, add a connection to an existing data source. See Connect to a Source. |
Add a connection
-
Select Add Connection.
-
In Pick Type of Your Connection, choose Amazon S3.
-
Provide the following:
-
Name: A meaningful name for your connection. This is used to indicate the location of catalog items.
-
Description (Optional): A short description of the connection.
-
Bucket name: Provide the name of the Amazon S3 bucket that you want to use.
-
Region: Select the region to which the bucket belongs.
-
Add credentials
-
Select Add Credentials.
-
Choose an authentication method and proceed with the corresponding step:
S3 Access Key Credentials
|
The connecting user requires the following permissions in AWS:
|
-
Provide the following:
-
Name (Optional): A name for this set of credentials.
-
Description (Optional): A description for this set of credentials.
-
Access key: Specify the AWS access key ID.
-
Secret key: Specify the AWS secret access key.
-
-
If you want to use this set of credentials by default when connecting to the data source, select Set as default.
One set of credentials must be set as default for each connection. Otherwise, monitoring and DQ evaluation fail, and previewing data in the catalog is not possible. -
Optionally, enable assuming a role, as described in [Assumed role].
-
Proceed with Test the connection.
AWS Instance IAM Role
|
If you want to authenticate using an AWS Instance IAM Role, Data Processing Engine (DPE) must be installed in your Amazon Web Services subscription on a virtual machine (VM) instance. If you have multiple DPEs running, you might need to specify additional constraints. See Constraints Configuration. |
-
Provide the following:
-
Name (Optional): A name for this set of credentials.
-
Description (Optional): A description for this set of credentials.
-
-
If you want to use this set of credentials by default when connecting to the data source, select Set as default.
One set of credentials must be set as default for each connection. Otherwise, monitoring and DQ evaluation fail, and previewing data in the catalog is not possible. -
Optionally, enable assuming a role, as described in [Assumed role].
-
Proceed with Test the connection.
Test the connection
To test and verify whether the data source connection has been correctly configured, select Test Connection.
If the connection is successful, continue with the following step. Otherwise, verify that your configuration is correct and that the data source is running.
Save and publish
Once you have configured your connection, save and publish your changes. If you provided all the required information, the connection is now available for other users in the application.
In case your configuration is missing required fields, you can view a list of detected errors instead. Review your configuration and resolve the issues before continuing.
Next steps
You can now browse and profile assets from your Amazon S3 connection.
In Knowledge Catalog > Sources, find and open the source you just configured. Switch to the Connections tab and select Document. Alternatively, opt for Import or Discover documentation flows.
Or, to import or profile only some assets, select Browse on the Connections tab. Choose the assets you want to analyze and then the appropriate profiling option.
Was this page useful?