We have a ETL pipeline which stores the results in a S3 bucket and would like to understand if we can use the great_expectations library to validate that the content present in the bucket is as per expectations. If this is a valid scenario, does GX provide any adapters to integrate with S3 or should we build libraries to read, parse files prior to using the library for validations?
Great Expectations supports a number of hosted filesystems, including Amazon S3, Azure Blob Store, and Google Cloud Storage. For guidance on how to connect your data, see this documentation.