This article is for comments to: https://docs.greatexpectations.io/en/latest/how_to_guides/configuring_datasources/how_to_configure_a_pandas_s3_datasource.html
Please comment +1 if this How to is important to you.
1 Like
+1
S3 datasource, not necessarily pandas
+1
Hi, the data asset is not picking up all files (only one is retrieved) that satisfy the regex rule in the target folder when creating the scaffold and checkpoint. is this expected?
Where may I add the reader_options: sep: "|" entries?
Essentially, I wish to specify that my .csv are pipe delimited, but I am a little confused about the context in which the reader_options are found or should be placed into.
from here: How to configure a Pandas/S3 Datasource — great_expectations documentation
I attempted to drop them into the great_expectations.yml file but that just threw a great_expectations.exceptions.exceptions.InvalidDataContextConfigError: Error while processing DataContextConfig: reader_options error.
Which file do I add these arguments/specifications into or maybe could we add an example of the complete file showing them tucked into their proper places?