Hi Team,
I am trying to setup validation of Doris database using Data Connection option in ‘OtherSQL’ as given in documentation.
When I am running a checkpoint:
batch_parameters={“year”: “2024”, “month”: “08”, “day”: “09”}
validation_results = checkpoint.run(
batch_parameters=batch_parameters)
I am getting error:
SAWarning: Did not recognize type ‘VARCHAR’ of column ‘date’ columns = inspector.get_columns(
I have not written any validation rule for this column but still this one is throwing error and checkpoint is not running.
Please can someone help me with this?
Much appreciated!
Hi there, welcome to our community!
The error message likely indicates that the SQLAlchemy engine is having trouble interpreting the VARCHAR type defined in the Doris database.
Doris databases may use SQL types that need additional configuration in SQLAlchemy to be properly parsed. You could investigate the SQL dialect compatibility for Doris, focusing on how SQLAlchemy retrieves column types but unfortunately, Doris isn’t a data source that we officially support at this time.