[Updated with new test on notebook] When I try to run the following code I expect the results to be generated inside the json files and stored inside the validations folder as well the data docs stored inside the data docs folder. I am using the same block of code on Pycharm as python script and on jupyter notebook.
Jupyter notebook:The same code in notebook returns the data docs files as well as the json files for the previously configured expectation suite test_example_3 but when I try to configure a new expectation suite WH29_tyrewear_expectation_set and run the validation over them in notebook data docs are not generated but json files are generated.
Pycharm : The same code in pycharm returns the json files for the previously configured expectation suite test_example_3 but when I try to configure a new expectation suite WH29_tyrewear_expectation_set and validation over them no json files and no data docs are generated.
Code :
import great_expectations as ge
from great_expectations.core.expectation_configuration import (ExpectationConfiguration)
context = ge.data_context.DataContext()
datasource_name = “WH29_measurement_dates”
path_to_folder_containing_csv_files = “Combined_singleday_data_for_each_measurement_dates”
datasource = context.sources.add_or_update_pandas_filesystem(name=datasource_name, base_directory=path_to_folder_containing_csv_files)
#Set the asset name as well as the regular expression for the dates when data is available
asset_name = “measurement_data_eachday”
batching_regex = r"37367b5d-daa9-4a6b-a6d7-683cddca44e0_(?P\d{4})-(?P\d{2})-(?P\d{2}).csv"
datasource.add_csv_asset(name=asset_name, batching_regex=batching_regex)
my_asset = context.get_datasource(“WH29_tyrewear_measurement_dates”).get_asset(“WH29_tyrewear_measurement_data_eachday”)
context.add_expectation_suite(expectation_suite_name=“WH29_tyrewear_expectation_set”)
context.get_expectation_suite(“WH29_tyrewear_expectation_set”)
batch_request = my_asset.build_batch_request()
validator = context.get_validator(batch_request=batch_request,expectation_suite_name=“WH29_tyrewear_expectation_set”)
expectation_validation_temp_r2 =validator.expect_column_values_to_be_between(“tire_pressure_controller_tire_temperature_r2”,min_value=0,max_value = 70,strict_min = True,strict_max=True)
validator.save_expectation_suite(discard_failed_expectations=False)
print(expectation_validation_temp_r1)
checkpoint = context.add_or_update_checkpoint(
name=“WH29_tyrewear_expectation_set”,
run_name_template =“WH29_tyrewear_trial_2”,
validations=[
{
“batch_request”: batch_request,
“expectation_suite_name”: “WH29_tyrewear_expectation_set”,
},
],
)
checkpoint_result = checkpoint.run()
In Image 1 it can be noticed that the folders for the newly created expectation suite and checkpoint is empty and the folders with previously made expectations suite are with json files whereas the data_docs is empty for everything.
Expected behaviour after running checkpoint:
- Build json files
- Build data_docs
The result I got for running the new expectation suite on Pycharm:
Error running action with name store_validation_result
Traceback (most recent call last):
File “C:\Users\memonju\PycharmProjects\Lubricant Control Web\venv\lib\site-packages\great_expectations\validation_operators\validation_operators.py”, line 478, in _run_actions
action_result = self.actions[name].run(
File “C:\Users\memonju\PycharmProjects\Lubricant Control Web\venv\lib\site-packages\great_expectations\checkpoint\actions.py”, line 100, in run
return self._run(
File “C:\Users\memonju\PycharmProjects\Lubricant Control Web\venv\lib\site-packages\great_expectations\checkpoint\actions.py”, line 885, in _run
run_return_value = self._basic_run(
File “C:\Users\memonju\PycharmProjects\Lubricant Control Web\venv\lib\site-packages\great_expectations\checkpoint\actions.py”, line 931, in _basic_run
return self.target_store.set(
File “C:\Users\memonju\PycharmProjects\Lubricant Control Web\venv\lib\site-packages\great_expectations\data_context\store\store.py”, line 226, in set
return self._store_backend.set(
File “C:\Users\memonju\PycharmProjects\Lubricant Control Web\venv\lib\site-packages\great_expectations\data_context\store_store_backend.py”, line 134, in set
return self._set(key, value, **kwargs)
File “C:\Users\memonju\PycharmProjects\Lubricant Control Web\venv\lib\site-packages\great_expectations\data_context\store\tuple_store_backend.py”, line 342, in _set
with open(filepath, “wb”) as outfile:
FileNotFoundError: [Errno 2] No such file or directory: ‘C:\Users\memonju\PycharmProjects\great_expectations_eg\gx\uncommitted/validations/WH29_tyrewear_expectation_set\WH29_tyrewear_trial_2\20240312T155739.587972Z\WH29_tyrewear_measurement_dates_2-WH29_tyrewear_measurement_data_eachday_2-year_2024-month_01-day_20.json’
Traceback (most recent call last):
File “C:\Users\memonju\PycharmProjects\great_expectations_eg\great_expectations_eg.py”, line 102, in
checkpoint_result = checkpoint.run()
File “C:\Users\memonju\PycharmProjects\Lubricant Control Web\venv\lib\site-packages\great_expectations\core\usage_statistics\usage_statistics.py”, line 266, in usage_statistics_wrapped_method
result = func(<em>args, **kwargs)
File “C:\Users\memonju\PycharmProjects\Lubricant Control Web\venv\lib\site-packages\great_expectations\checkpoint\checkpoint.py”, line 305, in run</em>*
self._run_validation(
File “C:\Users\memonju\PycharmProjects\Lubricant Control Web\venv\lib\site-packages\great_expectations\checkpoint\checkpoint.py”, line 530, in _run_validation**
async_validation_operator_result = async_executor.submit(
File “C:\Users\memonju\PycharmProjects\Lubricant Control Web\venv\lib\site-packages\great_expectations\core\async_executor.py”, line 106, in submit
return AsyncResult(value=fn(<em>args, **kwargs))
File “C:\Users\memonju\PycharmProjects\Lubricant Control Web\venv\lib\site-packages\great_expectations\validation_operators\validation_operators.py”, line 415, in run</em>*
batch_actions_results = self._run_actions(
File “C:\Users\memonju\PycharmProjects\Lubricant Control Web\venv\lib\site-packages\great_expectations\validation_operators\validation_operators.py”, line 508, in _run_actions**
raise e
File “C:\Users\memonju\PycharmProjects\Lubricant Control Web\venv\lib\site-packages\great_expectations\validation_operators\validation_operators.py”, line 478, in _run_actions
action_result = self.actions[name].run(
File “C:\Users\memonju\PycharmProjects\Lubricant Control Web\venv\lib\site-packages\great_expectations\checkpoint\actions.py”, line 100, in run**
return self._run(
File “C:\Users\memonju\PycharmProjects\Lubricant Control Web\venv\lib\site-packages\great_expectations\checkpoint\actions.py”, line 885, in _run
run_return_value = self._basic_run(
File “C:\Users\memonju\PycharmProjects\Lubricant Control Web\venv\lib\site-packages\great_expectations\checkpoint\actions.py”, line 931, in _basic_run
return self.target_store.set(
File “C:\Users\memonju\PycharmProjects\Lubricant Control Web\venv\lib\site-packages\great_expectations\data_context\store\store.py”, line 226, in set
return self._store_backend.set(
File “C:\Users\memonju\PycharmProjects\Lubricant Control Web\venv\lib\site-packages\great_expectations\data_context\store_store_backend.py”, line 134, in set
return self._set(key, value, **kwargs)
File “C:\Users\memonju\PycharmProjects\Lubricant Control Web\venv\lib\site-packages\great_expectations\data_context\store\tuple_store_backend.py”, line 342, in _set
with open(filepath, “wb”) as outfile:
FileNotFoundError: [Errno 2] No such file or directory: ‘C:\Users\memonju\PycharmProjects\great_expectations_eg\gx\uncommitted/validations/WH29_tyrewear_expectation_set\WH29_tyrewear_trial_2\20240312T155739.587972Z\WH29_tyrewear_measurement_dates_2-WH29_tyrewear_measurement_data_eachday_2-year_2024-month_01-day_20.json’
Environment :
OS : Windows 11
Great Expectations Version : 0.18.10
Python Version : 3.9.0
IDE : Pycharm & Jupyter Notebook
Error I receive for the previously configured expectation suite from which json files are generated and not the data docs on both IDE’s :
An unexpected Exception occurred during data docs rendering. Because of this error, certain parts of data docs will not be rendered properly and/or may not appear altogether. Please use the trace, included in this message, to diagnose and repair the underlying issue. Detailed information follows:
FileNotFoundError: “[Errno 2] No such file or directory: ‘C:\Users\memonju\PycharmProjects\great_expectations_eg\gx\uncommitted/data_docs/local_site/validations\test_example_3\WH29_tyrewear_trial_3\20240312T160338.728982Z\WH29_tyrewear_measurement_dates_2-WH29_tyrewear_measurement_data_eachday_2-year_2024-month_01-day_20.html’”. Traceback: "Traceback (most recent call last):
File “C:\Users\memonju\PycharmProjects\Lubricant Control Web\venv\lib\site-packages\great_expectations\render\renderer\site_builder.py”, line 493, in build
self.target_store.set(
File “C:\Users\memonju\PycharmProjects\Lubricant Control Web\venv\lib\site-packages\great_expectations\data_context\store\html_site_store.py”, line 269, in set
return self.store_backends[type(key.resource_identifier)].set(
File “C:\Users\memonju\PycharmProjects\Lubricant Control Web\venv\lib\site-packages\great_expectations\data_context\store_store_backend.py”, line 134, in set
return self._set(key, value, **kwargs)
File “C:\Users\memonju\PycharmProjects\Lubricant Control Web\venv\lib\site-packages\great_expectations\data_context\store\tuple_store_backend.py”, line 342, in _set
with open(filepath, “wb”) as outfile:
FileNotFoundError: [Errno 2] No such file or directory: ‘C:\Users\memonju\PycharmProjects\great_expectations_eg\gx\uncommitted/data_docs/local_site/validations\test_example_3\WH29_tyrewear_trial_3\20240312T160338.728982Z\WH29_tyrewear_measurement_dates_2-WH29_tyrewear_measurement_data_eachday_2-year_2024-month_01-day_20.html’**
".
Process finished with exit code 0
Is there possibility to solve this error or is it some bug ?
Image 1 :
