This AWS knowledge base article describes the steps of setting up an SSH tunnel and then pointing a client to the local post that will proxy to the private Redshift cluster:
https://aws.amazon.com/premiumsupport/knowledge-center/private-redshift-cluster-local-machine/
Great Expectations is the client in this scenario, so specify localhost and the local port of your SSH tunnel when the CLI prompts you for the Redshift credentials.