[FEATURE] Infer SQL types for Expectation tests #10601
Conversation
…o SQLBatchTestSetup
✅ Deploy Preview for niobium-lead-7998 canceled.
|
| from sqlalchemy import Column, MetaData, Table, create_engine, insert | ||
|
|
||
| # commented out types are present in SqlAlchemy 2.x but not 1.4 | ||
| from sqlalchemy.dialects.postgresql import ( |
There was a problem hiding this comment.
we shouldn't be importing these types directly off sqlalchemy
| str: postgresql.VARCHAR, # type: ignore[dict-item] | ||
| int: postgresql.INTEGER, # type: ignore[dict-item] | ||
| float: postgresql.FLOAT, # type: ignore[dict-item] | ||
| bool: postgresql.BOOLEAN, # type: ignore[dict-item] |
There was a problem hiding this comment.
@tyler-hoffman any suggestions here to fix these types?
Codecov ReportAttention: Patch coverage is
✅ All tests successful. No failed tests found.
Additional details and impacted files@@ Coverage Diff @@
## develop #10601 +/- ##
============================================
- Coverage 80.25% 67.93% -12.33%
============================================
Files 461 459 -2
Lines 40005 40019 +14
============================================
- Hits 32107 27187 -4920
- Misses 7898 12832 +4934 Flags with carried forward coverage won't be shown. Click here to find out more. ☔ View full report in Codecov by Sentry. |
| def infer_column_types(self) -> Dict[str, TypeEngine]: | ||
| inferred_column_types: Dict[str, TypeEngine] = {} | ||
| for column, value_list in self.data.to_dict("list").items(): | ||
| python_type = type(value_list[0]) |
There was a problem hiding this comment.
I'd rather merge this and then circle back on this suggestion, but checking just the first item seems a bit off. Pandas looks like it has an api to infer types. No clue on if it works for us. https://pandas.pydata.org/pandas-docs/stable/reference/api/pandas.api.types.infer_dtype.html
|
|
||
|
|
||
| class PostgreSQLDatasourceTestConfig(DataSourceTestConfig[PostgresColumnType]): |
There was a problem hiding this comment.
I think we should circle back on this loss of typing, but not now.
This PR adds functionality which allows test authors to skip explicit types when writing expectation tests.