You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
When processing a data source, we want to skip one row if it fails to process for some reason, and instead continue with the next row. If we are not in debug mode (i.e. in production), we should log the error and continue. IF we are in debug mode, the exception should be re-raised.
An implementation of this was done for the base_project 'create_project_subsidy_csv' method, where we try to decode the address for each row. All of the 'sources' classes also now have a 'debug' flag in their init method. We should use this flag in the data processing step to properly handle exceptions as they occur so that one errant data source does not crash the whole data processing step.
The example to build off is in a pending commit, to be put into PR today.
The text was updated successfully, but these errors were encountered:
When processing a data source, we want to skip one row if it fails to process for some reason, and instead continue with the next row. If we are not in debug mode (i.e. in production), we should log the error and continue. IF we are in debug mode, the exception should be re-raised.
An implementation of this was done for the base_project 'create_project_subsidy_csv' method, where we try to decode the address for each row. All of the 'sources' classes also now have a 'debug' flag in their init method. We should use this flag in the data processing step to properly handle exceptions as they occur so that one errant data source does not crash the whole data processing step.
The example to build off is in a pending commit, to be put into PR today.
The text was updated successfully, but these errors were encountered: