Skip to content

Commit

Permalink
fix (ssh connection) update transport
Browse files Browse the repository at this point in the history
  • Loading branch information
potench committed Oct 3, 2024
1 parent b41e91e commit 53c19b3
Show file tree
Hide file tree
Showing 2 changed files with 21 additions and 14 deletions.
10 changes: 5 additions & 5 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -44,7 +44,7 @@ Create a `config.json` file with connection details to snowflake.
"encoding": "utf-8",
"sanitize_header": false,
"skip_rows": 0,
"infer_schema": false,
"infer_schema": false
}
],
"xml_fields": [],
Expand All @@ -54,7 +54,7 @@ Create a `config.json` file with connection details to snowflake.
"gnupghome": "/your/dir/.gnupg",
"passphrase": "your_gpg_passphrase"
},
"private_key_file": "Optional_Path",
"private_key_file": "Optional_Path"
}
```
If using the decryption feature you must pass the configs shown above, including the AWS SSM parameter name for where the decryption private key is stored. In order to retrieve this parameter the runtime environment must have access to SSM through IAM environment variables (AWS_ACCESS_KEY_ID, AWS_SECRET_ACCESS_KEY, AWS_SESSION_TOKEN).
Expand All @@ -76,11 +76,11 @@ Create a `config.json` file with connection details to snowflake.
- `delimiter`: A one-character string delimiter used to separate fields. Default, is `,`.

The following table configuration fields are optional:
- `key_properties`: Array containing the unique keys of the table. Defaults to `['_sdc_source_file', '_sdc_source_lineno']`, representing the file name and line number. Specify an emtpy array (`[]`) to load all new files without a replication key
- `key_properties`: Array containing the unique keys of the table. Defaults to `['_sdc_source_file', '_sdc_source_lineno']`, representing the file name and line number. Specify an empty array (`[]`) to load all new files without a replication key
- `encoding`: File encoding, defaults to `utf-8`
- `sanitize_header`: Boolean, specifies whether to clean up header names so that they are more likely to be accepted by a target SQL database
- `skip_rows`: Integer, specifies the number of rows to skip at the top of the file to handle non-data content like comments or other text. Default 0.
- `infer_schema`: Boolean. If set to true (the default value), the tap will attempt to detect the data type of the fields. Otherwise all fields will be treated as strings.
- `infer_schema`: Boolean. If set to true (the default value), the tap will attempt to detect the data type of the fields. Otherwise, all fields will be treated as strings.

## Discovery mode:

Expand Down Expand Up @@ -115,7 +115,7 @@ $ tap-sftp --config config.json --catalog catalog.json --state state.json
pip install tox
```

2. To run unit tests:
1. To run unit tests:
```
tox
```
Expand Down
25 changes: 16 additions & 9 deletions tap_sftp/client.py
Original file line number Diff line number Diff line change
Expand Up @@ -52,10 +52,11 @@ def __connect(self):
LOGGER.info('Connection successful')

def _attempt_connection(self):
client = paramiko.SSHClient()
client.set_missing_host_key_policy(paramiko.AutoAddPolicy())
client.get_transport().get_security_options().key_types = ['ssh-rsa', 'ecdsa-sha2-nistp256']
try:
ssh_client = paramiko.SSHClient()
ssh_client.set_missing_host_key_policy(paramiko.AutoAddPolicy())
ssh_client.connect(
client.connect(
hostname=self.host,
port=self.port,
username=self.username,
Expand All @@ -64,12 +65,18 @@ def _attempt_connection(self):
compress=True,
timeout=120
)
self.sftp = ssh_client.open_sftp()
except (AuthenticationException, SSHException) as ex:
LOGGER.warning('Connection attempt failed: %s', ex)
if ssh_client:
ssh_client.close()
raise
self.sftp = client.open_sftp()
except paramiko.AuthenticationException as ex:
LOGGER.error('AuthenticationException, please verify your credentials: %s', ex)
except paramiko.SSHException as ex:
LOGGER.error('SSHException, could not establish SSH connection: %s', ex)
except paramiko.BadHostKeyException as ex:
LOGGER.error('BadHostKeyException, bad host key: %s', ex)
except Exception as ex:
LOGGER.error('Exception, failed to connect or establish SFTP session: %s', ex)
finally:
if client:
client.close()

def close(self):
if self.sftp is not None:
Expand Down

0 comments on commit 53c19b3

Please sign in to comment.