Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Using regex with escaped characters crashes with "yaml.scanner.ScannerError: while scanning a double-quoted scalar" #99

Open
iTaybb opened this issue Aug 1, 2021 · 1 comment

Comments

@iTaybb
Copy link

iTaybb commented Aug 1, 2021

When using classes with regexes, aiohttp-swagger attempts to convert these to patterns in the openapi v3 format.
If the regex got any escaped characters, such as \w, the code will crash:

Traceback (most recent call last):
  File "/c/git/app/rest/__init__.py", line 31, in setup_swagger  
    aiohttp_swagger.setup_swagger(app, api_version=__api_version__,
  File "/home/itay/.pyenv/versions/3.9.5/lib/python3.9/site-packages/aiohttp_swagger/__init__.py", line 66, in setup_swagger
    swagger_info = generate_doc_from_each_end_point(
  File "/home/itay/.pyenv/versions/3.9.5/lib/python3.9/site-packages/aiohttp_swagger/helpers/builders.py", line 129, in generate_doc_from_each_end_point
    swagger = yaml.full_load(swagger_base)
  File "/home/itay/.pyenv/versions/3.9.5/lib/python3.9/site-packages/yaml/__init__.py", line 142, in full_load
    return load(stream, FullLoader)
  File "/home/itay/.pyenv/versions/3.9.5/lib/python3.9/site-packages/yaml/__init__.py", line 114, in load
    return loader.get_single_data()
  File "/home/itay/.pyenv/versions/3.9.5/lib/python3.9/site-packages/yaml/constructor.py", line 49, in get_single_data
    node = self.get_single_node()
  File "/home/itay/.pyenv/versions/3.9.5/lib/python3.9/site-packages/yaml/composer.py", line 36, in get_single_node
    document = self.compose_document()
  File "/home/itay/.pyenv/versions/3.9.5/lib/python3.9/site-packages/yaml/composer.py", line 55, in compose_document
    node = self.compose_node(None, None)
  File "/home/itay/.pyenv/versions/3.9.5/lib/python3.9/site-packages/yaml/composer.py", line 84, in compose_node
    node = self.compose_mapping_node(anchor)
  File "/home/itay/.pyenv/versions/3.9.5/lib/python3.9/site-packages/yaml/composer.py", line 133, in compose_mapping_node
    item_value = self.compose_node(node, item_key)
  File "/home/itay/.pyenv/versions/3.9.5/lib/python3.9/site-packages/yaml/composer.py", line 84, in compose_node
    node = self.compose_mapping_node(anchor)
  File "/home/itay/.pyenv/versions/3.9.5/lib/python3.9/site-packages/yaml/composer.py", line 133, in compose_mapping_node
    item_value = self.compose_node(node, item_key)
  File "/home/itay/.pyenv/versions/3.9.5/lib/python3.9/site-packages/yaml/composer.py", line 84, in compose_node
    node = self.compose_mapping_node(anchor)
  File "/home/itay/.pyenv/versions/3.9.5/lib/python3.9/site-packages/yaml/composer.py", line 133, in compose_mapping_node
    item_value = self.compose_node(node, item_key)
  File "/home/itay/.pyenv/versions/3.9.5/lib/python3.9/site-packages/yaml/composer.py", line 84, in compose_node
    node = self.compose_mapping_node(anchor)
  File "/home/itay/.pyenv/versions/3.9.5/lib/python3.9/site-packages/yaml/composer.py", line 133, in compose_mapping_node
    item_value = self.compose_node(node, item_key)
  File "/home/itay/.pyenv/versions/3.9.5/lib/python3.9/site-packages/yaml/composer.py", line 84, in compose_node
    node = self.compose_mapping_node(anchor)
  File "/home/itay/.pyenv/versions/3.9.5/lib/python3.9/site-packages/yaml/composer.py", line 133, in compose_mapping_node
    item_value = self.compose_node(node, item_key)
  File "/home/itay/.pyenv/versions/3.9.5/lib/python3.9/site-packages/yaml/composer.py", line 84, in compose_node
    node = self.compose_mapping_node(anchor)
  File "/home/itay/.pyenv/versions/3.9.5/lib/python3.9/site-packages/yaml/composer.py", line 133, in compose_mapping_node
    item_value = self.compose_node(node, item_key)
  File "/home/itay/.pyenv/versions/3.9.5/lib/python3.9/site-packages/yaml/composer.py", line 64, in compose_node
    if self.check_event(AliasEvent):
  File "/home/itay/.pyenv/versions/3.9.5/lib/python3.9/site-packages/yaml/parser.py", 
line 98, in check_event
    self.current_event = self.state()
  File "/home/itay/.pyenv/versions/3.9.5/lib/python3.9/site-packages/yaml/parser.py", 
line 449, in parse_block_mapping_value
    if not self.check_token(KeyToken, ValueToken, BlockEndToken):
  File "/home/itay/.pyenv/versions/3.9.5/lib/python3.9/site-packages/yaml/scanner.py", line 116, in check_token
    self.fetch_more_tokens()
  File "/home/itay/.pyenv/versions/3.9.5/lib/python3.9/site-packages/yaml/scanner.py", line 251, in fetch_more_tokens
    return self.fetch_double()
  File "/home/itay/.pyenv/versions/3.9.5/lib/python3.9/site-packages/yaml/scanner.py", line 655, in fetch_double
    self.fetch_flow_scalar(style='"')
  File "/home/itay/.pyenv/versions/3.9.5/lib/python3.9/site-packages/yaml/scanner.py", line 666, in fetch_flow_scalar
    self.tokens.append(self.scan_flow_scalar(style))
  File "/home/itay/.pyenv/versions/3.9.5/lib/python3.9/site-packages/yaml/scanner.py", line 1149, in scan_flow_scalar
    chunks.extend(self.scan_flow_scalar_non_spaces(double, start_mark))
  File "/home/itay/.pyenv/versions/3.9.5/lib/python3.9/site-packages/yaml/scanner.py", line 1223, in scan_flow_scalar_non_spaces
    raise ScannerError("while scanning a double-quoted scalar", start_mark,
yaml.scanner.ScannerError: while scanning a double-quoted scalar
  in "<unicode string>", line 272, column 26:
                    pattern: "^s3://[\w]+/"
                             ^
found unknown escape character 'w'
  in "<unicode string>", line 272, column 35:
                    pattern: "^s3://[\w]+/"

This happens because the openapi.yaml jinja manifest exports the pattern as:

s3_bucket_key:
                title: "S3 Bucket Key"
                pattern: "^s3://[\\w]+/"
                type: "string"

and causes an error, since \ in double quote is parsed as escape sequence by YAML.

The solution would probably be to use single quotes instead of double quotes, probably in the nesteddict2yaml function.

@iTaybb
Copy link
Author

iTaybb commented Feb 9, 2024

This is still relevant.

Fix using sed:
sed -i 's|" \\"" + str(value) + "\\"\\n"|" \\"" + str(value).replace("\\\\", "\\\\\\\\") + "\\"\\n"|g' $(python -c 'import os.path, aiohttp_swagger; print(os.path.dirname(aiohttp_swagger.__file__))')/helpers/builders.py

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant