Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

license-expression does not handle invalid license key characters #1199

Closed
rnjudge opened this issue Dec 15, 2022 · 1 comment · Fixed by #1200
Closed

license-expression does not handle invalid license key characters #1199

rnjudge opened this issue Dec 15, 2022 · 1 comment · Fixed by #1200

Comments

@rnjudge
Copy link
Contributor

rnjudge commented Dec 15, 2022 •
edited
Loading

Describe the bug
When a license is reported with invalid license keys (i.e. anything besides letters and numbers, underscore, dot, colon or hyphen signs and spaces) the is_spdx_license_expression() function fails because the liense-expression library fails to handle the error.

To Reproduce
tern report -i debian:latest -f spdxtagvalue -o debian.spdx

Error in terminal

2022-12-15 13:35:22,056 - DEBUG - __main__ - Starting...
2022-12-15 13:35:22,056 - DEBUG - prep - Setting up...
2022-12-15 13:35:22,293 - DEBUG - rootfs - Running command: sudo chmod +x /home/rose/rel-2.11/lib/python3.10/site-packages/tern/tools/fs_hash.sh
2022-12-15 13:35:37,321 - DEBUG - run - Starting analysis...
2022-12-15 13:35:37,321 - DEBUG - skopeo - Attempting to pull image "debian:latest"
2022-12-15 13:35:37,321 - DEBUG - rootfs - Running command: skopeo copy docker://debian:latest dir:/home/rose/.tern/temp
2022-12-15 13:35:41,367 - DEBUG - skopeo - Inspecting remote image "debian:latest"
2022-12-15 13:35:41,367 - DEBUG - rootfs - Running command: skopeo inspect docker://debian:latest
2022-12-15 13:35:45,238 - DEBUG - rootfs - Running command: tar -tf /home/rose/.tern/temp/4948a51a9a3f176f30ac619014f4a2da453a943244eacb53096ee9742eb7cef1
2022-12-15 13:35:45,952 - DEBUG - rootfs - Running command: tar -x -f /home/rose/.tern/temp/4948a51a9a3f176f30ac619014f4a2da453a943244eacb53096ee9742eb7cef1 -C /home/rose/.tern/temp/1/contents
2022-12-15 13:35:46,686 - DEBUG - rootfs - Running command: sudo /home/rose/rel-2.11/lib/python3.10/site-packages/tern/tools/fs_hash.sh /home/rose/.tern/temp/1/contents
2022-12-15 13:36:09,906 - DEBUG - common - Loading packages from cache: layer "2852aecdc1"
2022-12-15 13:36:09,949 - DEBUG - common - Loading files from cache: layer "2852aecdc1"
2022-12-15 13:36:11,070 - DEBUG - generator - Generating SPDX document...
Traceback (most recent call last):
  File "/home/rose/rel-2.11/lib/python3.10/site-packages/license_expression/__init__.py", line 777, in validate
    parsed_expression = self.parse(expression, strict=strict)
  File "/home/rose/rel-2.11/lib/python3.10/site-packages/license_expression/__init__.py", line 539, in parse
    tokens = list(self.tokenize(
  File "/home/rose/rel-2.11/lib/python3.10/site-packages/license_expression/__init__.py", line 603, in tokenize
    for token in tokens:
  File "/home/rose/rel-2.11/lib/python3.10/site-packages/license_expression/__init__.py", line 996, in replace_with_subexpression_by_license_symbol
    for token_group in token_groups:
  File "/home/rose/rel-2.11/lib/python3.10/site-packages/license_expression/__init__.py", line 935, in build_token_groups_for_with_subexpression
    tokens = list(tokens)
  File "/home/rose/rel-2.11/lib/python3.10/site-packages/license_expression/__init__.py", line 597, in <genexpr>
    tokens = (t for t in tokens if t.string and t.string.strip())
  File "/home/rose/rel-2.11/lib/python3.10/site-packages/license_expression/__init__.py", line 921, in build_symbols_from_unknown_tokens
    for symtok in build_token_with_symbol():
  File "/home/rose/rel-2.11/lib/python3.10/site-packages/license_expression/__init__.py", line 901, in build_token_with_symbol
    toksym = LicenseSymbol(string)
  File "/home/rose/rel-2.11/lib/python3.10/site-packages/license_expression/__init__.py", line 1213, in __init__
    raise ExpressionError(
license_expression.ExpressionError: Invalid license key: the valid characters are: letters and numbers, underscore, dot, colon or hyphen signs and spaces: 'MIT/X11'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/rose/rel-2.11/bin/tern", line 8, in <module>
    sys.exit(main())
  File "/home/rose/rel-2.11/lib/python3.10/site-packages/tern/__main__.py", line 311, in main
    do_main(args)
  File "/home/rose/rel-2.11/lib/python3.10/site-packages/tern/__main__.py", line 123, in do_main
    crun.execute_image(args)
  File "/home/rose/rel-2.11/lib/python3.10/site-packages/tern/analyze/default/container/run.py", line 82, in execute_image
    report.report_out(args, full_image)
  File "/home/rose/rel-2.11/lib/python3.10/site-packages/tern/report/report.py", line 86, in report_out
    report = generate_report(args, *images)
  File "/home/rose/rel-2.11/lib/python3.10/site-packages/tern/report/report.py", line 41, in generate_report
    return generate_format(
  File "/home/rose/rel-2.11/lib/python3.10/site-packages/tern/report/report.py", line 56, in generate_format
    return mgr.driver.generate(images, print_inclusive)
  File "/home/rose/rel-2.11/lib/python3.10/site-packages/tern/formats/spdx/spdxtagvalue/generator.py", line 141, in generate
    report += mhelpers.get_image_block(image_obj, template) + '\n'
  File "/home/rose/rel-2.11/lib/python3.10/site-packages/tern/formats/spdx/spdxtagvalue/image_helpers.py", line 134, in get_image_block
    pkg_block = get_image_packages_block(image_obj, template)
  File "/home/rose/rel-2.11/lib/python3.10/site-packages/tern/formats/spdx/spdxtagvalue/image_helpers.py", line 37, in get_image_packages_block
    block += phelpers.get_package_block(package,
  File "/home/rose/rel-2.11/lib/python3.10/site-packages/tern/formats/spdx/spdxtagvalue/package_helpers.py", line 105, in get_package_block
    block += 'PackageLicenseDeclared: ' + spdx_common.get_package_license_declared(
  File "/home/rose/rel-2.11/lib/python3.10/site-packages/tern/formats/spdx/spdx_common.py", line 60, in get_package_license_declared
    if is_spdx_license_expression(package_license_declared):
  File "/home/rose/rel-2.11/lib/python3.10/site-packages/tern/formats/spdx/spdx_common.py", line 53, in is_spdx_license_expression
    return licensing.validate(license_data).errors == []
  File "/home/rose/rel-2.11/lib/python3.10/site-packages/license_expression/__init__.py", line 780, in validate
    expression_info.invalid_symbols.append(e.token_string)
AttributeError: 'ExpressionError' object has no attribute 'token_string'

Expected behavior
SPDX report should be generated.
@rnjudge
Copy link
Contributor Author

rnjudge commented Dec 15, 2022

I opened an issue for this with the license-expression library: aboutcode-org/license-expression#76

I will create a workaround for this in the meantime.

@rnjudge rnjudge changed the title Update to license-expression causes bug when generating SPDX reports license-expression does not handle invalid license key characters Dec 15, 2022
rnjudge added a commit to rnjudge/tern that referenced this issue Dec 15, 2022
@rnjudge
Replace inalid license key characters
711e7fa 
When a license is reported with invalid license keys (i.e. anything
besides letters and numbers, underscore, dot, colon or hyphen
signs and spaces) the `is_spdx_license_expression()` function fails
because the liense-expression library does not properly handle the
unknown characters. This commit is a workaround until the issue
opened in the license-expression library[1] is resolved.

Resolves tern-tools#1199

[1] aboutcode-org/license-expression#76

Signed-off-by: Rose Judge <[email protected]>
@rnjudge rnjudge mentioned this issue Dec 15, 2022
Merged
rnjudge added a commit that referenced this issue Dec 15, 2022
@rnjudge
Replace inalid license key characters
48e22cf 
When a license is reported with invalid license keys (i.e. anything
besides letters and numbers, underscore, dot, colon or hyphen
signs and spaces) the `is_spdx_license_expression()` function fails
because the liense-expression library does not properly handle the
unknown characters. This commit is a workaround until the issue
opened in the license-expression library[1] is resolved.

Resolves #1199

[1] aboutcode-org/license-expression#76

Signed-off-by: Rose Judge <[email protected]>
@rnjudge rnjudge mentioned this issue Jan 4, 2023
Closed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

Replace inalid license key characters rnjudge/tern
1 participant
@rnjudge