Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[pilot] Fix Large File Upload Issues in Mac pkg Deployment #21906

Open
wants to merge 1 commit into
base: master
Choose a base branch
from

Conversation

DanilKarpov
Copy link

@DanilKarpov DanilKarpov commented Mar 3, 2024

Checklist

  • I've run bundle exec rspec from the root directory to see all new and existing tests pass
  • I've followed the fastlane code style and run bundle exec rubocop -a to ensure the code style is valid
  • I see several green ci/circleci builds in the "All checks have passed" section of my PR (connect CircleCI to GitHub if not)
  • I've read the Contribution Guidelines
  • I've updated the documentation if necessary.
  • I've added or updated relevant unit tests.

Description

This Pull Request introduces enhancements to the PkgUploadPackageBuilder class in order to improve efficiency when handling large files. The main modifications include the implementation of chunked reading and writing for file operations.

Motivation and Context

  • Chunked File Copying: Modified the copy_pkg method to utilize chunked reading and writing during the file copy process. This approach helps optimize memory usage and improves the handling of large files.
  • Chunked MD5 Calculation: Updated the calculate_md5 method to perform MD5 calculation on file data in chunks. This ensures better efficiency when processing large files without loading the entire content into memory.

Testing Steps

  1. Create a test package with a large payload.
  2. Start command:
    fastlane pilot upload --skip_waiting_for_build_processing true --api_key_path keys.json
  3. Verify that the file copying process completed successfully without memory issues.

@DanilKarpov DanilKarpov changed the title [pilot] Upload Mac pkg more than 2gb [pilot] Improve Handling of Large Files in Mac pkg Upload Process Mar 3, 2024
@DanilKarpov DanilKarpov changed the title [pilot] Improve Handling of Large Files in Mac pkg Upload Process [pilot] Fix Large File Upload Issues in Mac pkg Deployment Mar 3, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

1 participant