chore(iast): add local debug scripts to find leaks

[avara1986]

Add to the repository the scripts that @juanjux and I use to debug C++ leaks. This folder (scripts/iast/) contains some scripts to check the memory usage of native code.

1. Build the docker image

docker build . -f docker/Dockerfile_py311_debug_mode -t python_311_debug

2. Run the docker container

2.1. Run the container with the script to find references (this script will run the memory usage check)

docker run --rm -it -v ${PWD}:/ddtrace python_311_debug /bin/bash -c "cd /ddtrace && scripts/iast/run_references.sh"
>> References: 1003
>> References: 2
>> References: 2
>> References: 2
>> References: 2
>> References: 2

2.2. Run the container with the script with memray usage check

docker run --rm -it -v ${PWD}:/ddtrace python_311_debug /bin/bash -c "cd /ddtrace && scripts/iast/run_memray.sh"
google-chrome file://$PWD/memray-flamegraph-lel.html

2.3. Run the container with the script with Max RSS

docker run --rm -it -v ${PWD}:/ddtrace python_311_debug /bin/bash -c "cd /ddtrace && scripts/iast/run_memory.sh"
>> Round 0 Max RSS: 41.9453125
>> 42.2109375

Checklist

• Change(s) are motivated and described in the PR description
• Testing strategy is described if automated tests are not included in the PR
• Risks are described (performance impact, potential for breakage, maintainability)
• Change is maintainable (easy to change, telemetry, documentation)
• Library release note guidelines are followed or label changelog/no-changelog is set
• Documentation is included (in-code, generated user docs, public corp docs)
• Backport labels are set (if applicable)
• If this PR changes the public interface, I've notified @DataDog/apm-tees.
• If change touches code that signs or publishes builds or packages, or handles credentials of any kind, I've requested a review from @DataDog/security-design-and-guidance.

Reviewer Checklist

• Title is accurate
• All changes are related to the pull request's stated goal
• Description motivates each change
• Avoids breaking API changes
• Testing strategy adequately addresses listed risks
• Change is maintainable (easy to change, telemetry, documentation)
• Release note makes sense to a user of the library
• Author has acknowledged and discussed the performance implications of this PR as reported in the benchmarks PR comment
• Backport labels are set in a manner that is consistent with the release branch maintenance policy

[datadog-dd-trace-py-rkomorn]

Datadog Report

Branch report: avara1986/iast_debug_scripts
Commit report: aa01e3f
Test service: dd-trace-py

✅ 0 Failed, 147843 Passed, 28595 Skipped, 9h 20m 57.19s Total duration (2h 53m 10.12s time saved)
⌛ 1 Performance Regression

⌛ Performance Regressions vs Default Branch (1)

• test_data_streams_kafka_offset_monitoring_auto_commit - test_kafka.py 1m 30.18s (+1m 26.27s, +2206%) - Details

[pr-commenter]

Benchmarks

Benchmark execution time: 2024-05-21 17:49:42

Comparing candidate commit 56f678e in PR branch avara1986/iast_debug_scripts with baseline commit e605bd3 in branch main.

Found 0 performance improvements and 0 performance regressions! Performance is the same for 209 metrics, 9 unstable metrics.

[codecov-commenter]

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 10.35%. Comparing base (e605bd3) to head (56f678e).
Report is 1 commits behind head on main.

Additional details and impacted files

@@             Coverage Diff             @@
##             main    #9318       +/-   ##
===========================================
- Coverage   76.22%   10.35%   -65.87%     
===========================================
  Files        1289     1259       -30     
  Lines      122509   120663     -1846     
===========================================
- Hits        93386    12499    -80887     
- Misses      29123   108164    +79041     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

[erikayasuda]

This looks handy, thanks for adding :)

[erikayasuda]

After a second thought and check, I'm realizing these scripts also get included when we build ddtracepy 🤔 Should we be storing these debug scripts elsewhere?

[avara1986]

After a second thought and check, I'm realizing these scripts also get included when we build ddtracepy 🤔 Should we be storing these debug scripts elsewhere?

Thanks for a second check @erikayasuda! I add an extra thought and, why are we not skipping all script folder when we build ddtrace-py? 😛

https://github.com/DataDog/dd-trace-py/blob/main/setup.py#L471

setup(
    name="ddtrace",
    packages=find_packages(exclude=["tests*", "benchmarks*"]),

[avara1986]

Like that @erikayasuda? 👉 #9324

[erikayasuda]

Like that @erikayasuda? 👉 #9324

Yesss 🥳 Thanks for updating!