Skip to content

[LifetimeSafety] Enhance benchmark script for new sub analyses #149577

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Draft
wants to merge 1 commit into
base: users/usx95/07-15-add-liveness-finally
Choose a base branch
from

Conversation

usx95
Copy link
Contributor

@usx95 usx95 commented Jul 18, 2025

Enhanced the lifetime safety analysis benchmark script with more detailed performance metrics and a new nested loop test case. This is a worst case for loan expiry analysis.

What changed?

  • Added a new test case nested_loops that generates code with N levels of nested loops to test how analysis performance scales with loop nesting depth
  • Improved the trace file analysis to extract durations for sub-phases of the lifetime analysis (FactGenerator, LoanPropagation, ExpiredLoans)
  • Enhanced the markdown report generation to include:
    • Relative timing results as percentages of total Clang time
    • More detailed complexity analysis for each analysis phase

Report

Lifetime Analysis Performance Report

Generated on: 2025-07-19 12:43:04


Test Case: Pointer Cycle in Loop

Timing Results:

N (Input Size) Total Time Analysis Time (%) Fact Generator (%) Loan Propagation (%) Expired Loans (%) Live Origins (%)
10 9.97 ms 28.13% 0.00% 27.36% 0.00% 0.00%
25 63.23 ms 87.12% 0.00% 86.83% 0.00% 0.00%
50 693.18 ms 98.51% 0.00% 98.42% 0.00% 0.00%
75 3.14 s 99.55% 0.00% 99.50% 0.03% 0.02%
100 9.33 s 99.86% 0.00% 99.84% 0.01% 0.01%
150 45.38 s 99.96% 0.00% 99.96% 0.00% 0.00%

Complexity Analysis:

Analysis Phase Complexity O(nk)
Total Analysis O(n3.89 ± 0.01)
FactGenerator (Negligible)
LoanPropagation O(n3.89 ± 0.01)
ExpiredLoans O(n1.54 ± 0.73)
LiveOrigins O(n1.67 ± 0.07)

Test Case: CFG Merges

Timing Results:

N (Input Size) Total Time Analysis Time (%) Fact Generator (%) Loan Propagation (%) Expired Loans (%) Live Origins (%)
10 7.87 ms 0.00% 0.00% 0.00% 0.00% 0.00%
50 36.09 ms 75.02% 0.00% 74.19% 0.00% 0.00%
100 240.05 ms 94.79% 0.00% 94.48% 0.00% 0.22%
200 1.63 s 98.91% 0.00% 98.78% 0.00% 0.10%
400 12.80 s 99.77% 0.00% 99.72% 0.00% 0.04%
800 102.48 s 99.95% 0.00% 99.92% 0.00% 0.02%

Complexity Analysis:

Analysis Phase Complexity O(nk)
Total Analysis O(n3.00 ± 0.00)
FactGenerator (Negligible)
LoanPropagation O(n3.00 ± 0.00)
ExpiredLoans (Negligible)
LiveOrigins O(n1.87 ± 0.02)

Test Case: Deeply Nested Loops

Timing Results:

N (Input Size) Total Time Analysis Time (%) Fact Generator (%) Loan Propagation (%) Expired Loans (%) Live Origins (%)
10 8.22 ms 0.00% 0.00% 0.00% 0.00% 0.00%
50 28.05 ms 52.29% 0.00% 46.37% 5.44% 0.00%
100 118.61 ms 84.16% 0.00% 74.95% 9.00% 0.00%
200 785.82 ms 95.46% 0.00% 85.12% 10.26% 0.00%
400 5.89 s 98.73% 0.00% 88.17% 10.53% 0.01%
800 44.26 s 99.65% 0.00% 88.40% 11.24% 0.00%

Complexity Analysis:

Analysis Phase Complexity O(nk)
Total Analysis O(n2.92 ± 0.00)
FactGenerator (Negligible)
LoanPropagation O(n2.92 ± 0.00)
ExpiredLoans O(n3.00 ± 0.00)
LiveOrigins (Negligible)

Copy link
Contributor Author

usx95 commented Jul 18, 2025

Warning

This pull request is not mergeable via GitHub because a downstack PR is open. Once all requirements are satisfied, merge this PR as a stack on Graphite.
Learn more

This stack of pull requests is managed by Graphite. Learn more about stacking.

@usx95 usx95 changed the title add-loan-analysis-to-benchmark [LifetimeSafety] Enhance benchmark script for end timing Jul 18, 2025
@usx95 usx95 changed the title [LifetimeSafety] Enhance benchmark script for end timing [LifetimeSafety] Enhance benchmark script for ExpiredLoans analysis Jul 18, 2025
Copy link

github-actions bot commented Jul 18, 2025

✅ With the latest revision this PR passed the Python code formatter.

@usx95 usx95 force-pushed the users/usx95/07-18-add-loan-analysis-to-benchmark branch from cb9a521 to 9e6a675 Compare July 19, 2025 08:41
@usx95 usx95 changed the base branch from users/usx95/07-14-users_usx95_lifetime-safety-add-loan-expiry to users/usx95/07-15-add-liveness-finally July 19, 2025 12:32
@usx95 usx95 force-pushed the users/usx95/07-18-add-loan-analysis-to-benchmark branch from 9e6a675 to d4f5457 Compare July 19, 2025 12:32
@usx95 usx95 changed the title [LifetimeSafety] Enhance benchmark script for ExpiredLoans analysis [LifetimeSafety] Enhance benchmark script for new sub analyses Jul 19, 2025
@usx95 usx95 moved this from Todo to In Progress in Lifetime Safety in Clang Jul 19, 2025
@usx95 usx95 self-assigned this Jul 19, 2025
@usx95 usx95 force-pushed the users/usx95/07-18-add-loan-analysis-to-benchmark branch 2 times, most recently from 6cb3654 to c55a8f9 Compare July 21, 2025 14:48
@usx95 usx95 force-pushed the users/usx95/07-15-add-liveness-finally branch 2 times, most recently from 8aa7431 to 4443dff Compare July 21, 2025 22:05
@usx95 usx95 force-pushed the users/usx95/07-18-add-loan-analysis-to-benchmark branch from c55a8f9 to 0b3061e Compare July 21, 2025 22:06
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
Status: In Progress
Development

Successfully merging this pull request may close these issues.

1 participant