-
Notifications
You must be signed in to change notification settings - Fork 462
test(openai): enable v2 testing and remove _v1 suffix #14905
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: main
Are you sure you want to change the base?
Conversation
|
Bootstrap import analysisComparison of import times between this PR and base. SummaryThe average import time from this PR is: 239 ± 2 ms. The average import time from base is: 241 ± 2 ms. The import time difference between this PR and base is: -1.62 ± 0.09 ms. Import time breakdownThe following import paths have shrunk:
|
Performance SLOsComparing candidate alex/MLOB-4188_openai-v2-support (38a389b) with baseline main (92937df) 📈 Performance Regressions (2 suites)📈 iast_aspects - 40/40✅ re_expand_aspectTime: ✅ 33.810µs (SLO: <40.000µs 📉 -15.5%) vs baseline: +5.2% Memory: ✅ 37.670MB (SLO: <39.000MB -3.4%) vs baseline: +4.8% ✅ re_expand_noaspectTime: ✅ 28.590µs (SLO: <40.000µs 📉 -28.5%) vs baseline: +0.3% Memory: ✅ 37.631MB (SLO: <39.000MB -3.5%) vs baseline: +4.7% ✅ re_findall_aspectTime: ✅ 2.891µs (SLO: <10.000µs 📉 -71.1%) vs baseline: -0.4% Memory: ✅ 37.690MB (SLO: <39.000MB -3.4%) vs baseline: +4.8% ✅ re_findall_noaspectTime: ✅ 1.410µs (SLO: <10.000µs 📉 -85.9%) vs baseline: -0.5% Memory: ✅ 37.729MB (SLO: <39.000MB -3.3%) vs baseline: +5.0% ✅ re_finditer_aspectTime: ✅ 4.708µs (SLO: <10.000µs 📉 -52.9%) vs baseline: +5.1% Memory: ✅ 37.631MB (SLO: <39.000MB -3.5%) vs baseline: +4.6% ✅ re_finditer_noaspectTime: ✅ 1.409µs (SLO: <10.000µs 📉 -85.9%) vs baseline: -0.4% Memory: ✅ 37.690MB (SLO: <39.000MB -3.4%) vs baseline: +4.9% ✅ re_fullmatch_aspectTime: ✅ 2.961µs (SLO: <10.000µs 📉 -70.4%) vs baseline: 📈 +11.1% Memory: ✅ 37.670MB (SLO: <39.000MB -3.4%) vs baseline: +4.8% ✅ re_fullmatch_noaspectTime: ✅ 1.312µs (SLO: <10.000µs 📉 -86.9%) vs baseline: +1.3% Memory: ✅ 37.670MB (SLO: <39.000MB -3.4%) vs baseline: +5.1% ✅ re_group_aspectTime: ✅ 2.951µs (SLO: <10.000µs 📉 -70.5%) vs baseline: -4.2% Memory: ✅ 37.572MB (SLO: <39.000MB -3.7%) vs baseline: +4.5% ✅ re_group_noaspectTime: ✅ 1.625µs (SLO: <10.000µs 📉 -83.7%) vs baseline: +1.2% Memory: ✅ 37.690MB (SLO: <39.000MB -3.4%) vs baseline: +4.9% ✅ re_groups_aspectTime: ✅ 3.072µs (SLO: <10.000µs 📉 -69.3%) vs baseline: -4.0% Memory: ✅ 37.650MB (SLO: <39.000MB -3.5%) vs baseline: +4.8% ✅ re_groups_noaspectTime: ✅ 1.697µs (SLO: <10.000µs 📉 -83.0%) vs baseline: -1.2% Memory: ✅ 37.690MB (SLO: <39.000MB -3.4%) vs baseline: +4.8% ✅ re_match_aspectTime: ✅ 3.118µs (SLO: <10.000µs 📉 -68.8%) vs baseline: +8.8% Memory: ✅ 37.670MB (SLO: <39.000MB -3.4%) vs baseline: +4.9% ✅ re_match_noaspectTime: ✅ 1.304µs (SLO: <10.000µs 📉 -87.0%) vs baseline: -0.3% Memory: ✅ 37.729MB (SLO: <39.000MB -3.3%) vs baseline: +5.2% ✅ re_search_aspectTime: ✅ 2.695µs (SLO: <10.000µs 📉 -73.0%) vs baseline: +5.1% Memory: ✅ 37.690MB (SLO: <39.000MB -3.4%) vs baseline: +5.0% ✅ re_search_noaspectTime: ✅ 1.211µs (SLO: <10.000µs 📉 -87.9%) vs baseline: ~same Memory: ✅ 37.709MB (SLO: <39.000MB -3.3%) vs baseline: +5.0% ✅ re_sub_aspectTime: ✅ 3.434µs (SLO: <10.000µs 📉 -65.7%) vs baseline: -0.2% Memory: ✅ 37.650MB (SLO: <39.000MB -3.5%) vs baseline: +4.7% ✅ re_sub_noaspectTime: ✅ 1.523µs (SLO: <10.000µs 📉 -84.8%) vs baseline: -0.5% Memory: ✅ 37.709MB (SLO: <39.000MB -3.3%) vs baseline: +4.9% ✅ re_subn_aspectTime: ✅ 3.638µs (SLO: <10.000µs 📉 -63.6%) vs baseline: -0.2% Memory: ✅ 37.650MB (SLO: <39.000MB -3.5%) vs baseline: +4.7% ✅ re_subn_noaspectTime: ✅ 1.609µs (SLO: <10.000µs 📉 -83.9%) vs baseline: +0.9% Memory: ✅ 37.690MB (SLO: <39.000MB -3.4%) vs baseline: +4.7% 📈 iastaspects - 118/118✅ add_aspectTime: ✅ 0.409µs (SLO: <10.000µs 📉 -95.9%) vs baseline: +0.9% Memory: ✅ 38.083MB (SLO: <39.000MB -2.4%) vs baseline: +4.9% ✅ add_inplace_aspectTime: ✅ 0.408µs (SLO: <10.000µs 📉 -95.9%) vs baseline: +0.1% Memory: ✅ 37.808MB (SLO: <39.000MB -3.1%) vs baseline: +4.2% ✅ add_inplace_noaspectTime: ✅ 0.322µs (SLO: <10.000µs 📉 -96.8%) vs baseline: +1.0% Memory: ✅ 38.083MB (SLO: <39.000MB -2.4%) vs baseline: +5.0% ✅ add_noaspectTime: ✅ 0.280µs (SLO: <10.000µs 📉 -97.2%) vs baseline: +1.2% Memory: ✅ 38.044MB (SLO: <39.000MB -2.5%) vs baseline: +4.9% ✅ bytearray_aspectTime: ✅ 1.376µs (SLO: <10.000µs 📉 -86.2%) vs baseline: +1.9% Memory: ✅ 37.650MB (SLO: <39.000MB -3.5%) vs baseline: +3.8% ✅ bytearray_extend_aspectTime: ✅ 1.455µs (SLO: <10.000µs 📉 -85.4%) vs baseline: +0.4% Memory: ✅ 37.985MB (SLO: <39.000MB -2.6%) vs baseline: +4.6% ✅ bytearray_extend_noaspectTime: ✅ 0.618µs (SLO: <10.000µs 📉 -93.8%) vs baseline: +0.7% Memory: ✅ 37.709MB (SLO: <39.000MB -3.3%) vs baseline: +4.1% ✅ bytearray_noaspectTime: ✅ 0.479µs (SLO: <10.000µs 📉 -95.2%) vs baseline: -0.3% Memory: ✅ 38.004MB (SLO: <39.000MB -2.6%) vs baseline: +4.7% ✅ bytes_aspectTime: ✅ 1.412µs (SLO: <10.000µs 📉 -85.9%) vs baseline: 📈 +10.5% Memory: ✅ 38.063MB (SLO: <39.000MB -2.4%) vs baseline: +4.7% ✅ bytes_noaspectTime: ✅ 0.492µs (SLO: <10.000µs 📉 -95.1%) vs baseline: +0.1% Memory: ✅ 38.044MB (SLO: <39.000MB -2.5%) vs baseline: +5.0% ✅ bytesio_aspectTime: ✅ 1.366µs (SLO: <10.000µs 📉 -86.3%) vs baseline: +0.9% Memory: ✅ 37.749MB (SLO: <39.000MB -3.2%) vs baseline: +4.0% ✅ bytesio_noaspectTime: ✅ 0.492µs (SLO: <10.000µs 📉 -95.1%) vs baseline: -0.9% Memory: ✅ 38.083MB (SLO: <39.000MB -2.4%) vs baseline: +4.8% ✅ capitalize_aspectTime: ✅ 0.737µs (SLO: <10.000µs 📉 -92.6%) vs baseline: ~same Memory: ✅ 37.867MB (SLO: <39.000MB -2.9%) vs baseline: +4.3% ✅ capitalize_noaspectTime: ✅ 0.434µs (SLO: <10.000µs 📉 -95.7%) vs baseline: -0.5% Memory: ✅ 38.083MB (SLO: <39.000MB -2.4%) vs baseline: +4.9% ✅ casefold_aspectTime: ✅ 0.735µs (SLO: <10.000µs 📉 -92.7%) vs baseline: ~same Memory: ✅ 37.670MB (SLO: <39.000MB -3.4%) vs baseline: +3.8% ✅ casefold_noaspectTime: ✅ 0.371µs (SLO: <10.000µs 📉 -96.3%) vs baseline: +1.0% Memory: ✅ 38.044MB (SLO: <39.000MB -2.5%) vs baseline: +4.9% ✅ decode_aspectTime: ✅ 0.729µs (SLO: <10.000µs 📉 -92.7%) vs baseline: +1.1% Memory: ✅ 38.103MB (SLO: <39.000MB -2.3%) vs baseline: +4.9% ✅ decode_noaspectTime: ✅ 0.421µs (SLO: <10.000µs 📉 -95.8%) vs baseline: ~same Memory: ✅ 38.044MB (SLO: <39.000MB -2.5%) vs baseline: +4.7% ✅ encode_aspectTime: ✅ 0.712µs (SLO: <10.000µs 📉 -92.9%) vs baseline: +0.1% Memory: ✅ 38.063MB (SLO: <39.000MB -2.4%) vs baseline: +5.0% ✅ encode_noaspectTime: ✅ 0.398µs (SLO: <10.000µs 📉 -96.0%) vs baseline: -0.8% Memory: ✅ 38.122MB (SLO: <39.000MB -2.3%) vs baseline: +5.1% ✅ format_aspectTime: ✅ 3.528µs (SLO: <10.000µs 📉 -64.7%) vs baseline: +4.9% Memory: ✅ 37.670MB (SLO: <39.000MB -3.4%) vs baseline: +3.8% ✅ format_map_aspectTime: ✅ 3.615µs (SLO: <10.000µs 📉 -63.8%) vs baseline: ~same Memory: ✅ 37.670MB (SLO: <39.000MB -3.4%) vs baseline: +3.8% ✅ format_map_noaspectTime: ✅ 0.784µs (SLO: <10.000µs 📉 -92.2%) vs baseline: +1.0% Memory: ✅ 38.122MB (SLO: <39.000MB -2.3%) vs baseline: +5.0% ✅ format_noaspectTime: ✅ 0.598µs (SLO: <10.000µs 📉 -94.0%) vs baseline: +0.1% Memory: ✅ 38.044MB (SLO: <39.000MB -2.5%) vs baseline: +4.8% ✅ index_aspectTime: ✅ 0.355µs (SLO: <10.000µs 📉 -96.4%) vs baseline: -1.0% Memory: ✅ 38.083MB (SLO: <39.000MB -2.4%) vs baseline: +4.9% ✅ index_noaspectTime: ✅ 0.281µs (SLO: <10.000µs 📉 -97.2%) vs baseline: +0.4% Memory: ✅ 38.063MB (SLO: <39.000MB -2.4%) vs baseline: +4.7% ✅ join_aspectTime: ✅ 1.402µs (SLO: <10.000µs 📉 -86.0%) vs baseline: +0.8% Memory: ✅ 38.142MB (SLO: <39.000MB -2.2%) vs baseline: +4.9% ✅ join_noaspectTime: ✅ 0.490µs (SLO: <10.000µs 📉 -95.1%) vs baseline: -0.3% Memory: ✅ 37.985MB (SLO: <39.000MB -2.6%) vs baseline: +4.7% ✅ ljust_aspectTime: ✅ 2.628µs (SLO: <20.000µs 📉 -86.9%) vs baseline: +5.9% Memory: ✅ 38.004MB (SLO: <39.000MB -2.6%) vs baseline: +4.6% ✅ ljust_noaspectTime: ✅ 0.404µs (SLO: <10.000µs 📉 -96.0%) vs baseline: +0.6% Memory: ✅ 38.083MB (SLO: <39.000MB -2.4%) vs baseline: +4.8% ✅ lower_aspectTime: ✅ 2.288µs (SLO: <10.000µs 📉 -77.1%) vs baseline: +4.0% Memory: ✅ 38.004MB (SLO: <39.000MB -2.6%) vs baseline: +4.7% ✅ lower_noaspectTime: ✅ 0.365µs (SLO: <10.000µs 📉 -96.4%) vs baseline: -0.3% Memory: ✅ 38.063MB (SLO: <39.000MB -2.4%) vs baseline: +4.7% ✅ lstrip_aspectTime: ✅ 2.580µs (SLO: <20.000µs 📉 -87.1%) vs baseline: 📈 +14.2% Memory: ✅ 37.729MB (SLO: <39.000MB -3.3%) vs baseline: +4.0% ✅ lstrip_noaspectTime: ✅ 0.385µs (SLO: <10.000µs 📉 -96.2%) vs baseline: +0.5% Memory: ✅ 37.985MB (SLO: <39.000MB -2.6%) vs baseline: +4.6% ✅ modulo_aspectTime: ✅ 1.007µs (SLO: <10.000µs 📉 -89.9%) vs baseline: +0.3% Memory: ✅ 38.004MB (SLO: <39.000MB -2.6%) vs baseline: +4.7% ✅ modulo_aspect_for_bytearray_bytearrayTime: ✅ 1.527µs (SLO: <10.000µs 📉 -84.7%) vs baseline: -0.2% Memory: ✅ 37.670MB (SLO: <39.000MB -3.4%) vs baseline: +3.7% ✅ modulo_aspect_for_bytesTime: ✅ 0.972µs (SLO: <10.000µs 📉 -90.3%) vs baseline: +0.6% Memory: ✅ 38.044MB (SLO: <39.000MB -2.5%) vs baseline: +4.7% ✅ modulo_aspect_for_bytes_bytearrayTime: ✅ 1.213µs (SLO: <10.000µs 📉 -87.9%) vs baseline: +1.5% Memory: ✅ 37.945MB (SLO: <39.000MB -2.7%) vs baseline: +4.4% ✅ modulo_noaspectTime: ✅ 0.625µs (SLO: <10.000µs 📉 -93.8%) vs baseline: -0.8% Memory: ✅ 38.083MB (SLO: <39.000MB -2.4%) vs baseline: +4.7% ✅ replace_aspectTime: ✅ 5.224µs (SLO: <10.000µs 📉 -47.8%) vs baseline: +9.4% Memory: ✅ 37.985MB (SLO: <39.000MB -2.6%) vs baseline: +4.6% ✅ replace_noaspectTime: ✅ 0.464µs (SLO: <10.000µs 📉 -95.4%) vs baseline: +0.9% Memory: ✅ 38.004MB (SLO: <39.000MB -2.6%) vs baseline: +4.6% ✅ repr_aspectTime: ✅ 0.907µs (SLO: <10.000µs 📉 -90.9%) vs baseline: +0.2% Memory: ✅ 38.024MB (SLO: <39.000MB -2.5%) vs baseline: +4.7% ✅ repr_noaspectTime: ✅ 0.417µs (SLO: <10.000µs 📉 -95.8%) vs baseline: +1.0% Memory: ✅ 37.985MB (SLO: <39.000MB -2.6%) vs baseline: +4.6% ✅ rstrip_aspectTime: ✅ 1.998µs (SLO: <20.000µs 📉 -90.0%) vs baseline: +3.0% Memory: ✅ 37.670MB (SLO: <39.000MB -3.4%) vs baseline: +3.6% ✅ rstrip_noaspectTime: ✅ 0.379µs (SLO: <10.000µs 📉 -96.2%) vs baseline: -0.1% Memory: ✅ 38.103MB (SLO: <39.000MB -2.3%) vs baseline: +5.0% ✅ slice_aspectTime: ✅ 0.494µs (SLO: <10.000µs 📉 -95.1%) vs baseline: -0.2% Memory: ✅ 38.122MB (SLO: <39.000MB -2.3%) vs baseline: +5.1% ✅ slice_noaspectTime: ✅ 0.442µs (SLO: <10.000µs 📉 -95.6%) vs baseline: -1.6% Memory: ✅ 37.965MB (SLO: <39.000MB -2.7%) vs baseline: +4.6% ✅ stringio_aspectTime: ✅ 1.539µs (SLO: <10.000µs 📉 -84.6%) vs baseline: +1.8% Memory: ✅ 38.063MB (SLO: <39.000MB -2.4%) vs baseline: +5.1% ✅ stringio_noaspectTime: ✅ 0.717µs (SLO: <10.000µs 📉 -92.8%) vs baseline: +1.1% Memory: ✅ 38.083MB (SLO: <39.000MB -2.4%) vs baseline: +4.9% ✅ strip_aspectTime: ✅ 2.451µs (SLO: <20.000µs 📉 -87.7%) vs baseline: +9.8% Memory: ✅ 37.965MB (SLO: <39.000MB -2.7%) vs baseline: +4.6% ✅ strip_noaspectTime: ✅ 0.388µs (SLO: <10.000µs 📉 -96.1%) vs baseline: +1.7% Memory: ✅ 38.083MB (SLO: <39.000MB -2.4%) vs baseline: +4.9% ✅ swapcase_aspectTime: ✅ 2.427µs (SLO: <10.000µs 📉 -75.7%) vs baseline: +0.8% Memory: ✅ 38.083MB (SLO: <39.000MB -2.4%) vs baseline: +5.1% ✅ swapcase_noaspectTime: ✅ 0.533µs (SLO: <10.000µs 📉 -94.7%) vs baseline: -1.0% Memory: ✅ 38.083MB (SLO: <39.000MB -2.4%) vs baseline: +4.7% ✅ title_aspectTime: ✅ 2.410µs (SLO: <10.000µs 📉 -75.9%) vs baseline: +3.9% Memory: ✅ 38.083MB (SLO: <39.000MB -2.4%) vs baseline: +5.0% ✅ title_noaspectTime: ✅ 0.501µs (SLO: <10.000µs 📉 -95.0%) vs baseline: -0.8% Memory: ✅ 38.044MB (SLO: <39.000MB -2.5%) vs baseline: +4.7% ✅ translate_aspectTime: ✅ 3.382µs (SLO: <10.000µs 📉 -66.2%) vs baseline: +5.6% Memory: ✅ 38.044MB (SLO: <39.000MB -2.5%) vs baseline: +4.7% ✅ translate_noaspectTime: ✅ 1.046µs (SLO: <10.000µs 📉 -89.5%) vs baseline: +0.4% Memory: ✅ 38.063MB (SLO: <39.000MB -2.4%) vs baseline: +4.8% ✅ upper_aspectTime: ✅ 2.199µs (SLO: <10.000µs 📉 -78.0%) vs baseline: -0.9% Memory: ✅ 38.103MB (SLO: <39.000MB -2.3%) vs baseline: +5.1% ✅ upper_noaspectTime: ✅ 0.369µs (SLO: <10.000µs 📉 -96.3%) vs baseline: -0.6% Memory: ✅ 38.063MB (SLO: <39.000MB -2.4%) vs baseline: +5.0% 🟡 Near SLO Breach (5 suites)🟡 djangosimple - 30/30✅ appsecTime: ✅ 20.421ms (SLO: <22.300ms -8.4%) vs baseline: -0.5% Memory: ✅ 65.508MB (SLO: <67.000MB -2.2%) vs baseline: +5.1% ✅ exception-replay-enabledTime: ✅ 1.350ms (SLO: <1.450ms -6.9%) vs baseline: +0.3% Memory: ✅ 64.488MB (SLO: <67.000MB -3.7%) vs baseline: +4.8% ✅ iastTime: ✅ 20.408ms (SLO: <22.250ms -8.3%) vs baseline: -0.6% Memory: ✅ 65.368MB (SLO: <67.000MB -2.4%) vs baseline: +4.7% ✅ profilerTime: ✅ 15.297ms (SLO: <16.550ms -7.6%) vs baseline: ~same Memory: ✅ 53.766MB (SLO: <54.500MB 🟡 -1.3%) vs baseline: +4.8% ✅ resource-renamingTime: ✅ 20.529ms (SLO: <21.750ms -5.6%) vs baseline: -0.1% Memory: ✅ 65.445MB (SLO: <67.000MB -2.3%) vs baseline: +4.7% ✅ span-code-originTime: ✅ 26.153ms (SLO: <28.200ms -7.3%) vs baseline: -0.3% Memory: ✅ 67.576MB (SLO: <69.500MB -2.8%) vs baseline: +4.6% ✅ tracerTime: ✅ 20.463ms (SLO: <21.750ms -5.9%) vs baseline: -0.2% Memory: ✅ 65.423MB (SLO: <67.000MB -2.4%) vs baseline: +4.9% ✅ tracer-and-profilerTime: ✅ 22.010ms (SLO: <23.500ms -6.3%) vs baseline: ~same Memory: ✅ 66.688MB (SLO: <67.500MB 🟡 -1.2%) vs baseline: +4.9% ✅ tracer-dont-create-db-spansTime: ✅ 19.315ms (SLO: <21.500ms 📉 -10.2%) vs baseline: -0.5% Memory: ✅ 65.441MB (SLO: <66.000MB 🟡 -0.8%) vs baseline: +4.7% ✅ tracer-minimalTime: ✅ 16.567ms (SLO: <17.500ms -5.3%) vs baseline: -0.5% Memory: ✅ 65.487MB (SLO: <66.000MB 🟡 -0.8%) vs baseline: +5.0% ✅ tracer-nativeTime: ✅ 20.503ms (SLO: <21.750ms -5.7%) vs baseline: +0.1% Memory: ✅ 71.363MB (SLO: <72.500MB 🟡 -1.6%) vs baseline: +4.8% ✅ tracer-no-cachesTime: ✅ 18.485ms (SLO: <19.650ms -5.9%) vs baseline: ~same Memory: ✅ 65.480MB (SLO: <67.000MB -2.3%) vs baseline: +4.9% ✅ tracer-no-databasesTime: ✅ 18.828ms (SLO: <20.100ms -6.3%) vs baseline: ~same Memory: ✅ 65.290MB (SLO: <67.000MB -2.6%) vs baseline: +4.5% ✅ tracer-no-middlewareTime: ✅ 20.175ms (SLO: <21.500ms -6.2%) vs baseline: ~same Memory: ✅ 65.482MB (SLO: <67.000MB -2.3%) vs baseline: +4.8% ✅ tracer-no-templatesTime: ✅ 20.292ms (SLO: <22.000ms -7.8%) vs baseline: -0.2% Memory: ✅ 65.435MB (SLO: <67.000MB -2.3%) vs baseline: +4.8% 🟡 errortrackingdjangosimple - 6/6✅ errortracking-enabled-allTime: ✅ 18.027ms (SLO: <19.850ms -9.2%) vs baseline: ~same Memory: ✅ 65.274MB (SLO: <66.500MB 🟡 -1.8%) vs baseline: +5.0% ✅ errortracking-enabled-userTime: ✅ 18.047ms (SLO: <19.400ms -7.0%) vs baseline: +0.1% Memory: ✅ 65.254MB (SLO: <66.500MB 🟡 -1.9%) vs baseline: +4.8% ✅ tracer-enabledTime: ✅ 18.082ms (SLO: <19.450ms -7.0%) vs baseline: +0.2% Memory: ✅ 65.205MB (SLO: <66.500MB 🟡 -1.9%) vs baseline: +4.7% 🟡 flasksimple - 18/18✅ appsec-getTime: ✅ 4.589ms (SLO: <4.750ms -3.4%) vs baseline: ~same Memory: ✅ 61.932MB (SLO: <65.000MB -4.7%) vs baseline: +4.8% ✅ appsec-postTime: ✅ 6.604ms (SLO: <6.750ms -2.2%) vs baseline: +0.2% Memory: ✅ 61.971MB (SLO: <65.000MB -4.7%) vs baseline: +4.7% ✅ appsec-telemetryTime: ✅ 4.590ms (SLO: <4.750ms -3.4%) vs baseline: +0.6% Memory: ✅ 62.030MB (SLO: <65.000MB -4.6%) vs baseline: +4.9% ✅ debuggerTime: ✅ 1.854ms (SLO: <2.000ms -7.3%) vs baseline: -0.3% Memory: ✅ 45.436MB (SLO: <47.000MB -3.3%) vs baseline: +4.7% ✅ iast-getTime: ✅ 1.866ms (SLO: <2.000ms -6.7%) vs baseline: -0.2% Memory: ✅ 42.369MB (SLO: <49.000MB 📉 -13.5%) vs baseline: +4.7% ✅ profilerTime: ✅ 1.907ms (SLO: <2.100ms -9.2%) vs baseline: ~same Memory: ✅ 46.439MB (SLO: <47.000MB 🟡 -1.2%) vs baseline: +4.8% ✅ resource-renamingTime: ✅ 3.370ms (SLO: <3.650ms -7.7%) vs baseline: ~same Memory: ✅ 52.296MB (SLO: <53.500MB -2.2%) vs baseline: +4.6% ✅ tracerTime: ✅ 3.366ms (SLO: <3.650ms -7.8%) vs baseline: -0.2% Memory: ✅ 52.258MB (SLO: <53.500MB -2.3%) vs baseline: +4.7% ✅ tracer-nativeTime: ✅ 3.362ms (SLO: <3.650ms -7.9%) vs baseline: +0.4% Memory: ✅ 58.257MB (SLO: <60.000MB -2.9%) vs baseline: +4.7% 🟡 otelspan - 22/22✅ add-eventTime: ✅ 40.785ms (SLO: <47.150ms 📉 -13.5%) vs baseline: -0.4% Memory: ✅ 44.111MB (SLO: <47.000MB -6.1%) vs baseline: +4.7% ✅ add-metricsTime: ✅ 317.108ms (SLO: <344.800ms -8.0%) vs baseline: -0.8% Memory: ✅ 616.948MB (SLO: <630.000MB -2.1%) vs baseline: +4.8% ✅ add-tagsTime: ✅ 289.026ms (SLO: <314.000ms -8.0%) vs baseline: +0.3% Memory: ✅ 618.336MB (SLO: <630.000MB 🟡 -1.9%) vs baseline: +4.8% ✅ get-contextTime: ✅ 80.885ms (SLO: <92.350ms 📉 -12.4%) vs baseline: ~same Memory: ✅ 39.752MB (SLO: <46.500MB 📉 -14.5%) vs baseline: +4.9% ✅ is-recordingTime: ✅ 38.229ms (SLO: <44.500ms 📉 -14.1%) vs baseline: -0.7% Memory: ✅ 43.621MB (SLO: <47.500MB -8.2%) vs baseline: +5.0% ✅ record-exceptionTime: ✅ 58.585ms (SLO: <67.650ms 📉 -13.4%) vs baseline: +0.3% Memory: ✅ 40.079MB (SLO: <47.000MB 📉 -14.7%) vs baseline: +5.0% ✅ set-statusTime: ✅ 44.883ms (SLO: <50.400ms 📉 -10.9%) vs baseline: +1.6% Memory: ✅ 43.558MB (SLO: <47.000MB -7.3%) vs baseline: +4.9% ✅ startTime: ✅ 37.610ms (SLO: <43.450ms 📉 -13.4%) vs baseline: +0.2% Memory: ✅ 43.607MB (SLO: <47.000MB -7.2%) vs baseline: +5.1% ✅ start-finishTime: ✅ 82.656ms (SLO: <88.000ms -6.1%) vs baseline: ~same Memory: ✅ 34.583MB (SLO: <46.500MB 📉 -25.6%) vs baseline: +4.9% ✅ start-finish-telemetryTime: ✅ 83.989ms (SLO: <89.000ms -5.6%) vs baseline: +0.2% Memory: ✅ 34.524MB (SLO: <46.500MB 📉 -25.8%) vs baseline: +4.7% ✅ update-nameTime: ✅ 39.294ms (SLO: <45.150ms 📉 -13.0%) vs baseline: +0.8% Memory: ✅ 43.924MB (SLO: <47.000MB -6.5%) vs baseline: +4.7% 🟡 span - 26/26✅ add-eventTime: ✅ 20.892ms (SLO: <22.500ms -7.1%) vs baseline: +1.1% Memory: ✅ 49.674MB (SLO: <53.000MB -6.3%) vs baseline: +4.8% ✅ add-metricsTime: ✅ 90.532ms (SLO: <93.500ms -3.2%) vs baseline: ~same Memory: ✅ 690.377MB (SLO: <961.000MB 📉 -28.2%) vs baseline: +4.9% ✅ add-tagsTime: ✅ 148.040ms (SLO: <155.000ms -4.5%) vs baseline: +0.6% Memory: ✅ 691.117MB (SLO: <962.500MB 📉 -28.2%) vs baseline: +5.0% ✅ get-contextTime: ✅ 19.105ms (SLO: <20.500ms -6.8%) vs baseline: +0.9% Memory: ✅ 48.415MB (SLO: <53.000MB -8.7%) vs baseline: +4.8% ✅ is-recordingTime: ✅ 19.348ms (SLO: <20.500ms -5.6%) vs baseline: +1.1% Memory: ✅ 48.365MB (SLO: <53.000MB -8.7%) vs baseline: +4.7% ✅ record-exceptionTime: ✅ 37.753ms (SLO: <40.000ms -5.6%) vs baseline: +0.8% Memory: ✅ 42.485MB (SLO: <53.000MB 📉 -19.8%) vs baseline: +4.9% ✅ set-statusTime: ✅ 20.867ms (SLO: <22.000ms -5.1%) vs baseline: +0.2% Memory: ✅ 48.379MB (SLO: <53.000MB -8.7%) vs baseline: +4.7% ✅ startTime: ✅ 18.798ms (SLO: <20.500ms -8.3%) vs baseline: +0.4% Memory: ✅ 48.431MB (SLO: <53.000MB -8.6%) vs baseline: +4.9% ✅ start-finishTime: ✅ 51.762ms (SLO: <52.500ms 🟡 -1.4%) vs baseline: +1.1% Memory: ✅ 32.086MB (SLO: <34.000MB -5.6%) vs baseline: +4.6% ✅ start-finish-telemetryTime: ✅ 53.088ms (SLO: <54.500ms -2.6%) vs baseline: +0.8% Memory: ✅ 32.165MB (SLO: <34.000MB -5.4%) vs baseline: +4.9% ✅ start-finish-traceid128Time: ✅ 55.137ms (SLO: <57.000ms -3.3%) vs baseline: +0.6% Memory: ✅ 32.126MB (SLO: <34.000MB -5.5%) vs baseline: +4.7% ✅ start-traceid128Time: ✅ 19.245ms (SLO: <22.500ms 📉 -14.5%) vs baseline: +1.1% Memory: ✅ 48.419MB (SLO: <53.000MB -8.6%) vs baseline: +5.0% ✅ update-nameTime: ✅ 19.541ms (SLO: <22.000ms 📉 -11.2%) vs baseline: +1.7% Memory: ✅ 49.099MB (SLO: <53.000MB -7.4%) vs baseline: +4.7%
|
0c40f1b
to
a9a2a70
Compare
Co-authored-by: Quinna Halim <[email protected]>
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
riotfile and release note changes lgtm
releasenotes/notes/test-openai-v2-support-49b3ca33d44ddb7c.yaml
Outdated
Show resolved
Hide resolved
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
LGTM, I defer to @brettlangdon 's judgement on the release note though!
Description
Enable OpenAI v2.x support in the test suite by adding
latest
to the riotfile configuration. Remove_v1
suffix from test files and snapshots since the tests work with both v1.x and v2.x versions.If we want to keep changes minimal, we can also not care about the suffix removal.
Testing
Test matrix now includes:
latest
(v2.x)<2.0.0
(v1.x)~=1.76.2
==1.66.0
Risks
None - purely additive change that expands test coverage to include v2.x alongside existing v1.x tests.
Additional Notes
If there is a better way to do it, lmk 👍