1.30.0

Known Issues

• This release contains two critical bugs that may break applications in the following cases:
  • Some Spring Boot applications.
  • Applications with ASM enabled (DD_APPSEC_ENABLED=true).
• In both cases, it is strongly recommended to upgrade to v1.30.1 or later.

Components

Application Security Management (IAST)

• 🐛 Fix dynamic invoke for string concat in IBM J9 (#6634 - @manuel-alvarez-alvarez)
• ⚡ Move away from soft references in sources (#6631 - @manuel-alvarez-alvarez)
• Add application vulnerabilities to telemetry (#6624 - @manuel-alvarez-alvarez)
• Detect insecure auth protocol (#6614 - @jandro996)
• Fix insecure cookies false positives when unsetting cookies (#6586 - @jandro996)

Application Security Management (WAF)

• Upgrade to libddwaf 1.16.0 (libsqreen 9.0.1) (#6658 - @ValentinZakharov)
• 🧪 Exclude WAF timeout from telemetry logs (#6618 - @smola)
• Add _dd.appsec.waf.timeouts tag (#6597 - @smola)

Continuous Integration Visibility

• 🐛 Fix null test framework name in JUnit 5 instrumentation (#6684 - @nikita-tkachenko-datadog)
• Do not ignore caught exceptions in CI Visibility code (#6671 - @nikita-tkachenko-datadog)
• Implement telemetry for code coverage operations (#6666 - @nikita-tkachenko-datadog)
• Implement telemetry for shell git commands (#6665 - @nikita-tkachenko-datadog)
• Implement telemetry for CI Visibility (#6664 - @nikita-tkachenko-datadog)

Database Monitoring

• Fix context store access with null key on DBM MySQL connection (#6616 - @mcculls)

Dynamic Instrumentation

• ✨ Add class filtering and exception probe conditions (#6641 - @jpbempel)
• ✨ Add instrumentation of exception stacktraces (#6609 - @jpbempel)

Profiling

• Upgrade to ddprof 0.99.0 (#6679 - @richardstartin)

Remote Configuration

• 🐛 Fix remote config activation configuration name (#6642 - @PerfectSlayer)
• Add target service+env check prior configuration activation (#6636 - @ygree)

Telemetry

• 🐛 Fix intake telemetry client URL for EU site (#6683 - @nikita-tkachenko-datadog)
• Update telemetry intake client to use CI Visibility agentless URL, if configured (#6653 - @nikita-tkachenko-datadog)
• Do not switch to agent telemetry client on detecting agent telemetry endpoint availability when intake client is preferred (#6644 - @nikita-tkachenko-datadog)
• Add support for periodic submission of telemetry distribution series (#6643 - @nikita-tkachenko-datadog)
• Use intake telemetry client by default when running in agentless mode (#6607 - @nikita-tkachenko-datadog)

Tracer core

• Implement a mechanism for adding ad-hoc listeners to OKHTTP requests (#6660 - @nikita-tkachenko-datadog)
• Override tracing preferred service name and pin to dynamic config (#6648 - @amarziali)
• 🐛 Fixes to trace agent service rates feedback - e.g. MAX TPS support (#6628 - @dougqh)
• Switching to case insensitive matching (#6613 - @dougqh)
• ✨ Add scheduledWithFixedDelay method (#6587 - @jpbempel)

Serverless

• Use connection pool for http communications with extension. (#6417 - @purple4reina)

Instrumentations

Apache Spark instrumentation

• Flush traces when spark application is finished (#6670 - @paul-laffon-dd)

JDBC instrumentation

• Fix context store access with null key on DBM MySQL connection (#6616 - @mcculls)

All other instrumentations

• Align apache http client 5 instrumentation names (#6596 - @amarziali)

1.30.0-RC1

Warning

This is a RELEASE CANDIDATE and is NOT intended for use in production.

Known Issues

• This release contains two critical bugs that may break applications in the following cases:
  • Some Spring Boot applications.
  • Applications with ASM enabled (DD_APPSEC_ENABLED=true).
• In both cases, it is strongly recommended to upgrade to v1.30.1 or later.

Components

Application Security Management (IAST)

• 🐛 Fix dynamic invoke for string concat in IBM J9 (#6634 - @manuel-alvarez-alvarez)
• Add application vulnerabilities to telemetry (#6624 - @manuel-alvarez-alvarez)
• Detect insecure auth protocol (#6614 - @jandro996)
• Fix insecure cookies false positives when unsetting cookies (#6586 - @jandro996)

Application Security Management (WAF)

• 🧪 Exclude WAF timeout from telemetry logs (#6618 - @smola)
• Add _dd.appsec.waf.timeouts tag (#6597 - @smola)

Database Monitoring

• Fix context store access with null key on DBM MySQL connection (#6616 - @mcculls)

Dynamic Instrumentation

• ✨ Add instrumentation of exception stacktraces (#6609 - @jpbempel)

Telemetry

• Add support for periodic submission of telemetry distribution series (#6643 - @nikita-tkachenko-datadog)
• Use intake telemetry client by default when running in agentless mode (#6607 - @nikita-tkachenko-datadog)

Tracer core

• ✨ Add scheduledWithFixedDelay method (#6587 - @jpbempel)
• 🧹 Introduce concept of an InstrumenterGroup (#6575 - @mcculls)
• 🐛 Fixes to trace agent service rates feedback - e.g. MAX TPS support (#6628 - @dougqh)
• 🐛 Switching to case insensitive matching (#6613 - @dougqh)

Instrumentations

JDBC instrumentation

• Fix context store access with null key on DBM MySQL connection (#6616 - @mcculls)

All other instrumentations

• Align apache http client 5 instrumentation names (#6596 - @amarziali)

1.29.0

Known Issues

• This release contains a critical bug that may break some Spring Boot applications. It is strongly recommended to upgrade to v1.30.1 or later.

Components

Application Security Management (IAST)

• 🐛 Fix issue with cookie parser and empty cookie values (#6588 - @manuel-alvarez-alvarez)
• 🐛 Fix org.json instrumentation (bad JSONObject advice) (#6578 - @smola)
• Improve header injection redaction (#6577 - @jandro996)
• ⚡ Cache IAST metric tag values to be used in metrics and spans (#6561 - @manuel-alvarez-alvarez)
• 🧹 Get access to tag values from the top of TraceSegments (#6560 - @manuel-alvarez-alvarez)
• ⚡ Improve the performance of the IAST cookie parser (#6558 - @manuel-alvarez-alvarez)
• Fix header injection exclusions (#6550 - @jandro996)
• ⚡ Do not compute DBInfo on every statement call (#6549 - @manuel-alvarez-alvarez)
• ⚡ Reduce the memory footprint of IAST sources (#6501 - @manuel-alvarez-alvarez)
• Kafka support as IAST source (#6465 - @manuel-alvarez-alvarez)
• Add hardcoded secrets detection (#6461 - @jandro996)
• Improve IAST coverage by not relying on an active context for tainting purposes (#6094 - @manuel-alvarez-alvarez)

Application Security Management (WAF)

• 🐛 Trigger WAF request metrics only at end of request only (#6595 - @smola)
• ✨ Configure API Security configuration via remote-config (#6340 - @ValentinZakharov)

Continuous Integration Visibility

• Report retried TestNG tests as failed rather than skipped (#6592 - @nikita-tkachenko-datadog)
• Implement flaky test retries for Karate framework (#6582 - @nikita-tkachenko-datadog)
• Add support for GZIP for CI Visibility intakes and EVP endpoints in agentless/evp-proxy-V4 modes (#6569 - @nikita-tkachenko-datadog)
• Adjust log level or error messages written by repo indexer (#6556 - @nikita-tkachenko-datadog)

Dynamic Instrumentation

• ✨ Add exception debugging for span error (#6568 - @jpbempel)

GraalVM native-image

• Fix profiling on Quarkus native-image (#6562 - @jbachorik)

Profiling

• Include aas related tags in profile tag-map (#6612 - @jbachorik)
• Fix profiling on Quarkus native-image (#6562 - @jbachorik)
• Upgrade to ddprof 0.96.0 (#6553 - @richardstartin)

Remote Configuration

• Handle remote config 204 responses gracefully (#6576 - @smola)

Telemetry

• 🧪 Do not send agent API errors to telemetry logs (#6605 - @smola)
• Fix dependency resolution for nested JARs (#6603 - @smola)
• Add Datadog-Entity-Id header (#6559 - @ygree)
• 🧪 Improve content of redacted stacktraces in telemetry logs (#6557 - @smola)

Testing

• Force test failure on unhandled instrumented exception (#6546 - @amarziali)
• Add latestDepTest for Jvm 21 on CI (#6542 - @amarziali)

Tracer core

• Implement ScopedContext and Baggage API (#6608 - @mcculls)
• ✨ Adding support for tag matching of non-String types (#6591 - @dougqh)
• 🔍 Capture JVM arguments in tracer-flare (#6581 - @mcculls)
• 🐛 Avoid NumberFormatException when logging 128bit trace ID (#6570 - @amarziali)
• 🐛 Fix for missing ToJson annotation SamplingRules deserialization exception (#6548 - @ygree)
• 🧹 Refactor how we apply custom transformers (#6541 - @mcculls)
• 🔍 Add requested and completed times to flare_info.txt (#6531 - @mcculls)

Instrumentations

Apache Spark instrumentation

• 🧪 Initial flush interval for long running spans (#6540 - @paul-laffon-dd)
• 🧪 Reduce long running default flush interval to 120 seconds (#6536 - @paul-laffon-dd)
• Use spark.app.name as default service if not set by the user (#6460 - @paul-laffon-dd)
• Capture relationships between spark SQL stages (#6459 - @paul-laffon-dd)

AWS SDK instrumentation

• 🐛 Avoid NumberFormatException when logging 128bit trace ID (#6570 - @amarziali)
• Add aws.object.key tag for aws-sdk-1 (#6555 - @paul-laffon-dd)

Jetty instrumentation

• ✨ Support jetty client 10 and 11 (#6590 - @amarziali)

Kafka instrumentation

• 🐛 KafkaConsumer: avoid null key context access (#6545 - @amarziali)

All other instrumentations

• Mute tracing for subprocesses executed in by the tracer (#6584 - @smola)
• Add aerospike 8 latestDepTests (#6543 - @amarziali)
• ✨ Allow axis server span to set local root resource name (#6537 - @amarziali)
• ✨ Add component and fix cmd.exit_code tag in subprocess instrumentation (#6502 - @smola)
• Attach command execution spans to current span if it exists (#6481 - @smola)

1.28.0

Components

Application Security Management (IAST)

• ⚡🧹 Refactor http headers to reduce memory allocations (#6516 - @manuel-alvarez-alvarez)
• Add IAST exclusion for OpenTelemetry SDK package (#6511 - @smola)
• ⚡ Replace string.format calls with regular concatenation (#6510 - @manuel-alvarez-alvarez)
• Add call sites in spring ldap template (#6509 - @manuel-alvarez-alvarez)
• Add IAST exclusion for org.openjdk.jol.vm (#6490 - @smola)
• Detect servlet configuration vulnerabilities with IAST (#6300 - @jandro996)

Continuous Integration Visibility

• Enable flaky tests retries based on remote settings (#6454 - @nikita-tkachenko-datadog)
• Implement flaky tests retries for Scalatest (#6447 - @nikita-tkachenko-datadog)
• Implement flaky test retries for JUnit 4 (#6435 - @nikita-tkachenko-datadog)
• Synchronously flush test session finish event (#6434 - @nikita-tkachenko-datadog)
• Implement flaky test retries for Test NG (#6347 - @nikita-tkachenko-datadog)

Dynamic Instrumentation

• ✨ Add TriageReport for Dynamic Instrumentation (#6519 - @jpbempel)
• Support special case of empty catch block (#6507 - @jpbempel)
• Use java.io.tmpdir for classfile dump directory (#6486 - @jpbempel)
• ✨ Add @exception as local var in snapshots (#6480 - @jpbempel)
• Rename isUndefined expression to isDefined (#6470 - @jpbempel)
• Enable diagnostics on debugger track (#6462 - @jpbempel)
• 🐛 Fix issue for parsing classes from debugger.symbol (#6455 - @jpbempel)
• Enforce ordering of probe instrumentation (#6444 - @jpbempel)
• Add finally block for @duration metric (#6436 - @jpbempel)
• Fix support of string primitives for index in EL (#6429 - @jpbempel)

Metrics

• 🔍 Provide summary of JMXFetch for tracer-flare (#6439 - @mcculls)
• 🔍 Tidy up tracer health metrics (#6438 - @mcculls)
• Keep historical tracer health/span counts (#6426 - @mcculls)
• 🔍 Add tracer health and span count summaries to tracer-flare (#6416 - @mcculls)

OpenTracing

• 🧪 Introduce blackhole span (#6326 - @amarziali)

Profiling

• Update the enablement logic for allocation and liveheap (#6492 - @jbachorik)
• add operation label to endpoint events (#6463 - @richardstartin)
• ✨ Proactively cleanup dangling JFR repositories on startup (#6441 - @jbachorik)
• Update ddprof to 0.94.0 (#6433 - @jbachorik)
• Stop collecting jdk.ClassLoadingStatistics (#6538 - @richardstartin)

Telemetry

• Improve telemetry log deduplication (#6525 - @smola)
• Decouple telemetry logs from minimum log level (#6493 - @smola)
• 🐛 Fix Telemetry App-Started event retry that prevents the application from exiting. (#6451 - @ygree)

Trace context propagation

• Be consistent when checking if tasks are excluded from wrapping/injection (#6446 - @mcculls)

Tracer core

• 🔍 Rename version.txt to tracer_version.txt in tracer-flare (#6532 - @mcculls)
• 🔍 Update Java tracer flare name to be dd-java-flare-<runtimeId>-<currentTimeMillis>.zip (#6506 - @mcculls)
• 🔍 Add thread-dump to tracer-flare when triage or debug mode is enabled (#6485 - @mcculls)
• 🔍 Support delayed trigger of local triage report (#6479 - @mcculls)
• 🔍 Add Instrumenter metrics to tracer-flare (#6472 - @mcculls)
• 🧪 Introduce blackhole span (#6326 - @amarziali)

Instrumentations

Apache Spark instrumentation

• Add spark SQL plan the spark.stage spans (#6305 - @paul-laffon-dd)

JAX-RS instrumentation

• 🐛 support PATCH and other custom jaxrs annotations (#6471 - @amarziali)

JMS instrumentation

• Add auto-instrumentation for Message Driven Beans (#6216 - @am312)

Kafka instrumentation

• Added messaging.kafka.bootstrap.servers to consume spans (#6327 - @kr-igor)

OpenTelemetry instrumentation

• ✨ Add support for tracestate extraction from OpenTelemetry API (#6362 - @PerfectSlayer)

Trace annotations instrumentation

• 🧪 Introduce blackhole span (#6326 - @amarziali)

All other instrumentations

• ✨ Extract user principal on jakarta servlets 5+ (#6527 - @amarziali)
• ✨ Add missing error tags to process spans (#6489 - @smola)
• 🐛 log4j: preserve mdc when copied and injection is disabled (#6487 - @amarziali)
• Don't create a process span for agent internal threads (#6478 - @bantonsson)
• 🐛 zio: fix context propagation (#6442 - @amarziali)

1.28.0-RC3

Components

Application Security Management (IAST)

• ⚡🧹 Refactor http headers to reduce memory allocations (#6516 - @manuel-alvarez-alvarez)
• Add IAST exclusion for OpenTelemetry SDK package (#6511 - @smola)
• ⚡ Replace string.format calls with regular concatenation (#6510 - @manuel-alvarez-alvarez)
• Add call sites in spring ldap template (#6509 - @manuel-alvarez-alvarez)
• Add IAST exclusion for org.openjdk.jol.vm (#6490 - @smola)
• Fix flaky test due to usage of Thread.sleep (#6457 - @manuel-alvarez-alvarez)
• Detect servlet configuration vulnerabilities with IAST (#6300 - @jandro996)

Continuous Integration Visibility

• Enable flaky tests retries based on remote settings (#6454 - @nikita-tkachenko-datadog)
• Implement flaky tests retries for Scalatest (#6447 - @nikita-tkachenko-datadog)
• Implement flaky test retries for JUnit 4 (#6435 - @nikita-tkachenko-datadog)
• Synchronously flush test session finish event (#6434 - @nikita-tkachenko-datadog)
• Implement flaky test retries for Test NG (#6347 - @nikita-tkachenko-datadog)

Dynamic Instrumentation

• Use java.io.tmpdir for classfile dump directory (#6486 - @jpbempel)
• Rename isUndefined expression to isDefined (#6470 - @jpbempel)
• Enable diagnostics on debugger track (#6462 - @jpbempel)
• 🐛 Fix issue for parsing classes from debugger.symbol (#6455 - @jpbempel)
• Enforce ordering of probe instrumentation (#6444 - @jpbempel)
• Add finally block for @duration metric (#6436 - @jpbempel)
• Fix support of string primitives for index in EL (#6429 - @jpbempel)

Metrics

• 🔍 Provide summary of JMXFetch for tracer-flare (#6439 - @mcculls)
• 🔍 Tidy up tracer health metrics (#6438 - @mcculls)
• Keep historical tracer health/span counts (#6426 - @mcculls)
• 🔍 Add tracer health and span count summaries to tracer-flare (#6416 - @mcculls)

OpenTracing

• 🧪 R&D week: blackhole span (#6326 - @amarziali)

Profiling

• Update the enablement logic for allocation and liveheap (#6492 - @jbachorik)
• add operation label to endpoint events (#6463 - @richardstartin)
• ✨ Proactively cleanup dangling JFR repositories on startup (#6441 - @jbachorik)
• Update ddprof to 0.94.0 (#6433 - @jbachorik)

Telemetry

• Decouple telemetry logs from minimum log level (#6493 - @smola)
• 🐛 Fix Telemetry App-Started event retry that prevents the application from exiting. (#6451 - @ygree)

Trace context propagation

• Be consistent when checking if tasks are excluded from wrapping/injection (#6446 - @mcculls)

Tracer core

• 🔍 Update Java tracer flare name to be dd-java-flare-<runtimeId>-<currentTimeMillis>.zip (#6506 - @mcculls)
• 🔍 Add thread-dump to tracer-flare when triage or debug mode is enabled (#6485 - @mcculls)
• 🔍 Support delayed trigger of local triage report (#6479 - @mcculls)
• 🔍 Add Instrumenter metrics to tracer-flare (#6472 - @mcculls)
• 🧪 R&D week: blackhole span (#6326 - @amarziali)

Tracer public API

• 🧪 R&D week: blackhole span (#6326 - @amarziali)

Instrumentations

Apache Spark instrumentation

• Add spark SQL plan the spark.stage spans (#6305 - @paul-laffon-dd)

AWS SDK instrumentation

• Log object key name to spans for AWS s3 (#6311 - @hokitam)

JAX-RS instrumentation

• 🐛 support PATCH and other custom jaxrs annotations (#6471 - @amarziali)

JMS instrumentation

• Add auto-instrumentation for Message Driven Beans (#6216 - @am312)

Kafka instrumentation

• Added messaging.kafka.bootstrap.servers to consume spans (#6327 - @kr-igor)

OpenTelemetry instrumentation

• ✨ Add support for tracestate extraction from OpenTelemetry API (#6362 - @PerfectSlayer)

Trace annotations instrumentation

• 🧪 R&D week: blackhole span (#6326 - @amarziali)

All other instrumentations

• ✨ Add missing error tags to process spans (#6489 - @smola)
• 🐛 log4j: preserve mdc when copied and injection is disabled (#6487 - @amarziali)
• Don't create a process span for agent internal threads (#6478 - @bantonsson)
• 🐛 zio: fix context propagation (#6442 - @amarziali)

Other changes

• Revert "Log object key name to spans for AWS s3" (#6529 - @bm1549)

1.28.0-RC2

Components

Application Security Management (IAST)

• ⚡🧹 Refactor http headers to reduce memory allocations (#6516 - @manuel-alvarez-alvarez)
• Add IAST exclusion for OpenTelemetry SDK package (#6511 - @smola)
• ⚡ Replace string.format calls with regular concatenation (#6510 - @manuel-alvarez-alvarez)
• Add call sites in spring ldap template (#6509 - @manuel-alvarez-alvarez)
• Add IAST exclusion for org.openjdk.jol.vm (#6490 - @smola)
• Fix flaky test due to usage of Thread.sleep (#6457 - @manuel-alvarez-alvarez)
• Detect servlet configuration vulnerabilities with IAST (#6300 - @jandro996)

Continuous Integration Visibility

• Enable flaky tests retries based on remote settings (#6454 - @nikita-tkachenko-datadog)
• Implement flaky tests retries for Scalatest (#6447 - @nikita-tkachenko-datadog)
• Implement flaky test retries for JUnit 4 (#6435 - @nikita-tkachenko-datadog)
• Synchronously flush test session finish event (#6434 - @nikita-tkachenko-datadog)
• Implement flaky test retries for Test NG (#6347 - @nikita-tkachenko-datadog)

Dynamic Instrumentation

• Use java.io.tmpdir for classfile dump directory (#6486 - @jpbempel)
• Rename isUndefined expression to isDefined (#6470 - @jpbempel)
• Enable diagnostics on debugger track (#6462 - @jpbempel)
• 🐛 Fix issue for parsing classes from debugger.symbol (#6455 - @jpbempel)
• Enforce ordering of probe instrumentation (#6444 - @jpbempel)
• Add finally block for @duration metric (#6436 - @jpbempel)
• Fix support of string primitives for index in EL (#6429 - @jpbempel)

Metrics

• 🔍 Provide summary of JMXFetch for tracer-flare (#6439 - @mcculls)
• 🔍 Tidy up tracer health metrics (#6438 - @mcculls)
• Keep historical tracer health/span counts (#6426 - @mcculls)
• 🔍 Add tracer health and span count summaries to tracer-flare (#6416 - @mcculls)

OpenTracing

• 🧪 R&D week: blackhole span (#6326 - @amarziali)

Profiling

• Update the enablement logic for allocation and liveheap (#6492 - @jbachorik)
• add operation label to endpoint events (#6463 - @richardstartin)
• ✨ Proactively cleanup dangling JFR repositories on startup (#6441 - @jbachorik)
• Update ddprof to 0.94.0 (#6433 - @jbachorik)

Telemetry

• Decouple telemetry logs from minimum log level (#6493 - @smola)
• 🐛 Fix Telemetry App-Started event retry that prevents the application from exiting. (#6451 - @ygree)

Trace context propagation

• Be consistent when checking if tasks are excluded from wrapping/injection (#6446 - @mcculls)

Tracer core

• 🔍 Update Java tracer flare name to be dd-java-flare-<runtimeId>-<currentTimeMillis>.zip (#6506 - @mcculls)
• 🔍 Add thread-dump to tracer-flare when triage or debug mode is enabled (#6485 - @mcculls)
• 🔍 Support delayed trigger of local triage report (#6479 - @mcculls)
• 🔍 Add Instrumenter metrics to tracer-flare (#6472 - @mcculls)
• 🧪 R&D week: blackhole span (#6326 - @amarziali)

Tracer public API

• 🧪 R&D week: blackhole span (#6326 - @amarziali)

Instrumentations

Apache Spark instrumentation

• Add spark SQL plan the spark.stage spans (#6305 - @paul-laffon-dd)

AWS SDK instrumentation

• Log object key name to spans for AWS s3 (#6311 - @hokitam)

JAX-RS instrumentation

• 🐛 support PATCH and other custom jaxrs annotations (#6471 - @amarziali)

JMS instrumentation

• Add auto-instrumentation for Message Driven Beans (#6216 - @am312)

Kafka instrumentation

• Added messaging.kafka.bootstrap.servers to consume spans (#6327 - @kr-igor)

OpenTelemetry instrumentation

• ✨ Add support for tracestate extraction from OpenTelemetry API (#6362 - @PerfectSlayer)

Trace annotations instrumentation

• 🧪 R&D week: blackhole span (#6326 - @amarziali)

All other instrumentations

• ✨ Add missing error tags to process spans (#6489 - @smola)
• 🐛 log4j: preserve mdc when copied and injection is disabled (#6487 - @amarziali)
• Don't create a process span for agent internal threads (#6478 - @bantonsson)
• 🐛 zio: fix context propagation (#6442 - @amarziali)

1.27.0

Components

Application Security Management (IAST)

• Add time and bucket size boundaries to the tainted map (#6389 - @manuel-alvarez-alvarez)
• Add Header injection vulnerability detection (#6299 - @DDJavierSantos)
• Add support for names in jackson propagation (#6132 - @manuel-alvarez-alvarez)
• Add support for SSRF in Apache HttpClient V4 (#6112 - @jandro996)

Continuous Integration Visibility

• ⚡ Optimise per-test code coverage for multi-threaded tests (#6412 - @nikita-tkachenko-datadog)
• Avoid submitting multiple identical coverage reports when test span has children that are top-level spans (#6411 - @nikita-tkachenko-datadog)

Data Streams Monitoring

• ✨ Track Kafka Avro schemas using Data Streams (#6370 - @piochelepiotr)
• Add Data Streams Monitoring support to SQS v1 (#6259 - @vandonr)

Database Monitoring

• 🐛 Fix context store access with null key on DBM connection (#6400 - @amarziali)

Dynamic Instrumentation

• ✨ Add instanceof operator in Expression Language (#6425 - @jpbempel)
• Remove retransformation on failed resolution (#6423 - @jpbempel)
• Introduce special toString for well known classes (#6419 - @jpbempel)
• ✨ Add specific support for Optional (#6406 - @jpbempel)
• ✨ Add emitting status for probes (#6365 - @jpbempel)

Metrics

• Introduce more local counters for tracer health metrics (#6424 - @mcculls)
• Update JMXFetch to 0.49.0 (#6397 - @mcculls)

Profiling

• Add a number of 'new' JFR events to be enabled by default (#6430 - @jbachorik)
• Capture the SELinux status as a profiler setting event (#6414 - @jbachorik)
• Auto-generate native-image argument to enable JFR when profiling enabled (#6369 - @jbachorik)

Telemetry

• Include dynamic_instrumentation product into the app_started telemetry event (#6422 - @ygree)
• Add install_signature to app-started (#6418 - @ygree)

Tracer core

• Add new javaassist proxy separator to ignores (#6413 - @amarziali)
• 🔍 Report instrumenter state in tracer-flare (#6387 - @mcculls)

Instrumentations

JDBC instrumentation

• 🐛 Fix context store access with null key on DBM connection (#6400 - @amarziali)

OpenTelemetry instrumentation

• ✨ Use OpenTelemetry SDK context key instead of Datadog proprietary one for local root span (#6399 - @PerfectSlayer)

All other instrumentations

• Add instrumentation to allow sending kafka payload size on produce (#6228 - @vandonr)

Other changes

• Bump integrations-core submodule to 7.50.0 (#6396 - @mcculls)
• Bump byte-buddy to 1.14.9 (#6395 - @mcculls)

1.26.1

Components

Continuous Integration Visibility

• 🐛 Update repo indexer to not follow symlinks that point inside the repo (#6401 - @nikita-tkachenko-datadog)

Trace context propagation

• Rate limit Datadog tags decoding warnings (#6385 - @bantonsson)

Instrumentations

• MongoDB 4.10-11 instrumentation fix (#6358 - @ygree)

1.26.0

Potentially Breaking Changes

Warning

Allow comma in request header tag values (#6376 - @bantonsson)

Components

Application Security Management (IAST)

• Match tainted objects with sources when checking unbounded vulnerabilities (#6122 - @manuel-alvarez-alvarez)

Application Security Management (WAF)

• Update ASM event rules to v1.10.0 (#6349 - @smola)
• Remove warning about unknown graphql.server.all_resolvers (#6344 - @smola)

Continuous Integration Visibility

• Implement custom code coverage mechanism for Intelligent Test Runner (#6390 - @nikita-tkachenko-datadog)
• Ignore configuration blocks when comparing skippable test identifiers (#6379 - @nikita-tkachenko-datadog)
• ⚡ Reduce per-test code coverage overhead (#6378 - @nikita-tkachenko-datadog)
• 🐛 Ensure module name is consistent between test.module, test.suite and test spans when build system is instrumented (#6354 - @nikita-tkachenko-datadog)
• Implement flaky tests retry for Cucumber JUnit 5 (#6339 - @nikita-tkachenko-datadog)
• Implement flaky tests retry for Spock (#6338 - @nikita-tkachenko-datadog)
• Implement flaky test retry for JUnit 5.3 (#6332 - @nikita-tkachenko-datadog)
• Implement Scalatest support (#6331 - @nikita-tkachenko-datadog)
• Added test suite level visibility for JUnit 3.8 test cases (#6320 - @nikita-tkachenko-datadog)
• 🐛 Fix source path detection in multi-project repos (#6318 - @nikita-tkachenko-datadog)
• 🐛 Fix root coverage packages calculation in large repositories (#6317 - @nikita-tkachenko-datadog)

Crash tracking

• cleanup crashtracking (#6363 - @richardstartin)

Data Streams Monitoring

• Add BP trigger for DSM Kafka Benchmark (#6173 - @ddyurchenko)
• Configurable DSM bucket duration (#6171 - @randomanderson)
• Extract Kafka cluster ID as tag for DSM (#6044 - @hokitam)

Database Monitoring

• 🐛 Append DBM trace context comments for SQL Server & Postgres (#6214 - @jmeunier28)

Dynamic Instrumentation

• Accept any config id from Remote Config (#6384 - @jpbempel)
• Disable debugger track while the UI is not ready (#6367 - @jpbempel)
• Add support of Java records (#6355 - @jpbempel)
• ✨ Send probe statuses to new debugger track (#6276 - @jpbempel)

GraalVM native-image

• Enable TimelineEvent in GraalVM native image (#6368 - @jbachorik)
• Have JFR setting event for native-image recording (#6348 - @jbachorik)

Profiling

• JFR stackdepth setting (#6335 - @jbachorik)

Trace context propagation

• ⚠️ Allow comma in request header tag values (#6376 - @bantonsson)

Tracer core

• 🐛 Fix span links serialization as tag (#6341 - @PerfectSlayer)
• Tag-based sampling rules (#6298 - @ygree)

Instrumentations

Apache Spark instrumentation

• Enforce new ingestion reason for spark traces (#6310 - @paul-laffon-dd)
• Add field parent_stage_ids to spark.stage spans (#6309 - @paul-laffon-dd)

Armeria Instrumentation

• 🐛 Armeria Jetty: ensure spans are finished (#6291 - @amarziali)

JAX-RS instrumentation

• 🐛 Make propagation happen on AsyncResponse using CFX JAX-RS (#6313 - @amarziali)

Spring instrumentation

• 🐛 Exclude spring-kafka runnables from spring scheduling wrapping (#6388 - @amarziali)
• 🐛 Make webflux compatible with latest spring 6 (#6352 - @amarziali)

Eclipse Vert.x instrumentation

• Add Vert.x 4 Redis client instrumentation (#6233) -- thanks @akshaypatidar1999 for the contribution!

Other changes

• 🔍 Populate tracer-flare zip with configuration details (#6359 - @mcculls)

1.25.1

Note

There will be no 1.25.0 release

Components

Application Security Management (IAST)

• 🐛 Fix config DD_IAST_TRUNCATION_MAX_VALUE_LENGTH value typo (#6303 - @jandro996)
• Add IAST support for org.json (#6165 - @DDJavierSantos)
• IAST support for Spring with gson (#6123 - @jandro996)
• IAST support for commons fileupload (#6089 - @DDJavierSantos)

Application Security Management (WAF)

• 🐛 Exclude OAuth from Spring Security instrumentation (#6278 - @smola)
• 🐛 Fix race in Spring Security instrumentation (hideUserNotFoundExceptions) (#6277 - @smola)

Build & Tooling

• Initial connection work for tracer-flare (#6133 - @mcculls)

Continuous Integration Visibility

• Skip Git data upload if backend already knows of the available latest commits (#6301 - @nikita-tkachenko-datadog)
• Reduce ITR overhead on default branches (#6296 - @nikita-tkachenko-datadog)
• Allow Jacoco auto-instrumentation even when ITR is disabled (#6294 - @nikita-tkachenko-datadog)
• Remove user info from SSH git repo urls (#6275 - @nikita-tkachenko-datadog)
• 🐛 Include feature URI in test suite name in Cucumber instrumentations (#6274 - @nikita-tkachenko-datadog)
• Ensure test.framework and test.framework_version with multi-values have fixed order (#6271 - @nikita-tkachenko-datadog)
• Propagate ITR tags from module to session in headless mode (#6269 - @nikita-tkachenko-datadog)
• Implement coverage for Cucumber .feature files (#6251 - @nikita-tkachenko-datadog)
• 🐛 Fix conflicting CI Visibility instrumentation names (#6197 - @nikita-tkachenko-datadog)
• 🐛 Fix root coverage packages calculation in large repositories (#6317 - @nikita-tkachenko-datadog)

Data Streams Monitoring

• Add DSM for google pubsub (#6156 - @amarziali)
• ✨ Improve KStreams support (#5697 - @kr-igor)

Database Monitoring

• Disable DBM full propagation for oracle & sql server (#6117 - @vandonr)

Dynamic Instrumentation

• 🐛 Fix inner class signature matching (#6270 - @jpbempel)
• ✨ Add support of @exception for uncaught exception (#6264 - @jpbempel)
• Handle WellKnown classes as primitives in EL (#6263 - @jpbempel)
• ✨ Add Remote Config based enablement for Symbol DB (#6252 - @jpbempel)

GraalVM native-image

• Support profiler in graalvm native image applications (#6145 - @jbachorik)
• 📖 Add GraalVM native image build instructions (#6265 - @ygree)

Profiling

• Upgrade ddprof to 0.90.0 (#6280 - @richardstartin)
• Add option to collect heap histogram after GC or periodically (#6240 - @richardstartin)
• Disable ddprof by default on GraalVM JIT (#6237 - @jbachorik)
• Surface cause when DD profiler can't be instantiated (#6223 - @richardstartin)
• Support profiler in graalvm native image applications (#6145 - @jbachorik)
• JFR based profiling context (#6261 - @richardstartin)

Remote Configuration

• Add new sampling rules criteria and support for dynamic config (#5164 - @PerfectSlayer)

Telemetry

• Report internal errors/exceptions via telemetry (#6057 - @ValentinZakharov)

Tracer core

• Add new sampling rules criteria and support for dynamic config (#5164 - @PerfectSlayer)
• Adds resource name matching to the trace sampling rules (#5900 - @dougqh)

Tracer internal logging

• Adds Tinylog instrumentation (#6267 - @zarirhamza)
• Support remote logging over logback's SocketAppender (#6229 - @mcculls)

Instrumentations

AWS Lambda instrumentation

• Add span error metadata to Datadog Lambda Extension request headers (#6061 - @DylanLovesCoffee)

Netty instrumentation

• 🐛 Add support for netty-reactor http2 client/server (#6226 - @amarziali)

OpenTelemetry instrumentation

• 🐛⚠️ Store OpenTelemetry span kind as tag instead of span type and drop reserved attributes from span tags (#6239 - @PerfectSlayer)

Reactor instrumentation

• 🐛 Add support for netty-reactor http2 client/server (#6226 - @amarziali)

Spring instrumentation

• 🐛 Spring scheduling: have actuator display the right task class name (#6249 - @amarziali)

All other instrumentations

• Enable google-pubsub by default (#6225 - @amarziali)
• Add DSM for google pubsub (#6156 - @amarziali)

Other changes

• Upgrade jnr-unixsocket to 0.38.21, jnr-posix to 3.1.18 (#6266 - @mcculls)
• 🔍 Configuration changes to support triage mode (#6262 - @mcculls)