DeepFactor Release Notes Follow

 

DeepFactor 2.0 

Improvements

• Improved runtime performance ensures DeepFactor’s telemetry collection will consume less CPU.
• Improved Language Specific Agent (LSA) performance when collecting Java application method tracing insights.
• Race condition between deploying webhook pod and enabling admission controller fixed. (The error condition caused pod deployment failures with error “Connection refused.”)

ZAP Scan Improvements

• DeepFactor portal 2.0 installed in K8s clusters can run multiple ZAP scans (default 10) simultaneously, resource permitting.

Insights

• DeepFactor platform now reports all OS components installed in host/container images where the application is running.
• Fixed deadlock in Java agent used for language specific insights when used to instrument JBoss.
• SBOM will report all incoming URIs at the application and component level.

Installation & On-boarding

• DeepFactor Portal can now be deployed in a Kubernetes cluster. Release 2.0 has been tested for deployment in EKS, AKS and GKE.

Miscellaneous

• Fixed files upload for symbol files greater than 100 MB.
• Running docker images with dfctl run now validates dfctl version installed on the host with version in installed in docker volume. The container run will fail if the version differs.
• Components/Instance details API now reports additional container information that includes image name and SHA.

DeepFactor 1.9.4 

Improvements

• Added clean restart of init containers used to initialize pods with DeepFactor.

• Added reportiing support for all OS packages installed on host/container image.

• Fixed crash when instrumenting statically linked application with multilibc option set to “auto.”

DeepFactor 1.9.2 

Improvements

• DeepFactor’s Mutating WebHook Admission Controller for instrumenting Kubernetes application is now deployed as a non-root container. The application initialization process is also executed with non-root privileges.

• Improved initialization of DeepFactor’s runtime to avoid deadlock when instrumenting Java applications in an Alpine image.

• DeepFactor Jenkins Plugin 1.0.1 released to add DeepFactor to a Docker image, initiate a DAST scan, generate reports, and review alerts in a DevOps pipeline.

DeepFactor 1.9.1

Improvements

• UI display options to select java agent for LSA.

• The smoke test in Dockerfile downloaded from the DeepFactor portal platform to package application containers with DeepFactor instrumentation caused a build failure for Alpine applications. The smoke test in the Dockerfile is now fixed with 1.9.1 release.

• Kubernetes Admission Controller, Mutating Webhook provided by DeepFactor was referencing an older version of the DeepFactor runtime container image. The smoke test is now fixed in 1.9.1 to avoid deployment failures.

DeepFactor 1.9

Improvements

• DF CLI (dfctl) supports options to select java-agent, or JVMTI, or both agents when instrumenting Java applications.
• DeepFactor runtime reports dynamically assigned port when the application binds on port 0.
• Application BOM reports well known service names for listening ports.
• DeepFactor removes duplicate URIs from the OWASP Zap scan reports.
• DeepFactor generates an alert (PS28) if the application modifies the environment variable LD_LIBRARY_PATH.
• Added an API to get all alerts occurrences at application/component level with version and date filters.
• Integration APIs now support taking the deployment environment name while running integration tests.
• GertDockerfileEndpoint() now supports app-name and component-name.

Miscellaneous

• Live events stream and stream API now report LSA events including Java classes loaded and methods invoked.
• Improved methods to determine unique stack traces in Alert evidence.
• DeepFactor platform maintains and reports audit trails per alert.
• If Slack integration is enabled, DeepFactor will send a scan completion notification.

DeepFactor 1.8.1 Kubernetes Technology Preview

Installation and Onboarding

• DeepFactor portal 1.8.1 can now be deployed in a Kubernetes cluster (EKS).

• Previous Kubernetes cluster portal releases can be upgraded to 1.8.1 release.

• To download ZAP scan logs, users do not need to disable DAST resource clean up as noted in the previous release.

DeepFactor 1.8.1

Improvements and Fixes

Insights

• Fixed Java agent issue in 1.8 release in failure to report application usage metrics for application jars and fat jars.

  • For detailed application stack traces and usage information, users need to use the --lang java command-line option to dfctl,

• Added support for DeepFactor runtime instrumentation of applications dependent on glibc runtime in the alpine container images.

Installation and Onboarding

• Added support for DeepFactor mutating webhook for instrumenting applications deployed in Kubernetes clusters version 1.19

• Added support for DeepFactor Portal installation on EC2 instances without a public hostname / IP address.

UI Enhancements

• Application Bill Of Material now displayed in a separate tile in the application dashboard .

• Application Bill Of Material now includes an application dashboard list of listening ports.

• Users can now apply build filters at the application level.

DeepFactor 1.8

Improvements and Fixes

Integrations

• Added DeepFactor Portal support for integration with on-prem Jira sever. This allows admins/AppSec engineers to raise tickets for DeepFactor Alert mitigation.

Insights

• Added Java agent support to collect and report application’s method call usage metrics. These usage metrics are included in the Application’s BOM (Bill Of Material)

• Added support for DeepFactor alerts generated from API scan reporting in the API Security alerts category and will be displayed in the API security tile on the dashboard.

• Added DeepFactor runtime improvements for the reporting of module versions for NodeJS applications. This improves accuracy in running dependency checks on NodeJS applications.

• Added DeepFactor runtime support for fat jar file processing while performing dependency checks. This improves accuracy while running dependency checks on java applications especially spring boot applications.

Alerts Improvements

• Added support for the Alert, “Attempt to connect to IP address not previously resolved,” to now also report the FQDN, service name, and protocol for which the connection was attempted.

• Added the ability for allowing users to configure thresholds for the rate of DNS. If the application's rate of DNS lookup calls exceeds configured value, DeepFactor will raise an alert.

• Added a built-in alert policy for applications developed in interpreted languages (Java, Python, etc).

• Added policy configuration support for configuring whitelist for geo-locations.

• Added the ability for users to add comments when changing alerts severity while configuring alert policies.

Miscellaneous

• Added support for downloadable Application BOM as CSV and PDF reports.

• Added support for triaging an alert by marking it as “not an issue.”

Limitations

• Modifying the alert policy or changing the assigned alert policy to a component does not trigger alert re-computation. These changes will take effect on the next run of applications.

• Java agent fails to load with instrumented JBoss/Wildfly servers.

• The DeepFactor runtime has limited CVE reporting for applications running on SuSE, Fedora 34, and Alpine 3.14.X.

• Java agent fails to report usage metrics for application jars and fats jar.

• Using the Java agent requires adding addition a Java command line option as follows:

  1-Xbootclasspath/a:/usr/lib/df-agent.jar -javaagent:/usr/lib/df-agent.jar

 

DeepFactor 1.7.2 K8s Technology Preview

Installation & Onboarding
The DeepFactor portal deployed in a Kubernetes cluster is installed with a prepopulated OS and dependencies vulnerability database. With this change, the DeepFactor platform will report vulnerabilities for instrumented applications without waiting for database to be populated.

Insights

• Users can upload applications' debug symbols for reporting detailed stack trace as alert evidence.
• DeepFactor portal upgrades can be performed using helm commands.
• Users can also download diagnostics using K8s commands.
  

ZAP Scan Improvements

• Added capability allowing DeepFactor to run ZAP scans for multiple applications simultaneously.
• Added ZAP scan log download capability using K8s commands.

Limitations

• Portal passwords are stored in plain text in config.
• ZAP scan logs are auto cleaned as soon as the ZAP scan completes. This is true for successful as well as errored scans. To collect logs please follow instruction from: Disable DAST Resource Clean Up When K8s Debugging – DeepFactor.
• ZAP scans may get scheduled on Kubernetes nodes with limited resources. This resource limitation may cause a failure to launch a scan driver, while the web scan state is set to running state after max retries.

DeepFactor 1.7.1

Improvements and Fixes

Alerts Improvements

• Fixed UI filter for alerts that causes count mismatch for vulnerable OS Packages. This issue is seen when DeepFactor portal v1.6 is upgraded to v1.7.

Instrumentation

• Added Mutating Webhook support for instrumenting Opensuse-based workloads with DeepFactor’s runtime observability.
• Added Mutating Webhook support for specifying application language to collect additional telemetry.
• Added support for Java applications.
• Fixed error in loading language-specific agent (LSA) for apps deployed using the Mutating Webhook.

UI Enhancements

• Added UI help for GitLab integration.

DeepFactor 1.7

Improvements and Fixes

Zap Scans

• Fixed ZAP scans generating partial scan reports due to connections issue.
• Improved scan management when a user deletes applications with scans in progress.
• Fixed reporting of HTTP/HTTPS scheme in alerts generated by direct scans (ZAP scans for non-df-instrumented applications).

Insights

• Improved DF runtime stability for applications that use scripts for app initialization.
• DF CLI (dfctl) now supports using an agent (JVMTI) to collect Java app stack frame details for alert evidence. This can be specified using -l or --lang argument to dfctl with “java” set as the value.
• DF CLI (dfctl) now supports specifying alert policy to be used for analyzing app telemetry. This can be specified using -p or --policy argument to dfctl with policy name passed as value.
• DF CLI (dfctl) now supports the command verb to list all available alert policies. This enables users to choose a policy for subsequent calls. This can be specified using the command dfctl policy list.
• DF now supports reporting the instrumented application’s Runtime Bill of Material (BOM) . This report includes dependent modules, OS packages, outgoing connections, usage information, and generated DeepFactor alerts.

Alerts Improvements

• DeepFactor now processes IOCTL commands used by the instrumented application and generates relevant alerts with evidence.
• Improved pattern matching for files accessed modified to reduce potential false positives in alerts. For example, alerts for files accessed/modified in /etc should be treated differently than files accessed/modified in /opt/deepfactor/etc.
• DeepFactor now allows users to custom set alert rule severity while defining alert policies.
• Vulnerable OS package and dependency alerts are generated per occurrence and reported in separate categories.
• Vulnerable OS package alerts now also report the shared object used by the application as evidence.
• Vulnerable OS packages/dependency alerts now honor CVSS v3/v2 score set in associated alert policies.

Installation & On-boarding

• Fixed DeepFactor portal upgrade failure when deployed behind a proxy for internet access.
• DeepFactor Kubernetes mutating webhook now supports using proxy required to access.
• DeepFactor package repository. Access to the package repository is required to complete pod initialization.

UI Enhancements

• DeepFactor now enables users to generate and download PDF reports of application runtime Bill of Material (BOM) and Alerts.
• The alert summary page allows a multi-select/multi-action way to acknowledge multiple alerts at once.
• DeepFactor dashboard now reports the count of active instances of an application component.

API

• Application stats API now reports the count of processes launched by instance.
• Added API to get alert list web services.

Miscellaneous

• Heart Beat live stream now contains process name for identification and correlation of parameters.
• DF CLI is able to instrument and launch the RHEL container using dfctl run --docker-run command.

DeepFactor 1.6.3

Improvements

• Added support to running DF instrumented application on SUSE Linux Distro.

DeepFactor 1.6.2

Improvements

• Fixed crash in DeepFactor Portal CVE service.

DeepFactor 1.6.1

Improvements and Fixes

ZAP Scan Improvements

• Added ZAP scan diagnostics log downloading for all scans.

Instrumentation & Insights

• Added the ability to delete cloned/custom alert policies.
• Added the ability to live stream telemetry from the application creation timestamp.
• Improved DeepFactor runtime performance using local caching and reusing TLS connections. Customers must upgrade the deepfactor-runtime package and re-instrument their applications to implement this improvement.
• Fixed DeepFactor runtime deadlock in applications with complex initialization sequence requiring multiple initialization scripts. Customers must upgrade the deepfactor-runtime package and re-instrument their applications to implement this improvement.
• Fixed regression introduced in 1.6 release regarding CVE alert reporting for vulnerable libraries loaded by application.
• Renamed Black/White lists to Allow/Deny lists.

Alerts Improvements

• Improved alert details for MM6 “program forked without clearing sensitive memory regions.
• Improved alert policy processing rules for AppSec policy Violations (PS8, LB4, LB &, LB8).
• Added DeepFactor platform performance alert if applications have high rates of launching child processes.
• LB4 Alert “Loading dynamic library from uncommon path” disables by default in alert policy. Users need to enable it based on their application composition.
• Fixed reporting error in rate of library load per second.

Integrations

• Added support for GitLab integration.
• DeepFactor now supports complete integration of DeepFactor alerts within the GitLab dashboard.

Installation & On-boarding

• Added beta version support for DeepFactor portal deployments in Kubernetes clusters (EKS).
  • Note: This feature is currently limited to maximum capacity of the worker node running the DeepFactor backend service (WebAppSvc).
• DeepFactor portals deployed with Kubernetes supports launching ZAP scans simultaneously for Web Applications.
• DeepFactor portal can be deployed in EKS clusters using Helm Charts.

UI Enhancements

• Added application dashboard reporting across various categories.

Limitations

• Following are limitations for DeepFactor portals deployed in EKS clusters.:
• Portal passwords are stored in plain text in config.
• UI will display a set of commands to be executed by Kubernetes admins to upgrade DF portal.
• UI will display a set of commands to download ZAP scan logs for DF portals deployed in Kubernetes.
• The beta version of DF portal deployed in Kubernetes does not support the uploading of application debug symbol.
• DeepFactor portals deployed in Kubernetes clusters are not setup with the pre-populated vulnerability database. Hence, instrumented applications may not show vulnerabilities until the database is fully populated.

 

DeepFactor 1.6

Improvements and Fixes

ZAP Scan Improvements

• Added support to launch ZAP scans from a portal deployed with proxy configuration.

Insights

• Improved event push architecture to scale to billions of events from a single component instance. 

• Improved evidence reporting to make alerts actionable. In addition, for OWASP alerts, you can now generate a curl command to replay the attack for debugging.

• Fixed bug in CVE processing engine that resulted in false positive reporting of vulnerable OS dependencies. 

Installation & On-boarding

• User can request a portal license upgrade from portal UI,

UI Enhancements

• Live telemetry streaming with event summary reporting.

Limitations

• Old CVE alerts for vulnerable OS dependency may be false positive and must be ignored. Users must use correct build filters to eliminate old potentially False Positive old alert data.
• Portals configured with proxy do not support Custom ZAP scans.
• Stack traces for Alpine applications are not reported by DF runtime. However, Java application stack traces will be reported.
• The agent to collect java application stack traces needs to be configured while launching java applications. Currently dfctl does not detect java application to automatically inject java agent while launching.

 

DeepFactor 1.5.1

Improvements and Fixes

Installation and onboarding

• DF Portal can now be deployed behind a proxy for internet access

ZAP Scan improvements

• Fixed failure of first external ZAP scans 

Instrumentation Improvements

• • DF runtime install scripts now support installing a specific version of runtime packages that include setting up docker volumes 

Limitations

• • DF portal deployed with a proxy configuration may have limited support for running ZAP scans on applications

DeepFactor 1.5

Improvements and Fixes

• ZAP Scan improvements
  • DeepFactor now supports specifying authentication parameters for DAST scans
  • Users can save and reload their scan configurations
  • Users can also configure a list of URIs to be included or excluded during scans. These are configurable scan parameters.
  • DeepFactor can now launch DAST scans against non-instrumented web applications
  • All of the above functions these can be exercised via APIs
• Instrumentation Improvements
  • DeepFactor now supports launching Docker images with the DF runtime instrumentation of container images at launch time
• Insights and Alerts
  • Improved evidence reporting in the DeepFactor alerts DF platform
  • Added APIs to report differences in alerts generated between application build versions
  • Added support for Java agents to collect application stack trace as evidence for alerts
• Limitations
  • The agent to collect Java application stack traces needs to be configured while launching Java applications. Currently dfctl does not detect java application to automatically inject java agent while launching.
    
    

DeepFactor 1.4

Improvements and Fixes

• ZAP Scan Improvements
  • Upgraded ZAP version from 2.9 to 2.10
  • Optimized ZAP plugin for HTTP request and response evidence collection and reporting
  • Added OWASP alert web confidence report  
  • Added WebApp scanning using Swagger (yaml or json) document 
• Insights
  • Added reporting of CVE alerts at occurrence level
  • Added stack trace alerting for Alpine applications
  • Added DF CLI help page
• Miscellaneous Fixes
  • Added ability to add comments to alert at each occurrence level
  • Added API to create Jira tickets at alert occurrence level
  • Added Web Scan repo in diagnostics logs
  • Added API to compare alerts difference between App versions
• Known Limitations
  • Stack traces are currently not reported for Alpine applications

DeepFactor 1.3.1

Improvements and Fixes

• Insights
  • Fixed Alert count mismatch in risk scenarios tile.
  • Disabled ZAP plugin to report HTTP request and response to address high memory consumption when scanning some web applications.

• Instrumentation

  • Added Kubernetes admission webhook for DF instrumentation of applications
  • Added parsing json blob as argument to application run using dfctl run.
    dfctl run now requires absolute path of the command.

• Integrations

  • Published Orb for integrating DF into CircleCI pipeline.  
  • Added OpenAPI document publishing on portal for downloads
    
    Note: DeepFactor does not publish OVAs and AMIs for patch releases. Customers can upgrade to 1.3.1 using their portal instances.

DeepFactor 1.3

The DeepFactor 1.3 release contains the following improvements and fixes. 

Improvements and Fixes

• ZAP scan improvements
  • Reporting HTTP request and response as evidence in ZAP alerts.
  • Added ability for users to exclude URI params in scan config.
  • Added ability for users to stop running ZAP scan and collect results.
• Insights
  • Reporting of stack traces in alerts
  • Improved alert workflow
  • Alerts and Alert policy regrouping
  • Alert acknowledgement at occurrence level.
• Integrations
  • Okta integration for standard edition
  • Published REST APIs for third party integration
  • Published Orb for integrating DF into CircleCI pipeline.  
• Miscellaneous fixes
  • DF_API_TOKEN has been renamed to DF_RUN_TOKEN. DF runtime will support DF_API_TOKEN for one more release before deprecating it.
  • Fixed install-dfctl.sh scripts to not upgrade all packages on the host.
  • Added support for “Scan Strength Config” while launching ZAP scans via API
  • Added ability for admins to change password for other users. This is also supported via API
  • Reporting of name of the process as evidence for alerts on loading of library that are not part of any package.
  • Added alert page summary info at top.
  • Email verification is mandated for users signing up for accounts on DF resolute server.
  • Fixed forgot password flow on resolute server.

Limitations

• Swagger document not available as yet. This should be published soon.

DeepFactor 1.2

The DeepFactor 1.2 release contains the following improvements and fixes. 

Improvements and Fixes

• Instrumentation
  • Added new application on-boarding flow.
  • Transitioned from build-time instrumentation to runtime instrumentation by adding support for
    dfctl run instead of dfctl create and manifest-based sealed application launch.
• ZAP Scan Improvements
  • Improved evidence in ZAP scans.
  • Increased handling of HTTP response by 3XX.
  • Enhanced reporting scan progress.
  • Added active scan policy support.
  • Added more evidence for SQL injection alerts.
• Insights
  • Added stack trace alert reporting.
  • Fixed reporting CVEs for Alpine applications.
• Installation and On-boarding
  • Users can now change data TTL from admin settings.
  • Added one-command installation of dfctl.
  • portalctl now supports restart verb to restart portal services.
• Integrations
  • Improved Slack Notifications.
  • Added API support for third-party integrations (Work in Progress).
• General Fixes
  • Fixed error in reporting CVEs for some Ubuntu versions.
  • Fixed ZAP Scan proxy fills out disk.

Limitations

• Stack trace for Alpine applications are not reported.
• DeepFactor instrumented Chrome application running NodeJS has undefined behavior.
• Jenkins plugin needs upgrade after changes in supported API.
• Launching of DF instrumented postgres service on CentOS 7 fails.
• MongoDB on CentOS7 VM does not send telemetry to the backend.
• DeepFactor instrumented service telemetry reported in default runtime environment.
• DeepFactor supports registering only one component per container image (more then one component executing inside one container image is not allowed).

DeepFactor 1.1

The DeepFactor 1.1 release contains the following new features and improvements. The build version is 1.1-482.

New Features

• Added User Flow in the portal UI replacing Invite User to improve the user onboarding workflow.
• Added alert reporting by sub-category.
• Added support to revoke tokens using portalctl.
• Added ZAP Scan performance improvements for web servers that do not set content-length or transfer encoding. (Requests that depended on connection close to terminate content previously took 60 seconds to time out.)
• Added more evidence for ZAP Scan Alerts.
• Added Jenkins CI Plugin for Deepfactor container build instrumentation.

Improvements and Fixes

• Fixed issues with Dependency Check for applications using JAVA as well as NodeJS components.
• Fixed issues in telemetry reporting for DF instrumented Redis service on CentOS7 platform.
• Fixed issues in telemetry reporting for DF instrumented Httpd service on CentOS7 platform.
• Fixed user token expiration issue.

Limitations

• Launching the Deepfactor instrumented postgres service on CentOS 7 fails with the following error:

Jun 09 08:22:49 localhost.localdomain postgresql-check-db-dir[22892] : sh: rpm: command not found

Jun 09 08:22:49 localhost.localdomain pg_ctl[22907]: sh: rpm: command not found

• • WebApp Scan may launch against old/dead instances.
  • Application instrumented using dfctl create, when launched, extracts an instance of libdf under /tmp. This may cause file system to fill up over a period of time.
  • DeepFactor Instrumented HAProxy Service may experience crashes.
  • DeepFactor supports registering only one component per container image. (The service cannot have more then one component executing inside one container image.)
  • DeepFactor instrumented service telemetry gets reported in default runtime environment.
  • MongoDB on CentOS7 VM does not send telemetry to the backend.