Sysdig | Daniella Pontes https://sysdig.com/blog/author/daniella-pontes/ Mon, 29 Apr 2024 22:56:15 +0000 en-US hourly 1 https://wordpress.org/?v=6.6.1 https://sysdig.com/wp-content/uploads/favicon-150x150.png Sysdig | Daniella Pontes https://sysdig.com/blog/author/daniella-pontes/ 32 32 Using Sysdig Secure to Detect and Prioritize Mitigation of CVE 2022-3602 & CVE 2022-3786: OpenSSL 3.0.7 https://sysdig.com/blog/sysdig-prepare-november-2022-openssl-vulnerability/ Thu, 10 Nov 2022 11:54:25 +0000 https://sysdig.com/?p=59574 This is an up-date to the blog post “Using Sysdig Secure to Prepare for the November 2022 OpenSSL Vulnerability”. We...

The post Using Sysdig Secure to Detect and Prioritize Mitigation of CVE 2022-3602 & CVE 2022-3786: OpenSSL 3.0.7 appeared first on Sysdig.

]]>
This is an up-date to the blog post “Using Sysdig Secure to Prepare for the November 2022 OpenSSL Vulnerability”. We have included a new section to cover how to detect and prioritize mitigation of the newly released OpenSSL 3.0.7 patch.


For more information about the OpenSSL vulnerabilities fix, see our blog posts from the Sysdig Threat Research Team 5 Steps to Stop the Latest OpenSSL Vulnerabilities: CVE-2022-3602, CVE-2022-3786 and How the Critical OpenSSL Vulnerability may affect Popular Container Images.

The awaited OpenSSL 3.0.7 patch was released on Nov. 1. The OpenSSL Project team announced two HIGH severity vulnerabilities (CVE-2022-3602, CVE-2022-3786), which affect all OpenSSL v3 versions up to 3.0.6. These vulnerabilities are remediated in version 3.0.7, which was released Nov. 1. The vulnerabilities fixed include two stack-based buffer overflows in the name constraint checking portion of X.509 certificate verification.

See below how you can use Sysdig Secure to look for the vulnerable container images using the reporting capabilities and how to prioritize mitigation by identifying containers with openssl vulnerability CVE-2022-3602 and CVE-2022-3786 in active packages at runtime.

How to detect CVE-2022-3602 &amp CVE-2022-3786 using Sysdig Secure

Let’s see how easy it is to find containers impacted by CVE-2022-3602 and CVE-2022-3786 (OpenSSL 3.0.7 patch) with Sysdig Secure.

On Sysdig Secure Home view, in the ToDo Recommendation box on the bottom left, under Sysdig Alerts tile, you will find the recommendation “How are you affected by the OpenSSL 3 vulnerability?.” Select it to see details and click on the “Generate Report” box in the top right corner to open a new report window. See the screenshot below:

OpenSSL Vulnerability


Alternatively, you can also generate a new report by selecting on the side menu Vulnerabilities > Reporting (see screenshot)

OpenSSL Vulnerability

Then, select the “Add report” button.

On the New Report view, fill the fields as follows to generate a report of container images impacted by CVE-2022-3602 and CVE-2022-3786 :

OpenSSL Vulnerabiilityx

The filtering is done for the container images that have the vulnerability IDs CVE-2022-3602 and CVE-2022-3786, and it is executed against all the infrastructure.

You can customize the scheduled frequency, set it to daily, and select preferred choice for notification (email, slack, webhook).

The report can also be manually generated by selecting the report in the “Vulnerability” -> “Reporting” section and pressing the “Generate now” button:

OpenSSL Vulnerabiilityx

After a few seconds (depending on your environment), the report will be ready to download.

The CSV file contains all the available information including the image name, version, and Kubernetes context (K8S cluster name, K8S namespace name, K8S workload type, K8S workload name & K8S container name).

The CSV file can be opened with LibreOffice for example.

Using the legacy engine

If using the legacy engine, the steps are quite similar (see the official documentation for more information). The report is created in the “Vulnerabilities” -> “Scheduled reports” section, and then specify the vulnerability ID in the data field as:

OpenSSL Vulnerability

Prioritize Mitigation of CVE-2022-3602 & CVE-2022-3786 Using Sysdig Secure Risk Spotlight

Only vulnerabilities that are tied to packages used at runtime offer a real chance of exploitation. Risk Spotlight uses deep visibility into containers to identify and prioritize vulnerable packages loaded at runtime. These are the ones that should have your immediate attention for mitigation.

Risk Spotlight prioritization is straight forward. From the side menu, choose “Vulnerabilities > Runtime” to see all your containers in production. Next, select the container of interest.

OpenSSL Vulnerability

Select the “Vulnerabilities” tab. Search for the vulnerability IDs CVE-2022-3602 and CVE-2022-3786. Filter packages used at runtime by selecting the “in-use” filter on the top menu.

OpenSSL Vulnerability

By knowing what is exposed and what isn’t, Risk Spotlight removes the noise and guesswork so your team can focus on really important issues that can’t wait.

To know more about Risk Spotlight check our blog Eliminate noise and prioritize the vulnerabilities that really matter with Risk Spotlight.



The following is our original article posted on October 29, 2022 “Using Sysdig Secure to Prepare for the November 2022 OpenSSL Vulnerability”.

A critical vulnerability with an expected high or critical severity rate of CVSS score is about to be announced on Nov. 1 on the OpenSSL project. There are still no details besides an announcement on the OpenSSL mailing list on Oct. 25, that says:

Hello,
The OpenSSL project team would like to announce the forthcoming release
of OpenSSL version 3.0.7.
This release will be made available on Tuesday 1st November 2022 between
1300-1700 UTC.
OpenSSL 3.0.7 is a security-fix release. The highest severity issue
fixed in this release is CRITICAL:
https://www.openssl.org/policies/general/security-policy.html
Yours
The OpenSSL Project Team

Given the criticality rating of the vulnerability, the commonality of OpenSSL, and prior history of broadly impacting OpenSSL vulnerabilities, it is a good idea to be prepared to update your software as soon as possible.

It is not clear if it will affect just the OpenSSL binary or the OpenSSL libraries as well, and may also be nested or transitive within other dependencies. So, expect more instances of it than what’s immediately obvious. To make things more complicated, different Linux distributions use different names for the OpenSSL libraries, such as libssl for Debian-based distributions or openssl-libs for RPM-based ones.

This article provides guidance on how you can use Sysdig Secure to generate an inventory of container images that contain OpenSSL within your environment that can be useful for executive reporting or prioritizing and aligning the teams for the inevitable remediation work.

Preparation work

The CVE ID hasn’t been disclosed yet, however, you can use the reporting capabilities to look for specific vulnerable packages and versions in your environment.

As mentioned before, different Linux distributions use different names for the OpenSSL libraries, such as libssl for Debian-based distributions or openssl-libs for RPM-based ones. To cover all the cases, you should generate two different reports: one for the package names that contains “openssl” and another for the package names that starts with “libssl.” See the official documentation for more information on how to create reports.

Let’s create a report to look for OpenSSL packages that can be affected (version 3) by going to “Vulnerability” -> “Reporting”

OpenSSL Vulnerability

Then select the “Add a report” button:

OpenSSL Vulnerability

The report we are looking for is as follows:

OpenSSL Vulnerability

We want to use the conditions to show the container images that have a package name that starts with OpenSSL and a package version that starts with 3. You can customize the scheduled frequency, set it to daily, and select preferred choice for notification (email, slack, webhook).


As explained before, the vulnerability only affects versions 3.0.X

Then, we can press the ‘Preview’ button to show a preview of the report.

The report can also be manually generated by selecting the report in the “Vulnerability” -> “Reporting” section and pressing the “Generate now” button:

OpenSSL Vulnerability

After a few seconds (depending on your environment), the report will be ready to download:

OpenSSL Vulnerability

The CSV file can be opened with LibreOffice for example to get all the details.

OpenSSL Vulnerability

Using the legacy engine

If using the legacy engine, the steps are quite similar (see the official documentation for more information). We should prepare similar reports, one for the package name “openssl” and another one for the package name “libssl.”

This time, the report is created in the “Vulnerabilities” -> “Scheduled reports”

OpenSSL Vulnerability

Then select the “Add a report” button:

OpenSSL Vulnerability

Fill in the data as previously explained, but this time, the condition is to filter the images that contain the package named “OpenSSL.” Here are the three simple steps to apply:

  • Create report
OpenSSL Vulnerability
  • Add condition (package name only, the package version can be filtered in the CSV/JSON file directly):
OpenSSL Vulnerability
  • Preview and Save
OpenSSL Vulnerability

Once scheduled and executed, it will be sent to the email you configured and be available for download.

In this case, it is a CSV file (it can be a JSON file) that contains all the available information, including the image name, version, and Kubernetes context (K8S cluster name, K8S namespace name, K8S workload type, K8S workload name & K8S container name).

The CSV file can be opened with LibreOffice, for example, to get all the details and filter by the package version desired:

OpenSSL Vulnerability
OpenSSL Vulnerability

We will keep the content updated as more information becomes available.

The post Using Sysdig Secure to Detect and Prioritize Mitigation of CVE 2022-3602 & CVE 2022-3786: OpenSSL 3.0.7 appeared first on Sysdig.

]]>
Detect cryptojacking with Sysdig’s high-precision machine learning https://sysdig.com/blog/detect-cryptojacking-with-sysdig/ Wed, 10 Aug 2022 12:55:59 +0000 https://sysdig.com/?p=52905 Is cryptojacking draining your resources and exposing your organization to financial and reputation damage risk? The rise in cryptojacking, which...

The post Detect cryptojacking with Sysdig’s high-precision machine learning appeared first on Sysdig.

]]>
Is cryptojacking draining your resources and exposing your organization to financial and reputation damage risk?

The rise in cryptojacking, which is an illegal form of mining cryptocurrency by the unauthorized use of someone’s computing resources, has reached alarming levels. According to the Google Threat Horizon report, 86% of compromised cloud instances in 2021 were used for cryptomining. That paints the picture quite clearly. So, going back to the opening question, it is wise to consider the possibility.

Cloud environments are attractive targets

Cloud environments are especially attractive to cryptojacking. They offer on-demand, auto-scaling resources for these computation-hungry invaders. While organizations struggle to implement effective security in the cloud due to the lack of visibility and control into what is going on with their cloud security settings, Kubernetes clusters, and inside their containers, threat groups like TeamTNT, WatchDog, Kinsing, among others, have been taking full advantage of this opportunity window. They have been feasting on cloud resources by exploiting cloud weak security controls, vulnerabilities in container workloads, as well as Kubernetes and Docker attack vectors.

Despite their prevalence in today’s cyber threat activity, cryptojackers are not easily detected. There are thousands of variations of cryptominers and they operate in stealth mode. Many organizations live with this kind of parasite oblivious of the danger that they truly represent. And although perceived as less dangerous than other types of attacks like ransomware, the risk incurred by cryptojackers goes way beyond scary unexpected six-figure bills.

Hefty bills are not the whole story

Cryptojackers can provide an entry point to the entire cyber threat landscape. While hijacking CPU and GPU resources, they also operate by disabling detection mechanisms facilitating the download and installation of other malware payloads, such as ransomware. The more sophisticated ones can look for credentials and SSH keys, spread to other systems like worms, and establish backdoors and connections to C2 sites, among other malicious activities. Attacks today have become multi-purpose, so waiting for the first spike in cost or application performance complaints to arrive to look for cryptominers in your environment, is late, more than you think. Time is of the essence.

Using machine learning to early detect cryptominers with 99% precision

Cryptominers are masters of evasion, that is their trade trait. But, as cryptominers, they have to fulfill certain operations necessary for mining coins, and that makes them recognizable from a behavioral perspective. By analyzing the behavior features of cryptominers, it is possible to detect even new variations of cryptominers throttling the use of cloud resources to stay under the consumption radar.

For this type of detection, which requires multiple data sources and the analysis of complex condition combinations to find a behavior match, machine learning offers the best approach. But, can all machine learning-based threat detection be effective to detect cryptominers? The short and truthful answer is no.

There is a lot of misleading information when it comes to machine learning. Anna Belak explains quite well in her article, The Beautiful Lies of Machine Learning in Security, how machine learning has been misrepresented as a security solution. But Anna also shows the benefits that machine learning can deliver when applied with focus.

How Sysdig does machine learning for cryptojacking detection

Sysdig’s machine learning solution to detect cryptominers stands out from other approaches to cybersecurity. Thus far, machine learning has been used to detect anomalies as an indicator of a potential threat. But, without a specific targeted use case, this leaves a large margin for false positives and also for missing more advanced stealthy threats, like cryptominers.

malware detection machine learning Sysdig

Sysdig solution is based on an ML model trained to recognize the anatomy of cryptominers from process activity in running containers. Sysdig uses deep visibility into containers at runtime to collect the necessary type data to be able to identify cryptominers behavior.

To know more about how Sysdig does it, check the blog Cryptominer Detection: A Machine Learning Approach by Flavio Mutti, Senior Machine Learning Engineer at Sysdig.

Enabling cryptominer detection with Sysdig Secure is very easy:

  • On the Sysdig Secure platform, go to the Runtime Policies.
  • Select Machine Learning Policy.
  • Enable cryptominer detection in your environments.
Create policy Sysdig Secure malware detection machine learning

Once the policy is enabled, the Sysdig machine learning pipeline will start detecting crytominers in running containers.

Sysdig Machine Learning feature detect malware

Real use case: Stealth cryptominer detection

On November 11, 2022, Sysdig received an alert that the machine learning miner detection system detected a potential threat. Our prior experiences indicate that classic miners usually provide us with a confidence probability above 96% and while it is still considered high confidence, this suspicious process alert had a probability of only 81%. The name of the suspicious process was associated with kernel activity workers. This was the first time a kworker process was detected as anomalous, so we decided to investigate further to determine if this was legitimate malicious activity or a false positive. 

The alerted process was in fact a miner, it was even classified as such on VirusTotal! The toolchain within the process was quite complex, involving tactics from reconnaissance to defense evasion. While machine learning can provide the first evidence of unusual activity, this case also reiterated the importance of managing machine learning and providing human oversight to improve automated efforts.

This event proved useful in demonstrating the effectiveness of machine learning to detect malicious activity as a complement to traditional rule-based detectors.

Key Benefits of Sysdig’s Machine Learning for Cryptojacking Detection

  • Detect cryptominers with 99% precision and enable a streamlined workflow to counter cryptojacking activities efficiently.
  • Prevent unexpected costs, reputation damage, and exposure to additional malware by early detecting the cryptojacking attack.
  • Strengthen security with a multi-layered approach to cloud detection and response by implementing rule-based detection and machine learning protection layers.

Conclusion

Sysdig took the approach of developing a machine learning solution that focuses on detecting cryptominers. And by doing so, it provides an effective detection mechanism with precision of up to 99%. The gains from a focused machine learning solution are multifold, from enabling fast detection and response to cryptojacking attacks to drastically reducing the number of false positives, which is very welcomed by security teams already stretched thin and dealing with alert overload and fatigue.

But machine learning is just one defense layer in Sysdig’s multi-layered detection approach. Sysdig’s rules-based approach based on Falco provides easily customizable out-of-the-box policies curated by the Sysdig Threat Research Team to maximize coverage. Other defense techniques, such as profiling, comprehensive indicators of compromise (IOCs), and Drift Control further strengthen security.

The post Detect cryptojacking with Sysdig’s high-precision machine learning appeared first on Sysdig.

]]>
Preventing container runtime attacks with Sysdig’s Drift Control https://sysdig.com/blog/preventing-runtime-attacks-drift-control/ Tue, 28 Jun 2022 13:30:46 +0000 https://sysdig.com/?p=51507 Containers revolutionized how we build, deploy, and run applications with increased speed, agility, and scalability. But, as often happens with...

The post Preventing container runtime attacks with Sysdig’s Drift Control appeared first on Sysdig.

]]>
Containers revolutionized how we build, deploy, and run applications with increased speed, agility, and scalability. But, as often happens with transformative technologies, they require an evolution to security strategy. Centralized deployments inside a protected perimeter gave way to continuous and distributed deployment of containers, creating a growing, dynamic, and distributed attack surface. IT and security teams were left blind and exposed in the cloud.

Adding to the pressure, reports such as Verizon’s 2022 Data Breach Investigation Report (BIRD) show that threat activity is increasing and targeting everyone. The growth of supply chain and third-party relationships used as attack vectors, a thriving market of stolen credentials, the rise of RaaS (Ransomware as a Service), and increase in the number of critical vulnerabilities uncovered, including Log4j and Spring4Shell, are a reminder that threat detection is critical. It is wise to consider a breach and protect your runtime environment from common and emerging threats, such as malware, C&C, backboor, and cryptominers, among others.

The challenges faced when securing container environments start with complexity and the lack of real-time visibility into what is going on inside containers. But cloud-native’s immutability principle can be a great asset to security teams. Immutability means that a container won’t be modified during its life: no updates, no patches, no configuration changes. If a change happens, it is considered a drift, and possibly a sign of an attack. Unfortunately, container drift can also be caused by legacy practices when IT administrators provide maintenance on running containers, blending their actions with the ones of adversaries. Threat actors and malware, as well as the signs of their attack, can then hide in the activity noise and stay undetected while executing their malicious codes.

However, with Sysdig’s Drift Control, teams have an easy way to detect, prevent, and speed incident response for containers that were modified in production to run new executables, also known as container drift. By blocking the drift, you prevent the attack.

Preventing attacks and legacy practices that increase risk

Attacks often include running adversary code on the victim’s machines. Following the initial access, actors try to run scripts or malware with embedded executables to continue the attack, which could include enabling remote command and control (C2), making data unavailable for ransom, launching other attacks, among other malicious actions. Being able to identify which code doesn’t belong is fundamental to thwart attacks before they can cause damage, and that is the job of Drift Control. It detects what is new or modified and blocks from execution.

Drift Control provides an easy way to prevent attacks at runtime by simply following security best practices of immutability and ensuring containers aren’t modified after deployment in production. It also helps organizations move away from legacy practices that don’t work in cloud-native environments, such as changing an application in production, downloading new packages, and installing IT tools for local maintenance. These practices increase risk by augmenting the attack surface that could be exploited and add to the noise by hiding similar actions, although malicious.

By detecting attempts to modify container executables in production and ensuring that those attempts are not successful, it forces operations to follow best practices and stops risky legacy practices from being carried over to cloud environments. Best practice calls for respecting the immutability principle, which means that you only fix at the source. This requires building a new container for deployment. Drift Control helps organizations put an end to ad hoc modifications, driving consistency from source to run and preventing actions that could be part of an attack.

How Drift Control works

Drift Control detects and prevents execution of executable files that were added or modified after a container is deployed into production. It uses real-time deep visibility into running containers to automatically identify those spurious executables.

Drift Control is a simple runtime security policy that can be quickly applied to the entire environment:

It can be enabled in detection mode to alert on attempts to run packages or binary files that were added or modified at runtime, such as:

  • Execute a package that was downloaded or updated with package manager
  • Execute a file whose permission/attribute has been changed to executable

And if in prevention mode, Drift Control blocks those detected new executables from running.

Key benefits delivered by Drift Control

  • Prevent attacks by blocking container drift in production: Drift Control automatically flags and denies deviations from the original container, blocking malicious executables before damage is done.
  • Enforce immutability best practice: Drift Control ensures that container software is not modified during its lifetime, driving good practices, consistency from source to run, and preventing actions that could be part of an attack.
  • Enable easy and effective security: Teams are often overwhelmed by cloud-native complexity and blind to container drift, especially at scale. With Drift Control, security teams and IT can just enable it on the entire container environments and immediately start protecting runtime.

The post Preventing container runtime attacks with Sysdig’s Drift Control appeared first on Sysdig.

]]>
Ten considerations for securing cloud and containers https://sysdig.com/blog/considerations-securing-cloud-containers/ Thu, 05 May 2022 15:00:30 +0000 https://sysdig.com/?p=48721&preview=true&preview_id=48721 Most organizations adopt cloud and containers to accelerate application development, but by adopting a secure DevOps approach and embedding security...

The post Ten considerations for securing cloud and containers appeared first on Sysdig.

]]>
Most organizations adopt cloud and containers to accelerate application development, but by adopting a secure DevOps approach and embedding security into the DevOps workflow, you can ensure security controls don’t slow down developers.

Check out these key considerations to keep in mind as you put together your plan for securing clouds and containers.

Securing containers and cloud for Dummies - Download now

1. Secure the CI/CD build

Shift-left security integrates image scanning into the continuous integration and continuous delivery (CI/CD) pipeline. Make sure to scan both operating system (OS) and non-OS packages for vulnerabilities in every stage of the CI/CD pipeline to avoid costly fixes at deployment. Security best practices and compliance should also be checked by scanners, and it’s a best practice to adopt inline scanning to avoid sending images outside of your environment.

2. Take advantage of Kubernetes native controls

Prevent containers with vulnerabilities and risky configurations from being deployed into production clusters by using Kubernetes admission controller. Kubernetes admission controller offers flexible policy setting, and, as a native construct in Kubernetes framework, it offers a powerful mechanism to control what gets deployed in Kubernetes clusters.

3. Secure IaC templates

You can shift security further left by implementing Infrastructure as Code (IaC) security. Enforce security policies in IaC by scanning cloud and Kubernetes templates for misconfigurations and violations of security best practices. Make sure to implement a pull request (PR) approach for drifts detected at runtime to automate remediation at the source.

4. Manage excessive cloud permissions

Ensure you have full visibility into cloud assets, identifying misconfigurations and drift across multi-cloud environments. Implement the least-privilege principle by detecting and removing excessive permissions on user roles (human and non-human). Look for tools that can not only automatically discover all identity and access management (IAM) roles and their permission configurations but also can detect roles with over permissions and recommend the right permission settings.

5. Implement cloud security monitoring

Keep track of cloud assets and their configurations. Cloud misconfigurations can easily happen and are a leading cause of security incidents. Make sure that cloud log auditing is enabled for all services and monitored for threat detection. Cloud logs are important forensics as well. Every cloud provider offers activity audit logs that show who did what and when.

6. Implement runtime security

Act fast on early indicators of compromise. Runtime threats are real and growing in sophistication. Adversaries are launching complex attacks to evade detection while infecting systems for maximum gain. Don’t miss weak signals. Get real-time, deep visibility into events to detect suspicious behavior and malicious activity in the cloud, container, and Kubernetes.

7. Enforce zero-trust network segmentation

Adhere to Zero-Trust principles by applying network segmentation and allowing only required communication between container services. Make sure that all network communications between pods, services, and applications inside Kubernetes follow network policies. Defining network segmentation manually is time consuming and error prone. Look for ways to automate.

8. Monitor container and Kubernetes performance and availability

Monitor resource consumption and application golden signals to stay ahead of performance, availability, and capacity issues in your Kubernetes clusters. Cloud-native environments are complex, so make sure that the monitoring is simplified with scoping capabilities to focus on a particular region, deployment, namespace, or workload. Also, look for out-of-the-box dashboards and alerts, as well as easy integration with other data sources.

9. Have an incident response framework for containers

Implement incident response and effective forensic investigation processes to understand security breaches, meet compliance requirements, and recover quickly in cloud-native environments. Make sure that a source of truth is available for providing deep visibility into system calls, as well as all activity in the container, and orchestration layers. Incidents don’t happen in a vacuum; granular data must be available to reconstruct the attack before and after the incident.

10. Use open-source tools to avoid vendor lock-In

Embrace solutions based on open-source to avoid vendor lock-in and take advantage of ecosystem integrations contributed by the community. Products based on open-source standards provide transparency and flexibility, so you can understand how detection rules are defined and customized to meet your needs.

By adhering to standards set by the community, you can protect your investment in technology and more easily find team members with the skills you need.

Want to learn more about cloud security?

Check out our “Securing Containers & Cloud” ebook, and discover topics like Infrastructure as Code (IaC), responding to threats, keeping your containers and cloud compliant, and more!

Securing containers and cloud for Dummies - Download now

The post Ten considerations for securing cloud and containers appeared first on Sysdig.

]]>
Preventing cloud and container vulnerabilities https://sysdig.com/blog/prevent-cloud-container-vulnerabilities/ Thu, 21 Apr 2022 15:00:29 +0000 https://sysdig.com/?p=48717&preview=true&preview_id=48717 Vulnerabilities are software bugs or weaknesses that could be used by an attacker. They could be present in the operating...

The post Preventing cloud and container vulnerabilities appeared first on Sysdig.

]]>
Vulnerabilities are software bugs or weaknesses that could be used by an attacker. They could be present in the operating system, application code, and third-party code dependencies, such as libraries, frameworks, programming scripts, and so on. By taking a secure DevOps approach and identifying vulnerabilities early in development, you avoid frustrating developers with delays when an application is ready for production. So, preventing vulnerable workloads from entering production is paramount, but keep in mind that new vulnerabilities and exploits can be discovered for software already in production. Scanning for vulnerabilities must be done throughout the entire workload life cycle, including at runtime.

Securing containers and cloud for Dummies - Download now





Discover how to identify vulnerabilities in container images and hosts, integrate vulnerability assessment and controls in your software development life cycle, and unify container and host vulnerability management.

Understanding vulnerabilities in containers

Modern cloud-native workloads run as microservices built on containers and deployed as Kubernetes clusters in the cloud. A vulnerability in a container can be exploited to grant attackers a foothold into your Kubernetes cluster, and from there, they can find ways to move laterally in your cloud environment.

According to the Sysdig 2022 Cloud-Native Security and Usage Report, 75 percent of images running in production contain patchable vulnerabilities of high or greater severity. The risk of exposing your organization through vulnerabilities present in containers is very real.

When securing containers it’s important to understand where vulnerabilities could be and how they could be introduced in your environment. Containers are built from read-only files called container images. Container images can be built totally from scratch or, more commonly, by using other available images, creating a layered structure. A base image (for example, Ubuntu, Debian, BusyBox, Alpine, and so on) provides the operating system (OS) dependencies and serves as a foundation for all other layers. The next layer could be a language or framework image, such as Ruby Gem, Python PiP, Node.js npm, and more. By using those readily available images, developers speed up development with a jumpstart. But, they could also be inadvertently introducing vulnerabilities into their containers.

Many developers aren’t aware that even official images stored in popular registries aren’t safe. Unfortunately, these repositories have been used as a channel to disseminate vulnerable software by malicious actors, like in supply chain attacks. For instance, DockerHub repository is known for having high-severity or critical vulnerabilities in a significant portion of its publicly available images. Some repositories offer vulnerability scanning; however, the scanning is often done just on OS images.

According to the Sysdig 2022 Cloud-Native Security and Usage Report, a high rate of high-severity or critical vulnerabilities exist in non-OS image layers. Both OS and non-OS images must be scanned.

Image scanning is a must-have in container environments. The application code and all image layers’ dependencies (OS and non-OS) in the container could have software vulnerabilities. As new vulnerabilities in existing software packages can be discovered at any time, scanning and auditing container images isn’t a one-time deal. The process requires continuous monitoring that starts with your code development and extends to production.

Integrating vulnerability scanning into DevOps life cycle

Integrating vulnerability scanning into the software development life cycle (SDLC), as for Infrastructure as Code (IaC) security, starts with a shift-left approach by embedding scanning early in the continuous integration and continuous delivery (CI/CD) process. Checking container images earlier ensures that development can run fast without costly fixes to the final build, and security isn’t a blocker for delivery.

Automate vulnerability scanning in the CI/CD pipeline

Integration with developers’ existing tools is key to successfully introducing security into DevOps processes. So, you should embed image scanning into the CI/CD pipeline to catch vulnerabilities (in all image layers, dependencies, and artifacts) early in the development life cycle. By doing so, you will also be saving engineering cycles required in more complex late fixes. Integration with CI/CD tools, such as GitHub, Google Cloud Build, and Amazon Web Services (AWS) Codebuild, can be done in two main forms, by pulling out the images for scanning or via inline scanning.

Inline scanning offers a more secure process. With inline scanning, your application code doesn’t leave your environment because you don’t need to send a copy to an intermediary repository for scanning. Only the metadata of the scan results are sent out for policy evaluation. Scan results are passed back to the pipeline, resulting in a pass or fail event.

Take a look into remediation cases that could be automated or facilitated, such as vulnerabilities with a fix requiring a low-risk version update of a package. And for those vulnerabilities without a fix or with a fix that would require more complex remediations, consider compensating controls with a risk assessment.

Automate registry image scanning

Keeping your private image repositories and registries free from vulnerable container images is another security best practice. Organizations commonly use registry services like DockerHub, Amazon ECR, Google Container Registry, Quay, and JFrog, among others. Even though these registries offer vulnerability scanning features, make sure that they’re comprehensive. Make sure that your registry covers each and every image layer and OS and non-OS dependencies. Most only offer OS image scanning, so your scanning solution should check for vulnerabilities in non-OS packages as well. Developers may be unknowingly pulling in vulnerabilities from non-OS open-source packages such as Python PIP, Ruby Gem, and more, which introduces security risks.

Protection against image tampering is also important. Make sure to digitally sign images and check signatures before using them. Verifying image signatures can assure that the specific image digest has been signed by the publisher.

As in the case of CI/CD pipeline integration, image scanning integration with registries can be done by pulling out the images to execute the scan or by scanning them inline. Organizations usually prefer inline scans that are done without pulling their images to a third-party repository for scanning.

Enforce vulnerability security with Kubernetes admission controller

Kubernetes admission controllers are the last line of defense before exposing the organization to container vulnerabilities in Kubernetes clusters. The admission controller evaluates requests to the Kubernetes application programming interface (API) server, then determines whether to allow the request. As an example, a request for a pod deployment could be denied if the admission controller receives a failed status on a check.

Admission controllers offer an extensible solution to enforce security policies using webhook integrations. Therefore, image scanners can be integrated with the admission controller to provide vulnerability check status prior to a container deployment in production. Kubernetes admission controllers also provide flexible rules to accept or reject requests based on the environment specifications. So, it is important that the vulnerability scanning solution also supports rich context within scan results so the environment context such as cloud, container, Kubernetes, and application criticality, among others, can be used to assess risk and define the appropriate policies.

Prevent vulnerabilities in serverless workloads

In serverless computing, cloud providers dynamically assign resources to run your workloads, but in a shared responsibility model, you’re responsible for protecting these workloads. Vulnerabilities in your code and dependencies can still provide attackers with a way to compromise your environment. You can and should implement vulnerability scanning on serverless workloads as well.

For instance, before launching a task on AWS Fargate — a serverless computing platform service to run containers as a service (CaaS) without the need to manage the underlying infrastructure — you could automatically trigger the scanning of the container image in the AWS ECR repository. And again, scanning can be implemented inline or by pulling out the image. Make sure that all layers are verified for vulnerabilities and that you can block deployment if high-risk ones are found.

Manage vulnerabilities at runtime

New vulnerabilities and exploits are discovered every day on new and old versions of software packages and OS distributions. If you’re only scanning at the build stage, you’re leaving yourself exposed to recently discovered vulnerabilities.

Managing vulnerabilities in your runtime environment is necessary. Your vulnerability management solution should keep track of images deployed in production and alert when an image is affected by new vulnerabilities in feed updates from vulnerability data sources. Security tools use different strategies to achieve this. One way is to rescan all images periodically, but that’s not an efficient use of your resources. Ideally, the image metadata of deployed containers is stored and new vulnerabilities can be detected without the need of a full rescan.

Let us just add this additional observation point: Alerting to vulnerabilities at runtime without context isn’t really effective. Environment information, such as cloud, Kubernetes, and application context, is necessary for risk assessment, remediation prioritization, and guidance regarding containment and compensating measures.

Implement host scanning

Hosts are the computing instances where your containers run. In Kubernetes environments, hosts are the nodes of a cluster where Pods are deployed, and an estimated quarter of organizations in the cloud have unpatched hosts with high severity and critical vulnerabilities. Although not as numerous as containers, vulnerable computing instances in public clouds expose the organization to serious risks.

Securing host virtual machines (VMs) is just as important as securing the containers running on them. Make sure that you have a tool in place that can scan hosts. For example, cloud VMs (computing instances such as EC2s) are hosts that need scanning. Host scans must also be done regularly, so your teams are given actionable information to prioritize and expedite remediation.

Adopting a single vulnerability management workflow for containers and hosts

Siloed solutions create security gaps and inefficiencies. Adopting a unified approach to vulnerability management for containers and hosts speeds the time to detect and remediate vulnerabilities, while generating fewer alerts. Organizations are often faced with thousands of vulnerabilities detected in their environments while their teams can only timely handle a small fraction of them. Prioritization and efficient remediation become fundamental to vulnerability management.

Risk comes in different forms. Not only do vulnerabilities in the code expose your environment to threats, but also it’s important to verify if security best practices are followed as well as all compliance standards and regulations that your organization adheres to.

Implement vulnerability prioritization

How to prioritize which vulnerabilities to fix is a constant challenge for resource-constrained security teams. Evaluating the severity of a vulnerability may only lead to cycles spent on vulnerabilities that don’t represent practical risk. When you’re doing prioritization, the crucial decision to make is about what can’t wait. So make sure to also include in your analysis contextual risk that reveals the true relevance of a critical vulnerability to your environments.

Think about the following questions:

  • Is the vulnerability located in packages loaded when the container is running, and therefore with real chances of being exploited, or in a package that isn’t used?
  • Is the vulnerability exposing critical applications?
  • Is the vulnerability present in environments that aren’t externally accessible?
  • Are there available exploits being used, and if so, what’s the reported threat activity?

As you answer these questions, it will become clear which vulnerabilities in your environment context are truly exploitable and incur high risk to the organization. Those areas are the ones where you need to focus your remediation efforts.

Managing vulnerabilities requires understanding how to effectively reduce risk in your environment. Make sure that your vulnerability scanning solution provides detection of vulnerabilities on containers and hosts. Check the quality of the threat data and context provided with the vulnerability alert for prioritization and remediation, tracking improvements in reducing exposure risk.

Check for compliance requirements

Unified host and container scans can be used beyond vulnerabilities checks. The scans can also validate regulatory compliance (for example, PCI DSS, NIST regulations, SOC 2) and security best practices, such as the Center for Internet Security (CIS) Benchmarks. Scanners should detect security risks, such as port misconfigurations, unprotected secrets, open-source software (OSS) licenses, and file integrity, among security controls.

Want to learn more about cloud security?

Check out our “Securing Containers & Cloud” ebook, and discover topics like Infrastructure as Code (IaC), responding to threats, keeping your containers and cloud compliant, and more!

Securing containers and cloud for Dummies - Download now





The post Preventing cloud and container vulnerabilities appeared first on Sysdig.

]]>
Eliminate noise and prioritize the vulnerabilities that really matter with Risk Spotlight https://sysdig.com/blog/eliminate-noise-prioritize-risk-spotlight-sysdig/ Wed, 20 Apr 2022 10:50:19 +0000 https://sysdig.com/?p=49205 Is your team drowning in container vulnerability noise? Are you spending a lot of time figuring out where to focus...

The post Eliminate noise and prioritize the vulnerabilities that really matter with Risk Spotlight appeared first on Sysdig.

]]>
Is your team drowning in container vulnerability noise? Are you spending a lot of time figuring out where to focus resources on and still missing dangerous vulnerabilities? Know that you are not alone.

Container environments revolutionized app development by enabling unprecedented velocity, but not without a price. The use of readily available container images of third-party and open-source code enabled much faster cycles, but also facilitated the introduction of vulnerabilities in the application. One single container could have hundreds of vulnerabilities; more complex application environments can reach tens of thousands.

Container vulnerabilities overload is growing as a challenge

Things are not getting any better. Since 2016, new vulnerabilities reported each year have nearly tripled, and as reported by US cybersecurity authority CISA, software vulnerabilities remained in the top three initial infection vectors for ransomware incidents in 2021. So, timely finding and fixing vulnerabilities are critical to prevent breaches.

Managing vulnerabilities in containers has become a complex equation of balancing risk, limited resources, and impact on development. Fixing everything is unrealistic and also unnecessary. Not all vulnerabilities incur risk, but finding the ones that cannot wait feels like looking for needles in haystacks.

DevOps and security teams know that handing a long list of vulnerabilities is a non-starter to developers. But leaving the applications exposed to attackers is not an option either. Effective prioritization is required to identify which vulnerabilities require immediate action.

Existing prioritization approaches are not effective

It is common to try to reduce the vulnerability load by focusing on the severity aspect as defined by the CVSS score. But this approach has critical flaws. First, it doesn’t reduce the load to a manageable size. Even just counting critical and high severity vulnerabilities, the number is still beyond what teams can handle, so further prioritization is still required. But it’s also important to realize that CVSS scores can be misleading.

As Miguel Hernández, security researcher, explains so well in his blog, vulnerabilities with high scores may not pose any actual risk to your application, they could be just noise. On the other hand, a medium vulnerability could provide an entry point to attackers, which could evolve to a broad and harmful impact. So, prioritization based only on CVSS scores is inefficient and ineffective.

Other prioritization methods try to apply additional risk factors but similarly fail to address overload because they don’t remove the noise from vulnerabilities that don’t pose any actual risk.

Risk Spotlight eliminates noise and automatically finds the vulnerabilities that really matter

Most of the vulnerabilities reported in container environments are actually noise. Containers are loaded with packages that are never used. Even though they are not used, their vulnerabilities are still reported!

Exploitability is a key determinant of risk. If a vulnerability is never exposed, it doesn’t offer a chance of exploitation and, therefore, doesn’t incur actual risk. Vulnerabilities in packages not active at runtime are just noise.

So, how do you know which vulnerabilities are exposed and pose real risk? By using runtime intelligence.

Risk Spotlight priorizate

Only vulnerabilities that are tied to packages used at runtime offer a real chance of exploitation. Sysdig’s deep visibility into system calls removes all the guesswork from container vulnerability prioritization by accurately identifying vulnerabilities in packages loaded at runtime.

By knowing what is exposed and what isn’t, Risk Spotlight removes the noise and prioritization guesswork so your team can focus on really important issues that can’t wait.

See how fast and easy you can identify which vulnerabilities pose a real risk with Sysdig.

Sending a report to DevOps and security teams listing hundreds of vulnerabilities in a container running in production is certainly not productive. Trying to prioritize them without eliminating noise is ineffective because just a handful offer a real chance of exploitation. So why overload your teams with vulnerabilities that pose no risk?

Risk Spotlight Sysdig vulnerabilities

With Risk Spotlight, you can focus mitigation efforts on the vulnerabilities that pose an immediate risk. All the other vulnerabilities can be deprioritized, allowing developers to fix important issues faster with minimum resources.

No longer scrolling vulnerabilities line-by-line, struggling to estimate risk through an endless spreadsheet of issues. With Risk Spotlight, you can easily find, focus, and fix the vulnerabilities that matter to you.

If you want to learn more, sign up for our upcoming webinar or request a free trial of our Sysdig Secure DevOps Platform.

Risk Spotlight Reduce Noise

Key Benefits of Risk Spotlight

    • Reduce vulnerability noise by up to 95%. Risk Spotlight eliminates the noise from vulnerabilities that pose no immediate risk by identifying the packages not used at runtime.
    • Manage risk with actionable insights. Risk Spotlight delivers rich vulnerability details – such as the CVSS vector from multiple sources, the fix version, and link to publicly available exploits – and a package-centric view that facilitates remediation and managing vulnerability risk at scale.
  • Comprehensive vulnerability management for containers from source to run. Risk Spotlight provides a single view of vulnerability risk across the container lifecycle, from build to runtime. Developers can take immediate actions to mitigate the few vulnerabilities that pose real risks and also apply security best practices early by removing unused packages during the build process.

No longer scrolling vulnerabilities line-by-line, struggling to estimate risk through an endless spreadsheet of issues. With Risk Spotlight, you can easily find, focus, and fix the vulnerabilities that matter to you.

If you want to learn more, sign up for our upcoming webinar or request a free trial of our Sysdig Secure DevOps Platform.

The post Eliminate noise and prioritize the vulnerabilities that really matter with Risk Spotlight appeared first on Sysdig.

]]>
Understanding cloud security https://sysdig.com/blog/understanding-cloud-security/ Thu, 07 Apr 2022 19:32:30 +0000 https://sysdig.com/?p=48704 Discover how to manage cloud permissions and configurations, detect threats in the cloud, and apply a unified approach for cloud...

The post Understanding cloud security appeared first on Sysdig.

]]>
Discover how to manage cloud permissions and configurations, detect threats in the cloud, and apply a unified approach for cloud and container threat detection.

Securing containers and cloud for Dummies - Download now

Managing cloud permissions and configurations

As organizations mature in the cloud, the number of cloud services their teams use and identity permissions that need to be managed grow exponentially. Many people refer to these services teams used to build and deliver applications as assets or resources. Configuring cloud assets, roles, and permissions doesn’t take long to become tedious, time consuming, and error prone.

Misconfigurations of assets and over-privileged identities increase the risk of security incidents and are the leading causes of security incidents, so make sure that you diligently manage these.

IT organizations usually don’t start with full control or even visibility over how their cloud assets are configured and which permissions were granted to identities in the cloud. Many cloud users manually configure cloud services as needed, like Simple Storage Service (S3) buckets, leaving IT out of the loop and potentially also leaving the company exposed to risky conditions. Other concerns include accounts of former employees, one-time users, and guest accounts that are left active, and also user identities with unused or unnecessary permissions.

Discovering cloud assets and configuration

Staying on top of existing cloud assets and their configurations can be challenging if done manually. IT sometimes misses the actual number by an order of magnitude. To keep up with the cloud’s constant state of change, scale, and complexity, a programmatic approach is required. Manual processes, besides leaving blind spots, also increase the risk of missing an exposed asset with weak security controls configuration.

Misconfigurations could be the result of both unintentional (legitimate users) and malicious (attacker) actions. Regardless of the nature of the actor, paying attention to security posture by checking the status of dynamic cloud configurations is practically a requirement for cloud environments.

Cloud Security Posture Management (CSPM) solutions offer cloud configuration management capabilities. For industry best practices, visit the Center for Internet Security (CIS) and review the CIS Benchmarks. This resource provides you with checklists for all major clouds.

Identifying overprivileged users

Overprivileged entitlement of human and non-human identities (applications, services, containers, and so on) is a top cause of data breaches. Applying the principle of least privilege — the concept of providing no more permissions than necessary to perform required actions — is a wise, but difficult, concept to implement. Cloud providers actually make permissions granular, which in theory would lead to least-privilege policies. However, the reality is much more complex.

In practice, permissions aren’t assigned in a precise manner. Often, existing rules are reused, noting only if the permissions are broad enough to avoid disruption. In modern organizations, nothing should get in the way of speed and performance — not even security. So, developers and IT often err on the side of excess. Manual fine-tuning would be excessively time consuming and still not precise.

Inactive identities are also a permissions threat. These identities often get left off because organizations simply lose track and lack the ability to be alerted to this inactivity.

Cloud Infrastructure Entitlements Management (CIEM) is a key tool to have in your cloud security toolbox. Look for solutions that can discover excessive permissions across active and inactive cloud identities and provide guided remediation to implement the least-privilege principle.

Monitoring cloud security controls and detecting threats in the cloud

Threats can be seen as the activities of cybercriminals, such as phishing, data exfiltration, cryptomining, distributed denial of service (DDoS) attacks, and so on. Cloud threats today are elaborate and complex and have become completely out of the reach of traditional, siloed security solutions that use coarse, out-of- context, and non-real-time data. To detect and contain attacks effectively, you need real-time visibility of the full spectrum of malicious activities applied in the attack. This includes monitoring cloud security controls, for instance, detecting configuration changes that increase risk. For more information on today’s global knowledge-base of adversaries’ tactics, techniques, and procedures, visit attack.mitre.org.

The good news is that adversaries leave a trail of their actions in some form of recorded events. One way to detect threats in the cloud is to monitor cloud audit logs for anomalous activities and malicious actions, such as unexpected configuration changes and permission escalations.

Threat risk doesn’t always result from malicious activity. Cloud configurations are changing constantly and must be monitored for impact to risk. When developers make configuration or permission changes as they debug or deploy applications, they may not consider the additional risk this adds to the organization, so cloud and security teams must continually evaluate configurations against best practices and their organizations policies.

Some organizations analyze activity logs out-of-band in scheduled pull intervals, or send their logs to a security information and event management (SIEM) system and then scan for threats, but these approaches have disadvantages:

  • They aren’t real time, which imposes a delay in the detection of a risky configuration or an intruder.
  • In the case of pull intervals, in addition to not achieving real time detection, it may miss a full sequence of malicious events.
  • SIEM tools are more suitable for forensics analysis, not for real-time detection.
  • Copying the huge volume of activity logs outside the cloud is costly and complex to manage and is a potential compliance violation in certain industries.

A more effective and efficient way to detect cloud activity threats is to apply stream detection. As the cloud audit records are generated, they’re analyzed against defined runtime policies. If a suspicious action is detected, a security event is triggered in real time. Only the security event data is sent out, not all logged records. Also, each newly recorded log is analyzed against the conditions of the detection rules, not the entire audit logs storage.

With stream detection, you can detect in real time signs of cloud threats, such as the following examples:

  • Turn-off of activity audit and logging services.
  • Change of user roles to add over permissive policies.
  • Make a S3 bucket public.
  • Access to S3 buckets from unusual accounts.
  • Change to weak or no encryption of data at rest or in transit.
  • Change to weak password settings such as no multi-factor authentication (MFA) or no password rotation.
  • Change to inappropriate firewall rules or network access controls.
  • Creation of application programming interface (API) accounts with anonymous or unauthorized access.

Adopting a unified threat detection approach

Today’s universe of cyber threats is complex. Tampering with cloud security controls, configurations, and permissions can just be a tactical step in an attack scenario that starts with the exploitation of a vulnerability in a workload, and being stealthy is the modus operandi. Adversaries adopt evasion techniques to get around legacy tools’ defenses and also take advantage of visibility gaps left by siloed solutions.

To detect and stop cyber threats in any environment, the first step is to see them. Trying to piece together data from siloed solutions slows down detection, and you may even miss the threat. If you don’t see the threat, you can’t stop it from spreading. As malicious activities could be happening in your applications, containers, Kubernetes, and cloud assets, servers and serverless platforms, a unified approach to threat detection is critical.

Single event store

In a unified approach, all detected events are in a single event store, enabling a sequenced event timeline that shows the attack in evolution. Siloed data slows down detection and may also be completely unsuitable to detect subtle and malicious aspects of a single action. When you use siloed security tools, you only see each activity in isolation, never putting together the complete attack. In sophisticated attacks where multiple systems are infected with malicious artifacts, like backdoors and malware files, without complete visibility of the attack components and blast radius, it could take months to completely remove the attacker from the environment.

A centralized view, revealing attackers’ sequence of steps from initial access through lateral movement and malicious actions across cloud and container environments, empowers security teams with the necessary information for immediate containment and removal of malicious artifacts.

Unified policy language

Detection policies can be set consistently across your cloud environment. Different policy languages add a learning curve and semantic gaps and may introduce parsing and translation loss when evaluating events. A single policy engine to detect threats across cloud and containers increases efficiency of security teams’ workflows, which not only makes policy management easier but also reduces mean time-to-remediate (MTTR) incidents.

Open-source validation

Security demands validation and transparency, so you want to avoid proprietary solutions. Proprietary tools are usually controlled by a single organization, and innovation is constrained by their resources and priorities. Solutions based on open-source standards usually have a primary contributing organization, supplemented by a community of motivated users and contributors that bring additional ideas and features. Open source also enables a more dynamic environment for providing feedback, reporting and fixing issues, and contributing with improvements. Technology ecosystems also grow much faster around open-source projects because having a community-based standard protects investments made in developing integrations. The synergies found in open source drive the collaboration, validation, and speed of innovation that are fundamental to combat modern cyber threats.

Just having security solutions in place isn’t enough; you need the confidence that they’re effective, can sustain protection against the full range of attacks, and continue to evolve to contain new threats at the same speed as the cybercrime world is evolving.

For cloud threat detection consider Falco. Falco, created by Sysdig, is the open-source standard for continuous risk and threat detection across Kubernetes, containers, and cloud. Falco acts as your security camera that continuously detects unexpected behavior, configuration changes, intrusions, and data theft in real time. Sysdig donated Falco to the Cloud Native Computing Foundation (CNCF), the vendor-neutral home for many open-source projects.

Want to learn more about cloud security?

Check out our “Securing Containers & Cloud” ebook, and discover topics like Infrastructure as Code (IaC), responding to threats, keeping your containers and cloud compliant, and more!

Securing containers and cloud for Dummies - Download now

The post Understanding cloud security appeared first on Sysdig.

]]>
Sysdig and Snyk use runtime intelligence to eliminate vulnerability noise https://sysdig.com/blog/sysdig-snyk-partnership/ Wed, 16 Feb 2022 13:00:53 +0000 https://sysdig.com/?p=46745 One of the greatest challenges in cloud environments today is to ensure rapid development cycles while keeping up with security...

The post Sysdig and Snyk use runtime intelligence to eliminate vulnerability noise appeared first on Sysdig.

]]>
One of the greatest challenges in cloud environments today is to ensure rapid development cycles while keeping up with security vulnerabilities. Sysdig and Snyk announced today a partnership to deliver integrated code to container runtime security that eliminates up to 95% of vulnerability alert noise, optimizes remediation, and protects runtime. Developers can be fast with security barriers removed, and yet without sacrificing security.

Vulnerability overload undermines security and productivity

The accelerating pace of cloud-native development is enabling faster innovation, but it is also leaving behind an increasing vulnerability backlog. Developers are overwhelmed with vulnerabilities without knowing their actual risk and where to focus remediation efforts. Just trying to make sense of the noise already takes precious time away from coding. Not to mention the frustration of dedicating time to vulnerabilities that don’t matter because they incur no real risk.

Security and operations teams monitoring runtime environments are also awash in vulnerability alert noise. Wasting resources on triaging vulnerabilities has a high price. It takes attention away from real threats.

The Sysdig 2022 Cloud-Native Security and Usage Report revealed that as much as 75% of containers with “high” or “critical” patchable vulnerabilities run in production. Vulnerability overload clearly makes remediation unmanageable, resulting in organizations having to deal with an uncomfortable average of about six months to remediate. This leaves a dangerously large window of exposure to vulnerabilities that can be actively exploited by threat actors.

Patchabe vulnerabilities

Fixing all vulnerabilities is an unrealistic goal, yet giving up on timely remediation is a dangerous bet. Prioritization is required.

Filtering out the noise with runtime intelligence

Snyk is the leader in developer security. With Snyk Container, developers get security feedback throughout the development process guiding them to build containers on more secure base images. But vulnerabilities are practically endemic in today’s applications assembled with open-source and third-party packages. The result is environments with tens of thousands of vulnerabilities in packages included in containers.

However, containers are often bloated with contents and packages that are not used when the application runs. So, trying to prioritize vulnerabilities without an upfront cut-off — to separate what matters from what just simply doesn’t — results in what you get from existing prioritization approaches. Noise in, noise out. That is why vulnerability overload pain is so prevalent. And, that is where Sysdig Secure container runtime security intelligence comes into play.

Sysdig is driving the standard for cloud and container security. We pioneered cloud-native runtime threat detection and response by creating Falco, the open-source standard for continuous risk and threat detection across Kubernetes, containers, and cloud. Applying Sysdig’s runtime risk intelligence from containers in production, vulnerability noise can be reduced by as much as 95%. This much-needed noise elimination is achieved by focusing on vulnerabilities affecting packages that are actually used when the container is running. These are the ones to fix first because they are at real risk of being exploited.

Runtime packages identified by Sysdig

Integrated prioritization enables optimized remediation

As evidenced by the persistently large number of vulnerabilities found in production, previous prioritization approaches render vulnerability reports still polluted with noise. Without the runtime context, developers end up overwhelmed by low-risk or irrelevant vulnerabilities, and may even waste resources fixing them. And, what’s worse, developers may miss critical vulnerabilities, leaving them unpatched, which can lead to breaches.

With Sysdig and Snyk integration, developers can focus. The runtime context pinpoints exploitable packages that are active in production applications. Because developers can now clearly see the few issues that cannot wait, they get more committed to remediating faster. Less guesswork and more done.

Snyk Container filtering running packages

Bridging the gap between development, security, and operations

We are very happy to partner with Snyk because a secure DevOps culture is fully embraced when it delivers a positive impact across teams. With our partnership, all teams get what they need to develop and run secure cloud-native apps while removing the barriers standing in the way of faster innovation.

The container security runtime integration is a good example of bridging gaps and delivering great value to developers, security, and operations. By providing container runtime visibility from production back to developers, vulnerability noise is eliminated and critical issues are fixed faster. With risk mitigated more efficiently, SecOps improves the organization’s risk exposure and can better focus on detecting early signs of threats. Plus, developers gain time back to code, advancing business goals.

From managing vulnerabilities to detecting and responding to real-time threats as well as monitoring and troubleshooting cloud-native environments, Sysdig and Snyk deliver the most comprehensive security to:

  • Secure containers from code to runtime: Integrate security into the container and Kubernetes lifecycle — from secure base images to vulnerabilities prioritization, to detecting real-time threats and new vulnerabilities at runtime.
  • Build secure from the start: Address vulnerabilities and remove unnecessary packages right in the build process based on what is really necessary for production.
  • Have runtime protection: Make sure that threat detection is in place to protect against attacks until new critical vulnerabilities and vulnerabilities targeted by zero-day exploits are remediated.
  • Unify prioritization: Get a unified view of risk, pairing runtime context with vulnerability checks, to prioritize alerts that matter. Developer and operations workloads become manageable when teams know what needs to be fixed now, versus in a week, and what is just simply noise that can be ignored.

Snyk and Sysdig are the first to bridge developer, security, and operations silos. When better security delivers increased productivity, it creates the perfect conditions for innovation, growth, cost savings, and customer satisfaction.

Check out the Snyk blog post on our shared vision to enable DevSecOps and the importance of this new integration.

Learn more about Sysdig and Snyk

Want to learn more and see our solutions in action? Request a demo or join us for one of our upcoming webinars:

The post Sysdig and Snyk use runtime intelligence to eliminate vulnerability noise appeared first on Sysdig.

]]>
Endpoint Detection and Response (EDR) for containers and Kubernetes – Sysdig Secure https://sysdig.com/blog/sysdig-edr-container-kubernetes/ Wed, 19 Jan 2022 16:00:44 +0000 https://sysdig.com/?p=45075 Sysdig Secure adds Rapid Response feature to streamline detection and response in container environments The increasing number of yearly reported...

The post Endpoint Detection and Response (EDR) for containers and Kubernetes – Sysdig Secure appeared first on Sysdig.

]]>

Sysdig Secure adds Rapid Response feature to streamline detection and response in container environments

The increasing number of yearly reported data breaches and new critical vulnerabilities, such as log4j, impacting both small and large businesses shows that cyberthreats are real and targeting everyone. You can minimize risk by implementing runtime security and having an incident response plan in place to contain attacks. But, in container environments, responding fast to incidents is challenging.

Cloud-native complexity and ephemerality leaves security teams without an easy way to perform investigations and detect suspicious activities in containers.

featured image EDR Sysdig rapid resoponse

Existing EDR solutions cannot solve this, because they were designed for hosts. As they don’t see containers, they just present a list of all processes running on the host. Which processes belong to each container is up to the response team to figure out. For instance, they don’t zoom in and isolate the troubled container in a Kubernetes cluster. Response teams are left with event alerts without context, data evidence, and quick access to containers for mitigation.

To address the need for a fast response in cloud-native and Kubernetes environments, Sysdig released Rapid Response, the first runtime security solution with EDR functionality for containers.

Security incidents’ MTTR is increasing in cloud-native environments

A mean time to respond (MTTR) to a security incident that spans days or weeks sounds like an eternity when you think in terms of the period an intruder stays in your environment.

But, unfortunately, it is a reality for many organizations dealing with lack of container visibility and limited resources to handle an increasing number of incidents. Life is not easy for those investigating security events and controlling and eradicating threats in cloud-native environments.

Think of an event triggered by a suspicious activity in a Kubernetes cluster. If you feel that you need a lot of visibility and context before killing containers to stop a potential attack, that is life for security teams. Often, it is necessary to perform local investigation, checking running processes, looking into log files, getting a memory dump, etc., to assess the threat, determine the right mitigation path, and then act. Killing containers and processes prematurely could cause unnecessary application downtime or degradation.

The response needed is delayed because security response teams don’t have an automated process to access a production container to do the necessary investigation and remediation.

Why aren’t existing EDR tools helping?

Existing EDR tools are not effective in cloud-native environments because they were not designed to deal with ephemeral infrastructure. They don’t provide container visibility and they don’t understand Kubernetes.

Current EDR tools are host-centric. They leave response teams with the last mile problem to solve, piecing information together to identify container processes, files, scope, and user activities. And when doing deep investigation looking into memory dumps, the security incident responder will not have any container delimitations to guide the analysis. While “what” and “how” are sorted out, MTTR is ticking.

What is needed?

Real-time response capabilities in Kubernetes container environments. Direct access to compromised containers right from the event alert will significantly improve MTTR from days and weeks to immediate response.

To address this requirement, Sysdig Secure introduced the Rapid Response feature. Rapid Response provides modern EDR functionalities for cloud workloads, enabling direct shell access to containers and hosts. It gives security teams the ability to act on incidents straight from the incident detection. No extra steps needed, no time wasted.

Rapid Response provides modern EDR functionalities for cloud workloads Sysdig Secure

What else is important?

Seamless integration into existing practices. Skills gap is a top reason for security to lag behind in organizations. Response teams, including the SOC engineer, IT operations, and developer should operate and collaborate in a shared and common knowledge environment.

Rapid Response feature doesn’t require any learning of additional language, commands, or scripting skills. The response team can mount their own investigation and remediation scripts in Rapid Response, as well as use host tooling already installed, such as kubectl, docker commands, and cloud CLIs. No learning curve, just out-of-the-box benefits.

Key Use Cases

Our customers have long expressed their frustration trying to use EDR tools in cloud-native environments. Here are some key Rapid Response use cases that address existing challenges faced by Tier 1 and Tier 2 responders when dealing with container threats:

One-click direct access to a container or host to investigate and troubleshoot locally with Sysdig Secure EDR

Use Case: Tier 1 on-demand shell access to investigate a suspicious Kubernetes container

One-click direct access to a container or host to investigate and troubleshoot locally.

Tier 1 responders team can take a deep look at malware files and suspicious activity, check if a suspicious file, or a malware file, is active in the system, and verify container escape and file-less attack by looking at CPU and memory data.

Use Case: Tier 2 advanced analytics and remediation

Shell in for fast mitigation without having to maintain admin role over the entire cloud environment.

Tier 2 responder can perform remediation with fine-grained actions such as kill processes and remove files, as well as perform quick containment like stopping the container.

Use Case: Enable collaborative work between developers and response teams

Get seamless integration with existing development, operations, and security remediation processes to apply the right containment, mitigation, and malware hygiene actions. Response teams can use the same cloud, host, and container utilities used by cloud, security, and development teams.

Sysdig Secure is the only cloud security solution with EDR functionality for containers.

A modern solution for incident response in cloud-native environments – Sysdig Secure EDR


Sysdig Secure is the only cloud security solution with EDR functionality for containers.

With Rapid Response, response teams can perform fast alert triage, deep investigation, and immediate remediation of threats, significantly reducing MTTR, risk exposure, and attack impact.

The log4j vulnerability has shown us the importance of being flexible and resilient. Therefore, it is important to have the best tool for future security incidents and to be prepared.


Check how fast and easy is to go from detection to contention with Sysdig Secure

Sign up for a free trial

Visit our page Container Forensics & Incident Response Solutions, or the Threat Reports from the Security Research Team.

The post Endpoint Detection and Response (EDR) for containers and Kubernetes – Sysdig Secure appeared first on Sysdig.

]]>
Top 10 Indicators of Compromise in Kubernetes https://sysdig.com/blog/indicators-compromise-kubernetes/ Wed, 01 Sep 2021 19:49:26 +0000 https://sysdig.com/?p=40993 In this blog, you will learn how monitoring data from your Kubernetes environments can be used to detect indicators of...

The post Top 10 Indicators of Compromise in Kubernetes appeared first on Sysdig.

]]>
Securing Kubernetes is challenging: Configuration flexibility, large clusters, ephemeral containers, and an ever-growing services ecosystem produce complex environments that open up your attack surface. Adversaries get an advantage because complexity is a natural enemy of security. You not only have to watch for misconfigurations that can facilitate attacks, but also for anomalous activity that hides behind the complexity. Top 10 indicators of compromise in Containers Organizations typically have separate tools for Kubernetes security and monitoring that often don’t talk to each other, resulting in siloed data that can leave your environment exposed. Ironically, monitoring systems often detect the early signs of a malicious attack as they are continuously monitoring various metrics and events. In order to be on top of indicators of compromise and respond quickly to security issues, it is important that security teams also have access to monitoring data. Let’s review the different types of monitoring alerts that serve as Indicators of Compromise (IoC) in Kubernetes environments that you should watch out for.

Kubernetes indicators of compromise

These top 10 Indicators of Compromise were put together by analyzing how Sysdig’s customers monitor their Kubernetes deployments, and the types of Kubernetes metrics they monitor that could have security implications. We also observed how they use Sysdig as their Kubernetes security tool to detect runtime threats and alert on abnormal activities which could be IoCs.

1. Container memory/CPU spike

Container memory and/or CPU spike is a notoriously common early symptom for resource hijacking. Resource hijacking is when an attacker gets access to your computing resources and performs malicious activity that is very resource intensive, the most common being cryptocurrency mining. Cryptocurrencies, like Bitcoin, Ethereum, etc., use resource intensive computations to validate transactions on the network. For an attacker, the ideal environment would be in an unsuspecting Kubernetes cluster where they can easily spin up several containers to do the job. The following PromQL query will provide the CPU usage for each workload:
sum (rate (container_cpu_usage_seconds_total{container=~".+"}[1w])) 
by (container) /  ignoring (container) group_left 
sum( machine_cpu_cores{}) * 100
That being said, CPU/memory spikes are not the only sign of cryptomining. Cryptominers can also kill your existing processes if they start to compete for resources. For greater confidence, you could verify the presence of process kill events to know you are experiencing a compromise. For an example on how a crypto currency attack works, and how to detect it, check our “Crypto miner attack – Sysrv-Hello Botnet targeting WordPress pods” article. Monitoring CPU and Memory usage with Sysdig Monitor

2. Anomalous outbound network traffic

Unusual outbound network traffic represents an anomaly. It is a commonly monitored Kubernetes metric and could be an indicator of compromise uncovering – for instance, an exfiltration or a compromised host used as a zombie in a botnet. Detecting anomalous connections in Kubernetes is complicated, since keeping track of who is talking to who (and why) is much harder with distributed containerized services. Your tool needs to be able to discover containers, hosts, Kubernetes nodes, services, and processes in full context (e.g., namespace, deployment, pod, process), and create a real-time topology map that will give visibility into all network connections in and out of a particular pod or service: Topology map of network traffic in Sysdig Monitor Then, detecting anomalous traffic can be done by monitoring connections made by a given service to a specific outbound IP address, and observing the total bytes in/out. Creating an alert to detect anomalous network traffic

3. Attach to a cluster-admin role

Attackers can take certain actions, like attaching to a cluster-admin role to conduct privilege escalation inside your cluster. In Kubernetes, a well-orchestrated attack usually starts with some level of privilege escalation (e.g., Tesla, WeightWatchers security incidents in 2018) and is followed by execution of commands or lateral movement through the network. One way to catch this indicator of compromise is to have a topology map that shows the average of all images running with a root user across your multiple kubernetes clusters, either on-prem or in the cloud. These are potential points of privilege escalation. Topology map of containers running as root in Sysdig Monitor In addition, having policy rules to alert you when privileged containers launched will further hone your detection capabilities. Rules library in  Sysdig Secure

4. Abnormal Kubernetes user activity

Abnormal user activity can also reveal an attacker in action and be an early sign of an indicator of compromise. Although a specific user activity could be a legitimate action, like an administrator opening a shell in a container to perform troubleshooting, any interactive user activity is suspicious. Interactive user activity is an anti-pattern and may indicate that something malicious is going on. Checking the course of actions and verifying if they fall into any observed adversary behavior path can provide higher confidence of an attack, and avoid false positives. For instance, auditing unusual account behavior and monitoring pods accessed, commands run, and connections made over a specific period can provide a clearer indication of a breach. Observing abnormal activity applies to insider attacks, as well as account takeover. Using a security tool with visibility into system activities, such as user commands including the command arguments, pid, directory, etc., which correlates them with a kubectl user session, you will be able to detect anomalous behavior. For a zero-trust approach, make use of fine-grained runtime detection that allows the building of very tight lists of expected activity, flagging anything outside the list. That can provide protection from known and unknown attack profiles. Activity Audit in Sysdig Secure Another important thing to keep in mind is that adversaries usually try to cover traces of their presence and actions by killing the container. Make sure that your tool keeps an activity audit even after the container is gone.

5. Abnormal inbound traffic

An abnormal amount of inbound traffic tied to a Denial of Service (DoS) attack can be another indicator of compromise to watch out for. In Kubernetes environments, a DoS attack on the API Server would lead to the server not being able to handle requests or have poor performance. One typical DoS attack generates a huge amount of requests to consume all the process capacity of the server (e.g., SYN flooding). So, an important indicator to detect Kubernetes API server attacks is abnormal inbound traffic. The Kubernetes API server inbound traffic size is dependent on your cluster size and the workload. So, you should see an inbound traffic hike, for example, when a mass deployment happens. Nonetheless, out of these unique situations, it is necessary to monitor your Kubernetes API server traffic status, identifying specific services with a traffic spike. Make sure that you can monitor Kubernetes API server inbound traffic size posture and easily get details such as container name and specific threshold crossed. For more details, read “Detecting the Kubernetes API server DoS vulnerability (CVE-2019-1002100).Events in Sysdig Monitor

6. Unexpected changes in file system or directory (FIM)

An unexpected change in the filesystem (FS) or directory could be a sign of an attacker executing malicious code on the system. This is why security teams need to implement file integrity monitoring (FIM) to spot these attacks. Attackers will leave behind signs that they’ve tampered with a host in system files and configurations. By looking for these kinds of changes, organizations can more quickly identify compromised systems. For example an attacker could create a container mounted /etc/ from the host filesystem to /mnt/etc/ inside the container. Then, by writing to files below /mnt/etc/ inside the container, they can write to files below /etc on the host. Ideally, your runtime security tool has rules already created to detect these types of actions since they are flagged as IOC by many security standards and frameworks, including MITRE, NIST, and SOC2. Rules library in Sysdig Secure As a next step, the attacker could install packet-sniffing software to harvest credit card data as it moves around the network. While the chances of catching the specific harvesting tool are slim — because attackers can easily avoid detection by changing the binary name — there are good chances to detect the malicious behavior, catching the unexpected changes to the system that houses the harvesting tool. In case of compromise, understanding the adversary’s course of action and the compromise blast radius is critical. Therefore, make sure that you have forensic capabilities. All pre- and post-attack container activities should be recorded to allow teams to analyze everything that happened (such as commands run, files touched, connections made, etc.), even after the container is gone! Browsing modified files in Sysdig Inspector You can learn more about this in our “File Integrity Monitoring best practices” article.

7. DNS request anomalies or large spikes in DNS requests from a specific host

Abnormal DNS traffic is identified as a potential technique under Command and Control (e.g., C&C or C2) tactics in the MITRE ATT&CK matrix. The unique patterns of this traffic can be recognized and are a very standard approach to identifying a compromise. Seeing a large spike in DNS requests from a specific host can serve as a good indicator of potentially malicious activity. Watching for patterns of DNS requests to external hosts, compared against geoIP and reputation data, and implementing appropriate filtering can help to identify and mitigate C&C tactics over DNS. One approach is to monitor Kubernetes coredns metrics, including coredns requests count. This is a good example of how monitoring metrics data from your environment can provide intelligence to unveil signs of an attack. Exploring a DNS spike in Sysdig Monitor Read more about resolving DNS with Falco

8. Unusual HTTP response sizes

Unusual HTTP Response Sizes could be a sign of an exfiltration. For instance, if an attacker accesses a web application in a PCI scoped namespace that holds sensitive credit card data, the HTTP responses exfiltrating data would be larger than a normal request. You can use metrics, such as http_response_size_byte, to monitor the Kubernetes deployments for unusual HTTP response size and detect an exfiltration. Exploring HTTP request sizes in Sysdig Monitor

9. Unknown binary processes spawned

Containers offer some security advantages, one of them being that they run a set of processes that are usually well known. So, detecting that an unknown binary process spawned could be an indicator of compromise and a sign of an execution or lateral movement technique. Detecting this type of compromise in action requires runtime protection and fine-grained visibility into the container’s runtime. Processes expected to be running in a container is something that can be easily defined, either manually or automatically, via image profiling. Once the list of processes is defined, the next step is simply to include it as a runtime policy rule for the container that would detect unknown binary processes spawned. Detecting abnormal processes with Image profiles in Sysdig Secure Here’s an example of a rule to detect unknown spawned process: Rules Library in Sysdig Secure Detecting is not all of it, however. Besides timely notification and remediation (e.g., killing, stopping, pausing the infected container), you also need auditable evidence for transparency and improved intelligence. Make sure that you have a tool that covers the complete workflow: real-time detection, remediation, and auditing.

10. HTTP 403 and 404 error codes spikes

Consecutive, failed HTTP requests, resulting in 403-forbidden or 404-unknown responses, can be seen as intrusion attempts and an indicator of an Initial Access TTP. Adversaries could be trying to access restricted areas or perform fingerprinting. Make sure you monitor HTTP metrics, including errors, alerting to a sudden increase in the number of error codes count.  Monitoring status codes for HTTP requests in Sysdig Monitor

Conclusion

Security incidents are not isolated events. They push limits, break through boundaries, make use of unexpected interactions (e.g., start processes and connections, modify accounts and permissions, etc.), and cause resource consumption disturbance (e.g., CPU, memory, bandwidth, etc.). Security incidents are anomalous in behavior and impact, leaving a trail of indicators of their malicious presence and procedures. You just need to be alerted to the combined signs to uncover a compromise, or attempt to do so, as quickly as possible. In this blog, we shared the top 10 indicators of a compromise in Kubernetes, and how you could detect them, by including monitored metrics and events from your Kubernetes environments in your security approach. Although detecting an early indicator of compromise is important, preventing the security incident would be preferred. That’s why security must be an integral part of the entire software development lifecycle (SDLC) – from code to production, a fully embraced Secure DevOps culture. You improve protection by enforcing security policies pre-deployment on Infrastructure as Code (IaC), having security integrated into the CI/CD pipeline, and continuously checking for drifts and threats at runtime.
With Sysdig Secure DevOps, you get unified visibility across workloads and cloud infrastructure from a single security and monitoring event store. Accurately alert on threats, misconfigurations, operational issues, and compliance risks pre-deployment and in production, and respond using a detailed activity record. Easily plug into your existing workflows with out-of-the-box integrations. It only takes a few minutes to get started! Request a free trial today!

The post Top 10 Indicators of Compromise in Kubernetes appeared first on Sysdig.

]]>