6 C
Tuesday, November 21, 2017
Home Tags Jitter

Tag: Jitter

EDINBURGH, SCOTLAND, May 12, 2017 – Calnex Solutions Ltd today shipped the industryrsquo;s first ever 100GbE Synchronous Ethernet (SyncE) Jitter measurement solution to a major North American equipment manufacturer. Providing 100GbE Jitter Generation and Jitter Tolerance measurement capability, the Paragon-100G enables the jitter performance of SyncE network equipment to be fully stressed and validated.The ITU-T standard for SyncE (G.8262) has strict limits for both Jitter Tolerance and Jitter Generation.
Itrsquo;s vital that network equipment can... Source: RealWire
After pages and pages of complaints in the Microsoft Answers forum, Microsoft has yet to come up with a fix, or even an acknowledgment, of the Surface Pro 4 screen shaking problem.

This isn't a pen jitter, which is common enough.
It's a full-on shak...
The Network Time Foundation's Network Time Protocol Project has patched multiple denial-of-service vulnerabilities with the release of ntp-4.2.8p9.

The last update to the open source protocol used to synchronize computer clocks was in June.   "NTP users are strongly urged to take immediate action to ensure that their NTP daemons are not susceptible to being used in DDoS (distributed denial-of-service) attacks," the project maintainers wrote in the security advisory. NTP is a widely used protocol, and has been hijacked several times over the past two years in distributed denial-of-service attacks.

Attackers harness the power of the servers running NTP and amplify the amount of traffic -- as much as 1,000 times the size of the initial query -- sent to victim systems. Research from network security company Arbor Networks estimated that 85 percent of volumetric DDoS attacks exceeding 100Gbps in size were NTP reflection attacks. Some of the vulnerabilities are easy to exploit, and there is a proof of concept already available for one of them.

Attackers are increasingly exploiting the limitations of older protocols like NTP to generate large volumes of junk traffic used for large DDoS attacks, network company Akamai said in a previous State of the Internet report. Issues fixed in ntp-4.2.8p9 The most serious vulnerability lets attackers crash Windows systems by sending "too big" packets (CVE-2016-9312).

The update also includes fixes for two medium, two medium-low, and five low-severity vulnerabilities, all of which can be exploited to cause a denial of service. One of the medium-severity flaws (CVE-2016-7431) was related to a regression in the handling of some Zero Origin timestamp checks. One of the low-severity flaws (CVE-2016-7433) was related to a previously fixed bug where the jitter value was higher than expected and affected initial sync calculations. Two vulnerabilities were related to the trap service in NTPD. While trap is not enabled by default, if the service is explicitly enabled, attackers can send specially crafted packets to cause a null pointer dereference (CVE-2016-9311) that will crash NTPD.

The configuration modification vulnerability in the control mode (mode 6) functionality of NTPD (CVE-2016-9310) can be exploited by a remote, unauthenticated attacker. "If, against long-standing BCP recommendations, restrict default noquery is not specified, a specially crafted control mode packet can set NTPD traps, providing information disclosure and DDoS amplification, and unset NTPD traps, disabling legitimate monitoring," according to the advisory. Two of the low-severity vulnerabilities exploited NTP's broadcast mode.

They were originally intended to be used only on trusted networks, but now attackers with access to the broadcast service can abuse the replay prevention functionality (CVE-2016-7427) and the poll interval enforcement functionality (CVE-2016-7428) to cause NTPD to reject broadcast mode packets from legitimate NTP broadcast servers. If NTPD is configured to allow mrulist query requests, a server sending malicious mrulist queries will crash NTPD (CVE-2016-7434).

The researcher who reported this vulnerability, Magnus Stubman, has publicly released the proof of concept, likely because the low-severity vulnerability, with a score of 3.8 on the Common Vulnerability Scoring System, is highly exploitable. "The vulnerability allows unauthenticated users to crash NTPD with a single malformed UDP packet, which causes a null pointer dereference," Stubman wrote. If the server response on a socket corresponds to a different interface than what was used for the request, NTPD updates the peer structure to use the newer interface. On hosts with multiple interfaces in separate networks and where the operating system isn't checking the packet's source IP address, attackers can send a packet with spoofed source addresses to trick NTPD to select the wrong interface (CVE-2016-7429). Repeating the attack even once per second is enough to prevent NTPD from synchronizing with the time server. Rate limiting prevents brute-force attacks on the origin timestamp, but if the server is misconfigured, it can be abused in a DDoS attack.

Attackers can send packets with spoofed source addresses and keep the rate limiting activated, which would prevent NTPD from accepting valid responses from other sources (CVE-2016-7426). Mitigations and recommendations Along with updating to ntp-4.2.8p9, the project maintainers recommended implementing Ingress and Egress filtering through BCP-38 to "defeat denial-of-service attacks." Many DoS attacks rely on IP source address spoofing to hide the packet's point of origin, making it hard for defenders to know where the network traffic is coming from.

BCP-38 filtering lets administrators block IP packets that have forged IP addresses.
If the device is not allowed to send packets with source addresses other than its own, then that device can't be hijacked to spew junk traffic as part of a DDoS attack. Other recommendations included: Monitoring NTPD instances and autorestarting the daemon without the -g flag if it stops running Using restrict default noquery in the ntp.conf file to allow only mode-six queries from trusted networks and hosts Using broadcast mode only on trusted networks Creating a firewall rull to block oversized NTP packets, especially on Windows Allowing mrulist query packets only from trusted hosts Configuring the firewall to control what interfaces can receive packets from which networks, especially if the operating system isn't performing source address checks Configuring rate limited with restrict source in the ntp.conf file, instead of restrict default limited The Department of Homeland Security's Computer Emergency Response Team at Carnegie Mellon University's Software Engineering Institute maintain a list of vendors implementing NTP.

At the moment, the status of most vendors is listed as "Unknown." Update or replace? Whenever there is a security vulnerability in a widely used open source project, a segment of IT and security folks pounce on the report as proof people should abandon using the software and switch to something else. NTP is no different, as the 30-year-old protocol lacks security features and is vulnerable to abuse.

There are many who think administrators should use secure alternatives, but most tend to be complex or not yet mature enough for widespread adoption. The "don't update, replace," argument is impractical. Replacing crucial services without thinking through how the new tool would be supported causes more administrative headaches.
It may be a simple enough task to uninstall NTPD, but if administrators don't know how to configure the new tool correctly, monitor the performance, and troubleshoot resulting issues, then the replacement tool doesn't do much to improve overall security.

Any replacement should come after a thorough review of the alternatives, be tested thoroughly in a nonproduction environment, and have controls to monitor and secure the software.

Don't replace; update instead.
NTP.org ntpd prior to 4.2.8p9 contains multiple denial of service vulnerabilities.
Whenever we talk about distributed denial-of-service attacks, we typically focus on internet of things botnets and shadowy attackers sending large volumes of junk traffic against networks and applications.

For many organizations, however, self-inflected DDoS attacks pose bigger dangers, a pair of Google engineers warned. Poor software architecture decisions are the most common cause of application downtime, Google’s director of customer reliability engineering Dave Rensin and Google site reliability engineer Adrian Hilton wrote on the Google Cloud Platform blog.

The mistakes aren’t in the code, but in the assumptions the developers make about applications and user interactions, especially as they relate to system load. “We think it’s a convenient moment to remind our readers that the biggest threat to your application isn’t from some shadowy third party, but from your own code!” Rensin and Hilton wrote. Software developers typically assume that load will be evenly distributed across all the users.

That isn’t a bad assumption, but if the developer doesn’t have a contingency plan to deal with situations when load is not evenly distributed, “things start to go off the rails,” the engineers said. For example, a mobile app developer writing code to fetch data from a back-end server every 15 minutes may not think twice about also putting in logic to retry every 60 seconds if the app encounters an error. On the surface, this looks reasonable and is actually a common pattern. What many developers fail to consider, or plan for, is what happens if the back-end event is unavailable for a minute. During that minute, there will be apps trying to fetch the data and encountering an error. When the back-end server shows up again online, it will be hit with the requests for data normally expected for that minute as well as the traffic from apps retrying after the one-minute delay.

That’s double the expected traffic, and load is no longer evenly distributed because “two-fifteenths of your users are now locked together into the same sync schedule,” the engineers said. For a given 15-minute period, the server will experience normal load for 13 minutes, no load for one minute, and double load for one minute. Service disruptions usually last longer than a minute.
In order to correctly handle a 15-minute service disruption, software engineers would need to provision at least 15 times normal capacity to keep the application server from falling over when coming back online.
If the back-end server responds more slowly to each request because of the growing load on the load balancer, the total number of requests can exceed 20 times normal traffic during this time. “The increased load might cause your servers to run out of memory or other resources and crash again,” Hilton and Rensin wrote. Tricks to avoid self-DDoS attacks Application developers can adopt three methods to avoid the self-inflicted DDoS attack: exponential back-off, adding jitter, and marking retry requests.
Incorporating these methods can “keep your one minute network blip from turning into a much longer DDoS disaster,” the engineers wrote. Exponential back-off refers to adding a delay that doubles with every failed retry attempt, to build in a longer time interval between failed connection attempts. Using a fixed retry interval almost guarantees that retry requests will stack at the load balancer.
In the case of the above example, after the first one-minute retry, the app will wait two minutes, four minutes, and eight minutes, and 16 minutes, before going back to one minute.

Exponential back-off lowers the number of overall requests queued from 15 to five. Jitter, the act of adding or subtracting a random time period to the next retry interval to vary the timing of the next attempt, helps prevent apps from getting locked into the same sync cycle.
In the example, if the next back-off interval is set to four minutes, adding jitter could result in the app sending a new retry attempt at some point between 2.8 minutes and 5.2 minutes instead of waiting a full four minutes. “The usual pattern is to pick a random number between +/- a fixed percentage, say 30 percent, and add it to the next retry interval,” the engineers said. In the real world, usage is almost never evenly distributed. Nearly all systems experience peaks and trough corresponding with their users’ work and sleep patterns, Rensin and Hilton said. Users may turn off their devices at night—which will result in a traffic spike when they wake back up. “For this reason it’s also a really good idea to add a little jitter (perhaps 10 percent) to regular sync intervals, in addition to your retries,” the engineers said, noting that this would have the most impact for first syncs after restarting the application. Finally, each attempt should be marked with a retry counter so that the application can prioritize client connections.
In the case of a service disruption, servers don’t all come back online at the same time and the application’s overall capacity is limited at first.

The application logic may focus on responding to requests with higher retry number because those clients have been waiting the longest, for example. “A value of zero means that the request is a regular sync.

A value of one indicates the first retry and so on,” Rensin and Hilton said. System outages and service disruptions happen for a myriad of reasons.
In a given 30-day month, a system consistently maintaining 99.9 percent availability can be potentially unavailable for up to 43.2 minutes, so a 15-minute disruption is completely within the realm of possibility. How the application recovers from these incidents determines how well it handles unexpected traffic volumes.

How to avoid DDoSing yourself

Google engineers offer guidance to keep application developers from shooting themselves in the foot. In the wake of the last month's distributed denial of service (DDoS) attack against Dyn, a DNS management service, Google engineers want to remind application developers that self-harm represents a more realistic risk. Just as US citizens have a greater chance of being crushed by falling furniture than to dying at the hands of terrorists, developers are more likely to DDoS themselves than suffer downtime from an external attack, or so claim Google site reliability engineers Dave Rensin and Adrian Hilton in a blog post. The Register asked Google if it could quantify that claim with specific figures, but we haven't yet heard back. Rensin and Hilton observe that software engineers commonly assume that system load will be evenly distributed and fail to plan for the alternative. As an example, the pair describe a mobile app that periodically fetches data from a backend server. "Because the information isn’t super time sensitive, you write the client to sync every 15 minutes," they suggest. "Of course, you don’t want a momentary hiccup in network coverage to force you to wait an extra 15 minutes for the information, so you also write your app to retry every 60 seconds in the event of an error." The problem with this approach may become apparent if and when there's a service disruption, which happens on occasion. When your backend comes back online, it gets hit with the expected periodic requests for data and with any delayed requests for data. The result is double the expected traffic. With only one minute of downtime, the traffic load becomes unbalanced.

Two-fifteenths of the apps on a 15-minute sync schedule users get locked into the same sync timing. "Thus, in this state, for any given 15-minute period you'll experience normal load for 13 minutes, no load for one minute and 2x load for one minute," said Rensin and Hilton. And, as the pair observes, most service disruptions last longer than a minute. With 15 minutes of downtime, all your users will get pushed into fetching data when service gets restored, meaning you'd need to provision for 15x normal capacity.

Add the possibility that repeated tries to establish a connection can stack on load balancers and there's more pressure on the backend.
So, preparing for 20x traffic or more may be necessary. "In the worst case, the increased load might cause your servers to run out of memory or other resources and crash again," said Rensin and Hilton. "Congratulations, you’ve been DDoS’d by your own app!" To avoid shooting yourself in the foot, the pair advise exponential backoff, adding jitter, and marking retry requests. Exponential backoff involves adding a delay that doubles with every failed retry, to create more delay between failed reconnection attempts. Jitter provides another way to vary the timing of retry attempts, by adding or subtracting a fixed amount of time to an app's connection schedule. Finally, by tracking retry attempts with a counter, application logic can prioritize which clients connect when there's a queue of unfulfilled connection attempts as a result of downtime. ® Sponsored: Customer Identity and Access Management
'Know Your Enemy' scheme uses switch flow tables to design attacks Software-defined networking (SDN) controllers respond to network conditions by pushing new flow rules to switches.

And that, say Italian researchers, creates an unexpected security problem. The researchers were able to persuade their SDN environment to leak information that sysadmins probably don't want out in public, including network virtualisation setups, quality of service policies, and more importantly, security tool configuration information such as “attack detection thresholds for network scanning”. Even a single switch's flow table, they write, can provide this kind of information, as well as serving as a side-channel for an attacker to exploit. The three network boffins – Mauro Conti of the University of Padova, and Sapienza University's Fabio De Gaspari and Luigi Mancini – are particularly concerned about SDN being exploited to help an attacker build a profile of the target network, in what they call a Know Your Enemy (KYE) attack. For example, they write, an attacker could potentially: Connect to the passive listening ports most SDN switches include for remote debugging, to retrieve the flow table (they offer HP Procurve's dpctl utility as an example); Infer information about the flow table from jitter (that is, round trip time (RTT) – variance); Sniff control traffic, because of inadequate protection (not using TLS, or not using certificates for authentication; Exploit vulnerabilities that might exist in switch operating systems, such as backdoors; or Copy the flow table or memory content of the switch to an external location. The paper points out that none of this is specific to particular devices: “the KYE attack exploits a structural vulnerability of SDN, which derives from the on-demand management of network flows, that in turn is one of the main features and strengths” of SDN. Because SDN is designed to respond to the network by pushing flow rules out to switches, the attacker can easily work out what conditions will make the controller push a flow rule out to the switches. Hence the KYE attack only needs to probe the environment (to discover what flow rules are on the switch they can access, either internally or remotely); and use the information gained in the “inference phase” to work out what policies will trigger activation of a particular rule. Their proposal is that SDN architects consider applying some kind of flow obfuscation to their network, to stop an attacker using SDN responses to reveal information about the network: “if it was possible to prevent the attacker from understanding which network flow caused the installation of which flow rule, the KYE attack would become unfeasible”. That's not too hard, it seems: one example given in the paper is that the controller push its instructions around the network on paths other than the most direct connection to switches, to make it harder for the attacker to correlate flow instructions with their fake stimuli. ® Sponsored: 2016 Cyberthreat defense report