Impact: 08:50 – 08:55, 09:50 – 09.55
Problem: Upstream/ISP router rebooted. Suspected DDOS. Operator is restarting services and restoring connectivity.
Future resolution: Replacing upstream router.
We’ve now omitted the ISP with connectivity issues and re-routed traffic to different ISP.
Uppströmsleverantör hade problem med sitt nätverk. Det drabbade vissa delar av Adminors nät.
Orsak var DDOS mot leverantör.
At 18:15 we were notifed that BGP sessions towards one of our upstream providers went down.
This caused some network dips for one of our core routers while routes were being rebuilt.
At 18.21 the BGP sessions towards upstream was restored.
We’ve asked upstream to investigat the unannounced loss of session.
19.00 Update: Upstream has replied and said that a high CPU situation on their router caused BGP sessions to ”flap” (disconnect and reconnect). The cpu usage is now stable and BGP sessions too.
notified that there is a new remote crash vulnerability many Linux systems .
The CVE has yet to be publicly released, it has just been reserved so far: https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2019-11477
The register has published more details: https://www.theregister.co.uk/2019/06/17/linux_tcp_sack_kernel_crash/
Other cloud hosting vendors have published steps on how to mitigate this flaw
and so is Adminor AB.
A patch to linux kernel will be issued by different vendors, meanwhile a
mitigation of these attacks is to disable tcp_sack (tcp selective
It’s possible that the recent reboot of systems already have mitigation for
this exploit but we have not been notified of such by upstream vendor as the
exploit is still not entirely released.
We recommend that you disable tcp_sack as a pre-caution.
TCP sack is used to speed up TCP transfer by allowing computers to tell the
server how much data is left to be sent. This should have minimal impact on
normal operations but we still recommend monitoring for any negative
run which should not require a system reboot:
echo 0 > /proc/sys/net/ipv4/tcp_sack
To make the change persistent across reboots a command such as the following
can be run:echo
’net.ipv4.tcp_sack = 0’ >> /etc/sysctl.conf
We recommend enabling tcp_sack when a kernel patch has been issued and system
Please let us know if you need assistance .
Start time: 00:00 CET (GMT+1) 18-NOV-2018
End time: 06:00 CET (GMT+1) 18-NOV-2018
Expected Outage/Downtime: 45-60 minutes (for Cogent transit, not Adminor services)
Cogent is performing network maintenance which will cause some network dips momentarily. Adminors network routing / BGP will move traffic over to other transits to minimize network outage.
At 20:26 a switch had an unplanned reboot which caused a short network dip.
We’re monitoring to find the cause.
As of 01:50 the maintenance job had been completed succesfully.
During the night, Adminor will perform routine maintenance on our core router and core switch. Intermittent outages will occur. We apologize for the inconvenience.
If your services still have issues after 05:00. Feel free to contact [email protected] or the on call telephone number if you have elevated SLA priority.
At around 04:30-05:00 tonight we performed an urgent update to our secondary router which is used for fail over.
This may have had some impact on network routes for a couple of minutes.
Adminor has scheduled a network maintenance on 12th of May between 01:00-05:00. We will attempt to keep restarts and network dips minimal
As part of increasing our security and GDPR compliance for the 25th of May deadline, we will be discontinuing Skype as a method of support / chat. It is not compliant with our privacy policies. This decision has been made after talks among our board and legal department.
That is because we can not delete information fully from our side, or control which personal information is stored on Skype.
From now on, all support info should be directed to [email protected] to relate tickets.
For priority support, use the on-call number that we’ve provided or our business number during office hours.
Cellphone / text messages to individual technicians are not monitored for support errands and all information will be disregarded for your security.
For example, user information, passwords or other critical details that should not be sent on insecure channels.
This will be in effect until we’ve found a suitable and secure communications system for tickets as a complement to emails.
When contacting Adminor via Facebook or similar, please be aware of what content you send to us. Do not send personal information via the chat. There is no option for us to delete it fully on Facebooks servers.