We found out (or rather re-discovered) that Azure Files CIFS implementation does not correctly support mmap function that is used in some softwares.
It has been previously reported on https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=900821 but we thought it might be worth clarifying this for people spending a lot of time googling why their apache2 fails to deliver files larger than 1MB.
The default setting in apache2 does not mmap files smaller than 1MB hence why the issue is not noticed on small files or files not hosted on the CIFS volume. Large files from the affected volume will however cause a ”Read error at byte X/ X (Bad address). Retrying.” error if you try to wget or download the files via apache2 on a web app using Azure Files mounted CIFS storage (path mapping).
Some searches might lead you to investigate apache2 file limitations preventing files larger than 1MB from beint transmitted.
LimitRequestBody is not a fix for this issue.
Scenario:
Azure web app or VM with Apache2 or other mmap using software fails to read files properly from Azure Files mounted ”path mapping” volumes which uses CIFS.
Workaround:
EnableMMAP off
This configuration line can be done per affected Directory, global apache2 config or in the vhost.
Författare: Alexander Norman
CloudFlare global DNS issues
CloudFlare are having issues with their DNS services.
This may impact sites and services globally.
We’re starting to see some of their services returning to normal but we still caution that there may be more impacts.
More information at https://www.cloudflarestatus.com/
Upstream network maintenance
One of our network upstreams has announced network maintenance.
Please note that routing changes may cause momentary traffic dip during this time frame.
Our own traffic should automatically failover to secondary ISPs.
Date and time for maintenance window
Start date and time: 2020-05-05 03:00 UTC
End date and time: 2020-05-05 04:00 UTC
Emergency maintenance
At 04.30 we performed a reboot of one of our core routers to ensure the issue is not on our end.
We will reboot some other equipment to troubleshoot issues.
We apologize for the disruption.
Update: One core switch was also rebooted manually at 05.50 – 05.52 . (short dip)
We have made adjustments to network traffic to find a stopgap solution to flapping network connection.
Network dip
One of our upstream transits suffered an outage at 19.05. This caused network re-routing to happen and may have been felt as momentary network dip.
Traffic has been re-routed to failovers. We’re monitoring the networks.
When: 19:05 – 19:10 (upstream has failed over)
Impact: 5min re-routing, traffic dip
What: 46.253.192.0/20
Update: Some more network dips occured during the evening. The network upstream suspected of causing high CPU usage in our cores have been isolated. We’re continuing to monitor the situation.
When: 20.30, 23.30 , duration (roughly 2-3min per incident)
Network dip
At 22.00 we experienced some routing issues. We are currently investigating the cause of this.
Cause: One of our Upstream Transit lost BGP connectivity / flapped
Region: Stockholm
Impact: 5 – 10min routing table rebuilds
Update (22.30): One of our transit operators having issues. Recovered after a few minutes.
Cyber Monday – highend dedicated server rental on sale 1400SEK/month
- 2 x E5-2620v3 Xeon Six-Core CPU
- 2.4GHz Processor Speed
- 128GB RAM
- 2 x 900GB 10k SAS 2.5″ Hard Drives
- PERC H730 RAID
- iDRAC8 Enterprise
- Up to 2 x 2.5″ Hot-Swap Drives
Contact us for customized setup
BGP session reset 04:00 – 04:05
At 04:00 we pushed an urgent patch to routers. This caused BGP to reset.
Traffic was disrupted momentarily until BGP was re-established.
Switch updates Västberga
Start: 00:50
End: 01:00
We’ve applied important security updates to some of our switches in Västberga.
Each switch requires less than 5 minutes to reboot.
Driftstörning 5.226.32.0/20
Impact: 08:50 – 08:55, 09:50 – 09.55
Problem: Upstream/ISP router rebooted. Suspected DDOS. Operator is restarting services and restoring connectivity.
Future resolution: Replacing upstream router.
Update 10:30:
We’ve now omitted the ISP with connectivity issues and re-routed traffic to different ISP.