Kategori: Driftinfo

There is currently pocketless towards Telia network from Västberga (5.226.32.0/21 and 46.253.192.0/20) datacenter. We’ve notified upstream transits who are looking into it.

If necessary we’ll disconnect Cogent temporarily to re-route traffic until it has been fixed, we’ll keep monitoring for now.

Update: Traffic for Västberga has been re-routed away from Cogent during the duration of the packetloss.

According to our provider, tonight’s outage was extraordinary. More information about the outage can be found here: https://status.glesys.com/

In short, both A and B power went down due to an issue in the data center. We are following up with them and will send out more information once the data center operator updates us on what went wrong.

The incident started around 20:45. Adminor’s virtual services were back online after a cautious restart between 21:00 – 21:30. SAN (storage) came back immediately after the power was restored (primary + backup in Västberga). Some services may have taken longer due to fsck checks and other factors, but efforts were made to restore everything as quickly as possible.

We monitored the situation overnight, reactivated Adminor’s replications for backups, and assisted customers in restarting their physical servers where there were issues.

When: 2024-08-12 20.45 – 21.30
Where: Västberga Datacenter
Impact: Outage

Packetloss was detected on inter-datacenter link. This was caused by a faulty fiberpatch in one of the locations.

The patch at VBDC was replaced and optics inspected.

Impact: 00.00 – 09.00 Minor packetloss on link, not causing outage notifications .
Redundancy was degraded during troubleshooting 14.00 – 15.00. No impact on service.

Resolved 15.10.

We’ve added a secondary storage platform to our hosting in Västberga. This will increase local failover capacity and speedup recovery incase of storage outage.

This improvement is for our VB VM hosted customers.

Core switch reboot was performed 01.30 – 01.35 due to memory leak (stack bug).
A tentative fix is the reboot and to avoid causing the ACL conflict which led to the memory leak.

At 04.00 – 04.05 another test was made to replicate the issue. The issue has most likely been found and a new fix has been applied.

Longterm solution will be to replace this switch line with new core routers , Juniper MX204.

At roughly 19.15 CET a network dip was detected due to an issue at a transit operator.
Traffic was automatically re-routed (BGP network re-calculation).

At 14:45 a core switch rebooted. This caused intermittent connection failiure between our two DC’s until paths were rebalanced. All traffic restored ca 14:55 .

swb2-hy

At 14:50 a PDU in HY DC failed.
This caused some equipment to lose grid redundancy.
At 20:30 , all equipment was moved to temporary failover grid redundancy until DC operator can remedy failed PDU.
The root cause was a short circuit in a server power supply causing fuse to trip.
On Tuesday during office hours, DC operator will attempt to remedy the tripped fuse and move equipment back to original PDU.
Impact: No service disruption of network connectivity (redundant power). Reduced Grid redundancy between 14:50 – 20:30.

Hello,

It has most likely not gone unnoticed by anyone that the electricity
prices are hitting record highs across Europe. During the winter
months, we know from experience that the price of electricity goes
up, something we always take in consideration when we price our
services, but this year we’ll need to adjust our electricity price
for those of you who have colocation, rent an entire rack or have dedicated servers.

Last year the price of electricity increased by over 500 percent in
Sweden, compared to 2020. The new price level has continued into
2022 and we see no end to these electricity prices in a foreseeable future.

The adjustment of the electricity price will be activated in the
months where the average electricity spot price exceeds SEK 0.75/kWh.
On these occasions, we will adjust the electricity price by SEK 10
per kW per month for each Swedish öre it exceeds SEK 0.75/kWh.

For customers in Stockholm and Falkenberg, the spot price will be
regulated according to electricity area SE3 resp SE4.

Price example: if the average electricity spot price during a month
is SEK
0.85 / kWh, we will charge SEK 100 extra per kW and month.

The price adjustment will take effect first of March this year!

This electricity price change will go in effect on agreement renewals for all co-location and/or dedicated server customers who have been notified, this this new electric price calculation will complement any previous included electricity and be based on known or calculated electric usage of customer.
This is to safeguard Adminor against the exceptional price swing on electric market which is plaguing Europe at this time. Adminors pricing is transparent and matches the electric price of our provider and aims to not make any additional money from the electric price billing. If our provider changes their price calculation we will adjust this as well, this may mean that your price will change quickly. We’ve taken all steps we can to offer reliable electricity , and alternative hosting sites – this is offered at first come, first served basis.

At roughly 04:50 we suffered a core switch outage.
Personel was sent to datacenter to investigate the issue.
At 05:50 a new core switch was installed.

The root cause is being investigated, but suspected hardware failiure.