| Date: |
01/22/2010 |
| Time: |
10:00 AM PT |
| Duration: |
Until Jan 27, 2010 |
| Affected Customers: |
SmarttNet Chilliwack Dialup Service customers |
| Reason of Outage: |
Chilliwack Dialup Pool Connectivity Issue. If you have any questions or concerns, please feel free to contact us. We apologize for the inconvenience. |
| Date: |
01/20/2010 |
| Time: |
05:30 PM PT |
| Duration: |
30 Minutes |
| Affected Customers: |
No customer should be affected by the monthly generator test. |
| Reason of Outage: |
Generator will be tested once a month on every 3rd Wednesday of the Month around 5:30 PM. The total test time will be 30 minutes. If you have any questions or concerns, please feel free to contact us. We apologize for the inconvenience. |
| Date: |
01/19/2010 |
| Time: |
09:30 AM PT |
| Duration: |
1 Hours 30 Minutes |
| Affected Customers: |
Specific SmarttNet Co-location Customers |
| Reason of Outage: |
Multiple compressor gas pressures dropped below desirable levels causing an increase in temperature. The problem was resolved by moving the load to standy-by compressors while the other units underwent inspection and maintenance. We apologize for the inconvenience this event may have caused. |
| Date: |
12/30/2009 |
| Time: |
2:50 AM PT |
| Duration: |
1 Hours 35 Minutes |
| Affected Customers: |
The 3 customers on the windows Cold Fusion server, Strontium |
| Reason of Outage: |
Failed windows update. If you have any questions or concerns, please feel free to contact us. We apologize for the inconvenience. |
| Date: |
12/28/2009 by Telus |
| Time: |
10:50 AM – 12:50 PM PT |
| Duration: |
2 Hours |
| Affected Customers: |
All ADSL Customers are down. |
| Reason of Outage: |
Telus Fiber Transceiver equipment issues. If you have any questions or concerns, please feel free to contact us. We apologize for the inconvenience. |
| Date: |
12/18/2009 |
| Time: |
21:00 – 22:00 PM PT |
| Duration: |
60 Minutes |
| Affected Customers: |
No customer should be affected by the server software updates. |
| Reason of Outage: |
Regular Server Software updates. If you have any questions or concerns, please feel free to contact us. We apologize for the inconvenience. |
| Date: |
12/18/2009 |
| Time: |
21:00 – 22:00 PM PT |
| Duration: |
60 Minutes |
| Affected Customers: |
All Windows web hosting customers |
| Reason of Outage: |
Regular Windows update and restarting server. If you have any questions or concerns, please feel free to contact us. We apologize for the inconvenience. |
| Date: |
12/16/2009 |
| Time: |
06:00 PM PT |
| Duration: |
30 Minutes |
| Affected Customers: |
No customer should be affected by the monthly generator test. |
| Reason of Outage: |
Generator will be tested once a month on every 3rd Wednesday of the Month around 5:30 PM. The total test time will be 30 minutes. If you have any questions or concerns, please feel free to contact us. We apologize for the inconvenience. |
| Date: |
11/27/2009 |
| Time: |
04:45 PM PT |
| Duration: |
10 Minutes |
| Affected Customers: |
SmarttNet Colocation Customers in Row F |
| Reason of Outage: |
Internetwork Operating System (IOS) Upgrade was scheduled at 11:30 PM PT. However, there was an Operation Error which made IOS Update take affect ahead of time. If you have any questions or concerns, please feel free to contact us. We apologize for the inconvenience. |
| Date: |
11/20/2009by Bell |
| Time: |
00:00 AM PT |
| Duration: |
2 Minutes |
| Affected Customers: |
SmarttNet Fiber Customers at specific locations in Vancouver |
| Reason of Outage: |
Update Trunking Configuration of Slot 5 Port 1-8 Ethernet. If you have any questions or concerns, please feel free to contact us. We apologize for the inconvenience. |
| Date: |
11/18/2009 |
| Time: |
05:30 PM PT |
| Duration: |
30 Minutes |
| Affected Customers: |
No customer should be affected by the monthly generator test. |
| Reason of Outage: |
Generator will be tested once a month on every 3rd Wednesday of the Month around 5:30 PM. The total test time will be 30 minutes. If you have any questions or concerns, please feel free to contact us. We apologize for the inconvenience. |
| Date: |
11/12/2009by Telus |
| Time: |
12:30 PM PT |
| Duration: |
8 Hours |
| Affected Customers: |
Specific SmarttNet Arista Customers |
| Reason of Outage: |
Telus is experiencing a problem with PGS (PPPoE) network for all ISP’s. RFO: a card’s CPU usage was pinned to 100%. Vendor was involved to resolve the issue. We apologize for the inconvenience. |
| Date: |
10/23/2009 |
| Time: |
11:30 PM PT |
| Duration: |
2 Hours |
| Affected Customers: |
SmarttNet Web Hosting Customers on Windows Server |
| Reason of Outage: |
Windows Update. If you have any questions or concerns, please feel free to contact us. We apologize for the inconvenience. |
| Date: |
10/21/2009 |
| Time: |
06:00 PM PT |
| Duration: |
30 Minutes |
| Affected Customers: |
No customer should be affected by the monthly generator test. |
| Reason of Outage: |
Generator will be tested once a month on every 3rd Wednesday of the Month around 5:30 PM. The total test time will be 30 minutes. If you have any questions or concerns, please feel free to contact us. We apologize for the inconvenience. |
| Date: |
10/14/2009 |
| Time: |
09:30 AM PT |
| Duration: |
15 Minutes |
| Affected Customers: |
SmarttNet’s automatic off-site backup customers |
| Reason of Outage: |
Server Upgrade. Customers will experience an outage of approx fifteen (15) minutes during the upgrade. If you have any questions or concerns, please feel free to contact us. We apologize for the inconvenience. |
| Date: |
09/17/2009 |
| Time: |
04:45 PM PT |
| Duration: |
10 Minutes |
| Affected Customers: |
Any Customers use mailadmin.smartt.com and mailreseller.smartt.com during upgrade periord. |
| Reason of Outage: |
Interface Upgrade for mailadmin.smartt.com and mailreseller.smartt.com. If you have any questions or concerns, please feel free to contact us. We apologize for the inconvenience. |
| Date: |
09/16/2009 |
| Time: |
05:30 PM PT |
| Duration: |
30 Minutes |
| Affected Customers: |
No customer should be affected by the monthly generator test. |
| Reason of Outage: |
Generator will be tested once a month on every 3rd Wednesday of the Month around 5:30 PM. The total test time will be 30 minutes. If you have any questions or concerns, please feel free to contact us. We apologize for the inconvenience. |
| Date: |
08/27/2009 |
| Time: |
2:00 – 4:40 AM, 5:09 - 5:28 AM PT |
| Duration: |
2:00 – 4:40 AM, 5:09 - 5:28 AM PT |
| Affected Customers: |
Customers connected to Switch A701 & E101 |
| Reason of Outage: |
Switch A701 went offline at approximately 2:00AM. There were no hardware or software faults in the log except that port01 went down first and then came back up several seconds later. Following this, all the ports on the switch went down, and complained about VTP domain being mismatched. The switch behavior has been changed to a routed port to prevent any further related VTP/STP issue.Switch E101 – failing GBIC. Had to reset the 2 ports to get the switch back online. Port gi0/2 didn’t come back up even other the reset to prevent interface flapping, it is now manually shutdown – ie only gi0/1 is up.We apologize for the inconvenience. |
| Date: |
08/24/2009 by Telus |
| Time: |
11:50 PM – 09:45 AM PT |
| Duration: |
10 hours and 55 minutes |
| Affected Customers: |
PGS based customers and VPOP customers on GigE 1/0 card in VANCBC01AR08 in Lower Mainland BC |
| Reason of Outage: |
PGS based customers and VPOP customers on GigE 1/0 card in VANCBC01AR08 in Lower Mainland BC experienced degraded service with their ADSL connections due to high CPU utilization. Trouble came clear while support was investigating the issue.Technicians working with the Vendor currently suspect there may be a defect in the code.If you have any questions or concerns, please feel free to contact us. We apologize for the inconvenience. |
| Date: |
08/19/2009 |
| Time: |
06:00 PM PT |
| Duration: |
30 Minutes |
| Affected Customers: |
No customer should be affected by the monthly generator test. |
| Reason of Outage: |
Generator will be tested once a month on every 3rd Wednesday of the Month around 5:30 PM. The total test time will be 30 minutes. If you have any questions or concerns, please feel free to contact us. We apologize for the inconvenience. |
| Date: |
08/05/2009 |
| Time: |
2:20 PM PT |
| Duration: |
10 Minutes |
| Affected Customers: |
Customers on the web servers Flanders and Hibbert |
| Reason of Outage: |
Hardware replacement. If you have any questions or concerns, please feel free to contact us. We apologize for the inconvenience. |
| Date: |
08/05/2009 |
| Time: |
3:10 AM PT |
| Duration: |
6 Hours 26 Minutes |
| Affected Customers: |
Customers on the web servers Flanders and Hibbert |
| Reason of Outage: |
Power supply died. Also a brief outage of around 5 minutes 11:15 – 11:20 due to kernel panic due to file corruption or a damaged motherboard. We are building a replacement for it. There will be a brief outage later when we switch to the new firewall. If you have any questions or concerns, please feel free to contact us. We apologize for the inconvenience. |
| Date: |
07/16/2009 |
| Time: |
06:00 PM PT |
| Duration: |
15 Minutes |
| Affected Customers: |
SmarttNet Website |
| Reason of Outage: |
Hardware migration. If you have any questions or concerns, please feel free to contact us. We apologize for the inconvenience. |
| Date: |
07/15/2009 |
| Time: |
05:30 PM PT |
| Duration: |
30 Minutes |
| Affected Customers: |
No customer should be affected by the monthly generator test. |
| Reason of Outage: |
Generator will be tested once a month on every 3rd Wednesday of the Month at 5:30 PM. The total test time will be 30 minutes. If you have any questions or concerns, please feel free to contact us. We apologize for the inconvenience. |
| Date: |
07/10/2009 |
| Time: |
11:59 PM PT |
| Duration: |
2-4Hours Intermittent |
| Affected Customers: |
All customers at various stages |
| Reason of Outage: |
Power distribution and rack upgrades. If you have any questions or concerns, please feel free to contact us. We apologize for the inconvenience. |
| Date: |
07/07/2009 |
| Time: |
12:03 AM PT |
| Duration: |
7 Hours 30 Minutes |
| Affected Customers: |
Customers on the web server Carbon |
| Reason of Outage: |
A customer was hacked. Hacker’s attack interfered with the standard automatic nightly maintenance causing a web service failure. An unrelated failure of the monitoring system caused the outage to be longer than it would otherwise be. We are continuing to investigate the hack incident. If you have any questions or concerns, please feel free to contact us. We apologize for the inconvenience. |
| Date: |
06/30/2009 |
| Time: |
06:36 PM PT |
| Duration: |
45 Minutes |
| Affected Customers: |
Customers on the web server Krypton |
| Reason of Outage: |
Uncertain. Failing hardware suspected. We will continue to monitor it and expedite the plans to replace it. If you have any questions or concerns, please feel free to contact us. We apologize for the inconvenience. |
| Date: |
06/17/2009 |
| Time: |
05:30 PM PT |
| Duration: |
30 Minutes |
| Affected Customers: |
No customer should be affected by the monthly generator test. |
| Reason of Outage: |
Generator will be tested once a month on every 3rd Wednesday of the Month at 5:30 PM. The total test time will be 30 minutes. If you have any questions or concerns, please feel free to contact us. We apologize for the inconvenience. |
| Date: |
06/16/2009 |
| Time: |
12:55PM PT |
| Duration: |
20 Minutes |
| Affected Customers: |
Abbotsford Dial Up Customers |
| Reason of Outage: |
Moving SmarttNet Abbotsford Dial Up Server to a new location. If you have any questions or concerns, please feel free to contact us. We apologize for the inconvenience. |
| Date: |
06/04/2009 |
| Time: |
12:04AM PT |
| Duration: |
28 Minutes |
| Affected Customers: |
Customers with sites hosted on the web server Carbon |
| Reason of Outage: |
Human error. If you have any questions or concerns, please feel free to contact us. We apologize for the inconvenience. |
| Date: |
06/03/2009 By Telus |
| Time: |
11:00PM PT |
| Duration: |
(24 Hour Clock) 23:00 – 05:00 AM PT |
| Affected Customers: |
Penticton, BC (PNTNBC02DE02) |
| Reason of Outage: |
TELUS technicians will be installing a new version of operating system software on our Penticton ADSL switch to address bugs with the current version and add additional functionality. This work will result in a brief 20 minute outage while the switch reboots. Please accept our apologies for any inconvenience caused. If you have any questions or concerns, please feel free to contact us. We apologize for the inconvenience. |
| Date: |
05/26/2009 |
| Time: |
10:00AM PT |
| Duration: |
10:00AM PT |
| Affected Customers: |
SmarttNet Webmail Customers |
| Reason of Outage: |
New minor software release for webmail.smartt.com. If you experience any issues with the web interface, please refresh the page or clear your cache. If you find messages not disappearing when you delete them, go to personal settings and uncheck the “Flag the message for deletion instead of delete” box and click save. If you have any questions or concerns, please feel free to contact us. |
| Date: |
05/20/2009 |
| Time: |
05:30 PM PT |
| Duration: |
30 Minutes |
| Affected Customers: |
No customer should be affected by the monthly generator test. |
| Reason of Outage: |
Generator will be tested once a month on every 3rd Wednesday of the Month at 5:30 PM. The total test time will be 30 minutes. If you have any questions or concerns, please feel free to contact us. We apologize for the inconvenience. |
| Date: |
05/19-20/2009 |
| Time: |
12:09PM PT |
| Duration: |
21 Hours and 35 Minutes |
| Affected Customers: |
Customers who use webmail to change their rules/vacation messages/passwords/forward settings. mailadmin.smartt.com was not affected by this problem. |
| Reason of Outage: |
A minor configuration change on the webmail server had the unanticipated side effect of causing errors in the pages that allow you to change the webmail preferences listed above. The problem was reported the morning on the 20th and soon corrected. Existing preferences continued to work, it was only the interface to change them that was affected. If you have any questions or concerns, please feel free to contact us. We apologize for the inconvenience. |
| Date: |
05/16/2009 |
| Time: |
Intermittent 11:30PM to 01:30AM PT |
| Duration: |
2 Hour Window |
| Affected Customers: |
Customers with Web hosting, email service, and IPI Photo services will receive intermittent disruption during the specified time period. |
| Reason of Outage: |
Software upgrades. Total downtime expected is approximately 5-15 minutes however a large window has been selected to account for any unexpected problems. If you have any questions or concerns, please feel free to contact us. We apologize for the inconvenience. |
| Date: |
05/16/2009 |
| Time: |
11:30PM to 11:45PMPT |
| Duration: |
15 Minutes |
| Affected Customers: |
All Arista Customers |
| Reason of Outage: |
Arista maintenance and hardware update. If you have any questions or concerns, please feel free to contact us. We apologize for the inconvenience. |
| Date: |
05/11/2009 |
| Time: |
Intermittent 10:00PM – 12:00AM PT |
| Duration: |
2 Hour Window |
| Affected Customers: |
Customers on the web server Carbon |
| Reason of Outage: |
Software upgrades. We expect multiple brief outages during the 2 hour window of a few minutes each and one 5 minute outage. If you have any questions or concerns, please feel free to contact us. We apologize for the inconvenience. |
| Date: |
05/09/2009 |
| Time: |
12:04 AM – 01:44 AM PT |
| Duration: |
1 Hour, 40 Minutes |
| Affected Customers: |
Customers on the web server Carbon |
| Reason of Outage: |
A customer site was hacked due to security problems with that customer. We are still investigating the situation but it appears to be a hacked ftp account. The malicious code inserted by this hacker interfered with the nightly web log rotation process causing some downtime. I suspect the code was designed to hijack the web server but was unable due to security restrictions resulting in a failure of the web server instead. If you have any questions or concerns, please feel free to contact us. We apologize for the inconvenience. |
| Date: |
05/05/2009 |
| Time: |
05:00PM – 05:10PM PT |
| Duration: |
10 Minutes |
| Affected Customers: |
All Arista Customers |
| Reason of Outage: |
Arista maintenance and hardware update. If you have any questions or concerns, please feel free to contact us. We apologize for the inconvenience. |
| Date: |
05/05/2009 |
| Time: |
11:59PM – 12:05AM PT |
| Duration: |
5 Minutes |
| Affected Customers: |
All Arista Customers |
| Reason of Outage: |
Arista maintenance. If you have any questions or concerns, please feel free to contact us. We apologize for the inconvenience. |
| Date: |
05/05/2009 By Telus |
| Time: |
01:00AM – 05:00AM PT |
| Duration: |
ESTIMATED OUTAGE PER CIRCUIT: 15 Minutes |
| Affected Customers: |
Vancouver, BC |
| Reason of Outage: |
TELUS PLANNED MAINTENANCE – TELUS Technicians will be replacing an ATM line card at our switch located in Vancouver, BC. The card is being replaced to expand the amount of available memory. This will result in a brief fifteen minute outage to your wholesale ADSL bridge groups provisioned on the TELUS network. Please accept our apologies for any inconvenience caused. |
| Date: |
04/15/2009 By Telus |
| Time: |
01:00AM – 05:00AM PT |
| Duration: |
ESTIMATED OUTAGE PER CIRCUIT: 10 Minutes |
| Affected Customers: |
This activity will affect your Managed Data/IP and or ADSL services in the White Rock, BC area. |
| Reason of Outage: |
TELUS PLANNED MAINTENANCE – Re-provisioning LAG port. TELUS technicians will be correcting a provisioning error on our ADSL switching equipment located in White Rock, BC. We will be updating the encapsulation on LAG 20 to correct the problem. This will result in a brief ten minute outage to all wholesale ADSL connections provisioned on the TELUS network. Please accept our apologies for any inconvenience caused. |
| Date: |
03/20/2009 |
| Time: |
11:30 PM |
| Duration: |
01 Hour |
| Affected Customers: |
IPI customers and SmarttNet website |
| Reason of Outage: |
Regular maintenance for IPI servers and scheduled upgrade for SmarttNet website server. If you have any questions or concerns, please feel free to contact us. We apologize for the inconvenience. |
| Date: |
03/09/2009 |
| Update: |
These outages have been confirmed to be caused by a bug in libwrap as I have documented in this bug report: http://www.freebsd.org/cgi/query-pr.cgi?pr=132705 |
| Time: |
04:05 AM |
| Duration: |
10 Minutes |
| Affected Customers: |
Database customers on the web server Carbon |
| Reason of Outage: |
We are continuing to investigate the cause of this recurring problem. If you have any questions or concerns, please feel free to contact us. We apologize for the inconvenience. |
| Date: |
02/26/2009 |
| Time: |
08:30 AM |
| Duration: |
1 Hour |
| Affected Customers: |
Database customers on the web server Carbon |
| Reason of Outage: |
The mysql lockup problems continue. We now have three techs investigating this problem. This may be a race condition or deadlock on mysql’s code. We will now be restarting the database every morning at 7AM in an attempt to work around this bug and resolving this issue. If you have any questions or concerns, please feel free to contact us. We apologize for the inconvenience. |
| Date: |
02/21/2009 |
| Time: |
11:30 PM |
| Duration: |
2 Hours |
| Affected Customers: |
SmarttNet Mail Server and Websites hosted on Carbon |
| Reason of Outage: |
Network Hardware Migration. If you have any questions or concerns, please feel free to contact us. We apologize for the inconvenience. |
| Date: |
02/19/2009 |
| Time: |
02:47 PM |
| Duration: |
21 hours (partial unreachability) |
| Affected Customers: |
The hand-full of Domains that use ns1/ns2.smartt.com as their authoritative name server. |
| Reason of Outage: |
The IPs of these DNS servers were changed, however their old IPs were still cached in the root name servers. When the old servers were shutdown, recursive server that were getting their cached IPs from the root servers would have problems resolving dns for those domains. Not all recursive dns servers were affected by this. We have updated the root servers with the correct IPs and turned the old servers back on to allow additional propagation time. If you have any questions or concerns, please feel free to contact us. We apologize for the inconvenience. |
| Date: |
02/17/2009 |
| Time: |
11:30 PM |
| Duration: |
Anytime between 12:00AM and 04:00 AM |
| Affected Customers: |
No outtage is expected. However, there is a very small chance that may happens. |
| Reason of Outage: |
Generator check. If you have any questions or concerns, please feel free to contact us. We apologize for the inconvenience. |
| Date: |
02/15/2009 |
| Time: |
05:51 PM |
| Duration: |
15 Hours |
| Affected Customers: |
The 3 mysql customers on Radon |
| Reason of Outage: |
This virtual server resides on the server Carbon which was rebooted in an attempt to resolve some other database issues. Due to a configuration error, it lost it’s IP and was not accessible. It was tested. However, because it was tested from the console, this issue was not noticed. If you have any questions or concerns, please feel free to contact us. We apologize for the inconvenience. |
| Date: |
02/15/2009 |
| Time: |
05:51 PM |
| Duration: |
5 Minutes |
| Affected Customers: |
Iridium and ns2 are secondary dns servers, so there should have been no significant interruption |
| Reason of Outage: |
These virtual servers reside on the server Carbon which was rebooted in an attempt to resolve some other database issues. If you have any questions or concerns, please feel free to contact us. We apologize for the inconvenience. |
| Date: |
02/15/2009 |
| Time: |
05:00 PM |
| Duration: |
1 Hours and 45 Minutes |
| Affected Customers: |
MySQL Customers hosted on the web server Carbon |
| Reason of Outage: |
The database problems continue. Tried switching the threading library to linuxthreads in the hope of improving stability, but that seemed to just make things worse, so I had to switch it back to process scope threads. Based on all things tried to bring it back up, I believe the problem may have been memory corruption. The databases will likely be migrated to another server in the near future to resolve this permanently. If you have any questions or concerns, please feel free to contact us. We apologize for the inconvenience. |
| Date: |
02/10/2009 |
| Time: |
01:09 PM |
| Duration: |
2 Minutes |
| Affected Customers: |
The remaining three customers who have mailing lists on the legacy Dowco mailing list system. |
| Reason of Outage: |
Migration to new hardware. If you have any questions or concerns, please feel free to contact us. We apologize for the inconvenience. |
| Date: |
02/07/2009 |
| Time: |
1:40 PM |
| Duration: |
30 Minutes |
| Affected Customers: |
Customers with mySQL databases on the web server Carbon |
| Reason of Outage: |
MySQL is locked up. If you have any questions or concerns, please feel free to contact us. We apologize for the inconvenience. |
| Date: |
02/06/2009 |
| Time: |
12:30 AM |
| Duration: |
30 Minutes |
| Affected Customers: |
Websites hosted on the web server Carbon |
| Reason of Outage: |
There was a problem with apache segfault. Reboot was not initiallized automatically. If you have any questions or concerns, please feel free to contact us. We apologize for the inconvenience. |
| Date: |
02/04/2009 |
| Time: |
12:53 PM |
| Duration: |
6 Minutes |
| Affected Customers: |
Websites hosted on Carbon |
| Reason of Outage: |
The new server does not like the encryption while trying to migrate and install a ssl. If you have any questions or concerns, please feel free to contact us. We apologize for the inconvenience. |
| Date: |
01/17/2009 |
| Time: |
11:00 PM |
| Duration: |
2 Hours |
| Affected Customers: |
Web hosting of Windows server |
| Reason of Outage: |
The latest updates will be made to all Windows server, the mail system and Web servers.
This will require various brief outages as each component is upgraded. Total downtime for each component is expected to be less than 5 minutes. |
| Date: |
01/13/2009 |
| Time: |
12:00 PM |
| Duration: |
No ETA |
| Affected Customers: |
ADSL Customers_Vancouver Area |
| Reason of Outage: |
There is a network problem in the RGNT CO right now with 2+ ports. There is no ETA yet from Central Office. If you have any questions or concerns, please feel free to contact us. We apologize for the inconvenience. |
| Date: |
01/09/2009 |
| Time: |
7:56 PM |
| Duration: |
13.5 Hours |
| Affected Customers: |
Customers with mySQL databases on the web server Carbon |
| Reason of Outage: |
MySQL locked up. If you have any questions or concerns, please feel free to contact us. We apologize for the inconvenience. |
| Date: |
12/28/2008 |
| Time: |
1:00 AM |
| Duration: |
11.5 Hours |
| Affected Customers: |
Websites hosted on the web server Carbon that use MySQL databases |
| Reason of Outage: |
Mysql server crashed. If you have any questions or concerns, please feel free to contact us. We apologize for the inconvenience. |
| Date: |
12/05/2008 |
| Time: |
12:02 AM |
| Duration: |
6 Hours |
| Affected Customers: |
Websites hosted on the web server Carbon |
| Reason of Outage: |
Configuration error. If you have any questions or concerns, please feel free to contact us. We apologize for the inconvenience. |
| Date: |
12/06/2008 |
| Time: |
11:00 PM |
| Duration: |
2 Hours |
| Affected Customers: |
Windows Web Server and IPI Server customers |
| Reason of Outage: |
Due to Scheduled Maintenance, interruption of the web site access could occur to some users. If you have any questions or concerns, please feel free to contact us. We apologize for the inconvenience. |
|
| Date: |
10/25/2008 |
| Time: |
12:37 PM |
| Duration: |
1 Hour |
| Affected Customers: |
Customers attempting to sent email through smtp.smartt.com. Incoming email slowed. |
| Reason of Outage: |
Component of mail system crashed. Server configuration has been adjusted to have additional checks to better automatically recover from such errors. If you have any questions or concerns, please feel free to contact us. We apologize for the inconvenience. |
|
| Date: |
10/22/2008 |
| Time: |
12:13 PM |
| Duration: |
42 Minutes |
| Affected Customers: |
Customers with domains on the legacy Dowco mail system including @dowco.com accounts |
| Reason of Outage: |
Disk capacity problems on smtp-be1 caused problems for users of imap (including webmail) and may have delayed incoming mail or caused errors when trying to send. If you have any questions or concerns, please feel free to contact us. We apologize for the inconvenience. |
|
| Date: |
10/16/2008 |
| Time: |
5:07 PM |
| Duration: |
23 Minutes |
| Affected Customers: |
Customers on the legacy Dowco mail server who have not yet migrated to the new system. |
| Reason of Outage: |
Raid card firmware crash. If you have any questions or concerns, please feel free to contact us. We apologize for the inconvenience. |
|
| Date: |
10/06/2008 |
| Time: |
Morning – 11:30 AM |
| Duration: |
See above |
| Affected Customers: |
Email users of the 4 domains remaining on the Lisa mail server |
| Reason of Outage: |
Saturday’s power outage caused the failure of one of the hard drives on Lisa and file system corruption on another. The corruption has been repaired and the failed drive removed from the system configuration. The remaining domains will be migrated off the server today. If you have any questions or concerns, please feel free to contact us. We apologize for the inconvenience. |
|
| Date: |
10/04/2008 |
| Time: |
2:38 PM |
| Duration: |
1 Hour 10 Minutes |
| Affected Customers: |
All Customers |
| Reason of Outage: |
On October 4th, 2008, a car crashed into a Burnaby bchydro substation at approximately 2:02pm, taking out the power in three neighbourhoods and affecting 2100 BC Hydro customers. SmarttNet’s UPS Battery backup carried the load until 2:38pm. The subsequent outage lasted until 3:45PM after which time full service was restored. We apologize for the inconvenience. Please direct all your inquiries to outage@smartt.com. |
|
| Date: |
09/24/2008 |
| Time: |
Early Morning |
| Duration: |
Early Morning to 9:00 AM |
| Affected Customers: |
Customers on the web server Carbon who require php4 enabled for compatibility reasons. |
| Reason of Outage: |
PHP4 configuration problem. If you have any questions or concerns, please feel free to contact us. We apologize for the inconvenience. |
|
| Date: |
09/07/2008 |
| Time: |
04:38 AM |
| Duration: |
3 Hours |
| Affected Customers: |
Customers using webmail on the new mail server, the mailadmin system. pop/imap and authenticated smtp may have also been affected. |
| Reason of Outage: |
Database problems. If you have any questions or concerns, please feel free to contact us. We apologize for the inconvenience |
|
| Date: |
08/18/2008 |
| Time: |
04:45 PM |
| Duration: |
85 Minutes |
| Affected Customers: |
Customers on the new mail server (Barium) |
| Reason of Outage: |
Hardware problems. The main board was replaced in an attempt to fix the hardware problems we have been experiencing. Upon replacement, problems were encountered with the raid card, so it was also replaced. The firmware on the new hardware then needed to be updated and the bios configuration set, which is unfortunately a very slow process. The server is now up and no more problems are expected. Due to the combined factors of the raid re-syncing, the fsck of the drives due to the ungraceful shutdowns, and the large influx of mail from the downtime, the server will be slow for the next few hours and you may experience timeouts or authentication errors due to the unusually high load. If you have any questions or concerns, please feel free to contact us. We apologize for the inconvenience. |
|
| Date: |
08/18/2008 |
| Time: |
11:00 – 11:05, 11:25 – 11:49, 11:49 – 12:09, 12:44 – 13:17, 13:55 – 13:59, 14:00 – 14:11, 14:20 – 14:24, 15:07 – 15:12 |
| Duration: |
See Above |
| Affected Customers: |
Customers on the new mail server (Barium) |
| Reason of Outage: |
Hardware problems. We have called in a tech from IBM to replace the server main board at 4:00PM today. There will be a scheduled outage at this time of approximately half an hour. If you have any questions or concerns, please feel free to contact us. We apologize for the inconvenience. |
|
| Date: |
08/14/2008 |
| Time: |
15:15-15:20, 15:45-16:10, 16:29-22:00 |
| Duration: |
See Above |
| Affected Customers: |
Customers using the new mail server (Barium). |
| Reason of Outage: |
Due to multiple spontaneous reboots and lockups, the server was brought for several hours down to diagnose and fix the underlying hardware problems. Upon deep investigation, the problem was determined to likely be the RAID card. On advice of IBM technicians, we upgraded the BIOS and the firmware on the RAID card to a version released last week which has solved the problems. We also brought in a replacement card to replace the current one in the event that further problems occur. |
|
| Date: |
08/14/2008 |
| Time: |
09:19 PM |
| Duration: |
21 Minutes |
| Affected Customers: |
Customers using the unix web server Carbon |
| Reason of Outage: |
To help ensure continued stability and reliability, the server was brought down for 20 minutes to upgrade the raid card’s firmware, system bios, and remote management bios. |
|
| Date: |
08/12/2008 |
| Time: |
11:00 PM |
| Duration: |
5 Minutes |
| Affected Customers: |
Unix web hosting customers on the server Carbon |
| Reason of Outage: |
Kernel Update_Scheduled Server Reboot-Carbon. If you have any questions or concerns, please feel free to contact us. We apologize for the inconvenience. |
|
| Date: |
08/12/2008 |
| Time: |
11:00 PM |
| Duration: |
5 Minutes |
| Affected Customers: |
Mail customers behind the Barracuda spam firewall. This includes all former Dowco customers, iMail customers, and customers who have moved to the new mail system. The affect of this will only be a slight delay in arrival of new incoming email messages sent during this time. The exact delay depends on the configuration of the sending mail server. It will also cause a brief delay for customers sending mail through or having aliases/forwards on the Tigger/datapark mail server. |
| Reason of Outage: |
Scheduled Firmware upgrade of Barracuda spam firewall |
|
| Date: |
08/05/2008 |
| Time: |
02:45 PM |
| Duration: |
15 Minutes |
| Affected Customers: |
Email customers using the mail server November (mail.smartt.com) |
| Reason of Outage: |
Configuration changes to help with spam problems caused brief outages due to service restarts and high server load. We apologize for the inconvenience. |
|
| Date: |
07/30/2008 |
| Time: |
11:30 AM |
| Duration: |
80 Minutes |
| Affected Customers: |
IMail Customers |
| Reason of Outage: |
Corrupted emails in the Queue manager. If you have any questions or concerns, please feel free to contact us. We apologize for the inconvenience. |
|
| Date: |
07/08/2008 |
| Time: |
12:56 PM |
| Duration: |
50 Minutes |
| Affected Customers: |
1/3 of the Aggregated ADSL customers |
| Reason of Outage: |
One of the Aggregated ADSL servers has a database/configuration failure. If you have any questions or concerns, please feel free to contact us. We apologize for the inconvenience. |
|
| Date: |
05/20/2008 |
| Time: |
10:57 AM |
| Duration: |
6 Minutes |
| Affected Customers: |
SmarttNet Datacenter |
| Reason of Outage: |
Human Error. If you have any questions or concerns, please feel free to contact us. We apologize for the inconvenience. |
|
| Date: |
05/10/2008 |
| Time: |
10:00 PM |
| Duration: |
15-30 Minutes |
| Affected Customers: |
Unix web hosting customers on the server Carbon, mostly former Dowco and Jurock/Featureweb customers |
| Reason of Outage: |
Replace a faulty hard drive backplane, adjust BIOS settings, confirm firmware updates. |
|
| Date: |
05/10/2008 |
| Time: |
07:16 AM |
| Duration: |
20 Minutes |
| Affected Customers: |
Unix web hosting customers on the server Carbon, mostly former Dowco and Jurock/Featureweb customers |
| Reason of Outage: |
Firmware upgrades. If you have any questions or concerns, please feel free to contact us. We apologize for the inconvenience. |
|
| Date: |
05/10/2008-05/11/2008 |
| Time: |
Between midnight until 6 am |
| Duration: |
6 Hours |
| Affected Customers: |
All Customers |
| Reason of Outage: |
Relocating fibre optic cable due to construction project. Cable must be moved or it will be cut. MTS Allstream must perform forced fibre relocations on our Starcom cable between Vancouver and Seattle. |
|
| Date: |
05/07/2008 |
| Time: |
12:09 pm |
| Duration: |
2 Hours |
| Affected Customers: |
ADSL customers in the Penticton and Kelowna area connected to the Kelowna Central Office. |
| Reason of Outage: |
Telus core ATM went down as a result of hardware failure. If you have any questions or concerns, please feel free to contact us. We apologize for the inconvenience. |
|
| Date: |
05/01/2008 |
| Time: |
7 am |
| Duration: |
2 Hours |
| Affected Customers: |
Jurock/featureweb email users |
| Reason of Outage: |
Hardware replacement of the mail server Lisa. The server hardware has been replaced. A new backup is currently being generated. No more downtime is expected from this server. |
|
| Date: |
04/29/2008 |
| Time: |
1 am |
| Duration: |
6 Hours |
| Affected Customers: |
Jurock/featureweb email users |
| Reason of Outage: |
Lisa is continuing to have intermittent problems while we prepare to migrate data to a new server. If you have any questions or concerns, please feel free to contact us. We apologize for the inconvenience. |
|
| Date: |
04/28/2008 |
| Time: |
5 am |
| Duration: |
5.5 Hours |
| Affected Customers: |
Jurock/featureweb email users |
| Reason of Outage: |
The mail server Lisa had a partial hard drive failure causing a major disruption of email services. Server is now running off backup drive. Mail service will be slow while the server catches up on the missed email. There may be an additional planned outage in the evening or night in the near future to add a new backup drive. Migration of accounts from this server to our new mail server will start soon to prevent further problems. |
|
| Date: |
04/27/2008 |
| Time: |
3:15 AM |
| Duration: |
7 Hours |
| Affected Customers: |
Jurock/featureweb email users |
| Reason of Outage: |
Further issues cause by failing drive. Server was brought back into a working state. If you have any questions or concerns, please feel free to contact us. We apologize for the inconvenience. |
|
| Date: |
04/25/2008 |
| Time: |
10:00 PM |
| Duration: |
3 Hours |
| Affected Customers: |
Jurock/featureweb email users |
| Reason of Outage: |
Planned emergency maintenance to attempt to repair failing hard drive. Replacement was unsuccessful due to unexpected technical problems and server was restored to previous state. |
|
| Date: |
04/22/2008 |
| Time: |
12:30 PM – 3:10 PM |
| Duration: |
1.5-2.5 Hours |
| Affected Customers: |
All ADSL customers. The non-ADSL+ network came back around 2:20pm. The ADSL+ network came back up around 3:10pm. |
| Reason of Outage: |
General outage on Telus ADSL backend. Telus still hasn’t given a specific reason for the outage. |
|
| Date: |
04/05/2008 |
| Time: |
11:00 PM – 5:00 AM |
| Duration: |
6 Hours |
| Affected Customers: |
Customers’ Managed Services that traverse or terminate on the indicated in New Westminster, BC |
| Reason of Outage: |
NETWORK UPGRADES – Upgrade of NWMR73. If you have any questions or concerns, please feel free to contact us. We apologize for the inconvenience. |
|
| Date: |
04/02/2008 |
| Time: |
6:00 PM |
| Duration: |
10-30 Minutes |
| Affected Customers: |
Burnaby Dialup, Wireless Customers |
| Reason of Outage: |
Final stage of datacenter move. If you have any questions or concerns, please feel free to contact us. We apologize for the inconvenience. |
|
| Date: |
03/09/2008 |
| Time: |
2:00 AM |
| Duration: |
10 Minutes |
| Affected Customers: |
ADSL, SDSL, and Extended LAN |
| Reason of Outage: |
Reason for Outage:Cisco IOS bug. Some MAC addresses were not resolved properly. If you have any questions or concerns, please feel free to contact us. We apologize for the inconvenience. |
|
| Date: |
03/02/2008 |
| Time: |
1:00 AM |
| Duration: |
4 Hours |
| Affected Customers: |
All Customers |
| Reason of Outage: |
Reason for Outage:As part of our infrastructure upgrade, we will be physically moving essential devices. The outages experienced by customers will be intermittent. If you have any questions or concerns, please feel free to contact us. We apologize for the inconvenience. |
|
| Date: |
02/16/2008-02/17/2008 |
| Time: |
2:00 AM |
| Duration: |
4 Hours |
| Affected Customers: |
All Customers |
| Reason of Outage: |
Each night 1 upstream provider will connect fiber to our new data center. This transition will result in intermittent outage to various services. If you have any questions or concerns, please feel free to contact us. We apologize for the inconvenience. |
|
| Date: |
02/07/200-02/08/2008 |
| Time: |
5:30 PM |
| Duration: |
5 Minutes |
| Affected Customers: |
Some DSSL, Fiber Customers |
| Reason of Outage: |
One of our upstream providers will be making some fiber based software. This will allow us to apply some of the recent infrastructure upgrades we have implemented. If you have any questions or concerns, please feel free to contact us. We apologize for the inconvenience. |
|
| Date: |
01/06/2008 |
| Time: |
10:05 AM |
| Duration: |
0.5 Hour |
| Affected Customers: |
SmarttNet Web Mail |
| Reason of Outage: |
One of our mail servers experienced an unexpected shutdown during standard daily maintenance. The service was restored within 35 minutes. The service outage did not affect all customers. We apologize for any inconvenience this may have caused. |
|
| Date: |
12/08/2007 |
| Time: |
Midnight to 01:00 AM |
| Duration: |
1 Hour |
| Affected Customers: |
SmarttNet Customers |
| Reason of Outage: |
Hydro, power upgrade. This activity will affect all the services provided by Smarttnet that includes all the connectivity and hosting services. CUSTOMERS WILL BE WITHOUT INTERNET SERVICE FOR 60 MINUTES WHILE WE UPGRADE POWER SUPPLY TO OUR NEW DATA CENTRE. Customers requiring further details description could contact our office Monday to Friday 8 AM to 6 PM by phone or email us at support@smartt.com We appreciate your patient while we are upgrading our systems to serve you better in future and again we apologize for any inconvenience this may cause. |
|
| Date: |
11/07/2007 |
| Time: |
11:55 AM |
| Duration: |
0.5 Hour |
| Affected Customers: |
SmarttNet Customers |
| Reason of Outage: |
At approximately 11:55 AM a Telus switch experienced a failure which spread to one of our switches. The problem was resolved within 30 minutes by SmarttNet and Telus staff with service being restored by approximately 12:25 PM. Customers may have experienced DNS related issues such as an inability to browse. We apologize for any inconvenience this may have caused. |
|
| Date: |
10/10/2007 |
| Time: |
9:35 PM |
| Duration: |
|
| Affected Customers: |
SmarttNet Customers |
| Reason of Outage: |
There was a system wide network outage which affected our customers and originated from one of our core routers. The problem was rectified at approximately 9:35 PM. We apologize for the inconvenience. |
|
