all times Toronto time unless otherwise noted
- 2013-03-21: Service degraded due to ISP issue, starting at 2:37pm Toronto time. Ticket opened with provider. (Service restored at 10:43pm)
- 2013-03-16: Infrastructure maintenance may result in brief disconnections/interruptions over the next 24 hours.
- 2013-02-17 2:27am: Some DNS changes are causing some domains not to resolve. I expect these problems to clear in the next 12-24 hours.
- 2013-02-16 7:10pm: Antivirus and antispam software is being upgraded. Some mailboxes are offline for the next 10-15 minutes.
- 2013-02-16 6:59pm: A mailbox server is being rebooted for security patches. Some mailboxes will be unavailable for the next 3 minutes or so.
- 2013-02-16 10:01am: Exchange service upgrade complete. Downtime was 46 minutes across all users.
- 2013-02-16 09:08am: Exchange services are being upgraded. Access to email may be intermittent for some users during this process. ETA for completion is 12:00PM noon, Toronto time. How long until service is restored?
Incident: 2012-05-18 -- 2012-05-29
- 2012-05-18: Packet loss reported. Line problem identified, opened ticket with ISP.
- 2012-05-18 2:01pm EST: Bell connectivity problem being worked on; connection lost.
- 2012-05-18 4:09pm EST: Currently completely disconnected (again) due to a Bell problem. Backup link is being re-established, ETA 3-4 hours. All inbound and outbound mail is currently not functioning. Sent emails will be delayed until service is restored. We expect the main internet line to be re-established in approximately 24 hours (awaiting Bell response.)
- 2012-05-29: Main link back online.
Incident: 2011-12-29 -- 2012-01-04
- A Bell Canada DSL line issue has taken the network offline. A ticket has been opened with the ISP and is awaiting resolution.
- Partial access to some services is currently being established. Some services will begin to slowly come online via a cellular backup link.
- Received emails during this period may have been returned to sender due to the extended outage -- please request emails to be resent once service is restored.
- Note that most services are SLOWER than usual since they are running over a cellular link. Your patience is appreciated -- we're working on the problem.
- Websites: Most are offline. In progress.
- Outbound email: Now functional. If you have a domain name hosted you may need to add mx2.justinho.com as a MX DNS record if it is not already there. Email me if you need help.
- Inbound email: Online.
- Mobile push email: Mostly online, depending on client configuration. Not all phones will be able to connect. Add JUPITERSTATION as the domain or as a prefix to your login name (i.e. JUPITERSTATION\your-user-name)
- Web access: Online at https://mail.justinho.com/owa . Authenicate using JUPITERSTATION\your-user-name.
- Blackberry BES Push email: Online.
- 2011-12-29 16:21PST -- ~5% packet loss being reported from monitoring, TekSavvy discovers lantern not mapped. Opened ticket with Bell.
- 2011-12-31 07:55PST -- Site goes offline. Another ticket opened shortly thereafter.
- 2011-12-31 11:20PST -- No update. Still waiting.
- 2011-12-31 13:34PST -- Bell confirms that line was incorrectly modified.
- 2011-12-31 22:48PST -- No update.
- 2012-01-01 06:05PST -- Still no service.
- 2012-01-01 14:23PST -- A backup cellular link is now being established to temporarily connect services. No ETA at this point, but hopefully things will be back online (but slow!) in the next few hours.
- 2012-01-01 23:43PST -- We weren't able to provision service on a long weekend, but will get this backup link established in the future.
- 2012-01-01 09:25PST -- A technician is onsite now to swap the modem to 'resolve' the first lantern not mapped problem. Site remains offline.
- 2012-01-01 21:20PST -- The backup cellular link is now routing some traffic. We are gradually re-routing some services via this (slow) link to bring email and some websites online at this time.
- 2012-01-03 06:05PST -- Ticket being routed to technician again.
- 2012-01-03 07:30PST -- Bell ticket is being worked on by tech, expect fix between 11am - 3pm today
- 2012-01-03 15:05PST -- Problem still not fixed. No update.
- 2012-01-04 06:01PST -- Update from Bell: Line mapping incorrect on ATM link, being fixed, ETA 08:00PST (11:00EST).
- 2012-01-04 07:56PST -- DSLAM reconfigured, ATM group believes everything should be working. Still no connectivity.
- 2012-01-04 08:21PST -- Still no connectivity. Power cycled modem again, and called ISP again to escalate the issue. DSL was moved to gigE port on Bell's side, off ATM links.
- 2012-01-04 09:25PST -- Service has been restored. DNS entries are now being updated to point back to the main DSL link -- expect full service restoration in the next 20-30 minutes while all routing entries are restored to normal configuration.