CoursePlus Outage on October 24, 2022
CoursePlus was unavailable from 6:45pm US Eastern time on Monday, October 24 until 7:37pm the same day. This was the first unplanned outage for CoursePlus since 2020. We know that any downtime for CoursePlus is frustrating and difficult for the thousands of students, faculty, and staff who rely on immediate access to CoursePlus every day. This frustration is only multiplied when final exams are in process. This post details what we know about the cause of the problem, and the remediation strategies currently being used.
Like all modern web applications that are not tiny, lightly-used applications, CoursePlus runs on multiple servers. When you make a request to the CoursePlus website, your request is routed through a tool known as a load balancer. The load balancer’s job is to evenly distribute requests to all the servers that make up CoursePlus. Your web browser does not speak to a CoursePlus server directly. Your browser talks to CoursePlus through the load balancer. At approximately 6:40pm, there was a spike in the number of connections to the CoursePlus load balancer. At the same time, connections between the load balancer and the individual servers that make up CoursePlus dropped significantly. This indicates that there was a connection issue between the load balancer and the individual servers that make up CoursePlus.
Fortunately, the BSPH IT team was alerted to this problem fairly quickly, and worked with the CoursePlus team to get the servers up and running again. CoursePlus was operating normally again within the hour. Additionally, the load balancer received a software update today – Tuesday, October 25 - which should bring greater reliability to the device as a whole.
The timing of this specific outage could not have been worse for students taking final exams. We appreciate the fast responses of so many faculty and TAs in addressing student concerns. The CoursePlus team is grateful for the quick response by the BSPH IT team to this issue, and for the tenacity of everyone who was using CoursePlus during this time.