It's not the SAN: ATO says mainframe needed a reboot
The Australian Taxation Office (ATO) has denied the outage experienced on Wednesday was related to the HPE SAN issue that has been plaguing the government department since December.
In a statement, the ATO said its systems are back up and running with full functionality, following a disruption that impacted its mainframe, five days into the new financial year.
"We identified intermittent system issues early this afternoon affecting our mainframe and impacting on our services to the community," the Taxation Office said. "This was caused by applications running incorrectly."
The ATO said it took "controlled action" to reboot its mainframe to resolve the issue, and said its systems were not compromised, that no data had been lost, nor that it was subject to a cyber attack.
"The issue experienced today does not relate to the ATO's capability and capacity to deliver for Tax Time 2017," the government entity explained.
"While an outage is not optimal, our decision today to address our degraded system performance was to ensure they were fully operational for the 8.00-9.00pm window when we know the majority of people choose to lodge their tax returns."
Services impacted by Wednesday's outage included ATO Online services, including myTax; The Tax Agent, Business, and BAS Agent Portals; Standard Business Reporting (SBR); the Australian Business Register (ABR); and the department's case management systems.
"The ATO operates a highly sophisticated and complex IT environment. We do whatever we can to minimise any impact on our ability to provide reliable services," the statement continued.
The ATO said Wednesday's outage, which occurred shortly after Australian Commissioner of Taxation Chris Jordan told the National Press Club the Taxation Office expects its IT will operate smoothly in the future, was not related to recent hardware issues or its storage area network (SAN).
A report on a series of outages the Taxation Office experienced as a result of the failure of the HPE owned and operated equipment at the end of 2016 said the SAN could not handle more than one drive or cage failure thanks to a design decision taken by HPE.
As noted by the report, an analysis of logs from the six months before the incident showed a number of alerts indicating problems with the SAN.
"Since May 2016, at least 77 events related to components that were observed to fail in the December 2016 incident were logged in our incident resolution tool," the ATO said previously. "We were not made fully aware of the significance of the continuing trend of alerts, nor the broader systems impacts that would result from the failure of the 3PAR SAN."
The exact root cause for the outage is pending a report from HPE due to arrive in late 2017, however the report placed blame on the degradation of a number of fibre optic cables used within the SAN.
Most damning was HPE's lack of preparation for an event of the kind experienced by the ATO in December.
"Recovery procedures for applications in the event of a complete SAN outage had not been defined or tested by HPE," the ATO said.
As a result of the incidents, the ATO has rebuilt its storage solution with a new 3PAR, and once data from the existing 3PAR SAN is transferred, it will be decommissioned in July for forensic analysis.