Australia’s 2016 Census … did not happen. Epic fail.
Australia was supposed to hold it’s first computerized census last night, Tuesday August 9. The idea was that we would all fill out our census forms online, giving a snapshot of the nation on that night. Previous censuses had all been filled out on one night on paper forms. So after dinner we would all go to the ABS website and fill in our forms. There are 24 million Australians and about 10 million households, mainly in one time zone.
Here is the problem: The Australian Bureau of Statistics (ABS) claimed they’d stress tested the Census system and were confident it could process one million forms per hour, yet they also expected two-thirds of Australians to complete their census forms online. Two thirds of 10 million households is about 7 million forms, in about 2 hours after dinner on the east coast, is 3.5 million forms per hour.
Chaos ensued. No wonder really important stuff like food production or making stuff like cars, computers, and airplanes is left to private industry (except in the old Soviet Union, but they died of shame).
The ABS site of course crashed, and now no one can fill out their census form as required by law. As of midday Australian eastern time the following day, the ABS website is still down and showing the previous day’s date:
The ABS were under fire for keeping our names and addresses on our census forms forever, thereby invading privacy like never before (four years as raw data and then codified after that). But (a) no database connected to the Internet is safe, (b) the ABS is revealed as utterly incompetent.
On a related note, some ABS officers have been convicted in recent years in multi-million dollar schemes to trade on sensitive economic data before it is released to the financial markets.
Enough of the serious stuff. What are real Australians saying?
Having wasted 45mins filling it in, I now want $180 from
@ABSCensus for every day that I can’t enter MY data!
Invading your privacy is important to us, so please be patient. You are number 21,000,002 in the queue.
If they can fine me for not putting my name on it, surely they can just put my name on it.
If I am ordered to fill out the Census, then I will fill it out with bogus information. And I won’t be shy about it. If they chose to throw the maximum fine of $1,800 at me for lying on the form then so be it, I’ll fight it out with them then.
The beauty of this approach is that there’s every chance they won’t try and fine me (as they love to point out, they’ve never actually fined anyone in the past), and the ‘$180 per day’ doesn’t start accruing until they send me an ‘order’, which is unlikely to happen. If they DO send me an ‘order’ to complete the Census, then I’ll complete their damn Census… I’ll answer every question so completely it’ll set their heads spinning… And none of it will be true.
But by answering (fake or otherwise) it will ‘stop the clock’ on the $180 per day fine, and cap my financial exposure to $1,800 + X days worth of the $180 per day fine depending on how long it takes to get around to filling out their damn form with all my bogus information after they issue the order. So call it $2,500 give or take, if they choose to chase me.
UPDATE: Census fail: ABS says hackers attacked website despite denials, after nearly $500,000 was spent on load testing servers, by Rod Chester.
Australian Census minster Michael McCormack insisted it was “not an attack”, just hours after ABS chief statistician David Kalisch declared it “an attack – it was quite clear it was malicious”.
Mr McCormack said the government shut down the site to avoid data being compromised. Yet A Denial of Service attack does not target data, it simply floods servers with incoming messages.
The government has also failed to explain why, if the reason for shutting down the servers was to stop a DDOS attack, why the servers continue to be down today.
The ABS this morning described it as a foreign attack, yet the ABS blocked traffic to international IP addresses at 11am yesterday.
In a DDOS attack, the information on the servers is typically not compromised. An analogy is to consider your letter box. A hack would be like someone opening your letters and reading them, a DDOS attack is like someone flooding your letterbox with so many letters the postman can’t get to it.
There are three clues to incompetence right there. Legally requiring 10 million households to fill out a form in a couple of hours on a computer system designed for a million forms an hour is not a DDOS attack, it’s just stupid. It a highly predictable digital traffic jam.
Matthew Hackling, a cybersecurity expert, said on Twitter today that there was no evidence of a DDOS attack, with international data maps showing no suspicious activity in Australia in that time.
“Reputable DDOS attack maps have not shown any abnormal traffic in this period.”
Not a conspiracy or hacker attack, just a stuff up. Government ministers and bureaucrats are not telling the truth.
The ABS is now working with the Australian Signals Directorate to determine the source of the attack.
“The first three (attacks) causes minor disruption and we received more than two million forms, submitted and safely stored, at the ABS,” he told the ABC.
“We did then have a fourth attack just after 7.30pm.
“That’s probably when many people had finished their dinner and were sitting down to use the online Census form, where we had a fourth attack and we took the precaution of closing down the system.”
So who built and tested the ABS system?
An Australian technology company [Revolution IT] with expertise in software testing was paid nearly $500,000 to ensure the Census servers would not crash under the load. …
Tender documents show that IBM Australia built the Census platform for a cost of $9,606,725.
UPDATE: Online attack poses problems for Malcolm Turnbull, his ministers and the ABS, by David Crowe.
The Census has become a test of basic government competence. At this stage, Malcolm Turnbull and his ministers are failing badly.
After years of planning, the project is crumbling under the weight of concerns over privacy, fears about security and now the alarm over foreign attacks on the Census website.
The Australian Bureau of Statistics has been too complacent about building community support for this huge and necessary stocktake of the nation. Worse, the responsible ministers were invisible until it was too late to settle the worries over personal privacy.
No one ever resigns any more. Remember the old principle of ministerial responsibility? Even if a minster knew nothing personally about some incident, if there was an awful failure in his department he resigned. Kept the ministers trying harder to prevent problems — worked like magic, the British built an empire on it. The current mob of weasels (and we are talking political culture here, for the last two decades) just blame it on some underling and keep their high paying, high status jobs. Mediocrity from government ensures.
In this case, even the cover-up afterwards is incompetent and mendacious. Hack? No. DDOS? No. Stupid design by government? Yes.
The only thing modern politicians and senior bureaucrats seem to do well is manage the 24 hour news cycle, manipulating by lying by omission and occasionally by commission. All show, with no principles or substance any more. Maybe their behavior would improve if we cut their wages to 1.5 times the average wage, instead of letting them set their own salaries.
UPDATE: In 2015 the federal government invested $250 million to upgrade ABS infrastructure, systems and processes (hat-tip Catallaxy).
UPDATE: Private companies in the US can sell you a report on any individual with all data in known public and some private databases, for as little as $10. Maybe the ABS could just buy the consensus info? Maybe get a quote from Chinese hackers? Might be cheaper.
