The conversation

Lessons learnedExperience excerpt · English

I'm not going to name any names: in the late 00s/early 10s I worked in one of the highest-profile, …

… I'm not going to name any names: in the late 00s/early 10s I worked in one of the highest-profile, high-growth tech startups of its era, and I've personally made a blunder that corrupted literally millions of user records in the database. This incident was known internally as one of the most disastrous technical things that happened in the company's history, among a few others. The nature of the product was one of very quickly updating data, and updates were critically important (e.g. is affected by user spends) and hence restoring from DB backups of even the night before was unfeasible. There was irreparable damage where a whole team of us had to spend the next few weeks painstakingly hand-fixing data for users, and coming up with algorithms/code to fix these things as users use the product as they go. As you expect in this anecdote, I did not get fired, I was part of the team that worked tirelessly following this incident to fix user data, and I continued to have a good, growing career in my remaining time in this company (the next few years).

Original discussionThe first time I was almost fired from Apple149 source comments · at import
8

Community replies 0

No community replies yet. Add your perspective.

Replies from the source 8

Direct replies available in this import. Original-language excerpts.

MI

There are, I think, different norms and risk levels in different types of employment. Somewhat ironically, it's much easier to get fired from a minimum wage fast food service job than it is to get fired from a six-figure-salary job at Google.

Full original
PJ

In most European countries it is certainly hard to fire people on a whim like in US and countries with similar weak labour laws. I have seen often enough puzzled looks from team managers from such countries, when leeding European teams for the first time, and routinely get a NO for whatever crazy requests they happen to do.

Full original
CL

> I think the general public (and by that I mean including software engineers too) overestimate the likelihood of a huge screw-up leading to being fired I've seen this misconception perpetuated by management even in countries with very strong worker protection laws. Works particularly great on people who already have strong work ethics and internal fears, it stokes that fire. Employees would work as if any failure could lead to termination, especially in emergencies. Those managers use this as a "motivational" tool (as much as a hammer to the face can motivate you). They can squeeze more results out of their teams and maybe edge out a promotion. It's illuminating to see when the people understand the pressure is many times fake, that even the internal SLAs are more relaxed than what they fear.

Full original
ID

I made a somewhat similar blunder to the original poster at a job some years ago. I got a dressing down from the CEO and that's it, went back to work and never heard of it again. As far as I can tell it had no effect on my career at that company.

Full original
SP

Great share! What do they say, "You can't make an omelette without breaking eggs". I think people can go through life trying to never make mistakes and ultimately I think that just holds them back.

Full original
DA

In the software engineering industry, what I have seen is people not afraid of being fired due to a mistake, but to be afraid of the cost of their mistake in their next performance review. Make two mistakes (not necessarily in a row) and you are putting yourself in a dangerous corner (e.g., you may be considered to be lay off in the next round of lay offs).

Full original
MA

Work long enough and a person will screw up. That's how things go. And if you never screw up, then you're not really pushing yourself. Personally, I've deleted customer tables, updated the whole table when meaning to update one record, even shipped our private key once. There are probably even more I can't remember, and others from people that I helped fix (one time a boss locked us all out of the db - only fix was restoring lol) In all cases I became a better engineer by owning the mistake quickly, fixing the issue, and then putting processes in place to avoid it in the future. I think it shaped who I was to be a better engineer, manager, and owner. The only time I've seen people fired on the spot is when they lie. After seeing someone fired, I asked one of the owners at an old company I worked at why that was the line. He responded that even the best performers make mistakes, but if he…

Full original
BR

> A young executive had made some bad decisions that cost the company several million dollars. He was summoned to Watson’s office, fully expecting to be dismissed. As he entered the office, the young executive said, “I suppose after that set of mistakes you will want to fire me.” Watson was said to have replied, > “Not at all, young man, we have just spent a couple of million dollars educating you.” [1] [1] http://the-happy-manager.com/articles/characteristic-of-lead ...

Full original
Continue on Hacker News

Respect different perspectives. Share concrete experiences.

Sign in to post, reply, repost, and quote.