“Victory comes only after many struggles and countless defeats.” — Og Mandino

I went to to the glasses place to get glasses. So I looked at frames for too long and picked some out.

I was hoping I didn’t need to clarify the purpose behind the visit, but I’m learning that if you provide a lack of specificity then people might think shit like maybe that I wandered into the glasses place to take a shit. I did not. I did pee though. In the toilet. In a closed restroom. Mostly clothed.

During checkout the sales representative asked if I wanted to donate to a glasses for poor countries non-profit. I opted out, not because I don’t believe in the cause, but because Luxotica owns the store, the frames, the lenses. Luxotica can lower the price of their eyewear and easily donate their product to those countries. And I’m willing to bet that Luxotica setup the non-profit or facilitated the creation and resourcing of it.

I have zero desire to verify the accuracy of that guess.

Instead they’re soliciting us for our money. Their customers. When they can easily afford to spend waaaaaaaaaaaaaaaaaay more than any single individual. I feel abused by these large corporations at times. Straight up customer abuse. Gaslighting and subterfuge.

It’s a rather unfortunate thing that the law refers to business entities as “legal persons.” Because these “legal persons” are fucking dicks.

I’ll contribute to altruistic causes in my own way, sans their “administrative fees.”

This post, again, was meant to be something else. About the meta story of The White Lotus. But I’ve had a moment of inspiration for something else.

No. Fucking. Shit. This is an ADHD thing. I’m writing fucking tangents to myself in H6. For now. I really wish it was easier to format shit in Day One. Sigh. H6 it is.

I’m alright with that though. I still have the 31415 other started posts that I can fall back to eventually.

I ain’t gots no deadlines that ain’t self imposed. I’m still making an effort to work to them.

This entire post has nothing to do with glasses though. lol. Except for the story.

Yup. Pretty much everything above this is a tangent. So is everything below. This post wasn’t meant to be.

The bald man, with a larger frame, with a higher voice - almost effeminate - mismatching his bigger bodily frame, with a long ginger beard poking out from under his mask, helped me after I spent way more time than I wanted skulking around hunting for frames.

I was immediate in requesting what I wanted, and as he did his thing on the buggy ass shit computer terminal we had my favorite kind of talk.

Not small talk. Not large talk. Just talk. Deep talk.

He shared with me that he’s an analytical kind of dude, and that his wife was a strategic kind of gal. Then brought up a spreadsheet he created for D&D on his phone. And wow. Just. Wow. Thousands of lines and tens of tabs. It was just nuts.

I felt somewhat emasculated by the depth of his spreadsheet skills. Except that I don’t like to spread sheet. I don’t really give af about “masculinity.” I’m more of a results oriented kind of dude.

I shared with him that I had been working on something similar but with Airtable, as well as what I do for my day role and a little about what’s involved, so he shared with me a story that he read on Reddit that was an Ask Me Anything thread with an IT professional. The topic was, “What’s the worst breaking change you’ve made?”

The answer was that there was a legacy system that had been running for 15 years.

Running stable.

Then a new employee provisioned their account on the system. But they used an emoji. The system did not support newer Unicode UTF-8 characters (emojis), and when the database wrote the username with the emoji to storage the entire system became corrupted and it took weeks to get it back up.

It was a major outage.

That story is just incredible. Aspirational. I wish I had been the person who did that. The person who did it did nothing wrong. That poor person very likely had to fix the failure they catalyzed though. Gotta embrace the suck is what they say, I hear.

Is it accurate? No clue. Zero desire to search verify. NonE. Have fun with that whoever wants to.

And then my fingers randomly take me to this topical topic.

One of the things that people in my role need to be very comfortable with is failure. And being able to work through failure. And being able to share the impact of the failure and how to make an effort to improve the stability of the system.

And more importantly, being able to be transparent about the failure, collaborative in fixing the fail, learning from failure, then sharing the learnings.

If you cause an outage to a production system, and that system is being used by thousands and thousands of customers, then a very small mistake can have a very large impact.

This is the nature of the Information Technology industry. Especially underlying infrastructure and cloud engineering.

Of which I am currently filling a DevOps-oriented role. Which kinda involves a bit of Cloud Engineering and a bit of leadership as well at this point.

This is where forming teams that hold each other accountable is vital. This is the reason there must be processes in place to improve operating procedures to minimize the risk of more failures occurring in the future. And there must be routines in place to ensure that the team is strong enough - competent enough - to work through failure.

I have fucked up so many things on production systems.

Fortunately, quite often, because I’ve been around the block for years and years, I manage to implement a fix before there is any sort of impact.

Sometimes these mistakes have been a part of a Standard Operating Procedure - which is just a change to the system that my team is allowed to make whenever they want - so no disclosure outside of our team needs to occur as long as our customers were not impacted. That’s a failure.

But sometimes I’ve done tiny things that didn’t have much, if any, of an impact. I still have to disclose that though so other people know about it. That’s a failure.

And sometimes I have caused a larger impact - one in which thousands of users of the system have been impacted - and that is definitely a failure.

And once, around Y2k, I caused a catastrophic failure. That was a wild ride. I accidentally unplugged the backplane of the network-attached storage NetApp ONTAP unit that contained thousands upon thousands of articles of porn.

Because I pretty much learned most of what I know because I worked to scale an “adult internet marketing” company. Or porn for the layman. Heh. Lay. Man. Heh. And omfg the emotional abuse I was subjected to. Yikes. That catastrophic failure was quite frequently thrown in my face. I had no team. I was the only person responsible for scaling from 4 to 200+ servers, as well as all the infrastructure required to support that.

Because even though the failure occurred I must now handle the consequences of that failure in real time.

Yes, when I’ve caused a larger mistake I feel kind of bad; and, at the same time I don’t.

Unless it’s an interpersonal mistake. Then. Sigh. Fuck. Damnit all. I do really make an effort to have less failures there.

I’m human. I’m fallible. I fuck up. And I fail a lot.

Soooooooooooooo much.

Within the scope of the role I have to be very comfortable with that. Especially when the system may be handling the load of millions of connections and active sessions.

And at the same time, as I’ve matured in seniority, and taking on leadership responsibilities, I now must do quite a bit more communication than implementation.

That is a post for another time.

But, I guess, the reason that I went on a total tangent is because I did what I thought I was gonna do. Wrote that I was going to do. Wrote what I needed to write. Scheduled the e-mail to be sent to the right people. Woke up. Cancelled the e-mail. Sent the e-mail to my manager.

Then Ssnt the agreed upon refactoring of the communication to our:

  • Director
  • Senior Director
  • Vice President
  • Chief Technical Officer

I sent that e-mail approximately 90 minutes before before the Director and CTO were about to have a 1:1, and my stars aligned just in time to support my ability to make sure that the audience required to be aware of, and potentially be required to make decisions regarding, were brought together to discuss an urgent topic that have serious ramifications to the schedule of an important project.

Some people would – often rightfully so – feel somewhat hesitant to directly approach that kind of audience. There were some important factors that I considered here.

  • I have pre-existing relationships with everyone.
  • I believe those relationships are in a healthy state.
  • I have been somewhat slightly provocative enough recently that some of the relationships may not be as secure as they should be… but…
  • Everyone knows that I have positive intent and that…
  • Everyone is aware that I just don’t give a fuck about traditional conceptualizations of communication processes.
I’ve always got options. My skillset is in so much fucking demand right now. I am so very fucking lucky, and very grateful, that the people who shaped me fostered the skills required to have those options.

So I just did it.

There is a very high chance of failure here in multiple and important ways.

Namely social capital.

If I just cried wolf then I would lose credibility. If I brought up an issue that was not of the kind of import that warranted an “emergency” meeting with a bunch of executives… then that would damage their faith in my ability to provide value.

But I had a few very important details already considered before I did this.

After I learned about the importance of the urgent topic for discussion, I immediately contacted my manager – who, as a Staff Engineer, is pretty much my peer and vice versa – so we could align (meet and figure out what to do next) on figuring out how the fuck we were going to remediate (solve) the problem.

Early disclosure of a problem with the appropriate person and alignment on urgency.

We worked for a few hours to come up with a basic strategy that we felt will be feasible.

Spent a couple of hours on a video conference doinking around in a fucking spreadsheet figuring out how to fucking schedule the require solutions to the problem.

And then I spent a weekend marinating.

They say, “lEAvE yOUr wORk aT HoME.” That’s fucking bullshit as a knowledge engineer. I can’t just fucking turn off my fucking mind. I mean… there are theoretically some mechanisms to do this to consider… I guess…

Then after a meeting packed with a bunch of surprises had the thought, “Oh shit. God fucking damnit. Yes, Me and my manager can fuck around with our teams time all we want, but that won’t matter unless the repercussions of us addressing this are felt by everyone else across the organization that our team supports.”

The team I’m a member of, and that I make an effort to lead, supports the entire development pipeline and platform for the products we work on. If we have to drop everything to solve a problem the entire organizations deliverable schedule will be impacted.

My brain goes into solution mode. The thought that comes to mind that I like is: “If this is going to impact the entire organization then we need to have a serious conversation about what we must consider.”

I allowed myself time to process the problem and imagine up solutions.

And so I wrote up an e-mail, did just a little cursory research into the depths of calendar scheduling hell, then sent an e-mail stating that we have an urgent problem that must be addressed, and that we’d like to crash one of the CTO’s meeting.

And we did.

And it was just awesome.

I love clicking send on stuff like that.

But there was a possibility that if I went in there that the topic I was broaching may not have been accurately processed in order to get the optimal decision made.

Fortunately, it seems, I hope, that the urgency of the topic I brought up is now being considered and that the best individuals to gather the resources required to expeditiously (quickly) solve the problem are now aware that this problem may have a relatively large impact to the organization.

And a large impact, within the scope of an organization the size that I work with currently, means a potential loss of thousands of man hours – and each man hour costs them money.

I have now made the most senior and authoritative decision makers within the organization that budgetary discussions must be had. And those are just a cross-organizational pain in the fucking ass.

I’m still now finding myself in a position in which I must be an active contributor at an engineering level, coordinating and leading the engineering leaders because there is stunningly questionably effective communication, and now having to fucking solve a problem because of that questionable communication pipeline.

So I sent the e-mail.

I highly advise anyone who might be reading this that is in a similar position and a similar scenario to tread carefully.

Be very strategic.

I already knew the executives that need to know, I knew the dramatic impact that solving the problem would have on my team and how not solving the problem would have a detrimental impact to the organizations capability to meet product deadline commitments, I knew my manager and I were aligned, and I know I’m in position where I can somehow get away with this shit.

I just have a fucking GED. I just. I just don’t get this. What is real life? I… Just don’t get it.

I also make an effort to remain positive and solution oriented.

My manager and I had already aligned, paired on a solution, and had the most important details worked through.

I lit a fire. I also had the firefighters on-call and ready to put that fire out.

If I’d approached this differently – if I’d just bitched about the situation we were in – I would have lost social capital, and it would have been a more ineffective use of everyones time.

And, within the context of traditional capitalistic business entities, time is money.

Wow. This entire post was just a huge tangent.

And written at the end of the day.

About to post at 23:53.

At least I did it.

3 posts.

I have, however, really fucked the schedule on what I wanted to do on TikTok. I’ll get back to it though.

Actually… I’ll probably revisit this. Ain’t no one keepin’ me from doing so.
And my Git commit is going to be inaccurate in its reflection as to the accuracy of when this was finished. I have to basically have to export this post and post process it.