Join Date: Oct 2009
Location: Midway, IN born & raised - Columbus, OH for the job
My teammate and I had the goal of turning our system on before heading out for the weekend. We set a deadline of 5, and were ready to for it up at about 4:54. (Takes about a minute to tear down a failed test and have the next one running and ready for inspection)
4:54| Ran through a test and hit a stack overflow. [Analysis]: AHA! The value stored in the database has a typo in it. Fixed, reran...[turnaround=45 seconds]
4:56| Ran again, hit a NullRefException. [Analysis]; AHA! Assigned the property in constructor before the backing field was added, so compiler "helped" by turning this into a property identity assignment. Fixed, reran...[turnaround=30 seconds]
4:57| Ran again, next error. Dont remember the root cause. [Analysis] We can get this straightened out in time and still hit the deadline...[turnaround=50 seconds]
4:59| Ran again, same error. Last fix didnt move us any closer. [Analysis] database fix from earlier fixed the immediate symptom but masked one downstream. Fixed, Nuked schema, Rebuilt, Repopulated...[turnaround=12 minutes]
5:11| Ran again....next error. We're vested in getting to watch this thing pass.
7:05| Ran again...logjam of "just one more"s cleared. All earlier failures were in the core stage of the business process and we've flown through it successfully to the postprocessor. Data Consistency Failure: Forward Curve doesnt contain as many data points as our Forecast expects. [Analysis] Cant get into specifics but we spent 15 minutes in Root Cause Analysis and simultaneously realized that this is *the* final potential error between us and a successful run...and that the fix will take us about 4 hours.
...It's waiting til Monday.
80 Muscort- "Wifebeater" (sorry if it offends. I didn't name it)
06 V6 Premium - "Hidalgo" (R.I.P.
67 C-code Fastback - "Tombstone"
My Mustangs don't get chick names. They get horse names.
When Hugh Hefner dies, will anyone actually believe the ubiquitous eulogy statement: "Well, at least he's in a better place"
Tombstone's Grille Ornament
Last edited by Alaric; 02-10-2017 at 09:21 PM.