2020
Tyler and I made the first Christmas cookies of the year.
The data in this graph is pretty amazing. From Why we are underestimating Zoom & it’s impact?.
I’m not sure what happened with our bandwidth usage this month but it was way up from any previous month and we just used our one “free" excess usage month from Xfinity. This would have cost $60 extra this month. I will have to keep an eye on usage and consider adding unlimited data for $30 a month. We didn’t make any notable changes in devices or usage so I’m not sure what drove the jump.
Family tradition to watch A Charlie Brown Thanksgiving on the evening on Thanksgiving. 🍿
We celebrated our first Pandemic Thanksgiving today. I cooked turkey breasts in the Sous Vides and finished on the Big Green Egg. Tammy put the rest of it together including the best sweet potato dish I’ve ever had and a delicious homemade pumpkin pie!
This mornings 45 min Turkey Burn Ride with Robin Arzon had 50,000 live riders! That is an entire good size city, spinning together. What a cool experience. #FitByFifty
This was the worst production outage I’ve experienced in Amazon Web Services, lasting over 21 hours. AWS has also published the full incident report.
Latest Update (1:18 AM PST): We have restored all traffic to Kinesis Data Streams via all endpoints and it is now operating normally. We have also resolved the error rates invoking CloudWatch APIs. We continue to work towards full recovery for IoT SiteWise and details of the service status is below. All other services are operating normally. We have identified the root cause of the Kinesis Data Streams event, and have completed immediate actions to prevent recurrence.
Previous Update (12:43 PM PST): We have restored all traffic to Kinesis Data Streams via all endpoints and it is now operating normally. We have also resolved the error rates invoking CloudWatch APIs. We continue to work towards full recovery for IoT SiteWise and Elastic Container Service; details of these services’ status is below. All other services are operating normally. We have identified the root cause of the Kinesis Data Streams event, and have completed immediate actions to prevent recurrence.
Previous Update (11:43 PM PST): We have restored all traffic to Kinesis Data Streams via all endpoints, and have resolved the error rates invoking CloudWatch APIs. We are continuing to closely monitor Kinesis and work toward full recovery of all services. We have identified the root cause of the Kinesis Data Streams event, and have completed immediate actions to prevent recurrence. Kinesis and CloudWatch are operating normally.
Previous Update (10:30 PM PST): We have restored all traffic to Kinesis Data Streams from Internet-facing endpoints, and we are continuing to incrementally restore all requests to Kinesis Data Streams using VPC Endpoints. We are also beginning to observe incremental recovery of CloudWatch metrics functionality for new incoming metrics, and working towards full recovery. The backlog of metrics will take additional time to populate.
Previous Update (9:06 PM PST): Over the past two hours, we have continued to bring more traffic in to Kinesis Data Streams, which is leading to gradual recovery of applications that use Kinesis directly, as well as dependent services within the US-EAST-1 Region. We are bringing traffic in more slowly than anticipated while we closely monitor each change to ensure continued stability. We expect that over the next few hours, we will complete restoring Kinesis Data Streams to normal operations.
CloudWatch metrics remain delayed in the US-EAST-1 Region. Once we have restored the throttles for Kinesis to previous levels, we will be restoring CloudWatch metrics functionality. We expect to see recovery of CloudWatch metrics at that stage for new incoming metrics, but the backlog of metrics may take additional time to populate.
Previous Update (6:23 PM PST): We’d like to provide an update on the issue affecting the Kinesis Data Streams API, and other dependent services, within the US-EAST-1 Region. We have now fully mitigated the impact to the subsystem within Kinesis that is responsible for the processing of incoming requests and are no longer seeing increased error rates or latencies. However, we are not yet taking the full traffic load and are working to relax request throttles on the service. Over the next few hours we expect to relax these throttles to previous levels. We expect customers to begin seeing recovery as these throttles are relaxed over this timeframe.
We got takeout from Petite León for the first time tonight and thought it was great. We enjoyed the Roasted Beets, Pollo al Carbon, Le Petite Cheeseburger and Frites. Recommend trying it for some delicious takeout.
Buttondown says that I have written 448,665 words in the first 166 issues of the Weekly Thing. The average non-fiction book has less than 100,000 words. Even if half of the Weekly Thing words aren’t mine, I’ve possibly written two books worth of words? 🤯
