Monitor Comcast Usage Data

The good folks at Comcast have decided to put a 250GB cap on monthly usage as a direct assault against my beloved Roku.



I decided to put a script together to notify me in case I start approaching the monthly limit. My first thought was that this was a perfect task for mechanize, a Ruby library for interacting with web sites. I put that aside, however, to make an attempt to script the scraping with curl.

Those folks at Comcast are whack. When logging in to the home page, what is sent back is a redirect - no not an HTTP redirect. You are sent back a page that has a form in it with a "cima.ticket" that submits itself in the body onload event. Here is the somewhat functional script to pull the your Comcast bandwidth usage:



I say somewhat functional because it doesn't work every time. It will fail to pull a result every now and then. There is a lot of wonkiness in the way the Comcast login works - and it appears this results in some kind of timing issue. Hopefully I have a chance to investigate a solution for this in the future.

Oh - back to mechanize. I googled for another solution to this problem and found this. It shows off the elegance and ease of use of mechanize, however, it seems to fail intermittently just like my script.

Backup Blogger Blog with cURL Script

I read Dave Winer's Where's Your Data? post today which was in response to a blog post by Craig Burton title The State of Blogging Sucks. These articles refer to the problems that may arise when someone else is hosting your data. The problem could be a massive data loss (see Ma.gnolia), a company going out of business (apparently a Userland product?), or just data lock in.

With that in mind I thought about my blog here. This is primarily a dumping ground for small but handy techniques I learn so that if I figure out a way to do something - I can quickly google for it and use it again later. At this point there are enough helpful hints in here that I would hate to lose it. Blogger provides a way to export your blog as XML but it involves logging in and clicking a button - the last thing a programmer wants to have to remember to do on a recurring basis.

So...I cooked up this bash cURL script to backup a Blogger blog. This script demonstrates the basics of using cURL. The first curl command fetches the login page and pulls out a security token using grep and sed. The second curl command logs in, providing the security token, login, and password among other arguments. The next curl request requests the home page so that your blogID can be determined, which will be used in the final request. The final curl request actually requests the backup dumping the result to standard out - you can redirect this to the file of your choice or modify the script to also accept an output file. Here is the script:



In other news the Cardinals almost pulled it off today. This was the first time in a long time where I really wanted one team to win over the other in the Super Bowl. Alas it wasn't meant to be. I've been waiting for the Chiefs to make it for almost 25 years and it doesn't look I'll be rooting for them in the Super Bowl anytime soon.