I got a decent amount of feed back and advice on my post the other day about getting a Dell warranty expiration with web scraping. It was recommended to change my scraping to use regex, Beautiful Soup or Scrapy. I figured I’d do all three and make a post on each one.

As you know if you read any of this blog so far, I’m just learning python, so I have a ton to learn. What better way than try the different options presented to me.

The first option I decided to try since I already did a little bit of it for other scripts I haven’t blogged about yet is scraping it via regex. This was quite challenging for a noob like me. I couldn’t seem to quite get the expression down to grab all the dates needed. The original expression I was using would grab the last date, but would skip right over the first one. I have no clue why.

One thing I learned from doing this scrap with regex is my original script was wrong. It was grabbing the date, but it wasn’t necessarily grabbing the correct date. Dell’s website can have multiple expiration dates. If you renew, it’s going to show the original warranty, and the old warranty. If you have a default warranty and upgraded to a better warranty, it’s going to show both. By using regex, I was able to grab the dates and compare them to find the correct expiration.

Another thing I learned about this task in particular is the slowness is not so much my code but Dell’s crappy website. As a network/systems guy, I have to go on Dell’s site a lot, and it is horribly painful to use because of the speed.

OK, so here’s a quick run down of the code and then the actual code.

First it grabs the url as a string. Then it performs a regular expression search looking for the dates and creates a list of tuples with the date being the second item in each tuple. After having a list of tuples, I have a while loop that runs through the tuples and grabs the dates out as integers and puts them into a list of tuples so they can be compared. After I have the dates in a list of tuples, I just use the max function to find out which is the correct date. I’m not sure this is the greatest way to do this, but it seems to work on the service tags I’ve tested out. Lastly, I convert the warranty back into a string to return the warranty as a string.

As I said with my original post, I’m sure this could be improved a million ways. I’m just learning, so any pointers would be appreciated.

 

 

As I mentioned in my blog on my first learning resources, right after finishing the Google training, the first script I wrote was a script to get a Dell hardware warranty expiration given the service tag. It was my first attempt at web scraping, something you learn with the Google python videos. I am not sure if this is the best or most efficient way to do this, but it was a way for me to test out what I just learned. If you have some advice, don’t hesitate to share it in the comments.

I’m actually going to do more with this as I learn more. The goal is to access our database of hardware, grab the service tags, get the warranty expiration from Dell, and lastly update a warranty field in the database. From that field, we’ll generate alerts to notify customers of their pending expiration date and hopefully get some warranty sales. This script will be a part of that bigger picture later.

For now, all you do is run dellwarranty.py SERVICETAG to get the date back in a string format.

This script grabs the html and then searches for TopTwoWarrantyListItem. From that starting point, it looks for four greater than brackets to find the beginning position of the date. Then it looks for the very next less than bracket as the ending point for the date.

When I started typing up this blog, I ran the script and it was giving me errors. It worked when I wrote it, but I’ve since rebuilt my laptop with Linux Mint. My laptop was previously running CentOS, which I believe runs Python 2.6 as default. I’m assuming something changed from there. Considering the current training I’m doing is in Python 3, I made the script check for the python version and error out if not version 3.

Without further ado, here’s the Dell warranty script code.

 

 

Before I get too far into writing posts about code I’m playing around with and the latest learning resources I’m using, I wanted to touch on what resources I found very useful to get started.

Depending on your background, you might need to take a different path. I had programming classes in my college days and tend to be fairly analytical. With that said, I can understand the basic flow of code and have been able to troubleshoot other peoples code periodically as part of doing network/systems administration. Because of this very basic understanding, I started learning using the videos over at TheNewBoston.

The videos from the new boston are pretty good to familiarize yourself with the language. I’d say it’s like learning words in high school foreign language classes. By the time you are done, you’ll know some good basics, but you probably won’t be able to do anything spectacular with it unless you already know programming and you’re just picking up a different language. I’m not trying to knock the videos. They are quick and to the point. They are a great way to get ready for the next resource, Google’s Python Class.

With the familiarity of the basics from the new boston, you will be able to follow Nick Parlante as he takes you though training and exercises that will have you writing real code. The exercises are what really separates the Google videos from the other tutorials online. They are fairly challenging, although maybe that’s more reflective of me than the actual exercises. The exercises have you add your code to the python files you download from the course website. The cool thing about the files is they have test functions built into them so after you add your code and run it, the exercise will check and see if it gets the correct information back.

The exercises start off with string functions and by the end of the training you are grabbing html from the web, doing regular expression searches through it, grabbing pieces of a puzzle and building the puzzle by assembling them in the right order and outputting the layout via html. When completed, if you open you html file and see the proper image, you completed the exercise successfully. These exercises were actually extremely fun and you learn a lot.

After using these two resources, I jumped into write a few things for work. I wrote the following:

1. A script that takes a Dell service tag and returns the warranty expiration date. This can be used to go through all the Dell workstations and servers that we manage and populate our database with warranty expirations so that we can inform clients of the approaching expiration. Typically, clients don’t renew the workstations, but they do typically renew the servers. That’s some extra sales that we might have let slip by. I did not get to the database portion yet, as I haven’t learned that part of python yet. I’ll add onto this later.

2. A script that exports all the contacts from the Global Address List in Microsoft Outlook. For one of our clients, I had to export their address list using the get external data function in Access. Then I’d export it to Excel. Lastly, I’d have to clean it up as it would include a bunch of garbage and email addresses weren’t formatted properly. This took probably 30-45 mins each time, and I had to do this monthly to send it to their office in Shanghai. Now with the script, it takes 20-30 secs and anyone can do it.

3. Next I started writing some code to go through our PSA to find workstations for a client that is no longer with us and changing their workstations to inactive. After banging my head against the wall for a few days, I finally was able to talk to the API and get the workstation IDs back. I thought all I had to do now was change the XML to the appropriate format to update the workstations and send the XML string back the same way I got the workstation IDs. Unfortunately, I was mistaken. I played with the updating portion for another day and then decided to hold off till I learn a bit more about XML and Python. I had another task that would help us, so I moved on to it.

4. Lastly, I wrote a script to access our disaster recovery system via the provided API and grab the backup status of each server for each client via XML. Then it formats an email with each client broken out with their servers, the current backup status and the last successful backup date and time. This saved us from having to login and drill down on each client every day. It also prevents us from forgetting to look if we walk into firefighting first thing in the morning. No matter what, you always end up going through email.

Before I moved on to other things, I felt I needed to learn more. Coursera had a class coming up called Crafting Quality Code. I figured I’d try that next. I’m just getting started in the class, but so far, it’s fairly interesting. I bought the book that the professors who teach the class wrote, Practical Programming (2nd edition): An Introduction to Computer Science Using Python 3 as it goes with the class. I’m also working my way through that. I’ll put a post out on the book when I’m done with it.

05. April 2013 · Write a comment · Categories: Code · Tags: , ,

The first thing I wrote in python was prompted by my son’s obsessive use of the Xbox and filling in the rest of his time watching Youtube videos of the games he plays.

I have no problem with his gaming as long as it doesn’t interfere with other interests. For some background, my son has cerebral palsy. He’s very bright and doesn’t have any major problems other than his motor skills. He walks, but not very well. He has always been far ahead intelligence wise and probably read more books by 10 than I read in my entire life. His reading skills are probably at college level and he’s 12.

Now every year it seems to be the same cycle. We give him some slack on the gaming. He ends up doing nothing but gaming, caring about nothing else, and acting quite nasty to his mother. Typically we just take it away. This time I just wanted to limit the time. I found the family time settings on the Xbox (pretty obvious I am not a gamer) and gave him 14 hours a week. That took care of the Xbox. The next thing was Youtube, because if he wasn’t playing Xbox, he was watching other’s play Xbox on Youtube. Here was my solution.

I have a Fortigate firewall at home. I created a rule that blocked Youtube from his IP address. Now, all I had to do was write a little python script to ssh into the firewall and enable and disable the rule as needed. I didn’t want it being something that only I could do. I wanted something my wife could run on her computer to enable his access as well. When ran, the rule would be set to disabled, allowing Youtube for 60 mins. After 60 mins, it would ssh back into the firewall and enable the rule. The firewall has scheduling functionality for rules, but I didn’t want this being tied to a strict schedule. When he asked for time, I wanted to be able to give him an hour and have it shut off automatically.

Here’s the python script to do this. Keep in mind, this is the first thing I wrote. It’s probably not the best way to do it, and the style is probably horrible. That’s kind of what I’m trying to focus on now.

This worked like a charm. For those of you who think I’m a jerk for doing this, I never actually implemented it. After setting the family time settings on the Xbox, my son started getting interested in other things again, in particular history. He’s even using Youtube for learning other things, which I’ve always told him to do. It’s how I learned most of what I know about python so far.

 

After about a month and a half or so of learning Python and finding a ton of help and information through blogs and YouTube videos, I thought it would be a good idea to blog about about my learning process and to return some information back to those who do as I am doing now, learning python. After all, what better way to reinforce what I am learning than to explain what I just learned.

My blog postings will be related to how I am learning python. I’ll post what works for me and what doesn’t. I’m sure most of it will be stuff that works, because if it doesn’t, I quickly move on and don’t waste time on it. If something works, I’ll want to share with people, so you’ll see more blogs about what works than what doesn’t.

Also, I’ll be sharing code I’m writing. Maybe someone else can use it, add to it, etc. So far, I’ve written a few useful things for my regular job, which is running a managed services company. Stay tuned for those items in future blogs.

Lastly, I’m hoping people will actually read this blog and I’ll learn more from the readers. I’m sure there are things I’m doing wrong, and any input would be appreciated. Just don’t be one of these jackasses I see on Youtube, “YOURE AN IDIOT”. Try to remember I’m learning and I’m doing it mainly for enjoyment. Also, my fingers seem to get out of sync with my mind, so don’t be shocked by the typing errors, misspellings, etc.