When the state runs things
I live in yet another state that controls the liquor sales. PA is the other state. After talking with a coworker about some booze they showed me the state liquor store website. Turns out that they post the prices, stock and location of all the booze they carry.
After a night of scripting I have 2 scripts. The first one finds the brands and prices. The second one finds the quantities of each brand and what store they are at. Then it stores it in a mongodb (many reasons i am using mongo but that is not for here)
So after unscrewing dreamhost with rvm I finally have these scripts running on my server. But you say "big deal becker. what can you do with that data?". To which I say ..
The store at PORT ANGELES 1331 E FRONT STREET PT ANGELES WA 98362-4616 currently has 176 .75 liter bottles in stock of MACNAUGHTONS 36 MTH CDN TRAVELER which by the way is 80 proof! It is being sold at 12.45 including tax. Its not on sale currently...
I have more data but its getting late and my screens are ready to run all night.
After a night of scripting I have 2 scripts. The first one finds the brands and prices. The second one finds the quantities of each brand and what store they are at. Then it stores it in a mongodb (many reasons i am using mongo but that is not for here)
So after unscrewing dreamhost with rvm I finally have these scripts running on my server. But you say "big deal becker. what can you do with that data?". To which I say ..
The store at PORT ANGELES 1331 E FRONT STREET PT ANGELES WA 98362-4616 currently has 176 .75 liter bottles in stock of MACNAUGHTONS 36 MTH CDN TRAVELER which by the way is 80 proof! It is being sold at 12.45 including tax. Its not on sale currently...
I have more data but its getting late and my screens are ready to run all night.
When the state runs things ii
I picked up my liq site scraping again. I reworked my db using sql. I am going to use ruby 1.9.X and rails 3. It should be up soon.
When the state runs things iii
As a project I have been working on a website that scrapes data from the wa.liq state site. It was to include search, location based info, json data dump files for clients, and other fun features the state is to outdated to understand. I have had fun with selecting a brand type (Scotch) and a store (#28). I have a page which shows you the brands in stock but my page also shows you which brands they do not have that store but have at other stores. It also includes an approximate mile count to the closest store.
But as of Friday the state updated their website. Hooray! Not really. The site is still poor at best. The update comes with a new look. A look I do not like. There is a slight search improvement where you can search per brand type per city (not store like mine does. I was going to offer city). The usability of this also very poor. The find store now lacks the product you were looking for.
To top it all off they updated their system to some new version of ASP. It now includes value checking and javascript for form manipulations. It also holds the search part in an iframe so you have to see the ugly layout and buttons.
Long and short of it is that I am rewriting my parsers in mechanize. Its been fun to learn mechanize and its Javascript shortfalls. I thought all was lost when I read the Javascript and had mechanize mimic the code. I should be reparsing data by weekend's end. I take the update to the state site as a personal challenge. I will give the drinkers of Washington more features then they can dream of. When I am done here I am turning my sites to Oregon!
As long as the state is running things.
But as of Friday the state updated their website. Hooray! Not really. The site is still poor at best. The update comes with a new look. A look I do not like. There is a slight search improvement where you can search per brand type per city (not store like mine does. I was going to offer city). The usability of this also very poor. The find store now lacks the product you were looking for.
To top it all off they updated their system to some new version of ASP. It now includes value checking and javascript for form manipulations. It also holds the search part in an iframe so you have to see the ugly layout and buttons.
Long and short of it is that I am rewriting my parsers in mechanize. Its been fun to learn mechanize and its Javascript shortfalls. I thought all was lost when I read the Javascript and had mechanize mimic the code. I should be reparsing data by weekend's end. I take the update to the state site as a personal challenge. I will give the drinkers of Washington more features then they can dream of. When I am done here I am turning my sites to Oregon!
As long as the state is running things.
