Last week I posted about my initial thoughts on Google App Engine. I'm going to follow up that by writing about something I did to overcome one of the drawbacks of App Engine.
One of the requirements for my App Engine application is to save some data by parsing one of the public websites. And this task has to be run at least few times a day to capture any changes on that website. I wrote the code to parse the website and save data. All I needed was a way to invoke that code periodically, like a scheduled task. But Google App Engine doesn't yet has a provision to schedule tasks to be run.
Then I thought of a workaround for this limitation. I thought of some cryptic URL on my App Engine application and when that URL is called, I executed my code that I wanted to run at least few times a day. I added some code to print random string every time this URL is called. Now, I went and subscribed for few services that convert HTML pages to RSS for the above cryptic URL. Essentially, the HTML to RSS service checks my cryptic URL page for changes every few hours and gives me an RSS feed of the updates. As this service calls my URL every few hours to check changes, my task of running the save task was taken care of.
This workaround has its own limitations. You can't actually call it a scheduler as you can't guarantee when it will be executed and at what interval. And the other issue is you can't do performance intensive tasks with this task as the page will timeout. But this served my purpose and is doing a good job.
Write a comment or send me a mail if any one of you have any better ideas or want to know more about it ...
1 comments: