Mind.random()

Wow Wow Wow. By God’s grace, this year had a wonderful start. Beginning with Jason’s 1st birthday on 7th and the Merrill interview on the same day, vacation to home and the best of it all – re-joining Merrill. Needless to say that i always enjoyed working in Merrill. Of course, it means less amount of family time but at least at the end of the day, i will have a satisfied heart that i have done something worth.

I fondly remember the first 4 months in ML India when working till 2 AM is almost a daily event. What the heck. I am awake till 2 AM even here in Singapore fighting with the code. I love programming and i’ll do it till death does us apart. That’s right, ML is aggressive but so am I.

Now that everything is settled and I have a definite and promising road in front of me, I am resuming my work on the personal projects – our lunarcodes.

I wanted to write an app which indexes all the ebooks, movies and songs on my machine and use amazon or google or any other webservice to fetch meta information about them – including the cover pictures, popularity, reviews and any other information that could be possibly useful. I seriously dont want a bloated webapp for this. A pretty SWT/Swing would be great. Flex would be brilliant but AFAIK i dont see any good search and index libraries for Flex. Flex and Java would require a server running which I dont want to. Python, on the other hand, has Lucene extensions and has good web service support too but felt that nothing would come close to using the original lucene libraries instead of extensions.

Ah… i really love this year.

This weekend, i managed to migrate a few core portions of Fetch2Me from Python to Java. It was an amazing experience and a wonderful feeling to be bilingual. Though the Java version looks more elegant (to me), i personally loved my Python. The “the first project in Python” feeling…

So, here are the upcoming tasks in the Java version :

1) SMS Service : SMS service is not a part of the the system right now. Implement SMS service into fej2me through Google Voice account

2) Encryption : The config.xml used in the project has plaintext “mail id and password”.  I was thinking on the lines of AES having an external private key stored in a non-project folder without which the password hash would not make any sense.

3) Use JavaMail instead of Commons Email : I happen to use Apache Commons email instead of for sending mails (SMTP) because it looked simple. But later i realised that we cant attach streams. What i am currently doing is that i am attaching an URL directly to the email like

emailAttachment.setURL(new URL(http://www.google.com));

which is good for our current needs but will limit our options. Plain JavaMail on the other hand will use a DataSource which will give us a lot of options.

4) Apache HttpClient : As of now, the URL is directly attached to the email. What if the target URL is down or isnt even an URL. We would want to use HttpClient to ping the URL and check whether the status code returned is 200.

5) Search functionality using Google AJAX search and JSON : Google search or Wiki search (idea by Rajesh) is not a part of the system right now. Need to use Google Ajax API and JSON to pull data from Google. Please refer to
http://code.google.com/apis/ajaxsearch/documentation/

You might get some additional ideas on Book search, Movie search from the following URL (Idea by Dinesh)

http://code.google.com/apis/ajax/playground/#hello_world

6) Interactive browsing using Apache HttpUtils (functionality similar to Mechanize/Beautiful soup) : Use HttpUtils/HttpClient to simulate user browsing with Java. This will open a whole world of accessing authenticated systems.

7) Recurring updates using Quartz scheduler : Everybody would like to have recurring updates from sites, say like cricinfo or nseindia. users should be able to set a recurring job with us and get updates. something like.. “between 1 pm and 8 pm, get updates from http://www.cricinfo.com/nzvpak2009/engine/current/match/423780.html” every 1 minute”. we should enable persisted scheduling using a backend database (which is also available with quartz)

Multithreaded processing : Currently, the processing and handling of requests is sequential. We have to think about exploiting ThreadPool API to process multiple requests.

9) Quartz scheduler with BigTable/HBase : This is totally optional, but somebody is yet to come up with a Quartz scheduler extension using the backend as HBase. May be… may be… we could think of that. Though, HBase is good for large data, it is still an idea worth considering. Optionally, if we have a need to store large data into our systems, we could consider HBase.. Just for the fun of it.