Thursday, March 27, 2014

[My projects] "jTextBrew" TextBrew implementation in JAVA


Thanks to Chris Brew we have this awesome customizable edit distance algorithm called TextBrew.
It can be used for fuzzy string matching and can be used even for shorthands and abbreviation matching. By changing a few parameters you can use it particularly for your purpose.
You can read more about TextBrew algorithm here http://www.ling.ohio-state.edu/~cbrew/795M/string-distance.html.

What I have done is to create a JAVA implementation of the modified https://github.com/relaynetwork/fuzzy-string-matching project, for my own purpose in one of my other project.
You can find jTextBrew at https://github.com/puneetsl/jtextbrew.

A few examples of the use of this library:

  1. TextBrew.compare("BRYN MAWR PIZZA II", "BR MA PI II");
    would return 52% match
  2. You can change costs using the following function:
    TextBrew.setCosts(Match, Insert, Delete, Substitute);

Why is it handy?
When combined with other fuzzy string matching algorithms taking weighted average can give really good results. I am currently using it for auto spell correction, combining it with n-gram. And this is giving really good result.

Don't forget to star this project on GitHub in case you use it. ;)

Friday, January 17, 2014

[Using code] If AAP was implemented in object oriented way


So I was wondering what if Aam Aadmi Party was implemented in Obeject Oriented way(AAP in OOP), thus I am writing here snippets of the code which I wrote in 5-10 minutes and would open source it, so that you guys can extend this effort :P.
I am starting here with class AamAadmiParty:

package indian.politics;
//Version 0.1
//Author: Puneet Singh
public class AamAadmiParty extends Congress implements Socialism, Communism, Populism, Mobocracy{

 //Created Kejriwal using singleton design pattern, can't create so honest man again
 public Kejriwal kej = Kejriwal.getInstance();
 public void getDonations(int money){
  this.putItInWebsite(money);
 }
 public void putItInWebsite(int money) {
  System.out.println(money);
 }
 
 private boolean doKhulaseAndRunAway(boolean isFormedGovernmentWithCongress)
 {
  if(!isFormedGovernmentWithCongress)
  {
   isFormedGovernmentWithCongress = this.madePeopleFoolEnough();
   return doKhulaseAndRunAway(isFormedGovernmentWithCongress);
  }
  else
  {
   return true;
  }
 }
 public void before2014()
 {
  kej.honesty();
 }
 private void after2014()
 {
  indian.politics.Congress i = null;//null as in the number if seats expected in 2014
  mergeWithCongress(i);
 }
 private void mergeWithCongress(Congress c) {
  System.out.println("We are against communal forces!!");

 }
 private boolean madePeopleFoolEnough() {
  return false;
 }
}


And the other class would be Kejriwal implemented using Singleton Pattern:

package indian.politics;

public class Kejriwal {
 private static volatile Kejriwal instance = null;

 private ()Kejriwal {
 }

 public static Kejriwal ()getInstance {
  if (instance == null) {
   synchronized (Kejriwal.class) {
    if (instance == null) {
     instance = new ();Kejriwal}}}
    
   
  
  return instance;}
 
 protected void honesty()
 {
  System.out.println("I am honest, every one else is dalal");
  honesty();//infinite until stackoverflow}}
 



You can get started with the complete source code using this link https://code.google.com/p/aapinoop
I hope you are familiar with SVN.


Tuesday, December 31, 2013

[Travelling Raconteur] My trip to New York City

NY skyline from empire state

To be frank I do not know how to write a travel blog, and this "wikiHow to write a travel blog" is of not so much help here. Anyways I would just start blabbering, So I went to New York with so much excitement and hope that I am going to "World's awesomest city". I took "Greyhound", with an 8 hour journey, and it was too tiring. Seats were not so comfortable, I used to travel with better seats in India. The only good thing was it having a restroom, giving my bladder a good time during the journey.
We reached NYC around 7 AM at Times square, but we were not able to find a Subway, so a guy came up to me and asked if I want any help, I said, "Sure, can you tell which direction is Subway station". He said come with me, and took me towards a street outside bus station and told me the direction. Just when I thought how good are NYC people, he shouted "Aye man!! Don't you think you owe me a tip" in an intimidating tone. I replied that I didn't have any money, and kept moving. So, I got this recollection where, if I had asked the same thing in India, many people would have come forward and helped me, trust me, telling direction is passion in India. 

Food cart guy selling falafel
First day, I went to Columbia University, my dream destination as a student (Anyways it rejected me). The university at first look was really beautiful, though a bit small and compact. One awesome thing I observed was, there was a Japanese family taking pictures there, as soon as the father clicked a beautiful picture of the rest of the family, his both daughters bowed and thanked him for clicking it, got amazed and moved at the same time. 

Before reaching Columbia we got lost and walked all over the streets and reached to Morningside Park (it was beautiful). After all this walking I got really hungry, and ate Falafel from a roadside food carts. 
In the evening we had an amazing dinner at Dhaba on Lexinton AV. The chilling wind made our life hell, while waiting outside for table getting cleared for us. 
On Christmas day (next day), we just went to see lightnings, on TSq. Let me share a few Google Awesome pics with you:

 



We also had an amazing time eating "chicken over rice" from halal food cart. It got special due to cold weather, we had to move into the subway, and ate sitting on the subway floor. If you visit NYC do not miss eating from here, one of my friends told that he had seen several Bollywood celebrities eating from here, sitting right there next to the cart, who knows you might get lucky.

I was damn cold on that day, but still it was one of my best Christmas. Last to last year on Christmas this was how I celebrated with family, which was equally special. We decorated Neem tree. I was too tired at night due to all the walking so we just bought this NYC pass, which helped us next day visiting almost all the awesome places in NYC at cheap rates. Do not forget to buy this before you are planning to visit NYC, would save you a load, and "Aye man! You can tip me later" (*wink*).

26 December, 2013 would never be forgotten because of two reasons, one I had an awesome time sight seeing NYC, and secondly walking almost 16 miles. In the morning after taking a NYC pass card we went straight to the Hudson River bank near "Pier 83". Where we got water taxi and we hopped in, went near the WTC. I really regret not visiting the WTC memorial pool, do not miss this if you are going to NYC. Anyways, from there we went to Liberty island, and got mesmerized by its beauty. It was a clear beautiful day, and we got the best shots in my camera. Unfortunately, I could not place a Chinese couple, by taking their pics. They were certainly not impressed.
Our last destination was Empire state building. I have no words to write how beautiful it was, looking down from the sky, watching tiny bulbs coherently creating an artful scenery. It was the moment which I would never forget. Just a piece of advice, visit ESB only after sunset, the night lights and the scenery is beyond words awe-inspiring. I have deliberately skipped writing about "Madame Tussauds", it was good too, and one must not forget to visit this. But there is not much to share about its experience, except the 4D show which was quite amazing.

In the end, there were many more moments I wanted to share but don't want to stretch my blog too far. All I would say is this was a refreshing visit I doubt I would ever forget. I would like to thank a few friends who helped me stay and have fun, making my trip as enjoyable as it can be, Amit and Prateek for helping me stay, Neha and Akshat for coming down and enhancing the fun factor, Arpit for giving me an awesome treat and conversation, Rahul for boring me to death as well as sticking out with me almost everywhere. 
Let me leave you with a few pics, and since this post is my last blog of 2013, happy new year to all, have a nice time ahead.
Times Square
A guy getting angry for taking his pic while I just wanted to
take picture of street with roadside shops just like we have in India













Not that different from India, eh!













Christmas Lightning


Saturday, December 14, 2013

[How To] Write a script which notifies and alerts you in Linux

Finally I got free from my End-Sem exams, phew! But not so fast, I have to manage many things in my one and half months of winter holidays and that too in this chilling weather of Buffalo.
One of the most important things on my priority list is to pay my next semester's fees. Now here is the problem, it is really hard to track USD to INR rate. I had to go to XE or Yahoo Finance to know the current exchange rate. This gave me hard time, when the rate is changing continuously with high variability. So I decided to write a script and run it as a daemon or cron job which will notify me if the Dollar falls or rises from a particular threshold.
For this purpose, I have chosen Elementry OS/Ubuntu which is installed in my system (and BTW, my laptop can almost serve as a server, as I rarely switch in off. At best I would just make it go to sleep mode for a few hours). I would use Python and Perl for this purpose.
So let me show you how I did it. First, let's make a script which would notify me on desktop if I am The rate falls below a threshold, we will call this a notifier script:


notify.py


#!/usr/bin/python
import sys
import pynotify

if __name__ == "__main__":
    if not pynotify.init("icon-summary-body"):
        sys.exit(1)

    n = pynotify.Notification(
        sys.argv[1],
        sys.argv[2],
        ##dont remove the below line
    "notification-message-im")
    n.show()

This script would simply use the Ubuntu notification system and would notify you on top-right corner of your screen. (See screenshot)

Now we are ready to make a script which would continuously fetch the current exchange rate, but before that I also want that if I am not at my desk or not using the computer I must be notified through email. For that I would install, sendmail. It is really easy to install sendmail in Ubuntu, just fire this command:
sudo apt-get install sendmail
also create a filter in your mail box which would not send mails sent from your computer to spam folder, and test everything works properly by firing this command:
echo "Dollar Rate" | mail -s "Rate is at 61.67, just to inform you" your.mail@gmail.com 
Now you must check your mail box, you would have received  a mail like this:

I wrote a Perl script which uses Yahoo Finance API and keeps check of current exchange rate:

check_dollar.pl


use strict;
use warnings;
use LWP::UserAgent; 
use LWP::Simple;
# my $url = "http://rate-exchange.appspot.com/currency?from=USD&to=INR";
my $url = "http://finance.yahoo.com/webservice/v1/symbols/INR=X/quote?view=basic";
my $oHTTPAgent = new LWP::UserAgent;
my $oRequest = HTTP::Request->new('GET'); 
my $i =0;
my $old_fh = select(STDOUT);
$| = 1;
my $rate;
select($old_fh);
while(1)
{
 $oRequest->url($url);
 my $sResponse = $oHTTPAgent->request($oRequest);
 if ($sResponse->is_success) {  
     my $sPage = $sResponse->content; 
     ($rate) = $sPage =~ m/price\">(\d+\.\d\d)/g;
     $i++;
     print $i."\t".$rate."\n";
     if($rate<61)#set your threshold
     {
      my $xpl = `python /path/to/notify.py "Rate is" "$rate for 1 dollar"&>/dev/null`;
      my $yoyo = `echo "Dollar Rate" | mail -s "Rate is at $rate, just to inform you" everlasting.puneet\@gmail.com`;
     } 

 }
 sleep(60);#set ping frequency, right now it would check in every 60 seconds
}

Now we are all set, all we need to do now is to start this process. If you can observe I have used an infinite while loop, if I want to make a cron job of this script I need to remove this loop.
Now either run the script as it is by inserting right path to "notify.py", or if you want you can remove while loop and write cron job like this:
Run
crontab -e
This opens an editor. Therein you type :
*/1 * * * * perl /path/to/check_dollar.pl
We are all done, I hope you can be creative and add more type of notifications, and alerts. 

Tuesday, December 10, 2013

A note on AAP's Success - Part 2

Source: Wikipedia
The problem with frivolous intellectualism is it always hyper-reacts. Before the Delhi election everybody was busy under-estimating AAP and now everybody is busy over-estimating and over analyzing the AAP phenomenon. We get to hear such humungous theories from TV studios as to how AAP would now be a national party and how Arvind Kejriwal would be the PM candidate of the third front! Nobody bothers to tell us how AAP will replicate its limited success of a rootless city like Delhi in other metros; for instance, how will AAP gather votes in Mumbai where the fault-lines are between Maharashtrians and outsiders, or say in Hyderabad where there are agitations for and against Telangana or even in say, Lucknow, where the politics of identity is foremost.

Even the success of the AAP in Delhi is a one-off phenomenon that has come about through a string of coincidences – a very unpopular government, agitations of the past year on a range of issues from corruption to women’s safety, passionate participation of youth (who have long-term attention deficiency as always) and a moribund opposition which woke up much later in the day and yet managed to be the single largest party. Arvind Kejriwal is making a classic mistake once again. He is taking this one-off support of the people of Delhi for granted and wants to go to the polls once again based on some ludicrous suggestions that AAP has the momentum and will gain a 2/3rd Majority in a re-poll.

What is more likely to happen is quite the opposite;
  • Voters passion would have come down a couple of notches in about 3-4 months and AAP would never be able to get the same kind of volunteer participation as they got this time, no matter what kind of funds they manage (this is exactly what happened to the Anna movement which eventually fizzled out despite the huge initial success) 
  • With the passage of time, basic every day governance delivery issues will become important and the greenhorn MLAs of AAP would be answer less to voters queries especially in a situation where there is no government 
  • The Voter anger against Sheila Dixit would also be much reduced in a few months’ time and all Congress needs to do is regain some 3-5% of the vote share to overtake AAP 
  • If Delhi elections are held along with LS polls, which is the only practical possibility as far as the EC policy on re-polling is concerned, then AAP would come a cropper as it is a well-established fact that almost 65% of AAP voters want to vote for Modi in the LS polls 
  • Even if Delhi elections are held separately, all that BJP/Congress has to do is manage better ticket distribution by giving more tickets to young professionals rather than tired old faces and AAP would lose its only USP 
  • At least half a dozen seats are such where AAP has won by a very narrow margin and as such it won’t be sustainable in another election 


At the end of the day, far from going national, AAP stands vulnerable in the only state that it has performed, as we move closer towards the 2014 elections.


[PS:
  1. I am not the Author of this blog. This blog is taken from excerpts of: http://5forty3.wordpress.com/2013/12/11/what-2013-means-to-2014/comment-page-1, Please go and read the amazing and neutral analysis of 2013 election done by DR PRAVEEN PATIL on this blog.
  2. This blog represents exactly what my opinion is about AAP's success. Not that I don't want it to succeed but the media are overdoing everything. AAP is important to the nation, it changed political discourse, but let's keep in mind that we must not overestimate in excitement.
]

Sunday, November 17, 2013

[In Reply To] Do not protect me from myself, Mr Modi

So I read this article in "Business Standard", where author Aparna Kalra wrote about all this Amit Shah and Saheb snooping controversy. First of all I am neutral to this news, because I do not trust the veracity of the tapes, and want the law to take its course, though the timing of raking such issues is dubious.
Here I don't want to defend anyone, but all I want to oppose the double standards, such self proclaimed feminists have.  Let me take her points, one by one.
She writes "Ten reasons why I don't want unwanted protection from the state or my family"

  1. Because I do not want my limited free time and my conversations with men or women (dates, friends, colleagues) to be monitored.
    I agree! No one wants to be watched when he or she is having some private moments, but what if you just met someone, and you do not know the other person completely. And then he or she (mostly he) takes you somewhere unsafe, loots you, rapes you. But after that it would be so convenient to write another blog on how state machinery and intelligence fails to stop such incidences. My question is "Why can't government be proactive?" If they feel they should stalk someone for his/her safety they must.
  2. 'Protection' is the worst form of control and the worst form of patriarchy

    Okay! You mean to say, mothers would never ask for their daughter's safety. Surely, asking by a father for his daughters protection would mean an increasingly man dominated society #sarcasm.
  3. It takes the forms of brothers spying on sisters and parents keeping an eye on their daughters (and sometimes even their sons) "for their own good" but actually exercising all forms of checks to see that they do not step out of line
    If the parents don't go overboard with this, and start hindering and keeping check on every decision their children make in life, I don't see a problem here. Recently, I read an article where a girl committed suicide, because she was not mature enough to deal with the problems she was facing, and neither she was brave enough to confront her parents about this. This stupidity could have been stopped if parents were proactive in monitoring. (Though I would have loved if there was a friend like bonding between parents and children, so that any problem you can share with each other, but unfortunately that is not the case in every home).
  4. The "stepping out of line", of course, is hanging out with, making friends with or meeting and marrying people of their choice as against suitable parent-approved and society-stamped candidates. 

    I agree with you on this. Many Indian families want a Parent-Certified spouse for their children, but a few parents would just want to check is if their child has taken the right decision and if not it's their duty to suggest or discuss as a friend. So you can not generalize this.
  5. Protection takes away a person's risk-taking, adventurous spirit by confining him to "safe choices", "safe areas" and "safe people"
    Before going any further, lets get the facts right. What is Protection:

    In the top three definitions of "Protection", I don't find any single sentence which protects you from going for adventure and taking limited risk. But, if in the name of taking risk you want to jump from a 15 story building, without protection. In the name of adventure you want to go to some dark places of the city, and some bad thing happens, would you be responsible for yourself, no! Everyone would start blaming the Government, and we would see the inundation on blogs, write-ups, news from some other self proclaimed feminists, that government cannot protect girls, make strict laws and blah blah. Protection doesn't always means jail, you can have your adventure, risks. Would anyone sleep with someone having STD knowingly, without Protection?
  6. Even if I get hurt, or make wrong choices in life, those choices are mine. Not something the state, or my family, can decide for me.

    With this argument, you lost even the slightest sense you were making earlier. So, what are your wrong choices, a bad husband? Committing suicide? Getting raped? There can be a number of bad choices you can make. Like the one made by nirbhaya, that she took the wrong bus. It was because the government was not proactive enough, it did not do its bit. Now I don't want to bring politics in here, but in 2011 TOI reported:

    The five best states or UTs in 2009, among those with a population of 100 lakh or more, were Gujarat, Karnataka, Tamil Nadu, UP and Bihar in that order, all of them having less than one rape victim per lakh population.
    Although even one rape is condemnable offense, but here just for the statistics, I want to show Gujarat is one of the safest place for women (even according to other reports). So I can deal with a bit of stalking, if that's what makes my country safe for its citizens.
  7.  All bullshit, they are not even any points to say that you don't want an unwanted protection. You just extended previous points to complete 10 points.

    (I agree with you on a few points, for e.g. Police harassing woman even when they try to register an FIR)
I found your post, totally unwanted, and yes you asked for this criticism.

PS: Where ever I have mentioned the word "you", I do not intend to mean that for the author of the BS article, but for all those who hold the same opinion as the author.

Thursday, November 14, 2013

[Food for thought] Big Data or Complex Computation??

When we hear of Big Data (which we are hearing a lot these days), always comes to our mind that we are talking about the data to be big. Is it really the case? Is data a problem or computation?
Well honestly, I don't know the answer. But in my opinion, It's not the size of the data which matters, it's the algorithm, currently available tools and speed of computation which matters. Suppose we have census data of 1.2 Billion people of India, contained in a 30-40 GB file, can we say this a Big Data? Well it depends. Dr. Gautam Shroff writes in this article, that to calculate the average age from a census data would have computation complexity of O(N).  Even to calculate moderate computations such as, "calculate the average age of all the husbands who have their wife's age in the range of 15-19 and belong to North India" would not be so computationally expensive to be called a Big Data.
On the other hand, calculating summary of hundred thousand Wikipedia articles (which would be in the size range of few hundred MBs)  using LexRank algorithm would not be so easy to compute, and might need some parallelization for getting the results in "human patience" time. So the question still remains, What is Big Data? Is it just size that matters or how complex is the computation?

PS: Big Data sounds awesomer than Complex Computation.