Open sourcing Social Photos

I have been working on a lot of open source projects lately (chromium, AOSP, ffmpeg, etc.) and I have had this tremendous change in the way i look at software projects now. I am all the more convinced that open source is the only right way of doing software.

With that in mind, I've made a pledge to myself that no matter what I do, I am going to put the source out there. As a first step, i'm open sourcing the one big project of mine, Social Photos.

The source can be found here: https://github.com/vickyg3/social-photos

It is a snapshot of the one that's currently powering the live site: http://socialphotos.net (with API keys redacted). Feel free to fork and use as you please. Although i'd appreciate a link back, it's not mandatory. Also, i'll be more than happy to look at Pull Requests.

One of the main reasons that developers (including myself) don't post our code out there is that we are ashamed of our code. I was really ashamed by the number of hacks i did in this project that i couldn't even think of making it public. I'm over it. I'm ready to accept people fixing my mistakes.

Happy Coding!

-Vignesh

Announcing Social Photos v2!

I have been working on this side project for quite a while now and I am happy to announce that it is launch ready now!

Visit http://socialphotos.net to manage all your digital photos across Social Networks in one roof using a simple intuitive user interface.

Features Include:

• Transfer Photos
• Transfer Albums
• Download selective albums as a zip file
• View Slideshow of your albums
• Monitor progress in a simple pane
• And much more!

Please do feel free to give feedbacks/suggestions. Use it and Share it!

http://socialphotos.net

-Vignesh

Pre School Problem - A programmer's approach!

Flashback - A few weeks ago

I came across this question in facebook. It said, "This problem can be solved by pre-school children in 5-10 minutes, by programmers in 1 hour, by people with higher education.. well check it yourself :)"

I shared this post and we had few nice sessions of fun with my colleagues in office. Myself and a colleague of mine were discussing about this and i told him, "the best part about this puzzle is that it cannot be solved programatically". And we carried on with our lives.

Train Journey - Last night


Last night when i was travelling by train, I had gotten a side lower berth and hence couldn't sleep (i have grown too tall to fit in to side berths :-( ). So, i had to while away time. I was tweeting, finished through a couple of stanford algo and nlp class videos. Then i had nothing to do, phone was almost running out of charge and so was the laptop. So i closed everything and started staring outside the window. Various thoughts came across, and one among them was "is there no way to programatically solve that facebook puzzle?".

Thus I started thinking about it, the first solution that came to my mind was to convert the numbers into a linear system of equations and apply linear regression on it. Then, the value of the co-efficients would give the mapping between a number and how we arrive at the solution (explained in detail below).

Coding Session - Today


I came home had a nice sleep. When i woke up, the first thing i wanted to do was to test out if my theory works. Obviously python (numpy) was the toolkit in my arsenal that would help me do this quicky. So here are the steps:

• Convert the input number into co-efficients of 0 or 1. For example, 2172 would translate to [0, 1, 2, 0, 0, 0, 0, 1, 0, 0]. (i th element denotes the number of times number i occurs).
• Now we have the coefficient matrix ( of size n x 10 where n is the number of inputs).
• We also have the constants column vector which are the known outcomes
• Now, we apply linear regression on this system to get the output
• Output will be a mapping between a number and how much it contributes to the output (0 maps to 1, 1 maps to 0, 2 maps to 0 and so on.)
• So to compute the outcome of a new number, merely sum the mappings of the digits in that number.

The whole program is embedded below:

The output of the program is:

['0 maps to 1', '1 maps to 0', '2 maps to 0', '3 maps to 0', '4 maps to 0', '5 maps to 0', '6 maps to 1', '7 maps to 0', '8 maps to 2', '9 maps to 1']

As we can see, that's a perfectly valid mapping!

Moral of the story:

1. Mathematics is really powerful. 
2. We can teach a computer much faster than we teach a pre-school kid. ;-)

-Vignesh

Replacing Lightweight Web Services with Twitter Bots!

Foresight
Here is a vague idea. I'm not sure if i can even call it an idea. I always had the tendency to create lightweight information portals as usable web-based services with a very simple user interface. It just used to get things done and nothing more.

Practical Use Case
For example, let us consider a simple problem of "PNR Status Enquiry" in Indian Trains (for the uninitiated: PNR status is nothing but the current status of a waitlisted ticket in Indian Trains). First of all, such a system is really useful because the official Indian Railways websites are a bit clogged and they don't provide any alert services. Also, their websites are not so catchy and mobile compatible, and we definitely don't want to switch on the computer just to check the PNR status.

Before today, if i was asked to build such a system, i would go for an elegant google-like web page, where there is just a text box and a button for the user to enter the PNR number and click go. The resultant page will be again a simple HTML table with the ticket details (of course all of them scrapped from one of the railway websites - pretty sure that this isn't legal, although i am not aware of any laws against site scraping).

Do we really need web for this?
But this thought stuck me today. Web is a wonderful platform, at times too good to host silly and redundant stuff like this. Tomorrow i may have to build a similar system for Buses or Aeroplanes requiring me to add more and more pages with almost same functionality but different information sources. We don't need the web for such silly things. My idea is that, why not use "Twitter" as a platform for such web services. I've heard of facebook as a platform before for many applications and games (Farmville), so why not twitter as a platform too?

Twitter as a Platform
So what exactly do i mean by twitter as a platform? Let us redesign the same PNR status enquiry system using what i mean by twitter as a platform. Instead of having a web page for inputs, lets host a twitter bot, say @pnrbot. Now, whenever you need to enquire the status of a PNR number, all you have to do is post a tweet mentioning that bot, for example "@pnrbot 1234567890" (where 1234567890 is the PNR number you wish to enquire).

Now, as long as your tweets are public, the bot can read your tweet almost instantaneously thanks to the vast amount of real time APIs provided by twitter. Now the bot does the usual site scraping from the railways website for your PNR status and it posts it as a reply to your tweet. Simple isn't it? To take it a level further, the bot can also autotweet your PNR status every day until your journey date, which is not very easy in the case of a web based app. Also, since you will not be following the bot and the bot will not be following you, you will not clutter any of your friends' timeline with this tweet as it won't appear in their timeline.

If any of the input data is sensitive and not to be exposed, then the same design can be adopted by just switching the term "tweet" with "Direct Message" (although in that case, both you and the bot need to be following each other, which can be easily accomplished).

Pros of the Platform
Twitter is a part of our day to day life since the inception of mobile internet. So, you get many useful information from such bots interactively rather than opening your webpage and waiting for the page to load. Moreover, twitter is known for its notifications. If you have activated SMS alerts for @ mentions, then you don't even need to have internet to make use of such a bot. You can just send the tweet through an SMS and read the reply from the bot as SMS. Also, twitter has email notifications which may also be of good use. As mentioned in the example, you can have one input with multiple periodical outputs (like time based notifications, etc.) which is not so easy to implement in web based systems.

Developer Standpoint
So from a developer standpoint, what do we ultimately gain by choosing Twitter as a platform over Web for lightweight services? The answer is quite simple and really advantageous. For one, you need not host a web server for lightweight web services. All you need to do is run a script that will act as the bot. The script will also be really lightweight since twitter APIs does all the pushing for you (no polling). You offload majority of the user interface and load to twitter and do only the actual processing in your server.

From the implementation perspective, it'd be really great to have a good library/framework in a nice scripting language (like php or python) for building such a bot so that the possible features (like twitter API access, etc.) could be abstracted out thereby the developers actually have to write just the logic of their actual bot and nothing extra.

Closing Thoughts
I am pretty sure such bots already exists. By building more and more of interactive bots Twitter can really stand tall as a good platform not just for communication but also for instantaneous information retrieval.

P.S.: I am in the process of developing a simple such bot as a proof-of-concept. I am also highly determined to come up with a generic twitter bot library as i mentioned in the post. So as always, interested developers are welcome to carry on if you like the idea!

-Vignesh

Orkut Deja Vu - The Technical Side!

Its been almost two months now since I first launched "Orkut Deja Vu" - A series of web applications and a chrome extension that helps you move your memories from orkut to facebook. Personally, I think the application was a good hit. In this article i will try and explain the technical side of it and the various hurdles faced on the due course of development.


Note: This article is for the technically inclined and if you don't want the technical details and just want to use the application, visit http://orkutdejavu.foamsnet.com!


Java and My server!


To begin with it, my server is a VPS running linux with a humble 700MB RAM and a shared processor. My search for an Orkut API ended with orkut os client - An official API provided by Google. This is more of a library than an API and only the java implementation of the library was very sophisticated (though there were php implementations, they weren't that good). Ever since i started learning computer science, if there was one thing i hated, it was java (now don't even get me started about perl).


So there lied the first problem in front of my eyes: Running java in my server. I didn't want to take up the pain of setting up JSP for this sake and decided to just call the java program that does the orkut calls from PHP using shell_exec. I know it isn't a safe option, but since there is no user passed data and the calls are hard-coded, i was sure that there was no injection vulnerabilities. Yet this approach isn't very efficient as it spawns a separate java vm for every exec call, but it was a compromise that i had to make for not setting up a java based server.


OAuth - The headache!


The orkut library's OAuth implementation was really messy. Fortunately, it had a method to explicitly set an OAuth access token obtained from elsewhere. That said, I used Zend OAuth library in PHP to perform the 3 legged OAuth and use that access token in the java programs. There were totally four java programs: one each to fetch the user's orkut name, albums list, photos and scraps. The java programs printed the output as JSON if it was successful or nothing if there was an exception. PHP then parses that JSON. I used JSON so that all the escaping will be taken care by the JSON libraries and thereby ensuring safety of data transmission.


Photo Album Migrator


The photo album migration was quite straight forward without much hurdles. All i had to do was integrate the following: Image gallery, Facebook API, Orkut API and write some simple javascript that made AJAX calls to transfer the photos. This was quite a cakewalk as i was already very familiar with the Facebook graph API.

Scrapbook Downloader


Again, the programming side of this was quite straight forward. But there was one major challenge involved in designing this. The java program generates a html file containing the scraps which is then converted to PDF. HTML to PDF conversion is CPU intensive and hence needs to be done with care. I could have used a resource manager like Sun Grid Engine, but i did not want to make things complex for a simple job to be done. Also, the conversion is not done programatically by a library as all the PHP PDF libraries were memory-wise very expensive (a file with ~100 scraps always exceeded PHP's memory limit of 64MB). So, I used a external command line utility (wkhtmltopdf) to accomplish this conversion. Again it was a simple shell_exec with hard-coded arguments.

Testimonials Migrator


This was the really challenging part. There was no Orkut API that gave access to users' testimonials. But i didn't want to give up. I really wanted to pull off a tool that can migrate testimonials to facebook. So I thought of attacking this problem from the heart of orkut - the orkut website. Obviously, the first thing that came to my mind was a Google Chrome extension. The extension will inject a javascript into orkut.com website and add a "Post to Facebook" button beneath eacch of your testimonials.


The first hurdle in accomplishing this was that there were two versions of orkut (old and new) with different page structures. I didn't want to write two different scripts to handle the versions. Instead i wrote another script that detects the version and if it is new, it prompted the user to redirect to the older version in order to use the extension. The next job was to understand orkut's DOM so that I can place the buttons. This ended up being quite an easy job too.


Another glitch in chrome extensions is that we cannot specify images directly in CSS for content scripts(for e.g.: background: url(a.jpg); is not possible), as the CSS will run in the scope of the website. So, it has to be either done programatically using javascript or encode the image in base64 and hard code it in the CSS. I chose the first option. From this point, it was fairly straight forward. When the user clicks on the post button, store the corresponding testimonial using HTML5's local storage and create a new tab where the user can choose the posting options. Again, the facebook authentication and API usage here were simple as i had enough exposure already.


Epilogue


Though it seems like a simple app, huge amount of thoughts are put into every single aspect in design of the application and the entire development process was a fun journey with a great learning curve. This application has made me feel my web presence. Web is really a great platform for amateur developers like me. I have got around 20 new followers in twitter and a person from brazil appreciating me for this application. I have really been motivated a lot by this and hope to continue the same stride in creating usable applications as this one!


This post will not be complete without a heartfelt thanks to all those who supported me on due course of development and all those who used and shared this with your friends!


-Vignesh

The Social Network - Build it Google, We will Come!

I recently read an article about Larry Page taking over as the CEO of Google and sending an internal memo to googlers about 25% cut in their bonuses if Google didn’t do well in social arena this year. I was disturbed a lot ever since I read that and wanted to pour out my thoughts about a dream social network from Google. Yes, I frankly think Google can still do much better than Facebook.

A headnote, this article is totally how i view a Google social network shall be made possible. I have tried to think of it as practically as possible. Some of it may sound silly/stupid for expert readers, so kindly bear with me.

Orkut was the trend setter! - What went wrong?

Orkut was the first and most famous "social network" of all time. It came during the period where the term "social network" was not really defined. Orkut gave that term a definition. And it was doing really well especially in places like India and Brazil. Ever since Facebook came in, Orkut started to lose.

I, personally, would say that the reason why Orkut never caught on was because Google tried to remain professional. They probably wanted to stick to the Google style of doing things and in the process forgot that people just wanted to have fun and party on in the social network unlike other Google services where professionalism kept people happy. The best example i would like to quote is, Orkut did not have scrap threading for a long time and that was available even for SMS by the time Orkut added that. Naturally, we don’t want to roam around in a blazer/suit 365x24x7. Facebook realized it and they just let people do whatever they wanted to do, literally no restrictions whatsoever.

Sign of Innovation - Google Wave

Google then launched wave and claimed that "it is simply going to change the way people communicate". Well, we know how true it is from the fact that wave has been axed by Google few months back and is now residing in its open source home of Apache.



Again, the problem with Google Wave has been that it was difficult for a layman to understand and use it on a day-to-day basis. Even when wave was axed, not a single layman cared about it, only computer professionals cared since the underlying technology and protocols were really well built. Yet again, Google had failed to capitalize the wonderful technology they built by coping it up to what everyone actually wanted. In short, Google tried to remain professional with wave too.

Sign of Desperation - Google Buzz

Then came Google Buzz, supposedly the "Twitter killer". One major factor that i feel Google doesn’t realize when building social products is that, You cannot force people down their throats to go Social. Yes, I love Twitter and can’t live without it but at the same time I don’t want to read my tweets in my Gmail inbox.



Similarly, Google Reader is one of the most wonderful software ever built. But with literally no connection whatsoever, Google decided to tie Buzz with Reader. This tie up lead to nothing but junk in both Reader and Buzz. And when people wanted to opt out of Buzz, it resulted in loss all their Reader follow list too. Making it opt-in is a really good vision by Google and at the same time they should also make sure that the opt-out is cleanly done.

What next? Social + Google = ??

So can Google ever surpass the mountain, that is Facebook, and succeed in social like they did in search? My answer is, yes they surely can. Sure Facebook has some 600 million users but Google is no poor lad when it comes to userbase too. Infact, Google is still a more prominent part of our life than Facebook is. Conduct a poll asking "Which of the following can’t we live without - Facebook or Google?" and i bet the winner will be Google.

Imagine a social network that revolves around all of Google’s services. Imagine a service that aggregates all of gmail, youtube, reader, maps, picasa, talk, news, voice and even orkut. Google, if you can build a rock solid service that does this and add your typical magical touch to it, then definitely you can climb the mount everest that is Facebook. Add to this the crazy, yet possible, thought of Google acquiring Twitter and integrating Buzz with it seamlessly. By recitifying all the mistakes that it did in the social arena in the past and building such a clean and fun service, it can surely overpower Facebook.

Google’s userbase is much more loyal when compared to that of Facebook. Google, as a company, with its policies is much closer to our hearts than Facebook is. Identify Facebook’s problems and fix it. Facebook’s main concerns today are spam and privacy. I can’t think of a product other than Gmail that can handle spam so near to perfection. Also, Google Talk has been an integral part of Gmail and Orkut for so many years now and not once we have seen any spam in it that can even be compared with the Facebook chat spam we have these days. Though Google have had their own share of controversies regarding privacy, they have somehow held on and been in the good books of their users when it came to privacy.




Of course all this is not going to be an overnight affair, it is going to take lots of effort and time. Given that Google is investing so keenly on getting that top spot in the social arena, proper focus, being unprofessional and learning from the past are the key factors that will decide the fate of Google in the social arena. They can’t afford to flop anymore. This may be Google’s last chance to make an impact. By heart, i sincerely hope that Google can just snatch that number one spot in the social arena!

So, my message is short and simple: "Build it Google - We will come!".

-Vignesh

Introducing Orkut Déjà Vu! - Orkut to Facebook Photo Album Migrator!

As we all know, Orkut was the trend setter of the modern "Social Networks". Few years back, Facebook took over that top spot in social networks from Orkut. Yet we all have lots of sweet memories stuck with Orkut in the form of Scraps, Photo Albums, Testimonials, etc. This website helps you pull out your old memories from Orkut and make them brand new by moving them over to Facebook!



This is just a night time project that i have been working on for almost a month now!

I don't want to bore you with stories. Try out the app straight away here: http://orkut.foamsnet.com or http://orkutdejavu.foamsnet.com

I am sure there are plenty of bugs. Please email me at vig..@gmail.com(click to expand) for any comments/feedbacks!

-Vignesh

InstaShare - Facebook Extension for Google Chrome! :-)

InstaShare - Share everything you find interesting on the web as and when you browse to your Facebook profile! The web you surf daily is now just a single click away from your facebook profile!

This is a mature version of yet another script that i have been using personally so far.

Features include:

• Post status updates through an always-present browser icon


• Share any image in the pages you visit in a single click


• Share any link in the pages you visit in a single click


• Share any piece of text in the pages you visit in a single click

Download You can download the extension from here: http://www.foamsnet.com/instashare

Screenshots
Sharing Images





Sharing Links





Sharing Selected Text





Posting Status Update




You are sure to find a lot of bugs. Feel free to contact me at vig...@gmail.com (Please click to view the full address) for any bug reports or suggestions!

-Vignesh

Why you should tweet?

I am one among the many millions out there who belong to the social era. The world is socializing much more than it ever did before. If facebook were a country, it’d be third largest in terms of population. Facebook (and even the damned orkut) is really famous in India. Twitter is a similar service with even a simpler goal, share all you can in 140 words.

I have found many teenagers (especially college goers) not use Twitter. It is not really surprising considering twitter doesn’t have many entertaining’y things like photo tagging, liking every damn comment, etc. Twitter is the best example for “simplicity”. The inception of twitter has rendered many bloggers lazy to even blog regularly, as they start tweeting more about things rather than writing lengthy blogs.

In this post, i want to publish my opinions on why teenagers should tweet or atleast use twitter.

Source of Information
Twitter is really a great source of information. Let it be technical information or some interesting news from/about your favorite celebrity, the first place where you can find it is twitter. If some hot news is being flashed in the tv channels worldwide, then you bet its definitely trending on twitter. You really get an instant, unbiased view of many people all around the world on any topic. This kind of rapid fire spread of a news is more common in twitter than in facebook because of twitter’s short and sharp nature.

Social Media is a fancy term used these days to indicate the facebook, twitter and co. Not only social media provides entertainment and helps you kill your time in the form of facebook, it also gives your brain some real value additions in the form of 140 characters.

Enterntainment
Many teenagers (atleast my friends) have the opinion that twitter is for geeks. Whenever i hear someone say this, i go WTH. Twitter is definitely much more than a source of useful information. You have entertaining discussion going on between not just your friends, but also with many other people with whom you might have just dreamt of before. (For example, i have got one or two replies from my favorite singer Chimayi).

Short lifetime of tweets
You post a status update in facebook, and it definitely lies around in your homepage (in the form of notifications, etc.) for a minimum of one day (mostly more than that). Whereas lifetime of a tweet is much shorter than that. Your tweet will remain in your followers’ home page for hardly a few minutes. So all the communication and sharing remains up to date and instant. This point could be argued either ways as both advantageous and disadvantageous. But in facebook, after some 50 odd notifications over a period of week, i’d regret why i liked a status in the first place.

I am not coming to say that facebook is bad. Facebook has its own use case. The status messages in facebook are meant to be lingering around one’s homepage for a longer period of time. Hence twitter should be the real tool that shall be used to post real time updates, where people can comment on it instantly too. Also, tweets come up in what is called Google Realtime Search results.

Internet is huge and a few years back Google revolutionized it by organizing the information available in the internet. That was purely an algorithmic approach, whereas today, twitter is trying to generate and organize information based on one’s personal context, supplemented by algorithms. So, join twitter today and i promise i’ll follow you :-P

You can follow me in twitter here.

-Vignesh

SMS - Talk through mobile from facebook !! :-)

Various application layer protocols always impressed me. Especially HTTP, as my understanding about it has been pretty good. There has been one thing that i have been doing pretty good, site scrapping.

What is site scrapping? Defining it is very simple. It is the process of extracting some information from a website that has complex interface (with flash, ads, etc.) and using that information for our own purposes. Though scrapping many websites (especially government ones) may be illegal.

Here is one such application that i have developed. This application is purely a brain child of the HTTP i have understood. It provides an interface to the famous Free SMS Gateways in India 160by2 and way2sms through a facebook application. It has been one of the dead works that remained uncared for a long time. Now that i have a decent amount of online presence, i have refined it and trying to popularize it through my blog (should i call it blogvertise ?).

Using this application is simple. Register yourself at Way2SMS or at 160by2. Goto http://apps.facebook.com/smstalk. Add your account here and you are good to go. Though this application is not a big feat, it just provides a simple and clean interface (exposed as a facebook app) for the ad clustered interfaces of the famous free sms gateways in India.

Here are a few screenshots:






Share your opinions/suggestions in the application's wall here or in comments.

-Vignesh

A Plant that lives on Facebook !! :-)

I recently came to know this plant that relies on facebook for its life. We all would have learnt in our secondary school that plants live with carbon-di-oxide and photosynthesis. But now, here is an innovation that connects the virtual and the physical worlds.

With the basic theme as, plants don't just grow with the things that we have learnt in our high school botany. Coming under the category of living things, they also require love, care and attention to grow. Connecting the offline and the online worlds has always been something i wanted to do and this is a project that has made me look at it with an awe.

This project, named Meet Eater, has been running since May and currently (September 2, 2010) has 3041 likes (including mine). This has been developed by John Bashkim belonging to the University of Queensland, Australia. The plant grows based upon the human interaction with it using facebook. It gets it nutrients when people become friends with it, write in its wall, etc. It even replies back to your posts (of course by a backstage human controlling its profile).



This is how Bashkim describes the plant:
"Meet Eater is a plant, but not one we're used to. This plant is watered upon the receipt of physical interaction, when it makes friends on Facebook and when people write on its wall. Behind this project is the idea that by introducing both physical and virtual levels of interaction the plant is able to make the move from being an object to being a creature. A creature you can become friends with on the Internet no less."

The facebook page has a link to view the live cam, where you can see the beautiful innovation in action. The Meet Eater garden is situated in the State Library of Queensland.

The facebook page also says:
"Meet Eater is rigged up with a system to register physical contact and social media interaction. Watering is triggered by human contact and nutrients are delivered when you start hanging out with it on Facebook."



So, go hang out with Meet Eater and thereby help it grow. Here is a link to the facebook page of Meet Eater.

Note: Thanks to mashable and allfacebook for the pictures.

-Vignesh