[v2.0] Ok Google, Open my curtains!

This post is a follow-up to my previous post http://vignesh.foamsnet.com/2018/06/ok-google-open-my-curtains.html.

Note: I've created a github repository which contains the code and some more details. I intend to keep that up to date with newer ideas, etc. https://github.com/vickyg3/curtain-automation

When i posted the previous set-up on reddit, one of them suggested a better way to do this that will overcome the cons of the previous method (noise and lack of manual control).

In a nutshell, these are the replacements i made:

• Use a stepper motor instead of a servo.
• Use a timing belt pulley and timing belts instead of a pulley and rope.

Here are the updated list of parts (with links to the cheapest ones i could find online to keep the cost minimum - although i didn't necessarily buy it from these links):

* Nema 17 stepper motor - $6.99

* TB6600 stepper motor driver - $9.96

* Mounting bracket for the stepper motor - $2.82

* Timing belt and pulley kit - $12.98

* Mounting bracket for the pulley on the non-motor end - I used a servo mounting bracket with a long screw. I am sure there are better alternatives for this one - $12.99

* Super glue - $4

* Vibration dampener for stepper motor (optional but highly recommended to keep the noise down) - $2.98

* NodeMCU (ESP8266) microcontroller - $5.98

* 12V DC adapter - $8.99

Total Cost - $67.69 (some of it can probably be gotten for a cheaper price on ebay/gearbest/aliexpress).

Step 1 - Screw in the mounting bracket

This is quite straightforward. I drilled in a couple of drywall anchors and screwed in the mounting bracket to the wall.

Step 2 - Install the motor and the vibration dampener

Place the vibration dampener on the motor. Make sure that the non-threaded holes on the dampener are touching the motor and use two M3 screws to attach the motor and the dampener. Attach the pulley wheel to the motor's shaft using the screws on the side of the pulley.

Now the motor can be mounted on to the mounting bracket using 2 M3 screws which connect to the dampener's threaded holes.

Motor attached to the mounting bracket (shown without the vibration dampener).

Step 3 - Install the pulley mounting bracket on the other end

I drilled a drywall anchor on the other end and screwed the pulley mounting bracket in. I then used a long screw with a couple of plastic fillers to hold the other pulley roughly in the center.

Step 4 - Loop the timing belt around the pulleys

I had bought an open ended timing belt. I looped it around either pulleys and tightened them as much as i could by hand and cut off the rest of the belt leaving about an inch of overlap. I then used super glue on the extra inch of belt to stick to them together with as much tension as possible. You could also use a clamp like this with a screw to close the loop on the belt.

At this point, you should be able to move the belt manually by hand by pulling it gently in either direction. If so, there is enough tension on the belt to move the curtains.

Step 5 - Attach the curtains to the timing belt

Just as before, i placed the curtains in the completely closed state and attached each curtain to either side of the timing belt using a safety pin.

Here's a sketch from before about how the pulleys and the curtains move:

Step 6 - Program the microcontroller

I used the same NodeMCU ESP8266 controller as before. Here's the gist that contains the Arduino sketch: https://gist.github.com/vickyg3/623ca535b565ed43c8b617a6f7c7c06f (it is a crude version and has a lot of room for improvement, but i left it for another day).

Step 7 - Microcontroller and Motor Driver setup

Here's the wiring:
* DIR+ to D2
* PUL+ to D5
* ENA+ to D8
* DIR-, PUL- and ENA- to GND (can either be bundled together or connected separately to individual GND pins on the NodeMCU)
* A+, A-, B+ and B- need to be connected to the motor (refer to the motor's spec for the wiring colors for your motor).
* The switches on the driver need to be set to ON, ON, OFF, ON, ON, OFF (S3 and S6 should be off).

Connect the DC power supply to the driver. I cut the ends of an old DC adapter and identified the positive and negative wires using a multimeter.

I then glued the driver and the NodeMCU board to the wall behind the curtain using small command strips. Any double sided tape should work as they aren't that heavy.

Step 8 - Home Assistant Configuration


That is pretty much it. For more details on how you can control it from Google Home/Alexa, please see my previous post.

The noise level of the old servo based system was around 60 decibels (as measured by an app on my phone). The new system with vibration dampeners works at 30-35 decibels and is barely audible. Most of the noise is from the curtains actually moving on the rod.

Here's how it works (The video is noisy because the roomba was running when i recorded the clip):

Here's a video without much of external noise:

Ok Google, Open my curtains!

Finally I've found a worthy enough reason to write a blog post.

I've been wanting to make the curtains in my home "smart" for a while now. I have what is called "grommet curtains"  and there are apparently no off the shelf solutions to automate this. There is this kickstarter project called Slide which retrofits to any curtain type, but that is still in its early stages and a bit pricey (~$150).

I ran into a tutorial which spoke about automating grommet curtains. But they used a 3d-printer to print custom designed components which wasn't an option for me (I don't have a 3d printer and i don't have any knowledge of how to design components for a 3d printer).

I borrowed the basic idea from that video. Two pulleys at either end with a rope running around them. One of the pulleys is attached to a servo which is controlled by a microcontroller. Here's an awful sketch of the idea:

Here are some of the challenges faced/things i did:

• There is no cheap off the shelf servo that has a pulley attached to it. So i had to tweak around to figure out a way to mount a pulley to the servo.
• The load on the servo to pull the rope along with the curtains is pretty high. So the servo has to be mounted securely to the wall. I first tried a heavy duty double sided tape, but that wasn't enough to withstand the tension of the rope. I then tried out various things and found a mount that could be screwed into the wall.
• The wall mount for the servo held it too close to the wall and there was no room for the pulley (whose diameter is larger than the breadth of the servo) to fit in and rotate smoothly. So i used a couple of washers behind each mounting screw to give enough room for the pulley.
• To tie the other pulley (which has no motor), i simply used a rope to tie the pulley on to the curtain rod's mounting bracket.
• The servo had to have metal gears in order to have enough torque to pull the curtains. But such servos need a minimum of 4.8 volts of power. I found a NodeMCU board which had a 5V pin to supply enough power to the servo.

Here are the list of parts with links to where i bought them:

* NodeMCU (ESP8266) microcontroller (model with 5V pin) - Amazon - $5.98
* Jumper wire (to connect to the 5V pin) - Amazon - $5.24
* MG995 Servo (360 degree, metal gear, digital model) - Amazon - $10.99
* Servo wall mount - Amazon - $12.99
* Washers - I just used a couple of old washers lying around. Equivalent link: Home Depot - $2
* Pulley (for attaching with servo) - Servocity - $3.49
* Pulley (for the other end) - Home Depot -  $2.47
* Rope - Home Depot - $3.98
* Safety pins - Amazon - $3.15

Total Cost - $50.33 (some of it can probably be gotten for a cheaper price on ebay/gearbest/aliexpress).

Ok, with all that out of the way, let me explain how exactly i built and programmed this.

Step 1 - Attach pulley to the servo

Attaching the winch pulley to the servo is quite straight-forward. I screwed in the 4 pieces of the pulley with 2 bolts (on diagonally opposite ends) and small hex nuts. I then attached one of the servo horns to the other two holes of the pulley with two small screws (that came with the servo).

Now, the servo horn can merely be screwed into the servo until it's fairly tight.

Step 2 - Install the mounting bracket

After marking two spots with the mounting bracket's holes, i drilled in two drywall anchors on those spots. Then use two screws to screw the mounting bracket into the wall placing two washers in between the mounting bracket and the wall. This makes sure there is enough room to mount the servo along with the pulley.

This is the bracket i used to mount the servo to the wall

Step 3 - Attach servo to the mounting bracket

This is very straight-forward. The only catch is i used two nuts instead of 4 to attach the servo on to the mounting bracket.

Servo mount with washer behind. Pulley attached to the servo.

Step 4 - Mount the pulley on the other end

I used a small rope to tie the pulley on to the curtain rod's mounting bracket on the other end. A nut and bolt can also be used but since the rope worked for me i didn't look further.

Step 5 - Tie the rope on to the pulleys

Loop the rope around the pulley without the servo once. In the servo end, loop the rope around the pulley at least twice and then tie a really stiff knot. Make sure that the rope is really tight. Cut off any extra rope since it could cause friction when the rope is moving.

Rope tied to the pulleys and attached to the curtains.

Step 6 - Attach the curtains to the rope

I used a couple of safety pins to attach the curtains to the rope. I set the curtains in fully closed positions and then attached each half to one of the ropes. When the pulley spins, both the ropes will move in opposite directions thereby opening each curtain outward. The same process is reversed to close it.

Curtains attached to the rope with safety pins.

Step 7 - Program the microcontroller

This is my first foray into programming a microcontroller and surprisingly enough it was really simple. The NodeMCU ESP8266 board is a $5 wifi enabled board. Wifi means that you can send commands to it through simple HTTP requests.

Here's the arduino sketch i used to make a simple HTTP server that supports 3 commands: left, right and off (which spins the servo left, right and turns off respectively): https://gist.github.com/vickyg3/94b80435fa10a58691f563f643aaf807

Once this is flashed into the board, controlling the servo is simply a matter of "curl http://<ip>/left".

Step 8 - Mount the microcontroller

I mounted the microcontroller near the servo (using a small double side tape on the wifi chip) and just attached the servo wires to the appropriate pins (i used a jumper wire to connect the power wire to the 5 volt pin). I then ran a long micro-usb cable that i had lying around to power the microcontroller.

Step 9 - Home Assistant configuration

I added this to home assistant as a command line cover that used curl. Here's a sample configuration:

Where curtains.sh contains the following:

I determined the time to sleep between rotating the pulley and turning it off by trial and error. It usually took more time to close than to open because the set up struggled a bit to pull the curtains in that direction. It now shows up in home assistant, like so:

That's it. You can now include this in your automations, etc. I have set up automations like close the curtains after sunset, when we are away and so on.

If you have your home assistant tied to google assistant, you can control the curtain by saying "Ok Google, Turn the <curtain name> on/off". If you want custom commands, you can also try ifttt with google assistant to have commands like "Ok Google, open my curtains!".

Here's the Eureka moment

Some cons of this system:

• The servo is a bit noisy. This is probably ok given that curtain control is not too frequent. It is probably bearable for 10 seconds or so once in a while.
• Hooking up the curtains this way makes it impossible to control the curtains manually (since the curtains are tied to the rope). I've worked around this by sticking a button next to the curtain by the wall for those scenarios where we don't want to use voice control. Pressing the button will open/close the curtain.

  

  Button to control the curtains in lieu of voice/phone control.

Comments and suggestions about improving this in any way are welcome!

Use your Chromecast/Roku on a Hotel WiFi

Chromecast and Roku are really wonderful devices which allow you to take all your media with you where ever you go. Especially, if you travel a lot, it's definitely way better to watch something on your Netflix subscription rather than paying exorbitant rates on the hotel's pay-per-view.

But there's a problem. Hotel WiFi's usually use web based authentication (i.e.) their wireless network is usually open and anyone can connect to it. Only when you try and open a webpage, they send you to an authentication page that asks for a username/password (which the hotel will provide you) and/or asks you to accept the terms and conditions.

The problem is that Chromecast and Roku do not have native support for such authentication. But fear not, in this post, i am going to write about a few ways in which you can work around this limitation to enjoy Roku/Chromecast on the hotel's WiFi network.

Find out your device's MAC Address

2 of the 3 workarounds that i mention requires you to know the MAC Address of your Chromecast/Roku device. Here's how you figure it out:

Roku

• Switch it on
• Navigate to Settings->Network->Wireless
• On the bottom right, you can see the MAC Address of the Roku

Chromecast

• Switch it on
• Connect your phone to the hotel WiFi and authenticate it
• Open the Chromecast Setup app on your android phone
• Tap on devices and wait for it to scan
• Tap on your Chromecast device in the list
• Tap "Set Up"
• Tap "I see the code"
• Tap "Set Name"
• The MAC Address of your chromecast will be displayed on the screen now. 

Workaround 1 - Call the Tech Support

Most hotels give you a small sheet of paper with your WiFi username and password. That sheet usually also has a "tech support" phone number. Here is what you can do: Call the tech support number and tell them exactly this: "I would like to use the roku media player device over the hotel WiFi. I wanted to check if you could whitelist the MAC Address of my device to by-pass the authentication page".

Depending on the mood and knowledge of the tech support person, he/she might say okay. But nevertheless, it's gonna take some time before the whitelist happens. And once the whitelist happens, you can enjoy Chromecast/Roku over the hotel WiFi.

Workaround 2 - Spoof your MAC and authenticate from your laptop

The whole web authentication system works around MAC addresses. Once you enter your username and password, your MAC and IP addresses are going to be whitelisted for a certain amount of time (about 24 hours) and your device will be allowed to access the internet in that time period.

One key weakness about this authentication mechanism is that MAC address is easily spoof'able. So we are going to to assign your Roku/Chromecast's MAC address to your laptop and perform the authentication like usual and then make use of that whitelist to access the internet from your Roku/Chromecast. Here are the steps (follow them precisely to the word):

• Find the MAC address of your Roku/Chromecast using the steps above. Note it down.
• Open the terminal (assumes linux).
• Run "ifconfig" and note down the MAC Address of the laptop. [1]
• Run this command: "sudo ifconfig wlan0 down".
• Disconnect from the WiFi network on the laptop.
• Run this command: "sudo ifconfig wlan0 hw ether <mac address of roku>".
• IMPORTANT: Turn off the Roku/Chromecast by unplugging it.
• Run this command: "sudo ifconfig wlan0 up".
• Connect to the Hotel's WiFi network on the laptop.
• Open the browser and navigate to google.com (or any other website).
• It will ask for authentication, provide username/password that the hotel gave you.
• Type google.com again and verify that you can access Google.
• Run this command: "sudo ifconfig wlan0 down".
• Disconnect from the WiFi network on the laptop.
• Run this command: "sudo ifconfig wlan0 hw ether <mac address of your laptop from step [1] above>".
• Run this command: "sudo ifconfig wlan0 up".
• Now you can plug your Chromecast/Roku back in and connect to the Hotel's WiFi network on that. It should now be connected to the internet!

You will have to repeat these steps after the whitelist expires (usually about once a day).

Note: The above two work arounds will connect your Chromecast/Roku directly to the hotel's WiFi 

Workaround 3 - Set up your own WiFi network

Buy a portable WiFi adapter and use that to share your Hotel's WiFi through your own access point.

Hope this helps!

-Vignesh

Chromecast URL Player

I was annoyed by the fact that the Chromecast SDK has been out for the public for quite a while and yet googling for a simple URL player for Chromecast did not yield satisfactory results. So I sat down to create my own simple web app for playing back any http video or viewing any still image in Chromecast.

URL: http://movies.foamsnet.com

Source Code: https://github.com/vickyg3/UrlPlayer

Would love to hear feedback. Feel free to fork, send pull requests.


-Vignesh

Introducing Super Secure File - One File, Completely Secure, Access Anywhere!

Super Secure File gets you a really secure password protected file that you can access anywhere with an internet connection. Smartphone, Tablet, Laptop, SmartTV, anything!

To cut to the chase and create your Super Secure File, go to: http://secure.foamsnet.com.

How does it work?

Super Secure File basically uses Google Drive to store your encrypted file and does all the encryption and decryption locally in Javascript. The key never ever leaves your computer. Once you close the tab, boom, the key is gone. Nobody knows the key but you (and probably your spouse if you are married).

Motivation

This is not a big feat or anything, this is merely a mashup of a few libraries to prove a point to myself. The motive behind this app is to build a secure mechanism to store my passwords and credit card numbers on the cloud so that I can access it anywhere. I do not trust anyone who links me up to a server when it comes to information like this. Which is why i wanted a completely static HTML page with no server access whatsoever to provide me with this functionality.

Feel free to examine the source code here (and please let me know if you find anything utterly stupid): https://github.com/vickyg3/super-secure-file

Hosting and Links

Since this is a static page, I have not hosted this on my server. This is merely hosted as a github page (if you look at the repo, you'll see the default branch to be gh-pages and not master). Hosting it as a github page also proves that it has no server interaction whatsoever and cannot steal your key by any mean.

So, all you need is to remember your password and one of these URLs (they all redirect to the same Github Page): http://bit.ly/securefile or http://bit.ly/supersecurefile or http://secure.foamsnet.com or http://vickyg3.github.io/super-secure-file/super_secure_file.html

Get your Super Secure File and make your life a little easier! :-)


-Vignesh

Serverless Downloads - HTML5 Awesomeness!

HTML5 brings an awesome feature known as client downloads. At first glance, the term "serverless downloads" may sound oxymoronic, but it sure makes a lot of sense. In the HTML5 era, we write a lot of thick clients. All the day to day web apps which we use are damn thick (Gmail, Facebook, etc.). So in the context of thick clients, serverless downloads make a lot of sense.

Content-Disposition HTTP Header

On the web 2.0 days, when you wanted to force a browser to download a file to disk rather than displaying it (especially if the file is of some format which the browser is capable of rendering), you used to set the Content-disposition header in the HTTP response.

Content-Disposition: attachment; filename="sample.txt";

There is no way for the browser to let the user download the file without contacting the server, even if the contents of the file were to be generated totally on the client side.


The Problem

This is old school. These days there are lot of web apps where you create the data on the app itself. For example, consider Google Docs, you create all sorts of documents on the web. Every time you click the download button, the data is sent to the server and the same data comes back along with required filetype's additions and you see the download dialog box.

This step seems too radical and unnecessary. Given how thick the clients are these days, it is atrocious to send some data to the server and get back the same data as a file from the server. The same holds for all sorts of web based photo editing apps, etc.


The Solution - "download" attribute

Now with HTML5 it is possible to generate a file on the client side and make the user download it without contacting the server at all. The simple addition of "download" attribute to the anchor(a) tag will tell the browser to download the content present in the "href" attribute rather than navigating it. Now, this can be combined with a "blob:" or a base64 encoded data url to create a file and download it, all on the client side.

I have put up a demo where you can enter text into a field and then download it as a pdf.

Demo: http://garage.foamsnet.com/static/pdf.html
Image Courtesy: http://www.html5rocks.com

-Vignesh

Google DevFest 2011 - Bangalore!

Atlast, I attended a professional developer meet! And it couldn't be better. It was the Google Developer Fest organized by Google at Lalit Ashok Hotels, Bangalore. Most of the speakers were from Google headquarters, working on key Google products like Android, Chrome, Maps and App Engine.

The registration process went on very easily as they had a QR code in the confirmation mail, and used an android phone to take a picture of the same and verify the registration. Also, they had queues based on alphabets range. The T-Z queue was empty while the other queues were having atleast 20 to 30 people in it. For the first time in my life, i experienced an advantage of my name starting with V!

In this post, I am sharing the raw notes which i took during the keynote and various other talks. Hope this will be of some use.

Keynote - Android
* 550k+ activations per day
* Complex Screen density vs Size matrix - unifying everything => android
* Always use DPI (density per square inch) for specifying assets
* Honeycomb features
* Fragments
* Loaders - wrapper on top of asynctask
* Action bar
* Tabs
* Animation framework
* Compatibility library
* Multiple APK Support

Keynote - Bleeding edge HTML5 - http://india-devfest-keynote.appspot.com
* Interestingly, this keynote isn't called as "Chrome", though most of the topics were Chrome specific
* 160M active users
* Page Visibility API
* Prerendering
* Offline capabilities
* Web animation - CSS3, WebGL
* Native Client
* Future
* Web intents
* Fullscreen content - Fullscreen API
* Web Audio API - Real time processing and analysis of audio
* WebRTC - Real time communication
* Be updated
* chromestatus.com
* updates.html5rocks.com

Android Market - Tony Chan
* 250k + apps, 6B+ downloads
* In app billing, carrier billing
* Licensing
* All licenses allowed
* Code Obfuscation
* Don't reuse sample code
* In app billing
* Sell only digital content
* Reports for developers
* Device Filter
* Ability to exclude handpicked devices
* Multiple APK
* Filter by specific segments (platform version, screen size, etc.)
* App available as one product listing
* Apps upto 4GB coming soon (50mb app + 2gb archive)
* Pay attention to graphics - Featuring in android market depends mainly on that

Android Fragments & Open Accessory ADK - Tony Chan
* Fragments
* Fragments need not always have UI
* Fragments are an investment for the future!
* Fragments vs <include/>
* Fragments (optional UI, has lifecycle, not standalone, no direct interaction with intents) vs Activity
* Open accesory & ADK
* USB Device basics (Descriptor, etc)
* Open Accessory -> A USB host device that can communicate with an android device (e.g. Arudino)
* Use Android Accessory Protocol for communication

Building Integrated Apps on Google Cloud Technologies - Alfred Fuller
* App Engine intro
* Google Storage
* Store data in google's cloud
* Access via RESTful API
* Objects of any type (100GB / object)
* OAuth / Web browser
* Prediction API
* Machine learning API
* Upload training data -> Build model -> Predict new data
* Many machine learning techniques, Asynchronous training, Many platforms access
* BigQuery
* Analyze massive amounts of data in seconds
* SQL like query language, REST, RPC, etc.
* Batch jobs (Mapreduce)
* App Engine Identity API (For secure authentication)

Google Apps Marketplace - Claudio Cherubino
* Google Apps as a pure platform
* Google Apps APIs
* Success Stories
* OpenID SSO & OAuth
* Provisioning API
* Gmail contextual gadgets
* Sidebar gadgets in Gmail/Calendar
Twitter: @ryguyrg @scottmcmullan @stevenbazyl

Chrome Developer Tools - Boris Smus
* Basics - Cheat Sheet
* Revision history for all changes made inline in dev tools
* Aim to transform this into an IDE
* Commandline API - console.log takes n parameters, copy(), inspect() and $0
* Javascript debugging - See call stack, Pretty print
* Breakpoints - Line, Conditional, Exception, DOM, Event, XHR
* Future
* Extensions can extend developer tools too!
* Use chromiumer
* Remote debugging (--remote-debugging-port=31337 , Blackberry PlayBook only)

Google Places API - Chris Broadfoot
* What is a Place -> Abstract & Concrete
* Available as a web service and as a part of javascript maps library

Designing UIs for Phones and Tables
* UI Patterns for Honeycomb
* Action Bar
* App Icon - Where am i?
* View details - What can i see?
* Action buttons - What can i do here?
* Multi Pane Layouts
* Take advantage of screen real estate
* Consolidate multiple related screens into a compound view (similar to iframe in web)
* Screen rotation handling - Stretch (Settings), Stack (Calendar), Expand/Collapse (Gtalk), Show/Hide (Gmail)
* App Navigation
* Beyond the List
* Think and use innovative UIs to replace traditional lists (carousel, slidestack, etc.)
* Do's
* Aim for single APK
* Use compatibility library
* Customize visual design completely, if straying from Holo theme
* Support both landscape and portrait
* Extract dimensions for phones and tablets
* Use theme/style/etc. to reduce redundancy
* Marry OS visual style with your brand
* Don'ts
* Assume API level >= 11 (tablet)
* Assume xlarge == tablet (7" inch tablet is large)
* Use small font sizes
* Overuse fill_parent; Avoid excessively long lines of text
* Think tablets are big phones

Btw, the lunch was awesome too! :-P Overall, a great experience! :-)

-Vignesh

Replacing Lightweight Web Services with Twitter Bots!

Foresight
Here is a vague idea. I'm not sure if i can even call it an idea. I always had the tendency to create lightweight information portals as usable web-based services with a very simple user interface. It just used to get things done and nothing more.

Practical Use Case
For example, let us consider a simple problem of "PNR Status Enquiry" in Indian Trains (for the uninitiated: PNR status is nothing but the current status of a waitlisted ticket in Indian Trains). First of all, such a system is really useful because the official Indian Railways websites are a bit clogged and they don't provide any alert services. Also, their websites are not so catchy and mobile compatible, and we definitely don't want to switch on the computer just to check the PNR status.

Before today, if i was asked to build such a system, i would go for an elegant google-like web page, where there is just a text box and a button for the user to enter the PNR number and click go. The resultant page will be again a simple HTML table with the ticket details (of course all of them scrapped from one of the railway websites - pretty sure that this isn't legal, although i am not aware of any laws against site scraping).

Do we really need web for this?
But this thought stuck me today. Web is a wonderful platform, at times too good to host silly and redundant stuff like this. Tomorrow i may have to build a similar system for Buses or Aeroplanes requiring me to add more and more pages with almost same functionality but different information sources. We don't need the web for such silly things. My idea is that, why not use "Twitter" as a platform for such web services. I've heard of facebook as a platform before for many applications and games (Farmville), so why not twitter as a platform too?

Twitter as a Platform
So what exactly do i mean by twitter as a platform? Let us redesign the same PNR status enquiry system using what i mean by twitter as a platform. Instead of having a web page for inputs, lets host a twitter bot, say @pnrbot. Now, whenever you need to enquire the status of a PNR number, all you have to do is post a tweet mentioning that bot, for example "@pnrbot 1234567890" (where 1234567890 is the PNR number you wish to enquire).

Now, as long as your tweets are public, the bot can read your tweet almost instantaneously thanks to the vast amount of real time APIs provided by twitter. Now the bot does the usual site scraping from the railways website for your PNR status and it posts it as a reply to your tweet. Simple isn't it? To take it a level further, the bot can also autotweet your PNR status every day until your journey date, which is not very easy in the case of a web based app. Also, since you will not be following the bot and the bot will not be following you, you will not clutter any of your friends' timeline with this tweet as it won't appear in their timeline.

If any of the input data is sensitive and not to be exposed, then the same design can be adopted by just switching the term "tweet" with "Direct Message" (although in that case, both you and the bot need to be following each other, which can be easily accomplished).

Pros of the Platform
Twitter is a part of our day to day life since the inception of mobile internet. So, you get many useful information from such bots interactively rather than opening your webpage and waiting for the page to load. Moreover, twitter is known for its notifications. If you have activated SMS alerts for @ mentions, then you don't even need to have internet to make use of such a bot. You can just send the tweet through an SMS and read the reply from the bot as SMS. Also, twitter has email notifications which may also be of good use. As mentioned in the example, you can have one input with multiple periodical outputs (like time based notifications, etc.) which is not so easy to implement in web based systems.

Developer Standpoint
So from a developer standpoint, what do we ultimately gain by choosing Twitter as a platform over Web for lightweight services? The answer is quite simple and really advantageous. For one, you need not host a web server for lightweight web services. All you need to do is run a script that will act as the bot. The script will also be really lightweight since twitter APIs does all the pushing for you (no polling). You offload majority of the user interface and load to twitter and do only the actual processing in your server.

From the implementation perspective, it'd be really great to have a good library/framework in a nice scripting language (like php or python) for building such a bot so that the possible features (like twitter API access, etc.) could be abstracted out thereby the developers actually have to write just the logic of their actual bot and nothing extra.

Closing Thoughts
I am pretty sure such bots already exists. By building more and more of interactive bots Twitter can really stand tall as a good platform not just for communication but also for instantaneous information retrieval.

P.S.: I am in the process of developing a simple such bot as a proof-of-concept. I am also highly determined to come up with a generic twitter bot library as i mentioned in the post. So as always, interested developers are welcome to carry on if you like the idea!

-Vignesh

Orkut Deja Vu - The Technical Side!

Its been almost two months now since I first launched "Orkut Deja Vu" - A series of web applications and a chrome extension that helps you move your memories from orkut to facebook. Personally, I think the application was a good hit. In this article i will try and explain the technical side of it and the various hurdles faced on the due course of development.


Note: This article is for the technically inclined and if you don't want the technical details and just want to use the application, visit http://orkutdejavu.foamsnet.com!


Java and My server!


To begin with it, my server is a VPS running linux with a humble 700MB RAM and a shared processor. My search for an Orkut API ended with orkut os client - An official API provided by Google. This is more of a library than an API and only the java implementation of the library was very sophisticated (though there were php implementations, they weren't that good). Ever since i started learning computer science, if there was one thing i hated, it was java (now don't even get me started about perl).


So there lied the first problem in front of my eyes: Running java in my server. I didn't want to take up the pain of setting up JSP for this sake and decided to just call the java program that does the orkut calls from PHP using shell_exec. I know it isn't a safe option, but since there is no user passed data and the calls are hard-coded, i was sure that there was no injection vulnerabilities. Yet this approach isn't very efficient as it spawns a separate java vm for every exec call, but it was a compromise that i had to make for not setting up a java based server.


OAuth - The headache!


The orkut library's OAuth implementation was really messy. Fortunately, it had a method to explicitly set an OAuth access token obtained from elsewhere. That said, I used Zend OAuth library in PHP to perform the 3 legged OAuth and use that access token in the java programs. There were totally four java programs: one each to fetch the user's orkut name, albums list, photos and scraps. The java programs printed the output as JSON if it was successful or nothing if there was an exception. PHP then parses that JSON. I used JSON so that all the escaping will be taken care by the JSON libraries and thereby ensuring safety of data transmission.


Photo Album Migrator


The photo album migration was quite straight forward without much hurdles. All i had to do was integrate the following: Image gallery, Facebook API, Orkut API and write some simple javascript that made AJAX calls to transfer the photos. This was quite a cakewalk as i was already very familiar with the Facebook graph API.

Scrapbook Downloader


Again, the programming side of this was quite straight forward. But there was one major challenge involved in designing this. The java program generates a html file containing the scraps which is then converted to PDF. HTML to PDF conversion is CPU intensive and hence needs to be done with care. I could have used a resource manager like Sun Grid Engine, but i did not want to make things complex for a simple job to be done. Also, the conversion is not done programatically by a library as all the PHP PDF libraries were memory-wise very expensive (a file with ~100 scraps always exceeded PHP's memory limit of 64MB). So, I used a external command line utility (wkhtmltopdf) to accomplish this conversion. Again it was a simple shell_exec with hard-coded arguments.

Testimonials Migrator


This was the really challenging part. There was no Orkut API that gave access to users' testimonials. But i didn't want to give up. I really wanted to pull off a tool that can migrate testimonials to facebook. So I thought of attacking this problem from the heart of orkut - the orkut website. Obviously, the first thing that came to my mind was a Google Chrome extension. The extension will inject a javascript into orkut.com website and add a "Post to Facebook" button beneath eacch of your testimonials.


The first hurdle in accomplishing this was that there were two versions of orkut (old and new) with different page structures. I didn't want to write two different scripts to handle the versions. Instead i wrote another script that detects the version and if it is new, it prompted the user to redirect to the older version in order to use the extension. The next job was to understand orkut's DOM so that I can place the buttons. This ended up being quite an easy job too.


Another glitch in chrome extensions is that we cannot specify images directly in CSS for content scripts(for e.g.: background: url(a.jpg); is not possible), as the CSS will run in the scope of the website. So, it has to be either done programatically using javascript or encode the image in base64 and hard code it in the CSS. I chose the first option. From this point, it was fairly straight forward. When the user clicks on the post button, store the corresponding testimonial using HTML5's local storage and create a new tab where the user can choose the posting options. Again, the facebook authentication and API usage here were simple as i had enough exposure already.


Epilogue


Though it seems like a simple app, huge amount of thoughts are put into every single aspect in design of the application and the entire development process was a fun journey with a great learning curve. This application has made me feel my web presence. Web is really a great platform for amateur developers like me. I have got around 20 new followers in twitter and a person from brazil appreciating me for this application. I have really been motivated a lot by this and hope to continue the same stride in creating usable applications as this one!


This post will not be complete without a heartfelt thanks to all those who supported me on due course of development and all those who used and shared this with your friends!


-Vignesh

InstaShare - Facebook Extension for Google Chrome! :-)

InstaShare - Share everything you find interesting on the web as and when you browse to your Facebook profile! The web you surf daily is now just a single click away from your facebook profile!

This is a mature version of yet another script that i have been using personally so far.

Features include:

• Post status updates through an always-present browser icon


• Share any image in the pages you visit in a single click


• Share any link in the pages you visit in a single click


• Share any piece of text in the pages you visit in a single click

Download You can download the extension from here: http://www.foamsnet.com/instashare

Screenshots
Sharing Images





Sharing Links





Sharing Selected Text





Posting Status Update




You are sure to find a lot of bugs. Feel free to contact me at vig...@gmail.com (Please click to view the full address) for any bug reports or suggestions!

-Vignesh

A flaw/feature in Google Chrome Extension design!

I am recently into customizing my road in the web with simple google chrome extensions to get and share content in the web the way i want. After having spent a good amount of time with the Google Chrome Extension Framework and their javascript API, i feel there is a flaw in the design of it. I still am not sure if this is a flaw or an intended feature.

Before describing the exact flaw, let me give a brief overview of things whose understanding is required to understand the flaw.

Browser Actions
Browser Actions are those using which your extension can do things along with the browser. Every browser can include a pop up page (yes the one which pops up when you click on the extension's icon on the top right corner). As of now, the only way to open up the pop up page is by clicking the icon. This point is important. Let me come back to it in a while.

Context Menus
Extensions can add custom context menus (the one that appears when you right click on something within the browser).

The flaw/feature
Now, as of writing this, there is no programmatic way of invoking the pop up page (other than clicking the icon manually as mentioned earlier). This makes sense because if a non-manual initiation of the pop up was allowed then it would be an irritation for users to see the pop up page annoyingly appear just when they install an extension.

Whereas, the restriction now enforced is a bit too strict. The restriction should look something like this, "The pop up page can be initiated only by a manual action", whereas the restriction that chrome enforces now looks like this, "The pop up page can be initiated only by clicking the pop up icon". This restriction is too much because, there are other manual actions than clicking the pop up icon, that might require an user interactive window for processing. For example, consider the case of a context menu. When a context menu is clicked, it completely makes sense for the extension to initiate the pop up with the context based content and enforce user interaction. This is simply *NOT* possible now.

Other options
Though the clean solution is not possible, there are one or two workarounds that i have found out to overcome this problem.

• If the user-interaction required is really simple, then merely use the javascript prompt('') function from the background page.


• You can always create a tab and make the user interaction


• Use a floating div or a jquery modal window to enforce the user interaction (this can be done using the executeScript API function).

-Vignesh

Playing with Technology (PWT) - Reach Alice's Website !

Hey folks,

This is my first series called "Playing with Technology (PWT)". I will post an interesting real time question related to some technology for you to play around with. It will involve anything and everything i am familiar with. You may have to write scripts, perform hacks, etc. There are no rewards involved. It is just to share the minimal knowledge i possess in the form of an interactive series.

PWT will kick off today with a very simple real time hack that i call as "Reach Alice's Website". Go ahead and enjoy the hack.

Scenario

Alice (yes, our same good old friend Alice) owns a website. Recently, her website was acquired by Hackers World Inc. (http://www.fakehackersworld.com). The CEO of Hackers World considers this acquisition as a very confidential one and decides that all traffic to Alice website should only come through Hackers World website. But the sad part is that, Hackers World website is accessible only to the employees of Hackers World. Your goal, being a hacker, is to somehow reach Alice's website through Hackers World website and acquire the secret key from there!

Alice's Website: http://foamsnet.com/pwt/visit_alice_site
Hackers World Website: http://www.fakehackersworld.com

Remember clues can be lying/hiding in anywhere!

Once you find the key, Post it as a comment in this post. I will approve all the comments once i reveal the answer and will aggregate all the correct responses along with the answer.

Update: The solution has been posted here

-Vignesh

Doodle it up - My first google chrome extension !! :-)

Google is really a part and parcel of our daily life. It has become so inevitable to our daily lives. I have taken a bit of my time to quickly develop a simple Google Chrome Extension to customize your daily visits to google. Wouldn’t you love to see your own picture instead of the boring google logo everytime you open google’s homepage? Yes. This is exactly what this extension does.

Here are step-by-step instructions for installing the extension:

• Download the extension from here


• Install the extension by clicking "Continue"


• Confirm the installation by clicking "Install"


• You will see the confirmation message


• Goto chrome://extensions and click on options under "Doodle it up" extension


• In the page that opens up, enter the URL of the image and then click on Save Settings


• Now, goto Google and Voila! you have your own image instead of the Google logo!

Please post your comments/suggestions as comments.

-Vignesh

Happy Birthday Google (12 years old) !! :-)

Today google celebrates its 12th birthday! Started in the year 1998 by two stanford guys in a garage, google has been the one thing that has transformed the whole world. Began as a search engine, today they work on almost all the cutting edge technology in the world (rather they invent most of the cutting edge technology).



Google has really become an integral part of almost all of our lives. Google co-founder Sergey Brin's dream when he initially thought about google was to "download the entire internet in his computer" and thats what they have achieved today!

As some rightly say, "when a noun becomes a verb, technology has really matured" and thats exactly what google is.

Happy Birthday Google !! May you make the life of more and more software engineers easy day by day !! :-)

Do you have a google moment to share ? Do it in the comments :-)

-Vignesh

Swype - The next generation of text inputs !! :-)

People following me in twitter, facebook or my blog would have known that i recently joined the android community with the purchase of my htc wildfire. Ever since I bought my phone, there is this one thing that I'm madly in love with. It is supposed to be the next generation of input system for touch screens. With a simple tagline "Why type when you can swype", it is really one of the most amazing technologies I have ever seen in my life.

When I first got my phone, being the first time of using a touch phone, I was really happy about the basic touch input that is built in with android as it had a really nice auto corrector and I just needed to key in the text very very approximately. But the moment I installed and started using swype I never turned back. In this article I will be sharing an overview about what swype is, how to get swype for various smartphones and a few alternatives to swype.

What is swype ?

Swype is the most recent innovative technology for text inputs in touch screen based systems. Just like we have the T9 input system for traditional mobile phones, swype is for mobile phones with touch screens. In T9, there are either 3 or 4 characters assigned to a single numeric key and to key in text, you just have to press the correct combination of numbers associated with those particular characters and voila you have the text you wanted keyed in very easily. If there are multiple words possible for a particular combination then you choose the right word you wanted by either pressing the "*" key (in traditional nokia phones), or by pressing the down arrow (in sony ericsson and irritating motorola) or by pressing the "0" key (in the unconventional samsung phones). Once gotten used to, you will feel that T9 is the best way to key in texts in a mobile phone. It is indeed true and people even type without seeing the screen like we do in computers.



Swype is a similar technology for touch screen mobiles. In swype, instead of tapping each and every character in the qwerty virtual keypad shown in the mobile, you just have to elegantly trace through the path of the word you intend to type. Similar to T9, if there are multiple words in the path you traced, swype offers you the list of possible words for you to drag it into the input. You can be very approximate in your path trace yet swype recognizes your word almost perfectly all the times. Though it might sound simple and not very appealing at first, on using it you will realize that almost 90% of the words you type in have unique paths and hence, unlike T9, you very rarely will come across the ambiguous word choosing pop up.



Who can enjoy swyping ?

• Android
  
  • Swype comes as a default input method along side touch input in many android devices.
  
  
  • If it doesn't come preloaded in your android device, then you can get the official swype beta from http://beta.swype.com. The official beta is closed now but you can download it through someone who has already registered with the beta was open (yes, it is legal to get any number of copies with a single registration! - and don't contact me as I didn't register when the beta was open).
  
  
  • If you didn't register for the official beta when it was open and don't know anyone who did, then this is the option for you. Download the apk file from some other source and install it in your device (which is what I've done in my phone). I am not posting any direct links here owing to legal reasons. It is available easily in 4shared.com. Find out the type of display your device has (VGA, QVGA, HVGA, etc.), download the appropriate version and install it.
  
  


• Windows mobile
  
  • Few devices with windows mobile comes with swype preinstalled. I am not familiar about other options of getting swype for windows mobile.
  
  



• Symbian
  
  • Swype has partnered with symbian and recently released a public beta for S60 5th edition based symbian phones (nokia 5800, 5230, 5233, N97, N97 Mini to name a few). You can download and install swype for symbian from Nokia Beta Labs. All the installation instructions are also briefed in that page.
  
  
  • One drawback i faced with swyping in symbian is that, most of the symbian devices have less sensitive resistive touch screens when compared to their capacitive counterparts in android devices. Although, it is convenient to swype with nails in the symbian devices which is not possible with the capacitive based android devices.
  
  

When is swyping easy ?

• When you are just done with a call and your phone is slightly wet out of your cheeks


• When have completely dry hands after a long rest of hands (e.g. after a sleep)

When is swyping not so easy ?

• When you have just washed your hands and wiped it dry using a kerchief/tissue


• When your hands are wet of course


• If your fingers are a little bit bigger, then you may have to switch orientation every time you key in text

Alternatives to swype

Every emerging technology is never unique these days. There are always multiple implementations of a single technology and swype is no exception. Here are a few alternatives if swype that I have tried out.

• Dasur SlideIT Keyboard
  
  
  • This is the best alternative for swype I have seen so far. I was using this before I could figure out swype installation in my device.
  
  
  • One feature that it had and swype lacks is shortcuts. You can store tiny shortcuts for frequently used lengthy words.
  
  
  • The dictionary is no where close to swype's preloaded dictionary of 60000 words. Although, many language packs are additionally available for free.
  
  
  • It is available for all major smartphone platforms (android, symbian, windows mobile and even for windows ce). It can be downloaded from the official market of your smartphone ( android market, ovi store, etc. ).
  
  
  • For more details visit here.



• T-Swipe Pro
  
  
  • This is a swype alternative that is available only for android devices as far as i have explored.
  
  
  • It is a stable piece of software that is terribly slow in recognizing what you swipe.
  
  
  • The trial version shows you irritating pop ups asking you to register for full version which is definitely not worth upgrading to.
  
  
  • If you badly want to swipe and both swype and slide it key board doesn't work in your device ( which is a very rare case ), then this may be your final destination. I am damn sure you need the patience of handling a tortoise to use this.

To conclude, Swype is definitely one of the major breakthroughs in mobile technology and I am pretty sure that soon enough, swype will be the de facto standard for text input in touch screens. Hoping to see more and more OEM installed swype devices soon.

Note:
This article has been composed entirely in a mobile device using the following input systems:

• This note is keyed in using the traditional touch input system for android


• The section about SlideIT Keyboard was keyed in using SlideIT Keyboard itself


• The section about T-Swipe Pro was keyed in using T-Swipe Pro itself


• The rest of this article was keyed in using none other than swype itself :-)

P.S.: Thanks to AK Notepad.

-Vignesh

SMS - Talk through mobile from facebook !! :-)

Various application layer protocols always impressed me. Especially HTTP, as my understanding about it has been pretty good. There has been one thing that i have been doing pretty good, site scrapping.

What is site scrapping? Defining it is very simple. It is the process of extracting some information from a website that has complex interface (with flash, ads, etc.) and using that information for our own purposes. Though scrapping many websites (especially government ones) may be illegal.

Here is one such application that i have developed. This application is purely a brain child of the HTTP i have understood. It provides an interface to the famous Free SMS Gateways in India 160by2 and way2sms through a facebook application. It has been one of the dead works that remained uncared for a long time. Now that i have a decent amount of online presence, i have refined it and trying to popularize it through my blog (should i call it blogvertise ?).

Using this application is simple. Register yourself at Way2SMS or at 160by2. Goto http://apps.facebook.com/smstalk. Add your account here and you are good to go. Though this application is not a big feat, it just provides a simple and clean interface (exposed as a facebook app) for the ad clustered interfaces of the famous free sms gateways in India.

Here are a few screenshots:






Share your opinions/suggestions in the application's wall here or in comments.

-Vignesh

Infamous common problem !! :-)

I have had a productive day so far today. Its 7'o clock in the evening when i am writing this and i really feel like i have learnt a lot of new things today. I had a really long discussion regarding work with my team lead and he bought up an interesting point that i thought is worth sharing. I was wondering why noone had not put this formally yet (may be somewhere in the DBMS course).

It is one of the most recurring scenario in a software engineer's life that he may have to perform an action and print a log message corresponding to that action. The question is, how to keep these two operations atomic. Ok, now for those who are not familiar with the term, atomic operation is one that follows the "all" or "nothing" principle. A good example would be, if i transfer 1 crore to you, it involves two separate processes: 1) Debit 1 crore from my account and 2) Credit 1 crore to your account. This operation definitely have to be atomic as doing just one of them and not the other will make the bank go bankrupt. So thats an atomic operation (I wonder why it is called so as atoms themselves are divisible into 3 sub particles!!).

So whats with this action and log message you were talking about? Yes, coming to that, usually we perform an action (say increasing 1000 rupees in one's account balance) and to indicate that we have done this operation we write a message (probably somewhere to a log file) saying that we have increased the account balance 1000 rupees (and possibly include the timestamp). In this case, the presence of the message in the log file indicates that one's account balance has been increased by 1000 rupees. If that message isn't there in the file, then it means that we haven't increased 1000 rupees in the account.

Lets say that this operation (of increasing 1000 rupees in one's account) is to be done strictly once. No matter how many requests come, the operation has to be performed only once. A layman or a newbie programmer might think, ah, this is simple, i just have to look at the log file and if the message does not exist then i have to increase the balance by 1000 or else i should print an error message. This is what most of the people do without analyzing the possibily that the first part of the operation (increasing the balance) might have completed successfully and the second part of the operation (writing that in the log file) might have failed (due to some reason) in which case the operation might be performed more than once on receiving successive requests which can really result in a nightmare if the singleton-ness of the execution is really critical.

Since nothing is ideally (100%) atomic, it is impossible to achieve the exact singleton-ness of execution. It can be ensured that an operation is carried out no more than one time and it can also be ensured that an operation is carried out atleast once but not both (It is sort of equivalent to saying, <=1 and >=1 is possible but =1 is never possible).

Here I make a comparison of two different approaches to tackle this problem.

Approach 1 (Action First - Log Next):
Action(Completely) -> Log
This is the default approach that many people take without realising the seriousness of this problem. As i mentioned earlier, here there is a possibility that the action might be complete but the logging might fail, resulting in possibly multiple executions of the operation. Since this approach performs the action first, no matter how heavy the action is, the logging is done only after the entire action is complete. So in case of a heavy action (where only certain part of the code is singleton critical whereas the rest of the code can be run multiple times without any problem. Whereas it is not possible to separate the action into singleton critical and non-singleton critical parts in this approach, as the entire action is carried out first followed by the logging. According to me, this is one major drawback of this approach. This method gives the guarantee that the action is executed atleast once.

Approach 2 (Log First - Action Next):
Preparatory Action(Need not be singleton) -> Log -> Action (Singleton)
This is the approach that many people don't even know the existence of. Why not log first and action next? As i already said, now we can divide the action into two parts - singleton critical part and the non-singleton critical part. Now the execution goes like this, 1) non-singleton critical part of the action (lets call it "preparatory action" as it can afford to run multiple times). 2) Log the message and 3) singleton critical part of the action. So what is better about this approach? First of all, as i said earlier, this does not solve the problem completely. This approach gives the guarentee that the action is executed atmost once, which is slightly an expected behavior than the previous approach. Moreover, we are making the action lighter by splitting it into two parts. Even if the split isn't possible (there is no non-singleton critical section in the action), this approach still guarantees that the action is executed no more than once.

This is a simple yet powerful fact. It is never possible to make (as of now and as far as i understand) an action ideally atomic (meaning, executed exactly once). Post your thoughts on comments :-)

P.S.: The basic idea of these two approaches is my colleague's and i have portrayed that in my own words with my own examples.

-Vignesh

Google Chrome's Second Birthday !! :-)

Google chrome celebrated its second birthday two days back. Its been exactly two years before I read the official post in google blog stating that it is going to launch an open source web browser. I was surprised and was very excited to try it out at that time. Ever since then, i have been using only google chrome for all my primary internet work. And as a trademark google product, it has shown very rapid scale of development and it has matured a lot since the first release.

Along with its second birthday, google chrome version 6 has been released on the same day. The first thing i liked about google chrome release is the comic strip that they released along with it which beautifully explains the internals of google chrome in such a way that from tom, dick and harry to gnu, kde and dll guys can understand. If you have missed the comic strip, you can have a look at it here.

Google chrome currently holds 7.52% of the total market share of web browsers (Source). Surely it is very less when compared to Internet Explorer (60.4%) and Mozilla Firefox(22.93%) whereas chrome is relatively new and has had just two years when compared to Opera which, after 10+ years in the market, holds just 2.37% of the share. The key to chrome's success was that it always focussed on speed and simplicity. Apart from that the HTML5 support is being top notch. Since the project was started almost when HTML5 started to evolve, it is easy for chrome to easily adapt to HTML5 rather than other browsers whose code roots dates back years.



Here is an excerpt from chrome's official blog on its second birthday:
"Since Chrome’s first beta launch for Windows, we’ve brought our Mac and Linux versions up to speed, and continued to make the browser faster, simpler, and safer across all three platforms. We’ve also introduced a boatload of features, including a more customizable New Tab page, browser themes, side-by-side view, password manager, better privacy controls, built-in Adobe Flash Player, Autofill, automatic translation, HTML5 capabilities and synchronization of various settings such as bookmarks, themes, extensions and browser preferences—just to name a few. Finally, there are now more than 6,000 extensions in our gallery to enhance your browsing experience."

It is really amazing how big a product can get in a matter of two years. To me, when you are a company as big as google, i don't think there is any such thing as a small scale product release. Everything you do is in the scale of billions and trillions. Google Chrome has made my road in the web much better.

Here are some useful extensions that i have been using:

• Google Mail Checker - Google Chrome version of Gmail Notifier


• Google Similar Pages - A beta tool that suggests you pages similar to the current page


• Google Translate - A really awesome plugin that suggests you and translates if a page is in a non-english language


• TabJump - An intelligent way to group and navigate between your tabs


• TinEye Reverse Image Search - More about this here


• Chrome Bird - A twitter extension

Share your two year memories with Chrome on comments :-)

-Vignesh