Developing WordPress Plugins in Github

As most of you know, I actively develop and main more than 20 WordPress Plugins. All of them are hosted in the official WordPress Plugin repository.

Until recently I was using the official WordPress repository’s svn to do the development. All my non-WordPress pet projects are hosted in my github account.

I was thinking of moving all my WordPress Plugins to github for quite sometime now, but didn’t do it thinking that it might involve huge effort to keep both the repos in sync.

After reading about how a couple of people moved (and didn’t face much problem afterwards), I decided to try it out for a couple of my Plugins. The following is what I did and in the end, figured out that it was worth the effort.

Github – more than just code

To be fair to official WordPress Plugin repository, there is nothing anything wrong with the way it works or with svn.

It is just that github as a platform is more suited if you are interested in collaboration and receiving contributions from users. It reduces the barrier to entry for people to contribute to projects and also it provides enough tools for project owners to easily merge (or reject) contributions from users.

I was more interested in this aspect of github rather than the ‘git’ part.

Using github for hosting WordPress Plugins

When I decided to move from svn to github, I wanted to do the following things.

  • I wanted to maintain the entire check-in history from svn. Some of my Plugins are more than 4 years old and have gone through lot of revisions and I didn’t wanted to loose them.
  • I also wanted to do the development in github (read doing atomic check-ins) and then upload to WordPress repository only when I tag the changes for release after testing them.
  • I also didn’t wanted to manually check-in in both the places to do the sync.

For the first I found that git-svn can fetch all the check-in history.

For the second and third point, keeping pragmatism in mind, I ended up creating shell scripts so that I don’t have to do anything manual.

Automation scripts

You can get the scripts that I wrote from my github account. The project consists of two scripts.

  • clone-from-svn-to-git.sh – Use this script to clone your WordPress Plugins from SVN into git/github
  • deploy-plugin.sh – Use this script to push your WordPress Plugin updates to SVN from gi/github

Right now the documentation is pretty sparse. I will fix it when I get some free time 🙂

Things to improve

The scripts pretty much do the job. Going forward, I want to improve the following to make it more useful.

  • The readme.txt file used by WordPress repository is not fully in markdown. Because of which the readme file displayed in github is slightly broken.
  • Ability to auto update the .pot file before releasing the new version
  • WordPress repository allows you to add a banner using the assests/ directory. Need to add support for that.

Update: All the above features are added. Check out the latest code in github.

Is it worth the effort?

So far I have moved the following Plugins to github. Will be moving the other ones as well when even I get some free time.

So now the real question. Is it worth the effort? It is worth every second that I spent on it.

You should check the pull requests I have received so far. I have received more contributions to my Plugins in the last couple of weeks than the last 5 years 🙂

If you are a WordPress Plugin developer, then try out my script and let me know if you have any feedback.

Posted in WordPress | Tagged , , , , | 8 Comments

Parse Apache error log and list down all missing images

Recently I had to parse Apache error log and find out all images that are missing.

After referring to a couple of man pages, I came up with this one liner. I am sure I will need it again, so thought of noting it here so that I know where to look when I need it again 😉

Feel free to use it in whatever way you want, if it solves your problem as well.

Assumption

This assumes that each line in your apache error log looks like this.

Explanation

Filter all 404 lines from the log file

The first step is to filter all lines that that contain the “File does not exist” text. This is done by using sed.

  • By default, sed prints out all lines. This is prevented by the -n.
  • The second option is the regular expression followed by the p flag. This option prints out all lines which match the text.
  • The third option is the name of the error log file.

Extract the last column of matching lines

The next step is to retrieve the file name from the matching lines. This is done by using awk.

  • By default awk uses space as the delimiter and splits the lines into different columns. If you look at each line, we want the last column.
  • NF is a special variable which points to the last column.
  • print $NF prints the last column

Filter only images

The next step is to filter out only the images. This is done again by using sed.

  • I use -n again to prevent sed from printing all lines.
  • The -r is added, so that we can use extended regular expression
  • The regular expression (jpg|jpeg|png|gif)$ filters out all images and p at the end prints out only lines that match

Sort and find uniques

The sort and uniq commands sort the list and find the unique lines.

Write to a file

The final output is written to a file by using the redirection > operator. If you want to append to a file then we may have to use >> operator.

More to come

It is really amazing like how you can combine these tools to do amazing things. I am planning to document other one liners which I end up creating to solve my problems. So stay tuned 🙂

Also if you think this can be improved, then do let me know as well.

Posted in Unix/Server Stuff | Tagged , , , | 5 Comments

Make sure you enable binary logs in MySQL

Today, I learned a very valuable lesson. Make sure you always enable binary logs in MySQL. I would have lost atleast 3 months of effort, if it was not enabled. Luckily it was enabled and I was able to do a point in time restore after someone did a wrong click.

If you maintain any kind of MySQL server (even if it is just a stage machine) make sure you enable binary logs in it. It is very easy to do and when something bad happens, you will be happy that you enabled it.

How to enable it

To enable binary logs all you have to do is to enable the following two settings in your my.cnf config file.

log_bin = /var/log/mysql/mysql-bin.log
expire_logs_days = 100

The first setting specifies the location where the logs should be kept and the second setting specifies the number of days to keep the logs. The default value is 10, but I changed it to 100.

How to restore from binary logs

You can do a point in time restore by using the mysqlbinlog tool, which comes with MySQL. Refer to this excellent tutorial to find out how to use mysqlbinlog tool.

Posted in Database Programming | Tagged , , | 1 Comment

Renewing Indian passport online

My passport was due for renewal and I just renewed it. Even though the process seems relatively simple on paper, there were quite a few things I learned on the way which would definitely help someone who has to go through the process again. I thought of documenting and sharing them so that it is useful for people who are in the similar situation.

Disclaimers

Couple of disclaimers before we start.

  • As with government offices in India and state of an atom, your experiences may vary 😉
  • I am documenting the steps that worked for me as of Jan 2013. It is possible that rules might have changed.
  • I renewed it in Bangalore. It might be different if you are doing it in another city/state.
  • Most of it apply for new passport also, but there might be some changes.

Additional changes needed in my passport

In addition to renewing the passport, I wanted to do these additional changes as well.

  • Change my address from Chennai to Bangalore
  • Change my marital status from single to married. Yes I got married about a year back 😉
  • Include my wife’s name in the spouse column
  • Remove ECR check. I got my old password while I was in college and it was enabled.

One thing I came to know is that, if you are married then it is mandatory to include your spouse name in the passport by providing the necessary documents. Sorry guys, if you are married you can’t skip it 😉

Tatkal and Ordinary

There are two methods to renew passport. Tatkal and ordinary. According to the official statement, if you choose tatkal, then you can expect to get your passport within 10 days and for ordinary it will take about 3-4 weeks.

My passport was due for renewal in Jan 2013. I submitted the online application on Dec 3, 2012 through ordinary method and received my passport on Feb 7, 2013. There were two major reasons for the delay.

  • Getting the appointment
  • Delay because of police verification

If you take both into consideration, it is fair to say that you will get your passport in 3-4 weeks even if you choose the ordinary method.

Documents needed

For passport renewal, you need the following documents.

  • Old passport
  • Address proof (need two separate proofs)
  • Marriage certificate or affidavit(to add your spouse name)
  • Degree certificate (for removing ECR check)

For address proof I took the tax document (katha extract) of my house, EB bills, Gas bills, letter from my HR dept and my company id card. Even though the website says only two is needed, they asked for all the above from me.

For adding spouse name, you need the marriage certificate or an affidavit. If you have a marriage certificate it is better to carry that instead of an affidavit. For me they objected since my affidavit was issued in Chennai and was registered using Tamil Nadu stamp paper. I told them that I got married in Chennai and they accepted it on the condition that they will keep the original.

To remove ECR check, I had both my BE and MS certificates and they were fine with that.

It is also safe to carry your 10th and 12th mark sheet and also some additional certificates with have your date of birth. This is not specified anywhere in the website, but they asked me for this. The reason they gave me for this was that, if you are renewing your passport in a new city, then they can’t retrieve your old documents and you have to submit them again. I was not able to argue beyond that but luckily I was carrying them that day.

Process

Okay so let me explain the process now. This is more or less the same for new passport also.

  • Submit your application online
  • Get an appointment for the visit to passport office
  • Visit the office and submit the documents
  • Complete police verification
  • Receive your passport

Submit your application online

You have to create a new account at http://www.passportindia.gov.in, download the application form, fill it and submit it back. The site is maintained by TCS, but don’t have any expectations. It is more or less at the same league of Irctc. Links open randomly in tabs, captchas every where, random session lock-in – you get the idea right?

One very important thing to do, if you don’t want any random calls from tele-marketers is to make sure you un-check the “Share my details with tele-marketers” checkbox, while uploading your filled in form. Yes, you read it correctly! Honestly I was shocked when I first saw it. I don’t know why they want to do this in a site where you upload sensitive information about you. I don’t know how much MEA (Minister of External Affairs) or TCS (who is maintaining this site) is going to earn by doing this. Anyways that is the state of things and make sure you uncheck the checkbox and hope they respect it.

Getting an appointment

Update: From May 28, 2013, Bangalore passport office has started to open the slots for 21 days.

If you thing uploading the document was tough, then you have not seen the real thing 🙂

Personally for me this was the most irritating, difficult and time consuming part of the entire process.

They give appointment only for the next two days. The booking starts at 6PM for Bangalore (varies for different cities) and gets over in less than 100 seconds! (not minutes!) You need to fill in 3-4 captchas during that time and most of the times the site just hangs or times out.

It took me one full month to get the appointment after trying it to get one exactly at 6 PM every day.

The following are some of the things I learned the hard way. Hopefully this should help you to get appointment easily.

  • Don’t try from your office or places where a single public IP is shared. They do filtering based on IP (I don’t know why)
  • Be careful while filling out the captchas.
  • Try exactly at 6:00 PM. Not at 5:59 or 6:01.
  • Close everything in your machine and have only this tab open.
  • Try to do it from a place where you have decent broadband speed. Trying from a slow connection usually fails.
  • Try to choose a time-slot between 12-2PM. Most people don’t choose this time period and the probability of getting this time slot is higher.

Visit the passport office

When you visit the passport office, make sure you are carrying all your documents and be on time. I guess you don’t want to loose the appointment that you got the hard way 😉

At the passport office, they will first check your documents and then will give you a file and token number. When the number is displayed on the display screens, you have to go to counter A, where they will check and scan your documents, collect the fees and take your photo. I saw a couple of people wearing blazers with tie and others using a hand mirror and comb. It’s up to you, if you want to do it 😉

After that you have to go to counter B when your token number is called. They also will check your documents and will ask you to go to counter C. In counter C, they will check your documents again, put a cancel seal in your old passport and will ask you to go to the final guy, who will give you an acknowledgment.

If you are lucky the entire process should be done in around 2 hours. I was very un-lucky the day I went and ended by waiting there for 6 hours 🙁 Hopefully you will be lucky enough 😉

Complete police verification

Let me state the theoretical process first of how things should happen and then the practical process of how things actually happen.

The theoretical process is that your application is received by the police station, then come to your home to verify you stay there and then forward the application.

But in practice, this never happens. You should consider yourself to be lucky if they atleast have a courtesy to call you. For me they didn’t even give me a call. I waited for more than 3 weeks and went to the police station after that. That’s one of the reason why my passport got delayed.

The police station receive the application in a day or two after you visit the passport office. Wait for a week and if you don’t get a call, it is better to visit them. Make sure you carry all your documents and two passport size photos. Be prepared to be treated badly and also expect to wait there if needed. It is a police station and I guess you get the idea.

You receive your passport

Once the police forward your application you can expect to receive your passport by post with-in a week. Make sure you are either at home or have someone at home with your photo id.

Congratulations you have renewed your passport 😉

Posted in Random/Personal | Tagged , , | 91 Comments

Changing the playback speed of Youtube videos

I generally like to watch lot of conference videos and lectures in youtube. Most of the time these videos are long (> 1 hour) and sometimes the speakers speak slowly. I wanted to see if there is a way by which I can increase the playback speed of the videos.

Luckily I stumbled upon the HTML5 experiment page of Youtube, which allow you to control the playback speed.

So, if you are interested in controlling the playback speed of youtube videos, then just go to youtube html5 experiment page and enable it. You should also be using one of the latest browsers for this feature to work. I think youtube is saving this setting per browser and not per user account.

Once you enable the experiment, there will be a small settings icon in your video player which will allow you to vary the playback speed.
Varying youtube playback speed
Hope this small tip is useful for you 🙂

Posted in Google/Yahoo | Tagged , , | 5 Comments

Sherlock Holmes and my childhood

Sherlock Holmes was one of my childhood hero (together with He Man 😉 ) and was the main reason why I started reading books, which I am still continuing.

At school, we had a chapter about Sir Arthur Conan Doyle in English Prose at 6th Std(grade) and a couple of short stories of Sherlock Holmes were include in non-detail. This influenced me a lot and I got interested in mystery fiction.

There were instance where I started to think how Sherlock Holmes would have solved the mystery, but I still couldn’t find out who stole my rubber or pencil sharpener 😉

The more I read about Sir Arthur Conan Doyle and Sherlock Holmes the more interested I got. But the library at school didn’t had many books and I was not able to read the full collection.

Years passed and one day I was browsing books at Crossword and came across the entire collection of Sherlock Holmes (4 novels and 46 short stories) and I immediately bought it. I think I finished the entire volume (more than 2000 pages) in less than a week 🙂

It was one of the best books I have read in my entire life. If you are interested in mystery and haven’t read Sherlock Holmes before, I highly recommend you to read it and I am sure you will like it.

Now, I am back to my bookshelf to find the book. Planning to read the entire collection one more time 🙂

Posted in Books | Tagged , , | 2 Comments

Completely rewrote WP-IRC Plugin

It’s been quite sometime since I released my WP-IRC WordPress Plugin. A couple of users complained about the increase in page load time and so I decided to fix it and ultimately ended up rewriting it.

By the way if you have not used it before, WP-IRC is a WordPress Plugin, that retrieves the number of users who are currently online in an irc channels and allows you to display this information in the sidebar using widgets.

Dynamic update using AJAX to decrease page load time

Since the Plugin has to connect to the irc server through sockets and do some protocol exchange to retrieve the user count in a channel, it was increasing the page load time drastically and was the major complaint that I was receiving about the Plugin.

I tired to decrease the page load time by lazily connecting to the irc server after the page is loaded using AJAX to retrieve the user count. This has drastically decreased the page load time and was one of the highlight of this rewrite.

You can check out this excellent article to find out how to use AJAX in WordPress.

Added caching to decrease page load time

In addition to using AJAX, I have also implemented caching so that the Plugin doesn’t have to connect to the irc server for every page load to retrieve the user count.

I implemented caching using transients in WordPress.

Support for multiple widgets

One of the other highlights of this release is that, I have enabled support for multiple instance of the widgets. This will allow users to use multiple widgets, each with its own configuration.

I implemented it using the Widgets API of WordPress.

Translation support

Finally I have also added support for internationalization in my Plugin. The .pot file is under the /languages folder and if you are willing to do translation for the Plugin, use the pot file to create the .po files for your language and let me know.

Download and source code

You can download the Plugin from its homepage and the source code is available at github.

Try out the Plugin and do let me know if you have any feedback or comments.

Posted in Plugin Releases | Tagged , , , | 1 Comment

Contributing to project hosted in Github

As most of you know, I host most of my projects which I have released as open source in Github. I am open for collaboration and generally accept most of the pull requests that people send me.

Recently I noted that not all people who want to contribute are proficient with git or Github. So here is a small guide to help people who are interested in contributing to projects hosted at Github.

Before we proceed just keep in mind, that this is not the only way to do it. But if you are just starting out using git or Github, then this is a good starting point.

Fork the project in Github

The first step is to fork the project at Github. Go to the project that you want to contribute to and then click the fork button near the top right corner.
github-fork

Clone the forked project to your machine

When you fork the project, Github creates the forked project in your account. Once you forked the project, you need to clone it to your machine.

You can use the following command to do it.

git clone git@github.com:sudar/wp-irc.git

Replace it with your actual username and project name

Create a new branch

The next step is to create a new branch. You can use the following command to do that.

git checkout -b branch_name

Commit your changes

After you have made the changes, you have to commit them to the new branch

git add file_name

or

git add -p file_name

Then do

git commit -m "Your commit message"

Remember to have a meaning full commit message. If possible follow this guide by Tim Pope on how to construct a good commit message.

Push the change to Github

Next you have to push the changes to Github. You can do it by using the following command.

git push -u origin branch_name

Send a pull request

Now go to Github and send a pull request, by clicking the pull request button in Github. The owner of the repo will be notified and he may choose to accept or reject the request.

Keeping your repo upto date

Once the pull request is accepted, you can merge the changes back to your repo by using the following commands.

git remote add upstream git://github.com/sudar/wp-irc.git

git checkout master

git pull --rebase upstream master

git push origin master

Posted in Unix/Server Stuff | Tagged , , | 5 Comments

Looking back at 2012

2012 had been a great year and it just ended. I thought of publishing my year end review for 2012 as I have done for the previous years.

Personal Highlights

If you have read my year end review of 2011, then you would have know that I got engaged in 2011 and I got married in 2012. Before marriage, pretty much everyone were saying that my life is going to change completely, but fortunately for me things haven’t changed much except that I have another person in my life who cares for me more than I care for myself 😉

Talking of marriage, I am very happy to know lot of people really liked my wedding invitation and recently a couple contacted me from Indonesia asking if they can use my wedding invitation design for their marriage.

Travel Highlights

Another thing which most people predicted for me is that after marriage I will not get much time to travel, which again proved to be wrong 😉 I continued my tradition of at least one international travel and a couple of domestic travel.

HardwareFun

2012 had been a great year for me from the hardware and robotics perspective. I continued working on hardware and Arduino related stuff which I picked up in 2010.

One of the important things regarding hardware that I did in 2012 is that I launched hardwarefun.com one of my pet projects that is close to me. I created hardwarefun.com to document my different adventures with hardware and also help other people who also want to learn hardware programming.

Talks and Workshops

Another thing which I improved in 2012 is my public speaking skills. I gave around 15 talks and also conducted my robotics workshop. The following are some of my favorite talks.

You can see the complete list of talks that I have given so far from my talks page.

Blog Stats

The following are stats generated for this blog by my Year End Stats WordPress Plugin.

Number of posts by year Number of comments by year Average length of posts by year Length of all posts by year

The number of posts have increased a bit and I am happy about that, since I am also writing posts in at hardwarefun as well.

I am planning to write more posts in 2013 and hope that with talks, workshop and personal life I get some free time to keep up this word 😉

What’s in for 2013?

So what’s in for 2013?

I am planning to concentrate more on my WordPress Plugins a bit and I already have some pretty good ideas. So stay tuned 😉

Regarding hardware stuff, I am planning expand HardwareFun.com and write more posts there and also continue working on Arduino and start playing around with Raspberry Pi as well. So it is going to be pretty exciting.

Regarding talks and workshop, I am planning to continue my robotics workshop (the next batch is going to be at the end of this month) and also start giving talks and workshops on other areas as well. Planning for a Hadoop and Pig workshop/talk soon.

From the looks of it, 2013 seems to be an exciting as well as a busy year and I am really looking forward to it.

So how was your 2012?

Posted in Random/Personal | Tagged , , , | 5 Comments

Getting started with Hadoop and Pig

I was supposed give a tutorial about Hadoop and Pig today at SRII-India ICSEM 2012 conference today which is happening at Infosys campus in Mysore.

But unfortunately I was not able to attend it due a logistic nightmare created by the travel company used by the conference organizers.

I spent more than 10 hours to prepare the presentation with the demo code. I thought of sharing them here so that it will be useful for someone who wants to get started with Hadoop and Pig.

Continue reading »

Posted in Events/Conferences, Hadoop/Pig | Tagged , , , | 4 Comments