Goodbye Yahoo

Today was my last day at Yahoo and for the first time in my decade long career I am “officially unemployed” ;)

When you work at a place for more than half a decade then lot of things in your life start to revolve around it. The countless amount of Yahoo swags at my home is a testimonial to it. I just did a look around and found the following Yahoo branded items around me.

  • Yahoo T-shirts (which I have enough for this lifetime ;) )
  • iPad
  • Couple of power banks
  • Couple of pen drives
  • Couple of bean bags
  • All kind of bags starting from backpacks to laptop bags, travel bags, carry bags etc
  • Couple of toys for my son
  • Pens and lot of other small things

I came out of the office today with lot of good memories (that I can cherish for my entire life) and I am pretty sure Yahoo would continue to be part of my life even though I don’t work there any more.

What’s next?

I have not made up mind yet and at this point I am planning to take up a break for a couple of weeks so that I can reboot myself before deciding anything. Meanwhile I might concentrate more on my pet projects and might spend a little extra time with Bulk WP.

Anyways I will keep you guys updated of what I am up to and meanwhile I am going to enjoy my little break ;)

Posted in Random/Personal | Tagged | 14 Comments

Automatically send unique errors (with count) from Apache error log as email

Sometime back I wrote about a simple awk script that I wrote which allowed me to find unique errors from Apache error log files.

After I wrote that script I found myself executing that script every day in the morning to figure out if there were any errors in my sites. After a couple of days I wrote another script to automatically parse the error log file and email me if there were any errors. As usual I thought of writing about it here so that it would be useful for someone else as well :)

Logrotate

First you need to install the logrotate program which will allow you to automatically rotate log files. In most servers it gets installed automatically. If it is not installed that you can install it using the command sudo apt-get install logrotate.

After you have installed the logrotate program you need to edit its configuration file. My configuration files looks like this.

The important part is the lines between postrotate and endscript. These lines get executed after the log file is rotated and I am calling my awk script to parse the error log file to get unique errors and then I am passing it to a python script. Kindly refer to my original post where I explained the awk script in detail.

Send email

The next step is to send the output of the awk script though email. For this I am using the following python program. This python program reads the lines from stdin and then send them through email. In my case, I am sending the email through mandrill. If you are going to use some other mechanism then you may have to change the send_email function appropriately.

That’s it. Once you have configured it, you would be getting email everyday with the unique errors from your apache error log.

Posted in Python, Unix/Server Stuff | Tagged , , | Leave a comment

Read data from Google Sheet into a Python Pandas DataFrame

Recently I have done lot of data analysis in Python (more details about this in another post) and have started to like Pandas a lot. The other day I had to process some data from a Google Sheet and was wondering whether I could read the data as a Pandas DataFrame and after a quick search found the gspread package and within a few lines of code I was able to read data from Google Sheet into a Pandas DataFrame.

I thought of sharing the code here so that it would be helpful for others and I would also know whether to search for it when I need it the next time :)

So here we go, this is all we need. Of course you need to install the gspread and pandas packages.

Posted in Python | Tagged , , | Leave a comment

WP Github Gist plugin now supports the new Gist API

Like my Twitter Avatar plugin, even my WP Github Gist plugin was broken recently due to changes in the underlying API that my plugin was using. This time it was because of the changes in the Gist Embed API.

Adding user to the gist embed url

In the earlier version of the gist embed API, the url to the embed script had only the id of the gist. But in the new version the user name is also added.

So I have added a new attribute called user to the gist shortcode where you can specify the user name.

Handling old Gist shortcodes

Unfortunately the above change breaks old gist shortcodes that you might already have in your posts where the user name is not specified. I have introduced a filter called wp-github-gist-user using which you can specify the user name for old gists. This will be helpful if you have embedded gist from the same user in all your old posts. If you have embeded gists from multiple users then I am sorry, you may have to go and manually add the user to all of them.

You can checkout the sample plugin that I have created that explains how you can override the user name using this filter. Essentially you have to just add the following code to your functions.php file or create a separate plugin like I did.

Download

This is a mandatory update. You can download the latest version of the Plugin from the Plugin’s homepage or install it using the Plugin installer.

Feedback

Please keep the feedback coming and if you want me to add any new features or find a bug, please leave a comment or post about it in twitter.

Posted in WordPress | Tagged , , , , | Leave a comment

Twitter Avatar plugin now supports the new Twitter API

Because of the recent changes in Twitter API, my Twitter Avatar Reloaded WordPress plugin stopped working. I rewrote the way my plugin communicates with Twitter and updated the code to use the new API and now my plugin works again.

New Twitter Backend

Instead of rewriting the entire Twitter backend again, I am not using the excellent wp-twitter-api, provided by @timwhitlock which allows you to easily communicate with Twitter API from your WordPress plugin.

If you are a WordPress plugin developer and want to communicate to the Twitter API from your WordPress plugin, then I would highly recommend you to use wp-twitter-api, instead of creating your own.

Setting up Twitter App

The new version of the Twitter API needs you to make authenticated calls even to get some basic information about a twitter screen name. So after you installed the plugin you need to follow these steps so that the plugin could interact with the Twitter API.

  • Register a Twitter application at https://dev.twitter.com/apps
  • Note the Consumer key and Consumer secret under OAuth settings
  • Log into WordPress admin and go to Settings > Twitter API
  • Enter the consumer key and secret and click ‘Save settings’
  • Click the ‘Connect to Twitter’ button and follow the prompts.

Download

This is a mandatory update. You can download the latest version of the Plugin from the Plugin’s homepage or install it using the Plugin installer.

Feedback

Please keep the feedback coming and if you want me to add any new features or find a bug, please leave a comment or post about it in twitter.

Posted in WordPress | Tagged , , , | Leave a comment

How to print unique errors(with count) from Apache error logs

After I moved to WordPress Multisite recently, I had to closely watch the Apache error log file for a couple of days to make sure that there weren’t any configuration mismatch.

Initially I used run the tail command to view the errors, but then I quickly realized that if there is a frequent error, then it becomes extremely difficult to find out other non frequent errors.

Command

After some fiddling with the awk and sort commands, I came up with the following one liner that prints the unique errors with their count.

Explanation

awk script is executed for each line in the error log file.

Field Separator

The -F directive is used to specify field separator. If you look at a single line in the apache error log file, you will notice that each field is enclosed by [ and ].

awk allows us to specify multiple field separators by enclosing them between []. That’s what we are specifying in -F[\[\]]

Filtering out lines that contain error

After splitting the fields, we need to filter out the lines that contain the term error in the fourth field. That’s what happens in the next part of the command. Basically the part $4 == "error"{print $7} prints the seventh field, which has the real error message if the fourth field is equal to the string “error”.

Sorting the error messages

The next step is to sort all the error messages. This is done by piping the output to the sort command.

Finding unique error messages and counting them

The next step is to find the unique error messages and then count them. This is done by piping the output to the uniq command. The -c flag does the counting.

Sorting the messages by frequency

The last step is to sort the messages again by frequency and print them. This is done by the last sort command. The -n flag sorts by numbers and the flag -r does the sorting by descending order.

Posted in Unix/Server Stuff | Tagged , , , , , | Leave a comment

How To Unfreeze Frozen SSH Tabs In Mac Terminal App

I use the Mac Terminal App a lot to ssh into remote servers. It all works great except for a single annoyance. If I leave the ssh tab open without any activity for sometime or put my Mac to sleep, then the tab gets frozen and becomes unresponsive. It takes a couple of minutes for it to log out of the server and become responsive again, so that I can reuse the tab again.

Whenever this happens, I have to close the unresponsive tab and open a new one, or wait for 5-10 minutes. I did some research about it and found that Mac Terminal App provides an escape command that can be used to forcefully log out of the remote server. All you need to do is to type the following command in the unresponsive tab.

~.

(Tilda followed by a dot)

I tried it and it works like a charm. It immediately logs you out of the frozen ssh tab and you can reuse your tab again without waiting for it to become responsive again.

Hope this small trick is useful for others as well :)

Posted in Unix/Server Stuff | Tagged , , , | Leave a comment

How To Properly Create Tables In WordPress Multisite Plugins

Recently, I added full WordPress Multisite compatibility to my Email Log WordPress plugin and during the process, I learned a lot about how to create and handle tables in WordPress Multisite plugins. I thought of documenting it here so that it would be useful for other plugin authors.

Creating tables in Single site installations

Before we get into WordPress Multisite, first let’s see how we can create a table in a single site WordPress installation. You have to hook into the register_activation_hook action, which will be called every time you activate your plugin and then check if your table exists or not. If it doesn’t exist, then you can create your table.

The following code shows how you can do it.

The above code will work even in WordPress Multisite installations, if your plugin will be activated individually for each blog. But if your plugin is networked activated, then the above code will create the table only for the primary blog.

Creating tables for all blogs in a WordPress Multisite installation

Now that we know how to create the table for a single blog, let’s see how we can create the table for all the blogs in a WordPress Multisite installation. Even in this case, we have to hook into the same register_activation_hook action, but should loop through all the blogs in the network.

The following code shows how you can do it.

Creating table when a new blog is created

The above code will create the table only for the blogs that were created before the plugin got network activated. We should also make sure that we create the table for every new that gets created. In order to do that we can hook into the wpmu_new_blog action.

The following code shows how you can do it.

Deleting the table when a blog is deleted

Now that we are creating the table for every new blog, it is our job to make sure that the table is deleted when the blog is deleted. In order to do that, we can hook into the wpmu_drop_tables filter.

The following code shows how you can do it.

Querying the correct table

When we are querying the table, we should always use $wpdb->prefix . $table_name. If we do that, then WordPress will automatically query the correct table based on the current blog. We don’t have to manually find out the blog id and add it to the table name.

Now your plugin should be completely compatible with WordPress Multisite :)

Alternate approach

If you look into the above code closely, you will notice that we are creating one table for each blog in the network. For most WordPress Multisite installations, this shouldn’t be a problem. But some WordPress Multisite installation may have thousands and even hundreds of thousands of blogs. In those cases, we might end up creating huge number of tables which might become a bottleneck. Also my plugin needed just one table. Some plugins might need more than one table, which might also increase the number of tables that gets created.

One alternate approach is to create just one table for all blogs and then separate out data for each blog using a blog_id column. While querying the table, we can filter out based the blog_id column.

If I had started my plugin from scratch, I would have done that instead of creating separate tables.

Removing the tables when the plugin is deleted

The other thing to keep in mind is that we should delete all the tables when the plugin is deactivated and deleted. I will write a separate article explaining how we should do that.

With WordPress Multisite becoming more popular these days, I hope this information was useful to you. Do let me know if you have any question or comments. Also you can checkout the entire code of my Email Log plugin in github.

Posted in WordPress | Tagged , , , | 6 Comments

Email Log plugin is now fully WordPress Multisite compatible

I just released version 1.7 of my Email Log WordPress plugin, which adds full compatibility with WordPress Multisite.

About Email Log WordPress plugin

Email Log is a WordPress plugin that allows you to log every email sent through WordPress and provides a UI where you can view them. The logged emails can be searched based on date, email address or subject.

WordPress Multisite compatibility

Earlier versions of Email Log plugin had some compatibility with WordPress Mu (older version of multisite), but I didn’t test it for long time. Also when WordPress Mu got merged into WordPress core, certain things have changed and my plugin was not fully compatible after that.

I came to know about it recently and then started to work on it to make it fully compatible with WordPress Multisite, since I have moved this blog to WordPress Multisite as well. I also learned a lot about WordPress Multisite during the process and will write a separate blog article about it soon. Update: As promised, here is the new blog article explaining how to properly create tables in WordPress Multisite plugins.

Dev Time

This release took me about 2.5 hours of development time. You can find more details about the dev time tracking which I have recently started doing in a separate blog post.

Download

You can download the latest version of the Plugin from the Plugin’s home page.

Feedback

Try out the Plugin and if you have any comments or if you want to report any bugs, please leave a comment below.

Posted in WordPress | Tagged , , , , | 1 Comment

Prevent users from adding new terms to custom taxonomy in WordPress

Recently I faced a scenario where I had to prevent certain users from adding new terms to some custom taxonomies that I created. After some research I found that there isn’t a straight forward way to do this using user capabilities or roles.

The only way is to use the pre_insert_term hook and black-list taxonomies based on user capabilities. I am sharing the code here so that it would be useful for people who want to something similar.

In the above code change the taxonomy name and capability to suit your usecase.

Posted in WordPress | Tagged , | 2 Comments