The “Automation” Guide for Web [Part 2]
Let the web do the work. A wide variety of tools are dedicated to saving you time. From email to social media to your own website, you could be wasting time doing things free services could do for you, automatically. Getting these services working takes a bit of knowledge, time and creativity, but they almost always pay off.
If you want to save time later, and are willing to do some setup work now, this manual can help.
Table Of Contents
Welcome to Part 2 of the MakeUseOf guide to Web Automation!
3.1 Automate your repetitive tasks in the web browser
These days, the web browser is probably the application you use most, but many of the tasks you perform using it are repetitive – checking on the same website everyday, remembering passwords, filling out forms, information gathering or testing websites over and over again. With iMacros, we can record tasks once and then let iMacros execute them whenever you need them. iMacros can even assist you during the recording with visual feedback. The iMacros extension is available for Mozilla Firefox, Google Chrome and Internet Explorer. After installing, you’ll see an iMacros icon in the browser toolbar and it will open in the sidebar.
The fundamental function of the iMacros is to record a task, and this can be performed by the Record button. Then just do your thing: iMacros keeps track of what you do and can play it back later. Let’s try this out. We’ll create a basic macro, to show you how it works.
- Let’s say you want to track the release channel information of Firefox, to see the changes in upcoming versions.
- Click the Record button by activating the second tab (Rec). It will start recording.
So as you see, I am in the Firefox Wiki page and my interest is to study the Release Tracking page. So iMacros will record the mouse click and sequence from this webpage onwards.
We can see that iMacros sidebar window consist of the variable information:
a. Version Build of iMacros.
b. URL go to – Firefox Wiki.
c. Release Tracking.
d. HTML Tree Editor.
- After recording the sequence, go to the first tab and click Play to see the magic.
- Now we’ll add this macro to a bookmark. Right-click it and select Add to Bookmark. Now we can launch this macro from your bookmarks with single click.
- We can Rename it, or we can see this macro in a New Folder.
If one of my friends is also interested in this information I can click the third tab (Edit) and click Share Macro. iMacros also offers the option to save the macro, to take the screenshot of the webpage and can even delete the cache and cookies. We can click Edit Macro to view all the code it generates during the recording. An entire reference of the commands and how to use them can be found here.
3.2 Automatically fill in repetitive web forms
We all use them constantly: Web forms. They’re any form on a webpage that allows a user to enter data. They can be used:
- to enter shipping or credit card data to order a product.
- to register in forums for interacting with the community.
- comment forms to submit your viewpoint on the topic.
- customer support forms to enter additional information like product serial numbers or models.
In Opera and Google Chrome the auto-fill functionality is available out of the box. In Mozilla Firefox, we have the option of using the InformEnter, Fireform, and Autofill Forms extensions, while for Google Chrome you can use Autofill.
InFormEnter is a flexible add-on that can semi-automate the process of filling out forms in the browser. It adds a small, clickable icon next to every input field in a web form, from where you can select the item to be inserted. You can configure this add-on to display your frequently used information. What I like about this add-on is its simplicity and support for different profiles so that I can use it in various situations. As you can see, you can add as many profiles you want.
- Create a new profile: Comment Form as shown. Click Ok.
- In this comment form we will add our name, email address, and a spam check entry (it may not be valid for every comment form).
- Input the menu item per line, which is name in one line, email in another and so on. After entering all details click Ok.
- If I wish to comment in any article of MakeUseOf, just right click andenter the details. Do you see the blue-colored marker in every field?
Simple, right? It’s a little more useful for longer forms, but I’m sure you get the idea. You could also make profiles for logins, such as Gmail and Facebook, but remember: this add-on does not give any option for encryption. Therefore I recommend you use a password manager for this functionality. Personally, I use Keepass to manage all my passwords. It is open source and gives you an option to auto-fill login information. Just take a look at this documentation.
While you can sync the Keepass database with another computer by using Dropbox, it would be convenient if the database can be synced with any browser and device and also if you could access the database from anywhere in the world. For that there’s Lastpass and Dashlane , with the option of storing the password, automatically, fill the web forms with multiple profiles and even apps for your mobile. So, it’s up to you to find the right application for the job.
- It’s happened to all of us: something goes wrong while filling out long web forms. There could be network issues, server issues, browser crashes, or power failures. So what you can do? Type it all again. There’s no need to, just install the add-on Lazarus: Form recovery. It will save your form securely as type, so you will never lose your work.
- Most auto-fill web form tools permit us to create multiple profiles, but do we know how much sensitive information is stored in their database? Not only that, how do you clear private information, or take a backup of it? Now, it is possible to entirely control what is being stored and what needs to be cleaned or backed up. Just install the add-on form history control to manage web form data.
- Do you register in forums by using your personal email address? You may not think there is a problem in doing this. However, if your email inbox is loaded with spam messages from that forum, then it’s really a problem. My recommendation is to create a disposable email address. You can use the add-on Trashmail.net [Broken URL Removed] or you can even create a separate disposable address for Gmail and Yahoo .
- Do you ever find yourself grabbing the mouse seconds after going to a webpage, just to select an input form that should have been made active by the website? Cycle Input Focus removes this strain and you can keep your hands on the keyboard. I know we can use tab function on the keyboard, but with this extension you can scroll back and forth between input forms with a keyboard shortcut.
- If you want to test any of these extensions before introducing it into your workflow, I highly recommend creating multiple profiles in Firefox . This way you can keep your main profile tidy and test add-ons in a different profile.
3.3 Automate Your To-do List
A to-do list is just a list of things you need to do. Simple, but if used well, can help you process and exercise conscious control over the amount of time spent on specific activities. This is done so as to increase effectiveness, efficiency or productivity. What should an effective to-do list include? How should I write them? How to get the most from them? These are the frequent questions, all of which we will encounter while creating to-do lists.
- Your calendar is your friend. Plan in advance.
- Priority tasks should be in an upper hierarchy. See whether it can be divided into chunks.
- Color code the prioritization, but don’t make everything red.
- Choose the application that works for you best. Don’t consider an app that is overloaded with features. You may think it’s good, but at the end of the day, if you still can’t figure out how to use a particular feature it’s not useful.
- A to-do list is a formulated plan, not a mundane or ancillary task list. For example, this list is not useful:
- read email.
- make coffee.
- read daily newspaper.
- write an email to the following person.
- Make your to-do list goal oriented. Avoid describing the action and instead pinpoint the result.
- A to-do list is not your journal or diary, so arranging it in a chronological order will probably be of no benefit.
With that in mind, let’s discuss some tools that help keep your to-do list everywhere.
Remember The Milk
Remember the Milk is a simple, easy to use, feature-rich online to-do list that you can access from multiple devices. This include Android, iPhone, Blackberry and even Twitter. It also offers integration with Gmail and Google Calendar. You can tag and prioritize your tasks with different colors, and you can create different to-do lists for various purposes. Signing up for Remember the Milk is simple: just give your name and email address, then choose a desired screen name. After you’ve verified the email address you can log in to the dashboard and you can start entering tasks. There are three basic tabs (personal, study, work) that can be used to organize your tasks.
After adding a particular task, you can add a smart-add shortcut to it. Basically, this is just a syntax that will be a lot of help when you use a web app in relation to Remember the Milk (such as Gmail and Twitter). As you can see, I added a task with priority 1 with syntax (!). Now we can add a due date, tags and notes to this particular task.
There are two other tabs (Inbox, Sent) that are useful if you use this web automation tool with colleagues in a workspace environment. If someone sends you a task, it will show up in the inbox with all information included. If you need to send a task to someone else, you can do so by clicking on the task you’ve created, click the drop down “More Actions” button and send it to one of your contacts. At any point you can postpone a task or mark it as complete. If any task is due during that day or is overdue, Remember the Milk will automatically bold and underline the task.
One of the best parts of this tool is the availability of keyboard shortcuts, and apps for mobile devices and email. Your to-do list is automatically present everywhere and you can access it anytime. You can study some more tips about using Remember the Milk here .
This one is similar to Remember the Milk. It manages to incorporate Gmail, Launchy and even mobile support in a seamless way. After you sign up, you’ll be taken to a dashboard with a quick tutorial on task creation. After I enter a project name, you can add a task to that particular project, with a due date. You can sort the task by date, priority or name.
Todoist is available as a plugin for Firefox and Chrome with integrated support for Gmail and Outlook. With Todoist Premium you get additional features: email and SMS reminders about upcoming tasks, label auto-completion, add email as tasks and SSL security. One of the best parts of this tool is again their availability of keyboard shortcuts and apps for mobile device and email with auto-reminding feature.
Above all, these are the two awesome tools, but sometimes a printed to-do list is best. Here is a cool website, where you can take a printable to-do list of various categories (simple, complex, multiple to-do list, informal to-do list, checklist and so on).
While a to-do list is an efficient way to plan and prioritize your activities or tasks, I must also remind you that it can make you inefficient and stop you from getting things done. Confused? I was reading this article on cognitive psychology and I learned of an effect called the “Zeigarnik Effect” which “is the tendency to experience intrusive thoughts about an objective that was once pursued and left incomplete” (Baumeister & Bushman, 2008, pg. 122). The automatic system signals the conscious mind, which may be focused on new goals, that a previous activity was left incomplete. It seems to be human nature to finish what we start and, if it is not finished, we experience dissonance. So what can you conclude from this? Have you experienced dissonance?
This is quite bad enough since your subconscious mind is still working to remind you that you aren’t working effectively. I am not a big fan of GTD tools, as I can effectively utilize the features of Followup.cc. If you are able to make it work, then you are on the right track.
3.4 Retrieve Citations For Your Research
A citation is a way of giving credit for a quote or idea. A number, usually in square brackets, points to a list of sources at the bottom of the paper. They look something like this:
“Facebook was founded by Mark Zuckerberg with his college roommates and fellow students Eduardo Saverin, Andrew McCollum, Dustin Moskovitz and Chris Hughes” 
If you use Wikipedia you know what I’m talking about: numbers like these refer to citations in the “References” section at the bottom of every article. In the academic world various types of citation systems and styles are used in research papers (it depends on the subject). Citation is important because:
- It is the best way to avoid academic plagiarism.
- The information becomes reusable. When we cite, credibility is returned to the source of information that helps readers to go back to the source for further in-depth study.
- Citation helps to respect and uphold intellectual property.
Gathering citations can be the most mundane part of academic writing – and that’s saying something. Let’s discuss some tools to manage citations.
Citelighter is a fully automated bibliography, research, citation and text highlighting tool that organizes your content quickly. It works as a plugin, which you can install in Firefox, Chrome and Safari.
To get started, register with an email account or through Facebook Connect. Next, you have to download the plugin. It shows up as a toolbar after you restart your browser.
Start a project by clicking the “Create” button. You can also set a due date for project completion and set reminders.
Start collecting the information. Currently, it works with web pages but not yet with PDF files.
Highlight all relevant information with the mouse by selecting the text and then clicking the “Capture” button.
After you captured the content, you have the option to edit the citation where you can fine-tune the information by adding some more comments (Publication Date, Author Name, etc.).
As you collect the information, you can click the “View” button anytime, and the pop-up window has three columns. The left column is related to additional settings with the added information (Send to Word or Email). The middle column is the actual content, wherein you can add some additional comments. The right column offers “Suggested Articles from Citelighter”, which I think is a key feature of this tool.
Scroll down and you can see the generated bibliography for that information with 3 types of citation styles (APA, MLA, Chicago), you can edit it if necessary.
Once you have completed your research, press the “Export” button to send all the organized data or any selected entries to your preferred word processor or Email.
You can make as many web automation projects you want. There are even advanced search capabilities.
Recently, Citelighter partnered with Cengage Learning, to allow easy access of millions of credible academic articles including journals, magazines, newspapers and transcripts for $10 a month.
Zotero is a free, easy to use tool to help you collect, organize, cite and share your research sources. It lives right where you do your work – in your web browser. You can take notes on sources, create groups for collaborating on gathering sources with other colleagues/Zotero users and much more. To get started, register with an email account. Download the add-on for Firefox, install and restart the browser.
Collecting Sources – The ease of collecting sources from the Web browser is one of Zotero’s strengths. If I am doing a project on Information Overload I have to collect many resources (articles, books, journals). Zotero instantly recognizes sources: if any webpage has content, you will notice a little icon in the address bar of web browser, as shown.
It’ll either be a little folder or a different icon, depending on the type of source (a book icon for books, text page for an article and if it’s a folder that means the web page has multiple sources). In the case of a folder, a window will open asking you to check the sources you would like to add to your library.
Organizing Sources – Click the icon in the address bar and the source will be instantly added to library. Click the “Zotero” icon in the add-on bar and you will notice your book in the library, as shown.
You can add information to each item record:
- Info: The bibliographic information used in citation and bibliographies.
- Notes: Jot down any supplemental information here.
- Attachments: Attach any file such as PDF to this item.
- Tags: Use tags to categorize your reference.
- Related: Use this tab to define relationship between resources.
On the far left you can see a folder. Using this, you can create any number of folders or sub-folders and organize them in a tree hierarchy.
Citing Sources – The first thing you need to do is to download the appropriate plugin for your preferred word processing software. Right-click the resource you would like to cite or hold down Ctrl to select multiple items, as shown.
Now choose your preferred citation style and choose Copy to Clipboard from the dialog box, as shown.
In your word processing program, Paste the citation. Done!
- You can configure Zotero to show duplicates. This is a hidden feature; to access it, type about:config in your address bar and hit enter. Right-click anywhere in the preferences window and choose New>Boolean. Enter the preference name
and click Ok. Choose True for the value and click OK. Restart Firefox and in the Zotero preferences Window you’ll see the option “Show duplicates”. It’s a very helpful feature if you want to filter out the duplicate articles in your library.
- You can use the plugin Zotfile to let you move, rename and attach PDF files to Zotero items. You can even sync PDFs from your library to your mobile PDF reader (iPad or Android tablet) and extract annotations from PDF files.
- Zotero is free and open source. You must remember that syncing the database is free, but if you are attaching PDF to any research paper, the limit becomes 100 MB. You can use WebDAV server to sync your PDF documents and my recommendation is to use CloudMe. It works with Zotero, but you have to configure it.
Do you think that automating the citation process will make you lazy? I feel that simplifying the tedious steps in the research process, like creating references and bibliographies, redirects your efforts to the steps that will actually increase your understanding, analysis and writing of the material.
3.5 Send your bookmarks or Evernote notebooks to your Kindle
Whenever you read an interesting article and plan to share it with your friends or save for your own reference later, a simple solution is to bookmark it. This is perfect if you work only using desktop computers or laptops, but most people today also have smartphones, tablets and e-readers. Why can’t we have access to our bookmarks everywhere? There are many services for managing bookmarks but online bookmarking has been always troublesome for me: you need to click through an extra 3 or 4 links just to save a bookmark. What if your bookmarking tool could do the heavy lifting for you, so your favorite stuff is bookmarked automatically and without hassle?
Pinboard, the popular bookmarking web app, includes a number of tools for doing just that. You can create new bookmarks, edit your existing bookmarks, and search your collection. Drag the Pinboard bookmarklet to your bookmarks bar and you can expect this service to work as advertised. What makes it different from other bookmark services is that it can integrate with other web app accounts, including Delicious, Instapaper, Pocket, Readability and Google Reader. Pinboard will automatically bookmark the sites and pages you add to such services, so you’ll always find the articles in Pinboard no matter where you saved them.
Pinboard has excellent support for Twitter, too. Just sign into your Twitter account via oAuth (up to three Twitter accounts per Pinboard account) and it will automatically save an archive of everything you post on Twitter. Twitter doesn’t save an unlimited history, so this feature gives you an awesome archive. You can also automatically save any links you share on Twitter, or links in your Twitter favorites.
The resources page lists all services that can be integrated with Pinboard.
I’ve been using Pinboard for months and I am very satisfied with the service. My biggest use for Pinboard is to save bookmarks from Pocket app to Pinboard and secondly to save starred articles from Google Reader.
By default, Google Reader doesn’t offer any Pinboard integration, but it does let you add other apps if you know what to enter. Just open your Settings page in Google Reader and click the Send To tab. Near the bottom, click Create a Custom Link and enter the following:
Icon URL: http://pinboard.in/favicon.ico
Save your changes, and now you’ll have a Pinboard entry in your Send to box, which is always at the bottom of RSS feed entries.
Another strategy for sending Google Reader items to Pinboard is using IFTTT, which we mentioned in Part 1 of this guide. You can use IFTTT to create a trigger with Google Reader and a number of other services, so head over to IFTTT and create your Pinboard recipe.
There is a Pinboard bookmarklet too. It’s available on Github, and works for expanded items in Google Reader as well as for web pages.
Sending Articles to Kindle
With bookmarking sorted out, now I want to send articles to my Kindle. There is a service called as Crofflr, which specializes in automated wireless e-book deliveries from your favorite reading list directly to your Kindle. It collects all unread articles from your personal reading list and converts them into a Kindle-compatible text version. This service currently supports Pinboard, Pocket, Longform.org, Longreads, and Give Me Something To Read. Its main features, according to its website, include:
- Mixing articles from different reading lists.
- Unlimited article count/delivery.
- E-book available in periodical format.
- Daily or weekly delivery of your e-books.
- Filtering your articles by tags and word count.
- Include optimized article images.
There are a plethora of free webpage-to-Kindle services, so you may be wondering why you should pay for this service. The best reason is that Crofflr will collect all your unread articles, convert them and send them to Kindle. So, the entire process is automated.
We can even combine lists of our bookmarks to make an eBook by using a tool called Readlists. This tool will bundle your group of links into an e-book and send it directly to your Kindle, iPad or iPhone. Every Readlist is shareable on the web and you can even embed it into your blog.
Visit Readlists.com and click “Make a Readlist”.
Now you see two columns. In the Left hand column you can enter a “Name and Description” to the Readlist; on the right side you can add any URL. Click “Add” and paste whatever you like; you can even edit the title and description, if you want. Once you have added the URL, in the left column you can see a plethora of options.
You can either “Download e-book” as EPUB and convert it to MOBI using Calibre or you can directly “Send to Kindle”.
Evernote to Kindle
In Evernote, we create notebooks to store our ideas and articles. While I was using this service on trial basis, I created lots of notebooks – which I wanted to send to my Kindle. Enter en2ki, a simple portable utility that can seamlessly convert Evernote notes into a Kindle supported format. You also don’t have to download your Evernote notes to local disk in order to convert them to the MOBI format. The only requirement of this app is having an Internet connection, since it has to fetch information from the Evernote API. Enter your Evernote ID and password and select the output folder where MOBI files are to be saved. Once done, click Create to start the process. It may take time if you have lots of articles in notebooks. Once it is finished, you can use Send to Kindle application to send the MOBI file.
3.6 Create your own personalized RSS feeds
RSS (Really Simple Syndication), as you probably know, is a family of web feed formats used to push updates to people. It’s used for blog entries, news headlines, audio podcasts and video in standardized formats. It includes full or summarized text, plus meta-data such as publishing dates and authorship.
There are a plethora of apps available to access RSS feeds, and out of these I really love Feedly and Taptu . While RSS feeds are great to access the content on daily basis, we cannot deny the fact that explosion of the Web has led to an overwhelming amount of content, making it more difficult to locate the best and most relevant. If there are 1000 feeds in your RSS app it is nearly impossible to digest all the information. That’s the reason for the popularity of content-curated sites. But other than this, is there any way to filter content? As it turns out, there is. We can create our own personalized RSS feeds. Here are a few services to check out.
Feedrinse is an easy tool that lets you automatically filter out syndicated content that you aren’t interested in, just like a spam filter. Sign-up is simple, and with a free account it allows you to filter up to 500 feeds. You can begin adding feeds either by adding one subscription URL per line or by importing an OPML file from your RSS reader.
If you use Google Reader it’s easy to download an OPML file. Just login in to Reader and navigate to Reader Settings > Import/Export.
Download the OPML file. Save it to any directory of the hard disk and import this OPML file into Feedrinse.
Now we’ll select some feeds so as to setup rules for your filter.
You have to set conditions for the post. In the first chapter we talked about how filters are composed of conditions and actions. So herein, the condition will be to “block the post” “if any” “post” “starts with” and action will be “Geek Comic”. All the conditions are customizable with pre-defined set rules and action depends upon the content. Of course, you can add more filter rules by clicking the (+) icon. After setting the filter “Save Changes”. You can still modify the filter later on.
In the next step, we’ll create a channel for that feed, start by giving channel a name. Click “Continue” and select the feed; you can add as many feeds here as you like.
In the final step, we have to select the reader from which I will get the rinsed feed.
The process is simple, but it may take time if you have many feeds to process. Right now I have created one channel, a technology section, in which there are 10 feeds with some specific filter rules.
Technorati is a popular blog search engine. It is an important tool for both readers and bloggers: readers benefit from a one-stop search engine to find articles about their favorite topics while bloggers get the benefit of visibility.
You may wonder: how is this different from Google? Technorati uses a system not hugely dissimilar to Google to rank its blogs. It’s called “Authority” [Broken URL Removed] and is calculated by looking at your blog content, what links are incoming and which links are outgoing alongside the type of categories and meta-tags you are using. It is based on a scale of 0-1000 with 1000 being the highest. You will see the blog Authority level only when you have added your blog to Technorati [Broken URL Removed] and if the reviewers approve it. The review is actually divided into two parts: (1) A claim token will be sent to you via email and you will have to put that claim token within a new blog post for a Technorati crawler to verify; (2) Once the verification is done, your blog will be reviewed by a human for approval.
You can predict the importance of the two level verification system. First: the content will be good. Second: you can create your own personalized RSS feeds from that content.
Sign-up for the account and verify your email. It is recommended you complete your profile (bio, photo, organization, etc) because this really helps once you decide to index your blog in Technorati. Type a topic in the “Search box” in order to receive RSS feeds specific to that search.
I have added “Facebook” in the search box (refine if necessary). Click the RSS icon, and in the next step you can select from where you would like to receive updates. Select the RSS feed and finally it will give you a link to that feed, which you can add to your favorite reader.
So now you can create your own powerful, personalized RSS feeds. If neither of these services work for you, check out Feedweaver (sadly, it’s not accepting any new registrations) and Yahoo pipes. I haven’t used Yahoo pipes personally; you can find some tips here .
4.1 Question yourself before choosing any cloud service
“The Cloud” – a term that became ubiquitous during the past few years – is now in every consumer’s mind. I just casually asked my geek friend after my laptop hard disk crashed “where are you storing data these days?” He said “The Cloud”. I was pretending that I don’t know anything about this technology, so he started explaining to me in detail about The Cloud’s advantages and unique features. I asked if it’s the same as “Cloud Computing” and he said “Yes”.
Of course, “The Cloud” lacks a clear definition, and after reading this article from Wikipedia I got a headache. I think “The Cloud” is a sort of a trendy, technical-sounding term for any consumer-facing applications delivered over the Internet. Does it match the characteristics of cloud computing in terms of agility, scalability, elasticity and multi-tenancy? Certainly, the cloud hype has blurred a lot of concepts. Nevertheless, these services are growing steadily in the popularity from small to even large corporations. Hosting your data in the cloud has consequences, and you need to be mindful before choosing any cloud service.
- Is the backup seamless? Is it correctly monitoring my folders on the computer and automatically uploading any content that it discovers?
- Do you understand the terms and conditions of service? Before choosing any service, you must agree to abide by their terms of service. Does the service claim any ownership of your data (i.e. will they use this data for advertisements or analysis)? Does the service have any provision for migrating the data in case they go out of the business?
- Will this company survive? Large, stable corporations will probably never end their service abruptly, but what about smaller firms?
- What will happen to your data if you are not using their service or cancel your account? Will it be securely deleted? What technology is used to securely delete files? If the data is still on their server, how long will it remain – a specific duration, or forever?
- If your account is terminated, what will happen to other services linked with that account? For example, if a Skydrive account is linked with Xbox or email account, will losing one account eliminate the others?
- What security is used to encrypt files? You must have heard about the security mishaps regarding Dropbox. Could your data be at risk? Does the service have two-step verification
if you want it? How is the data transferred between your device and servers? Is it encrypted?
- Where are the servers hosted? Is it in the USA, China or somewhere else? If the location is far, will that affect download and upload speeds, or latency? And what are the legal implications of hosting data in this country?
- What is the billing model? Is credit card charged automatically, or will you have to pay manually? Are charges based upon traffic, usage or storage limits?
Have you frankly ever asked yourself these questions before choosing a service? You should.
4.2 Automate backups of your email and sort your files in the cloud
Do you regularly back up your files, documents, photos, and videos – but not your email? Emails are equally important, and if you don’t back yours up you should.
There’s a simple way: just set up a POP3 client and regularly download your emails. If you use Gmail, for example, simply enable POP3 access by going to Settings> Forwarding and POP/IMAP and see if the status of the POP is enabled or not. Use any desktop email client to download the copy of all your messages from the cloud to your disk.
If local backup is so simple, why should you backup your emails to the cloud?
- Perhaps setting up the desktop email client is too complicated.
- If the main Gmail account gets hacked, your backup stops working.
- If you have deleted some important emails by mistake you might lose them forever.
- If the Gmail service goes down, recovering will be hard.
With this in mind, here are a few simple ways to back up your Gmail account to the cloud.
A. Back up to new Gmail account:
Create a new Gmail account. Open it, then under Settings > Accounts and Import > Check email from other accounts (using POP3) > Add a POP3 email account, enter the email address of your main Gmail account that you want to backup. Within few hours, the Mail Fetcher will pull messages from the main Gmail account and will copy it into the new backup account.
B. Backup to Hotmail:
You can copy email from Gmail into Hotmail by using TrueSwitch. It can backup your email address, address book, calendar and contacts. Setup a new Hotmail account and fire up TrueSwitch. It will copy all your emails and attachments from Gmail to your new Hotmail address.
Of the services discussed, they have one common drawback – they’ll backup your entire Gmail account and if your inbox contains many mails and attachments, it can take a significant amount of time. If you want to backup specific labels/folders then try Backupify. The free account will allow using 3 services with storage of 1GB weekly.
C. Backup to Dropbox:
If your Gmail storage is filling up very fast and you just thought about backing up those attachments then complete backup is not the solution, and neither is backing up any particular label. Don’t panic: you can send email attachments to a Dropbox account just by forwarding them to a special address, using Send to Dropbox. Any emails that you forward to it will be processed and their attachments saved to your Dropbox account. It also provides you other features: automatic archive unzipping, folder organization and message copying.
Another such tool is MailDrop. It checks your email account via IMAP; any attachments in a specified folder/label in Gmail are then directly saved to your Dropbox. For this to work you must create a label in Gmail named “Dropbox”, then anything labeled “Dropbox” will be downloaded. This way you are automating the process of copying attachments from your email.
You can also use the extension attachments.me – available for Firefox and Chrome [No Longer Available] – to save attachments from your Gmail account to Dropbox. With this tool a toolbar appears on the right hand site of the browser whenever you open an email with an attachment. This lets you save the attachments by clicking on “Save to Cloud”. This application is also available for iPhone. It also allows you to share a file from Dropbox without leaving Gmail.
If you’ve been using Dropbox for a long time, you may observe that your files and folders in the cloud are disorganized and it’s becoming difficult to find files. There is a service that allows you to sort the files and folders using filters: Sortbox. Let’s look into the details:
- Authenticate your Dropbox account with Sortbox.
- Grant permission to Sortbox so that it can access your Dropbox profile.
- As soon you authenticate it, you will see a note, as shown. It mentions that a folder named Sortbox has been created in your Dropbox account.
- By default, you would see three filter rules created as examples. You can add new ones or delete them as per your needs.
- Now, open your Dropbox folder and paste any file or folder into the Sortbox folder. If it matches rules, it will be moved accordingly (it refreshes every 15 minutes and this cannot be customized).
- Now, add the Sortbox shortcut to the right-click “Send to” menu in Windows. This way you can send any files/folders to Sortbox and the Sortbox in turn will sort it according to rules. Visit their FAQ to see more sorting options.
4.3 Schedule cloud hosting data backups
We have already discussed why it is important to question yourself before choosing any cloud service, and right now every small and big company is investing in cloud storage solutions. You probably know about Dropbox, Google Drive, Microsoft Skydrive; let’s discuss their plans and storage space. What is more important now?
- How do you use that extra storage? Do you keep only documents?
- Do you keep an entire library of photos, videos, music and eBooks synced to the cloud?
Extra storage is awesome, isn’t it? Before using any of the cloud services there are some factors and features to consider.
- Does it upload new files immediately?
- Does it allow syncing and sharing of data?
- Can it backup files while you have them open?
- Is there a Web client for restoring files?
- How much does it cost?
- Does it provide private encryption?
Choosing one service won’t meet all our needs or our budgets. Finding the right combination of web automation services isn’t always easy but nevertheless it is necessary to protect our important and growing collection of data. My recommendation, however, is Amazon S3.
Amazon S3 (Simple, Storage, Service)
Amazon S3 is an easy and inexpensive online hard-drive from Amazon Web Services (AWS). You may think that Amazon S3 storage service is meant for web start-ups or industry, but that’s not entirely correct because anyone can get benefit from it. For example: you can back up your eBook or music collection on S3. Webmasters can use S3 to store different type of media without worrying about bandwidth bills.
To use the S3 service you need an Amazon account and an Amazon AWS S3 account. After you sign-up, you have to generate a unique Access key ID + Secret Access key pair (don’t share this key with anyone). Create a “bucket” (like a folder). You can name it anything you like but the name must be unique across the Amazon system. Now we have to find some backup software that can connect with S3 after providing the keys to log you in.
Bucket Explorer is a graphical user interface manager for managing “Buckets” in Amazon S3. With this tool, you can safely store your files off-site on S3. You can access your files from anywhere and share it too. You can schedule uploads, download and copy operations. It has a versioning system to maintain the archive of files. It has two versions; personal and team edition. With team edition you can share files with your team using Amazon S3.
Cloudberry Lab is a tool available for Windows to automate encrypted and compressed data cloud backup. It supports Amazon S3, Amazon Glacier, Microsoft Azure, Google storage and Rackspace. Currently its features include Scheduling Backup, Virtual disk, Block level backup, Networking backup and support for the command line interface to manage backup.
S3 Backup is a similar tool that supports Amazon S3. You can schedule the back with Windows Task scheduler. It has tree-like interface to include folders in the backup set.
Probably the best part of Amazon S3 is the low cost; you pay accordingly to the storage required. They also provide a CDN (Content Delivery Network) service through Cloudfront and if you are designing a website you need look no further. Its prices are inexpensive when compared to other CDN services.
Another service by Amazon is Glacier, which is again a low cost storage service that provides secure and durable storage for data archiving and backup. In order to keep costs low, Amazon Glacier is optimized for data that is infrequently accessed and for which retrieval times of several hours are suitable. With Amazon Glacier, you can reliably store large or small amounts of data for as little as $0.01 per gigabyte per month, which is a significant saving compared to on-premises solutions. If you are searching for low cost solution, this service looks quite good, though I haven’t tried it yet.
You may be thinking that there are other backup services as well, which I haven’t discussed here. Honestly speaking, once I tried Amazon S3 I did not look for another solution; even Dropbox runs on Amazon S3. If you want to try other services, you can go through this article on Wikipedia.
4.4 Automate your daily tasks in the cloud
Living in the age of the Internet is amazing. We use computers to browse Facebook, watch our favorite movies on Netflix or work in collaboration with someone halfway around the world using Google Docs. All of these are applications delivered to you using the collective power of thousands of supercomputers around the world.
There are many other daily tasks we use the internet for. For example, converting documents, images and videos to various formats, converting eBooks, renaming or encrypting/decrypting files, signing a PDF and many more. Wouldn’t it be wonderful to automate these things? Happily, there is a service called Wappwolf, which offers a suite of apps that connect to certain cloud services. It allows you to automate more of your life.
Step 1: Wappwolf can connect to Google drive, Dropbox and Box (sorry, Microsoft Skydrive users). When you first use the web app, you have to connect your preferred cloud service with Wappwolf:
Authenticate Wappwolf to either access only a particular folder (Limited access) or all of your folders (Full access).
Step 2: Select the folder that you would like to add actions to. Select from the tree structure (limited access) or explore folders and sub-folders in the detailed view (full access) as shown. When finished, click on “Next”.
Step 3: Now you have to choose from the list of available actions.
I will choose “Convert it to PDF”, then click “Add Action”.
As you can see, it is compatible with many formats. When Dropbox Automator runs an action, it typically creates two folders; one is the “processed” folder that consist of all the documents and a “result” folder for converted copies of the original files. So every time you add documents to this folder it will automatically convert it to PDF.
The result is instantaneous; I am yet to see a major delay (though the delay depends on the size of the file). There were some issues, however:
- The setup and organization of the app is confusing. Wappwolf has a different setup for Dropbox, Google Drive and Box. So if I want to make an automation in Google Docs then switch over to Dropbox, I have to log out from one and log in to another every time.
- The “Send to Twitter” action never worked for me; I don’t know whether it’s a recent change in the Twitter API or something else.
- Conversion of photos resulted in loss of quality. While it’s not an issue if that photo is being uploaded to Facebook, if you want to put that photos on a website it may not be a good option.
Wappwolf Account and Mobile Apps
You can also sign up and create an account in Wappwolf.com to view all its free and paid actions. The free account is limited up to 100 files or 100MB of data per day with 10 active automations. The file size is limited to 25MB and the check runs every 15 minutes (not customizable). Wappwolf also has its own iPhone and iPad app; an Android app is still pending.
If we combine IFTTT and Wappwolf then you can expect to automate lots of tasks.
- Process files via email: Wappwolf can trigger some actions only if you put files in a particular Dropbox folder. What if you want to process some files with Wappwolf coming in via email? Just set up an IFTTT trigger to save a particular file to Dropbox, then select the appropriate action in Wappwolf.
- Google Reader to Kindle: I use Google Reader a lot, and there are always some articles I want to save for later. What I do: star the article in Google Reader and then use IFTTT to save the article as a PDF in Dropbox. From there I use Wappwolf to send this article to Kindle. It’s a little convoluted, but it gives you an idea of how these services can be combined.
- Copy files from IFTTT to Skydrive: Recently IFTTT added a recipe for Skydrive users. So go ahead, create your own recipe and share it with world.
There are endless possibilities of actions when you combine these two services – after all, both services have their own pros and cons. Setting up such services takes time and moreover the delay of frequent checks. But that’s the limitation of both these apps. I recommend creating at least one chain of automated mechanisms and learn to get better at using these services.
5.1 Why is Google Analytics paramount for webmasters?
Every website exists for a reason. For the most part, they exist because you want to (a) sell a product (b) generate a sales lead (c) generate ad revenue (d) share knowledge and gain online presence or (e) provide support. Whatever your site’s reason for existing, you should start measuring your website’s guests to better understand them. Google Analytics provides a core set of tools for just that. This simple service can teach you:
- How many daily visitors you receive, and how many are new.
- Your average conversion rates, i.e. registration, downloads and purchases.
- Your most visited pages.
- The average visit time on the website and how often they come back.
- The average number of pages a visitor reads before leaving your website.
- Geographic distribution of visitors and what language settings they are using.
- Do visitors stay, or leave quickly?
- Traffic sources, i.e. which websites send visitors your way.
- What are your top selling products and the average order value of it.
- Where are your customers coming from.
- How much revenue your website is generating.
These are the basic points, but don’t be overwhelmed. Figuring out what everything means takes time, so become familiar with the terminology. The complete list of terms can be found here.
Here’s a quick rundown of features in Google Analytics:
- It enables you to track and compare all your visitors – from non paid organic search, paid ads, email newsletters, referrals, social media and links from within the website.
- It integrates nicely with Google Adsense, and can generate reports of the content earning you the most revenue.
- You can trace the transaction to campaigns and keywords and identify the key revenue sources. It will help you find data on how users interact with the product page till the order. It will also help you in identifying the key usability issues of website, if any.
- Analytics includes customized Dashboard where you can go through the abridged reports from the main sections of the Google Analytics.
- Site Overlay reports are the graphical representation of the popularity of links on your webpage.
- Map Overlay reports are the graphical representation of geographic data. It gives you a detailed, fascinating look at where your visitors are located on the globe. All the regions are color-coded.
- Data export and scheduling. It can be exported as CSV (best for its analysis in Microsoft Excel) and PDF (for printing). Reports can be even scheduled which will be directly sent to your email address. From there I forward to Evernote for storage and for writing key points.
5.2 Automate Google Analytics reporting
Are you manually grabbing data from Analytics and adding it to spreadsheets? I used to spend a lot of time doing that, and it was bad. I was not able to spend time generating quality content. I was a novice user of Excel and never got used to the complicated functions and formulas.
So I set up a way to get analytics data automatically .
GA Data Grabber
GA Data Grabber is a powerful automated Analytics tool used in Excel for Windows or Mac – it pulls the data directly from Google Analytics, AdWords, Facebook and Microsoft adCenter.
- Allows logging in simultaneously with multiple user accounts, and combining data over user accounts.
- Fetches the metrics that are available through the Analytics API. The results can be split by dimensions, such as traffic source or search keyword.
- Automatically generates charts and other visualizations from the data.
- Automatically calculates how the figures have changed compared to the previous week or month or to the same date range a year earlier.
- Is Compatible with all the versions of Microsoft Excel, including the ones for Mac.
Excellent Analytics is a simple Excel add-on/data grabber that lets you import analytics data from Google Analytics into an Excel spreadsheet automatically. It is an open source tool and has the following main features:
- Apply filters to create advanced queries with all dimensions and metrics in Google Analytics.
- Combine data from multiple sources.
- Define and calculate customized KPIs.
- Share workbook with other users.
- Compatible with Microsoft Excel 2007 or 2010; not 2003. Mac version not available.
The paid version of Excellent Analytics Pro can import all the rows from Google Analytics (not limited to 10,000 rows).
Google Apps Script
Google Docs is very popular among users but as of now its use is limited among webmasters. However, now you can use this Google Apps Script template to put Google Analytics Data into Google Spreadsheets. Once the data is inside a Spreadsheet, you can manipulate it, create new visualizations and building internal dashboards:
The dashboard is automatically updated and no coding is required. You can write your own script and get the important analytics data in a spreadsheet. Also check out the additional reference docs for Google Apps Script.
Besides these automated tools, we must understand that merely getting the analytics data does not help. You have to analyze it to get important insights about your website – or to troubleshoot certain aspects of your website.
- Optimizing 404 pages: How should you monitor a 404 page with Google Analytics? Set up a Google Alert so you can optimize things later.
- Performance of website: Have you thought about the website’s performance? There are many tools available to check the performance of your website, but you can get great information from Google Analytics.
- Tracking Form-Filled errors: They can be the common source of errors, and it hampers the user experience of website. Just think: the user encounters an error while filling out a form. This is horrible experience for the user, so track it with Google Analytics.
This is again a very small part of troubleshooting, but there are also tools that help you to spot problems in collaboration with Google Analytics.
- Annotations Manager: This is the Greasemonkey script that will provide Google Analytics annotation management tools for your Google Analytics interface.
- Web Developer Toolbar: This is an indispensable tool for webmasters and it offers an excellent browser error console and DOM inspector. You can quickly look up tools for cookies, form validation and errors, source code and many more.
5.3 Publish your blog content to social media automatically
We have already discussed that social networking is a great way to get your content out to a larger audience and engage with new people, but generating and sharing quality content is not an easy task. The situation becomes aggravated if you manage multiple blogs and social networking accounts . However, there are some excellent services that allow you to assist with this process.
Networked Blogs [Broken URL Removed] is a user-generated blog directory that provides a simple way to get more exposure for your blog. In their system your blog gets its own page, listing in their directory and can be easily syndicated to Facebook and/or Twitter.
Start at networkedblogs.com and click the “Add Your Blog” button. You need to authenticate your Facebook with NetworkedBlogs
Now you have to register your blog:
Fill in the relevant data such as Topics, Language and Tagline and accept their Terms of Service.
Confirm that you are the author of you blog; later on you can add co-authors if you wish. In order to confirm it, you have to either Install a widget in your blog or ask your friends to confirm it.
The next step is tedious; you have to install that widget in your blog by inserting a code in the blog template.
You have to do it only once for verification purpose.
I am using Google Blogger for the final verification step; your steps may vary. Login to Blogger Dashboard, click a small arrow as shown.
Then go to Layout. Once you are in the layout page, decide the position where you want to add your widget and click “Add a Gadget”, as shown here:
The process is slightly different for WordPress.
The best part of this tool is that you can add a “Blog Tab” to your Facebook Page. In my experience, this tab isn’t updated automatically. You have to wait or refresh your Facebook page to see the new post in that tab. When I was using this service it was completely free; now they’ve introduced some premium plans, so check it out.
Dlvr.it is yet another service. Pronounced “Deliver it”, it has a simple interface and gives you a whole lot of options to connect with services like Twitter, Facebook, LinkedIn, Tumblr, Delicious and RSS feeds. Once you will sign-up you simply need to add your blog or feed URL and select the options that are appropriate for you.
You can then route your content to any of the social networking sites.
With Hootsuite, in order to post RSS feeds to social media automatically, you have to first create an account and at least add one social networking account to it.
Once you have added the account, you will see each of your accounts listed in its own tab in the Hootsuite web interface. Locate the publisher link on the left sidebar and select RSS feeds from the available options there.
To add a new RSS feed simply click on the (+) icon and fill out the configuration options that pop up. You need to add your Feed URL, frequency of feed (hourly, 2 hr, 3 hr, 6…and so on) and the URL shortener for links – then click “Save Feed”.
Once that is done, the service will automatically post to the social networking profiles. As of now, Hootsuite will allow you to add 2 RSS feeds and 5 social profiles, which I think is sufficient for free users.
IFTTT, using RSS to Buffer. We have already discussed both of these services; now you can combine them. IFTTT has a recipe to connect your blog’s RSS feeds to Buffer. You can then schedule your post to be shared in both Facebook and Twitter. This is awesome but be aware: there is sometimes a delay. It also takes time and patience to set this all up. I tried this service and recommend you only use it if your blogging frequency isn’t high. Moreover, if you have more than one blog, creating and managing a recipe for each is a total hassle.
If you carefully follow all the steps, then your blog content is certainly reaching your audience. But have I specified how many members of your audience? Theoretically, you may think that once posted something should reach all your followers and fans instantly; this is not the case. The Facebook Edge Algorithm is constantly modified, and you may not even realize that your content is not reaching your fans. Moreover, there is still a controversy about whether Facebook penalizes third party tools, if you use them frequently.
So why depend on these web automation tools? It can damage your visibility, but there’s a potential fix. I tell my fans to add my page to their interest list, because Facebook is more likely to push content to readers if they do so. This is an opportunity to increase the exposure of your content and potentially expand your reach beyond your fan base. I would also recommend adding a tutorial video to one your tabs in Facebook, if you think this will help.
5.4 Schedule your website backup
How do you feel when you realize that something has just disappeared from your grasp forever? The proliferation of digital goods in our lives is a double-edged sword. On one hand it provides simplicity, portability and convenience; on the other, there’s the fear of losing it. Backing up your essential documents, photos and videos, online assets (email, Twitter archive backup) have become an important part of our digital lives. If you are a webmaster, you have one more thing to backup: your website.
Why backup a website?
- It’s not your web host’s responsibility to make backups available to you. I have gone through terrible experiences with certain hosts (I won’t name them here). They give you the illusion that they are backing up your website, but what if you decide to suspend the service? Will they give back your data? So when you select a web host, consider these three aspects,
- Are they making daily or weekly backups?
- What security measures do they use to store backups?
- Will they give your data back, or not?
- Storing the original website locally is not enough – If you are developing a website using a CMS (Content Management System), you may already have a setup in a local machine. But what if your computer stops working one day; what if there is a hard drive failure?
- Peace of mind – If you have a full backup of your site you can expect a good night’s sleep. I was once awake for two consecutive nights restoring my blog, so I know the value of sleep.
- Protection against security mishaps and hackers – The unfortunate reality is that no website is completely safe from being hacked. While you can perform certain steps or take precautions to lessen the likelihood, there are no guarantees. Anyone who runs a CMS must be aware about the vulnerability of databases (where all the sensitive information is stored). What if one day your database is compromised due to a XSS/SQL injection
? Some other reasons could be vulnerable extensions, file and directory permission issues or malware. Backups are your friend.
- Upgrade woes – Your CMS probably has a regular release schedule. While it’s good to get new features and security fixes, we cannot deny that some extensions can break, or otherwise experience compatibility issues. Be safe: back up before upgrading.
Here are some automated backup solutions for your site.
Backupmachine is a feature rich backup solution. It offers free backups, plus automated daily website and database backups. Its features are –
- Automated backups, depending upon the package type.
- Support of all web apps (blog, shopping cart, forum).
- Archive of the backups.
- Exclude any folders, file types or database tables.
- Download backups in zip format.
Backup Box offers a full-featured backup solution. It does backup your entire website (covers WordPress, Joomla and Drupal) but other than that you can move files from one cloud service to another, e.g. Dropbox to Microsoft Skydrive. They support FTP as well as SFTP. You can create automatic transfers of your database or files between the cloud servers and have the ability to archive and timestamp your transfer. As of now its free accounts are limited to 1 GB/transfer with 10 transfer/month. The pro accounts have unlimited data transfer up to 25GB total/month. Only the pro accounts have to ability to create a fully automated transfer of files and database.
CodeGuard focuses their features on malware detection, and alerts when your website is changed. CodeGuard offers both manual and automatic website backups and can even rollback changes. Their dashboard lets you see what is happening on your site and you can monitor it for any intrusion or malware. You can schedule the monitoring hourly, daily or weekly. Currently, there are no free plans and their premium plans starts with a limit of 10 websites.
Cron is a very useful, but sometimes confusing, tool for managing websites. According to Wikipedia “Cron is the time based job scheduler in Unix-like computer operating system. Cron enables users to schedule jobs (commands or shell scripts) to run periodically at certain times or dates. It is commonly used to automate system maintenance or administration, though its general-purpose nature means that it can be used for other purposes, such as connecting to the Internet and downloading email”.
In simple terms, Cron can be described as a scheduling calendar. Cron is a feature of many servers but is sometimes unavailable to the user. For example: GoDaddy does not give users access to Cron, whereas cPanel supports Cron. I am going to set a basic Cron function for Joomla website using JPrc Cronjob. It does an accurate simulation but technically it is not a Cron function.
- To set up a Cron function, download the extension and then we are going to install it using the Joomla Extension Manager. Go to Extensions > Plugin Manager.
- Search for JPrc Cron and Enable the plugin.
- Now we’ll configure the plugin, go to Components>JPrc Cron to access the controls.
- Click the Categories tab and click New.
- Give a Title. Save & Close.
- Click on Tasks tab and click New.
- We have to define the Cronjob. Give the task a name: I have given mine Akeeba Backup (currently it is an awesome backup solution for Joomla, and this Cronjob will help me to automate the backup). Select the category and choose the task type, in this case it is SSH command. Now we have to enter the command so that it will run in background.
The command is;
php -q/home/user/administrator/components/com_akeeba/backup.php &
If you have any issues, just see the documentation of Akeeba backup.
- Now here is the fun part. You have to set the scheduler. Just select your preferred input in each case, i.e. Minutes, Hours, Days, Months and Weekdays. As you set the set the schedule, the UNIX Crontab will fill the data by itself, so you don’t have to worry about it. Set the Status to “Published” and Enable Logs to “Yes”, so every time a Cron runs a line will be written in your log file. It is useful if any error arises during Cronjob.
- Save and Close.
Now your website will be automatically backed up. Since this is a development site I can now test and run as many extensions and I have set the cron to run hourly. If you are running a website with cPanel, you can set the Cronjob there itself, I haven’t done because the developmental site is free and the hosting company is not giving me the permission to run cron.
Setting up cron is complicated, and my instructions above are only one example. Your mileage may vary.
So now you have finished reading this manual. I hope you learned something valuable! Setting up web automation requires time and patience, but if most of your life is digital it’s worth doing. If not, you can decide which tools can help you in your regular workflow. Don’t blindly choose any tools. I would highly recommend that you make a strategy and use it on trial basis before you include it in your workflow.